Early work on a subterranean tunnel environment.

I wore myself out with several interconnected changes to how maps and tiles work internally. Maps now have more features and fewer arbitrary hard-coded limits, which is great. Unfortunately, I underestimated the amount of effort that this required. Between that, some hardware issues, and COVID-19 arriving in my city, I got very little done on the design side of the project since my last devlog post.

More Camera Work

There are a few limitations to how things can be framed with the camera system, caused by the ‘activation area’ being the same Tiled rectangle object that specifies frame boundaries. I split these into two separate object layers: the main layer for most typical scroll zones, and a secondary layer of shared boundary rectangles, for which zones in the main layer can serve as a proxy.

In this picture, there are two horizontally-scrolling rooms connected by vertical poles on the left and right sides. The main scroll zones are indicated by cyan rectangles, and the secondary shared zones are black (they’re kind of hard to see here because the cyan rectangles overlap them. In Tiled, to help navigate, you can change the opacity of a layer or make it invisible.) The top room is represented by Zone 1, and the bottom room by Zone 3. Zone 2 represents the vertical tunnel to the left with the pole. When you climb the pole to the right, the camera remains locked to the boundaries of the scroll zones, snapping as you pass into the next room. When climbing the left pole, the camera follows the player up and down. Climbing the two smaller poles to the left does not cause the camera to scroll vertically because they are contained within proxies of Zone 3.

This is a bit of a hassle to set up and maintain, especially if the shape and dimensions of a room need to be altered, but it does give more control over how the camera behaves within certain subsections of a room.

De-hardcoding Map Layers

Stress-testing on a map with 22 layers.

This is the kind of thing that I should probably have shelved for the next game, but I was feeling confident after the camera system improvements. I thought it would be cool to allow the player to travel through areas that overlap with other areas, like ride an elevator that travels behind other rooms. Bolero couldn’t really do that, except maybe through trickery with special-purpose actors.

This went from “I’ll just extend it enough to support dual terrain layers” to “I’m going to remove the layer hardcoding, allow for an arbitrary numbers of layers, and tag actors to different layers with a depth variable.”

Getting this to work was a slog:

• Nearly all of the collision detection code assumes that layer 1 is the only array where collisions should be checked
• The scene render function expects to find only a few specific, named layers that each serve one function
• The veiwport code expects layer 1 to contain the main scroll offset information, but layer 1 is now just whatever layer is at the bottom of the list in Tiled
• Navmasks only build map caches for layer 1
• Actors that modify the map (such as for pasting one region of tiles to another) only operate on layer 1

And on and on. As mentioned above, since actors can be on any arbitrary layer, they need to have a depth value to determine which layer they should collide with. Every layer can also have a depth value, and the scene runner will match the actor depth to a map layer with the same depth before processing the actor’s tick code.

If the layer doesn’t have a depth setting, then nothing can collide with its tiles. (Matching actor depth to just the layer’s index — that is, where it happens to reside in the layer array — would cause issues whenever a layer is re-ordered in Tiled.) When the scene spawns an actor, it takes on the map’s default actor depth setting, or 1 if not defined. When an actor spawns another actor, the new actor inherits the invoker’s depth. So every actor should get a sensible starting depth and not fall off the screen. (Hopefully.) Two layers on the same map sharing the same depth value is an error.

The mechanism by which actors transition between different depths is going to take some work. Currently I have an invisible “gateway” actor that changes the depth of other actors as they pass through it. But if the user finds an unexpected way out of an area that isn’t covered by one of these gateways, they will not transition to the correct depth layer, and risk falling off the entire map. I have the basics working now, and can circle back to this once I have more use cases.

Flipping and Rotating Tiles

Tiled supports flipping and rotating tiles by encoding this information into the three upper bits of each 32-bit tile value. I was hesitant to pursue supporting this because of having to deal with reading and clearing those bits in a programming language with no built-in bitwise operators, and uncertainty about how to do so within the context of the existing codebase (keep a separate array of flipping info? Read and clear the bits every time?)

Ultimately it wasn’t much work to get it operational, as I had code elsewhere for Tiled image spawnpoints that also reads flip bits using the LuaJIT bit library. One thing that did trip me up a bit: the third flipping flag is anti-diagonal, making the top-left coordinate become the bottom-right and vice versa. Combined with horizontal and vertical flipping, these flags are enough to get the eight different layouts that are possible with a square graphic.

Using the letter F for testing flipping and rotation, as you can tell its orientation and flipping at a glance.

At first, I couldn’t figure out the correct scaling and rotation to make anti-diagonal tiles show the same orientations as what was shown in Tiled. I gave up and wrote out all eight possible combination of all three flags to set the correct orientation. Later, after reading some posts on the Tiled forum and Github repo, and looking at how other LOVE Tiled implementations do it, I got the rotation and scale calls in the correct order.

Texture Atlas

The output of all currently-registered graphics packed into a single texture atlas. If all art doesn’t fit into one texture, the atlas tries again with a texture resized to the next power of two.

Speaking of things that probably should have waited until next time, I began work on a texture atlas module. An atlas is a large texture that holds many smaller textures. Atlases offer a performance advantage over storing many smaller textures separately, but this comes at the cost of increased setup and maintenance.

I had a much dumber reason for pursuing this, unrelated to performance: basically the engine only works with one background tileset. (Code existed to load multiple tilesets, but it was limited to selecting a tileset while rendering different viewports, which is not a useful feature in the context of the current game.) At some point, I expanded the tileset’s size from 256×256 to 512×512, and later to 2048×2048 (so 128×128 tiles that are 16×16 pixels.) A tileset of this size is virtually impossible to work with in Tiled. I’m only using a fraction of that space right now, and it’s already difficult to locate what I need in the tileset panel. It would be nice to work with multiple smaller tilesets in Tiled, and then have the game stitch them all together into one single tileset during startup (or during the build-and-package process), while the rest of the engine continues operating on the assumption that only one tileset is loaded.

But let’s not stop there. What if we relocate all sprite data to the same atlas as well? Tiles on top, sprites along the bottom. We can also trim sprites down to remove transparent padding, and sweep every tile and every sprite to check for duplicate image data which may be flipped or rotated. (More things that should be done at build time!)

I got a demo of the texture packer working as a separate LOVE project, but successfully generating an image with sub-images packed into it is not the same thing as actually getting it to cooperate with the rest of the existing codebase.

Splitting Tiles into tileDefs and cellDefs

The first step towards atlas integration and multi-tileset support. Tiles were part of the engine’s tileset structures, but if they are going to be merged and migrated to an atlas, then the data structure that defines a tile’s overall in-game properties needs to be separate from the structure that defines the graphic, and both of them need to be stored separately from the tilesets that they originate from.

tileDefs contain anything relevant to drawing a tile on the screen, including dimensions and animation parameters. cellDefs contain the terrain ID and spawn tag strings, and an index number for the associated tileDef. As these structures are populated, tilesets also maintain a mapping of their local cellDefs to their global 1D index value.

Tile and sprite deduplication

For every registered tile, compare its imageData against the imageData of every other tile, and merge tiles that are duplicates. I had flipping and rotations checks implemented, but I couldn’t get flipped deduplicated tiles to play nicely when the map attribute data also specifies additional flipping and rotation. I’ve disabled that for now, as the ability to rotate tiles in Tiled is a lot more useful than deduplicating them after the fact. (Tiled has the ability to flip and rotate a selection of tiles all as one structure, which is really nice for when constructing backgrounds.) The deduplication is still useful in that it merges all empty tiles on all tilesets with the same dimensions into one tileDef.

The function that handles this will become slower and slower for every unique tile detected, as every remaining tile needs to be compared against a growing list. To reduce time spent comparing pixels against other pixels, the sum of all RGB values in a tile is stored, and only tiles with matching sums and dimensions are examined further.

I had deduplication of sprites working in my initial atlas demo, but I haven’t touched it in the main engine branch after experiencing a lot of issues getting it to work with tiles. The sprites do get trimmed though, so that outer columns and rows of sprites which contain only transparent pixels are removed before being placed onto the atlas.

When placing sprites on the atlas texture, all rectangles are sorted by their dimensions squared, with the largest rectangles being placed first, right-to-left, bottom-to-top. (To ensure that they’re always ordered the same way given the same inputs, there are some additional tiebreakers based on the source spritesheet name and XY position on the sheet.) The placement algorithm is pretty dumb, but the majority of sprites in this game so far are pretty small, and there’s currently no shortage of room on the atlas. To speed up placement a bit, I added a spatial partition to reduce the number of rectangle-overlapping-rectangle checks to a 128×128 window around the current rectangle.

Anyways, once all tileDefs have been reindexed and the duplicates removed, we have to go through every cellDef and update their tileDef references. Then, we can deduplicate identical cellDefs, and update each tileset’s tile-to-cellDef mappings, so that when maps are imported later on, their layer data will be reassigned the correct values.

Multiple Tilesets Per Map

Finally, after a bunch of detours, I got this working.

I’ve noticed that while Tiled’s TSX editor allows you to copy and paste custom properties between tiles, it does not seem to support doing so with selections of multiple tiles in one shot. Copying and pasting 32 tile properties from one tileset to another in Tiled one at a time is a wrist destroyer, and I need to think of ways to avoid having to do that.

For structures like slight angled floors, I set up this template tileset:

It includes temp art for all angled variations of floor (green), ceiling (blue), walls (cyan), “steps” without a slope component (dark cyan), and bars and jump-through platforms (all that junk in the upper-right.) The hope is that I can just copy this as needed for new kinds of areas, and not have to painstakingly fill in the terrain properties over and over. I screwed up though, and made the first layout (bottom) without considering the need for additional ‘trim’ tiles, and had to reassign every custom Tiled property to reflect the new layout. Argh!

Tile Animation Rewrite

I disabled tile animations while integrating the texture atlas stuff. While commenting out code and reviewing the tileset files (and feeling sad that my grass graphics no longer swayed), I revisited the Tiled animation editor and the table data that it exports. I realized that it wouldn’t be a huge amount of work to migrate over to just using this editor, and not writing every tile animation by hand as source code. Besides being easier to work with, this also carries the benefit of arranging frames in any order, not just left-to-right. The base tileDef frame doesn’t even have to be part of the displayed sequence.

My main concern when looking at this earlier was that the timing for animations in Tiled is in milliseconds, and my game runs at 256 ticks per second internally. If I make two animations that need to be in sync, and one of them is twice as fast as the other, and there is a discrepancy because of remainders after the division, then they will begin to drift apart from each other. However, multiples of 1000, down to 125, do appear to divide down to integers when divided by 3.90625.

I’m thinking that this hazard is outweighed by the convenience of using the editor, and even if it does turn out to be a problem, I can just override the animation speed by adding a new property field to a tile. One limitation I am sticking to is to have only one duration speed for all frames of a given animation. Tiled gives you per-frame control, but I want to continue driving animations off of the scene’s elapsed_ticks timer, and not have to deal with additional state or iterating through sequences to get the correct duration in the middle of the tilemap drawing loop.

I might have to rethink how tile animations work if I need to cache chunks of the map to reduce draw calls. For now, I’m happy with the results, and happy to find that it plays well with flipping and rotation.

Slight Performance Decrease

I was disappointed to find that after implementing this stuff, the framerate on my (pretty sparse) testing map took a hit from about 500-550 FPS down to 300-330 FPS. I realize this isn’t really a huge deal, and there are still multiple optimizations that I can try implementing, but I was bothered by the fact that I can’t isolate what specific change caused the drop, and I’m paranoid about every instance of stutter. It’s definitely related to the render function for map layers, as removing all but one layer brings it back up to about 400 FPS. I went through the entire layer render function and localized any remaining table fields that are accessed more than once and things improved a bit.

What I really need is to not think about this right now.

Optimizing the Collision and Platformer Toolkits

Oh no I’m still thinking about it. Unrelated to the render times, the collision and platforming code can sometimes take a varying amount of time to execute. Again, I don’t know why, so I’m just attacking anything that looks like it could be optimized. I went through every collision function, removing ones that are never called in the current codebase, and attempting to flatten out some of the nested function calls.

I added a terrain cache to the navigation mask layers which stores the direct terrain tables for every cell in a layer, and then I went through the collision and platforming toolkits and migrated all terrain lookups to use the terrain cache. Before, whenever the code needed to check terrain attributes for a location on the map (if something is an obstacle, or is water, or harms the player, or what its slope angle is, etc), it did the following:

• Get cell value from map at XY
• Look up the cellDef for that cell, and get its terrain ID string
• Look up the terrainDef that matches the terrain ID string, and return it

Now it does this:

• Get terrainDef table from terrain cache at XY, and return it

The navigation mask already caches of a small set of attributes, namely whether a solid tile should block ingress actors on its top, bottom, left and/or right sides. (For example, acting as a solid while next to a sloped tile can cause actors to ‘snag’ on it, so we would disable ingress blocking in that case.) But these caches quickly eat up a lot of RAM, basically as much as a real map layer, and I can’t make a parallel table for every single attribute.

Performance Monitoring

In light of how this month went, I need a more disciplined approach to performance monitoring. I have been using this profiler library since Hibernator, which has been helpful in many cases, but it carries a pretty serious performance impact while active, so it isn’t something that you can run in the background while playtesting a level. There’s also JProf, which is fast, and provides a neat visualizer to inspect trace files. However, if you capture a lot of data within hot loops, it will struggle when it comes time to write that data out to disk. (There is also a realtime output mode which sends data to a port that I haven’t tried yet.)

Sometimes I will pepper code with love.timer.getTime() and print the output on every tick or frame to get a better idea of how long something takes. I decided to time love.update() and love.draw(), store the last 1024 samples, and render those out as graphs overlaid on the screen.

Testing the performance monitor. Light green graph: time to complete love.update(). Dark green graph: time to complete love.draw() (not including time spent drawing the graphs.)

300 creatures huddled together like in the screenshot above is not a realistic scenario that will be in the final game, but I was nonetheless curious why CPU time spikes as they bump into a wall and turn around. I looked over the code that handles turning, and found some things that could be optimized, mainly using more local variables and switching the contents if statements around so that the conditions most likely to be false are evaluated first. This brought a small reduction in CPU usage.

 

I could reduce but not eliminate those CPU spikes. Turns out it’s not being caused by the wall collision code at all. When the group turns around, more of them are briefly colliding with each other, which results in more CPU load as each one logs a collision event with each other on every tick.

I haven’t decided on how to monitor subsections of love.update() and love.draw() yet. This is probably OK to start with though.

System Failure

The PC that I use to test Windows builds stopped POSTing. Troubleshooting has been unsuccessful, and I don’t have any spare parts to swap in to test the remaining culprits (PSU, CPU, RAM, motherboard.) Of course, even if I wanted to buy parts right now, they’d be delayed for a long time. Back to the closet it goes.

Luckily, I was able to install Windows 10 on a spare laptop, so I’m back in business from a Windows testing standpoint.

That makes two major system failures in the past six months. The other was a failed OS upgrade that required a reinstall to get back to the previous version. I’ve been delaying the upgrade since then, but I’ll have to bite the bullet and try it by May, when support for this version ends.

Background Matte Tilemaps

Earlier, I had set up the main gameplay scene to link to a ‘background’ sub-scene with its own map dimensions, tile dimensions, and wrap + scroll-scale settings. I got the basics working, but commented it out at the time for some reason, maybe due to concerns about performance. It seems to run OK now (maybe recent optimizations have helped, or maybe it was never a huge problem to begin with), so I’ve started integrating it into the game as an optional property on a per-map basis.

This allows working around Tiled’s limitation of one tile width + height per map. Using 32×32 tiles for the background seems to have slightly better performance than 16×16. I also tried 64×64, and didn’t see much difference in that case, but it might help in situations where the background matte itself has multiple layers. On a lark, I also tried 8×8, 4×4, 2×2 and 1×1 tile dimensions, each of which requires more and more processing overhead. 2×2 stuttered a lot, 1×1 was basically unusable.

 

Itty bitty background tiles.

While implementing this, I realized that the tile atlas packer had a big oversight: it didn’t fully account for multiple tile sizes. I found a quick fix: sort all tiles from smallest to largest. As the current tile’s dimensions are used for the step values, nothing overwrites anything else this way, though it does leave substantial horizontal gaps in the atlas in some cases. I’ll have to come back with a better solution at some point (probably just treat tiles like sprite chunks and make the chunk arrangement function position them all), but this is good enough for now.

One neat side effect I hadn’t considered is that I can place one-off actors within the background scene for a bit more variation (for example, creating a sun or moon actor that doesn’t scroll with the rest of the tilemap(s).) Those actors would be separate from the actors in the main action scene.

Other Stuff

I spent time looking into ways of implementing seamless map-to-map traversal by either splitting maps up into chunks, having primary + secondary maps that swap priority at transition points, or having one large wrapping internal map that periodically has chunks grafted onto it. Going from place to place without a hard cut is really appealing to me. But I don’t think I can easily retrofit the current project to accommodate it. Even if I could, it might come at the cost of compromising all of the gameplay / session code that I’ve completed up to this point, and I am very much not interested in that happening. So maybe next time.

LOVE is limited to writing files to a specific save directory path, so you can’t load images, modify them, then save them back out to the project directory. (Lua’s standard i/o libraries are available from within LOVE, but their use is discouraged, and I don’t want to build up code that relies on it in case they are are somehow disabled in later LOVE releases.) I looked at options for image manipulation from Lua, and found the lua-vips package which provides bindings to libvips. It’s probably more powerful than what I need, but it can get RGB values for specific image pixels and save files very easily, so I’ll probably look into that when migrating the tile / sprite deduplication code to the build system later on.

I started migrating some core modules to a ‘shared core’ path. Files in this area are expected to be functional for both the main engine running in LOVE, and also the build system running Lua or LuaJIT from a terminal session. Eventually the build system will take care of map and tileset conversion, and also part of the atlas construction (maybe pre-arrange chunks on sets of 512×512 images, then the game can arrange those images on the atlas texture) so that it doesn’t slow down the game’s startup. Converting modules to work this way will take some time, as many of them rely on the game engine’s logging module (which in turn relies on love.filesystem), and the build system doesn’t have access to that.

Project Outlook

Progress has slowed. I’ll try to make more disciplined development choices in the next few months and hopefully make headway on this project.

Hibernator’s one-year anniversary is April 8th. There are some bugs related to the save system that I still haven’t fixed. I’d like to do a revamp of the game at some point, maybe once this project is nearing completion.

Anyways, that’s all I have for now. I’ll try to post another devlog at the end of April.