I’ve got nothing terribly photogenic to show this month, as most of the work I did was at the command line / terminal level. I did find an old build of the game from 2019, and I uploaded some footage of it here. (Warning: unfortunately, it has a very loud explosion sound that triggers intermittently throughout the video.) This build predates the player being able to grab his own thrown shot — in fact, the shot is hazardous to the player once thrown.

Devlog

Troubleshooting palette and map problems after the build system rewrite.

Summary

Rewrote Build System to be a secondary LÖVE application
Partial rewrite of resource management, moving away from global assets
Worked on Gale image loader; should have a proper 1.0 release in March
Worked on restructuring codebase to support multiple projects in tandem. Partially complete.

BuildSys

BuildSys is a headless LÖVE program that builds and packages Bolero 2 / LÖVE projects. It replaces a mix of Bash, Python and Lua scripts, and uses a nifty 3rd party i/o library, nativefs, to get around LÖVE’s file system limitations. Since it’s built on the same application framework as the games themselves, it’s able to use shared Lua libraries, and to load and modify Lua assets from the games directly. As one monolithic application, it’s also able to cache work that has been completed in a given session, so that future steps can just grab the cached objects in RAM instead of having to reload them from disk.

As it currently stands, the build procedure goes like this:

(If requested, flush all previously-generated output data)
Convert .gal images to .png, so Tiled and other dev tools have image files they can work with
Process one-off art (stuff that doesn’t require further modifications — palettes, ImageFonts, etc.)
Probe, palettize BG images associated with tilesets
Convert Tiled TSX tilesets to Lua source
Update, repack tilesets and BG images
Convert Tiled TMX tilemaps to Lua source
Update tilemap layer data to reflect new tileset order
Probe, palettize spritesheet images associated with AnimDefs
Load, process (trim) AnimDefs, and repack spritesheet images
Copy + preprocess nearly-finalized content to output directory

In basic chart form:

This is a bit simpler than the initial plan. There was going to be an additional step near the end, where all assets were combined into one global unit. Frames from all tilesets and spritesheets were combined into a single texture atlas; tiles from all tilesets were combined into one big list; all layer data in all maps was updated to refer to the global tile list. This is fine at run-time — in fact, the original boot-up process basically did this before loading the title screen — but it created a huge bottleneck in the build system where if even a single bit of associated data changed since the last successful build, then all of this globalized stuff would have to be rebuilt from scratch.

Part of the globalized texture atlas.

Consider the texture atlas as an example. During the build process, we deduplicate tilesets and animations, remove empty tiles from the resource list, and then assemble the remaining frames on the atlas. Yes, that’s better than redoing it every time at boot-up, but the whole atlas still needs to be regenerated:

If any tileset changes, even a tiny bit
If any background image associated with any tileset changes, even a tiny bit
If any animation def changes, even a tiny bit
If any spritesheet linked to any animDef changes, even a tiny bit

When the game is in development, 99% of all test runs are going to be incremental build-and-run cycles. I really want the time to build after tiny changes to be as short as possible.

Resource Loading and Unloading

This is new ground for Bolero 2. Until now, everything got loaded at boot-time. That’s incompatible with non-global assets, so I’ll have to come up with a scheme for loading and unloading assets on-demand. I was not able to actually accomplish this before the end of the month, but I’ll write a bit here about how I’m planning to handle it.

Just Globalize At Boot Time?

That’s certainly an option. Maybe I’ll throw my hands up and just go this route. It won’t scale with larger projects, though, or even projects which happen to use a larger base resolution for textures.

When to Load?

If we’re going to have assets loaded and unloaded at runtime, when should those actions occur? I’m probably going to tie them to world structures. Worlds in Balleg are basically a collection of rooms with some header data that ties them together as one area. Only one world can be loaded in a scene at a time, and while you can transfer between rooms in a world without resetting the scene state, transferring between rooms in different worlds requires the entire action scene to reload from scratch. (It’s also a hard cut.) So this seems like a good point to do some loading and unloading.

What to Load?

Maps and textures are things I absolutely want to be loaded and unloaded as needed. I could keep the header info in RAM, and just load the layer data and PNGs respectively as needed. I think I could reorganize the code to allow actor definitions and per-tick callback functions to be loadable and discardable, but these things don’t really, truly take up that much memory.

What if the game tries to read something that isn’t loaded?

The most likely scenario for this happening is an actor attempting to set an animation which does not have the appropriate texture populated in the atlas. I have three options:

Crash the game!
Log an error and use a stand-in graphic
Attempt to load the texture on-demand
- If unsuccessful, pick 1 or 2

I’ll have to get further into this before I make a call.

Texture Atlas

There are a few approaches I could take with populating a texture atlas on demand at run-time. I don’t think arranging every individual tile and sprite-frame is going to be feasible, so instead, it’s going to work on the basis of packed tileset and spritesheet textures.

Atlas arrangement would be dead-simple if I committed to making all tilesets and spritesheets one uniform size, like 256×256. In that case, the atlas could just be a fixed-size 2D grid. I want a bit more flexibility to load textures of different sizes, though.

The existing binary tree packing algorithm that arranges the global atlas is perfect for loading once and never reloading, and reasonable for loading once, then starting from scratch whenever different assets need to be assigned. It’s bad if you want to occasionally unload assets out of order, because freeing a node disconnects all of its children from the tree. Deleting a node near the root could require recreating most of the tree.

I’m going to try using a quadtree. The atlas will be a power-of-two square, and I’ll divide it into nodes with four children, each 1/4th the size of its parent. In this scheme, an occupied node cannot have any children, and so there won’t be dependency issues when freeing up a node.

The downsides are that there will be many more nodes created and left empty, and if the textures aren’t close to power-of-2 size, there could be a lot of wasted space on the atlas. The above diagrams are exaggerated examples, though, as the atlas target size is 4096×4096 and most tilesets and spritesheets are 256×256, with a few being 512×512. This leaves quite a bit of room for assets:

Gale Converter

An animation I made to help test the GraphicsGale image loader.
(Yeah, I’m struggling to come up with any relevant visuals at all for this post.)

So I fixed some more issues with the converter:

Added 15bpp and 16bpp modes.
Added some functions to combine layers (working on ImageData objects.)
Got per-layer transparency hooked up.
Added the dithered transparency mask which is used instead of alpha blending when an image uses indexed color.
Added a function to generate an ImageData containing only the GaleImage’s background color. It isn’t used directly with getPixel(), but it can be used with a custom compositing function to implement the background color.

The more I work on this, the more complicated the getPixel() function becomes. This is mostly because of how the Gale format contains multiple transparency rules. Some of them override the others, and they don’t necessarily behave the same across different color depths and format exporters.

I found this excellent breakdown of the Gale format on a Japanese blog (using Google Translate to read it), and this accompanying Gale decoder written in Haxe. The decoder only handles indexed color modes, but it does have support for older Gale file formats (made in versions of GraphicsGale older than 2009 or so.)

Cool, so why isn’t this on GitHub? The answer is I just haven’t had the time and energy to clean it up and write tests. (Or rather, I do have some tests, but they were written as part of BuildSys, not as a standalone script.) I’ll try and get this properly out the door by the end of March at the latest. In the meantime (or if I don’t follow through with that claim), if you want a beta copy, you can find it on the LÖVE Discord showcase from February 26th, or email me.

EDIT 13 March 2022: It is now on GitHub.

Multiple Projects Living In A Shared Apartment

I’ve been staring at this anthropomorphic egg jerk for a very long time, and I want to try out some other game ideas. Maybe something top-down.

I’ve got a problem: the code I’ve written is locked into the Balleg project. The engine is technically separate, but the folder structure basically only allows the engine to interface with one project.

It looked something like this before:

./bolero-2
  /res -- Project data
  /lib -- 3rd party libraries
  /src -- Engine is here
  main.lua -- LÖVE entry point

I’m reorganizing it to be like this:

./bolero-2
  /buildsys   -- Build system is here
  /projects   -- Project data goes here
    /bal      -- Balleg
    /demo     -- A bare-bones demo project
  /engine     -- Engine is here
    /rev-1-0  -- (Keep multiple revisions in case of breaking changes)
  /inc_lib    -- Pool of shared 3rd party libraries
  /inc_src    -- Pool of shared 1st party code and assets
  /output     -- Finalized projects go here
  main.lua    -- LÖVE entry point for build system

The project source and output folders look like this:

./bolero-2

  /projects      -- The source directory.
    /bal
      /assets    -- Resources that need prep/baking
      /res       -- Code and assets which do not need extensive prep work.
      /pack-in   -- Loose files to be included in final package. (README, etc.)

  /output        -- The output directory.
    /bal
      /dev       -- Development target. Will have another dir for Release (rel)
        /cache   -- Stores work-in-progress build data
        /game    -- Completed contents, suitable for a .love file
        /package -- Per-OS packaged versions of the game.

In the long term, it won’t be feasible to maintain numerous projects in a shared manner like this. Things will eventually break and require one-off fixes at a low level. When that happens, I will have to fork the entire bolero-2 folder. What this is intended to help with is the early, exploratory, experimental stages of projects, where fixes to the engine are broadly applicable to multiple codebases.

Closing

So that was a yawn-fest. Overall I’m feeling good about this direction (even if I’m not totally happy with my slow rate of progress), and I’m very satisfied with nativefs. I’ll try to get this all wrapped up and have something more interesting to talk about for the end of March.

(e 2022/03/13: Add repo link for galeOps.)

Feb 2022