Emulation Multi-platform builder scripts for GBA emulators

[patters]

I have coded some Python 3 emulator compilation builder scripts for most of the best GBA emulators. These mostly have the same features as the original Win32 executables, but the advantage is that they're multi-platform so if you're a macOS or Linux user you won't have to mess about making a Windows VM in order to try out the emulators.
https://github.com/patters-syno/gba-emu-compilation-builders

 

[FAST6191]

Might as well include what it handles

Python 3 builder scripts for the following Gameboy Advance emulators, in date order of initial release:

• PocketNES for Nintendo NES (by Loopy, Jan 2001?, later FluBBa, Dwedit)
• PCEAdvance for PC Engine (by FluBBa, Apr 2003)
• Goomba for the original Gameboy (by FluBBa, Oct 2003)
• SMSAdvance for SEGA Master System, Game Gear, SG-1000 (by FluBBa, Jul 2005)
• Cologne for ColecoVision (by FluBBa, Jan 2006)
• Goomba Color a Goomba fork to add Gameboy Color (by Dwedit, Jan 2006)
• MSXAdvance for MSX-1 (by FluBBa, Mar 2006 - v0.2 is best)
• NGPAdvance for Neo Geo Pocket / NGP Color (by Flubba, Jul 2008)
• Jagoomba enhanced Goomba Color fork for Gameboy / Gameboy Color (by Jaga, Nov 2021)

Anyway nice to see such a thing. Will give them a test when I can as some things there got tricky at points but even if it only avoided concatenating files then I would have been happy, if it dodges some of the actually needs to build a menu then so much the better.

 

[patters]

Contents​

Python 3 builder scripts for the following emulators for Gameboy Advance:

Emulator	Target System	Author(s)	1st Release
PocketNES	Nintendo NES	Loopy, later FluBBa, Dwedit	Jan 2001?
ZXAdvance	Sinclair ZX Spectrum 48K	TheHiVE	May 2001
PCEAdvance	NEC PC Engine / Super CD-ROM²	FluBBa	Apr 2003
Goomba	Nintendo Gameboy	FluBBa	Oct 2003
HVCA	Nintendo NES / Famicom Disk System	outside-agb?	Sep 2004
Wasabi	Watara Supervision	FluBBa	Nov 2004
SNESAdvance	Nintendo SNES	Loopy, FluBBa	Feb 2005
SMSAdvance	SEGA Master System / Game Gear / SG-1000	FluBBa	Jul 2005
Cologne	ColecoVision	FluBBa	Jan 2006
Goomba Color	A Goomba fork to add Gameboy Color	FluBBa, Dwedit	Jan 2006
MSXAdvance	MSX-1 (version 0.2 is most compatible)	FluBBa	Mar 2006
Snezziboy	Nintendo SNES	bubble2k	May 2006
NGPGBA	SNK Neo Geo Pocket / Pocket Color	Flubba	Jul 2008
Jagoomba	An enhanced Goomba Color fork	FluBBa, Dwedit, Jaga + various	Nov 2021

Purpose​

The scripts combine the emulator binary with the game ROMs and their required metadata into a .gba executable for Gameboy Advance. Since the scripts are written in Python 3 they will run on most present-day platforms, helping to preserve these technical marvels.

Features​

• All scripts:
  • -h for help
  • Drag and drop a selection of ROMs onto the shell window after typing the script name, to easily add multiple ROMs
  • Blank SRAM save file of the appropriate size can now be created automatically using -sav option
  • GSS patch file for EZ-Flash IV firmware 2.x (to force 64KB SRAM saves) can now be created automatically using -pat option
  • Patch file data is encoded within the script body - no external dependency
  • Optional overrides of file paths
  • Can clean brackets from ROM titles with -c option
• Some scripts (as applicable):
  • Splash screen support with -s option
  • Auto-detection of ROM types for emulators that support multiple types with specific header requirements
  • Region options and PAL timings are now auto-detected based on ROM naming
  • Boot-to-BIOS support with -bb option
  • Small ROMs suitable for link transfer (<192KB) can be marked in the game list with -m option
• MSXAdvance:
  • Detects appropriate mapper for added ROMs and records this in a previously unused byte in the header, can opt out using -nomap option
  • Use my new fork of MSXAdvance v0.2 which adds auto mapper selection
• PCEAdvance:
  • ISO and TCD tracklist support for PC Engine CD-ROM²
  • Some sprite follow settings for "Unscaled (Auto)" display mode (those featured in gamelist.txt)
  • CD BIOS automatically added when an ISO image is added, and titled with the ISO name
  • Can trim compilation to fit within 16MB PSRAM with -trim option (needed for certain CD-ROM² titles)
• Goomba:
  • Works around an EZ-Flash issue where some ROMs would cause duplicate game list entries
  • Can prefer ROM filenames in the game list rather than original ROM game titles, with -f option
• PocketNES:
  • Will, if present, lookup ROM checksums in PocketNES Menu Maker database (pnesmmw.mdb) for optimal game settings, sprite/memory follow for "Unscaled (Follow)" display mode
  • Can prefer game titles from PocketNES Menu Maker database with -dbn option
  • 256 byte alignment of all ROM data for optimum performance
• SNESAdvance:
  • SuperDAT database is mandatory (snesadvance.dat, some additional supported titles here)
  • Default graphics assets are encoded in the script, external skin files are ingested if present
  • Can accept headered or unheadered ROMs (.smc/.sfc)
  • Can export header-stripped ROMs with -strip option
  • Can prefer game titles from SuperDAT database with -dbn option
  • Verbose mode with -v option
• Snezziboy:
  • Each game must be bundled with its own emulator instance, though multiple games can be processed in one command line
  • Dat file database is mandatory (snezzi.dat), but can use SNESAdvance SuperDAT also (snesadvance.dat, some additional supported titles here)
  • Can accept headered or unheadered ROMs (.smc/.sfc)
  • Can export header-stripped ROMs with -strip option
  • Verbose mode, to mimic original snezzi.exe builder with -v option
• HVCA:
  • Reconstructs FDS ROM headers if they are missing, which HVCA requires
  • Adds an exit function for EZ-Flash IV / 3in1 / Omega flashcarts
• ZXAdvance:
  • On first run use the -e builder option to extract the emulator from the original ZXAdvance exe file, will accept v1.0.1 or v1.0.1a
  • Adds an exit function for EZ-Flash IV / 3in1 / Omega flashcarts
  • Will retrieve game-specific controls configurations from ZXA.INI
  • Can create a Pogoshell plugin integrating the game configurations from ZXA.INI with the -p builder option
  • Can clean and sort the inifile using the -c option

 

[patters]

Added PocketNES Menu Maker database support

 

[patters]

I have made a builder for SNESAdvance available at the above GitHub repo.

I was able to code it to produce the same output as the old Win32 application, relying on the SuperDAT file, thanks to this handy technotes.txt document. You can also replace the graphics assets in the same way as before, provided the relevant graphics files are present in the same folder. I did however extract and embed the default skin into the Python script to avoid having loads of file dependencies.

There is one thing I can't seem to figure out though. The official builder application is altering the autoscroll Scale value which is written out to the compilation. All the other flags and settings are written out to the ROM header exactly as the values in the SuperDAT but for some reason this one is being altered. I cannot figure out why, nor what the pattern is. For instance:

• Super Mario World - SuperDAT scale value 0A becomes 19 via SNESAdvance.exe
• Chrono Trigger - SuperDAT scale value 05 becomes 0C via SNESAdvance.exe
• Gradius III - SuperDAT scale value 32 becomes 80 via SNESAdvance.exe

My script follows the spec and writes the value from the SuperDAT, but I can see that the behaviour of the official builder gives more optimal display results. These bytes are the only divergence between my script and the original Win32 builder.

@FluBBa - do you remember why this would be? The relevant part of my code.

 

[patters]

Comparing hex numbers is hard on the brain. A bit more analysis does reveal a pattern, though as a coding amateur I'm not seeing how to replicate it perfectly:

Game​	SuperDAT hex​	SuperDAT dec​	output hex​	output dec​	scaling factor​
Super Mario World​	0x0A​	10​	0x19​	25​	x 2.50​
Chrono Trigger​	0x05​	5​	0x0C​	12​	x 2.40​
Gradius III​	0x32​	50​	0x80​	128​	x 2.56​
Castlevania - Dracula X​	0x1E​	30​	0x4C​	76​	x 2.53​
Secret of Mana​	0x64​	100​	0x100​	256​	x 2.56​
Super Metroid​	0x32​	50​	0x80​	128​	x 2.56​

 

[patters]

Ok got it. As the numbers grow larger the scaling factor trends towards 2.56 which resembles a power of 2, not x2.5.
The easiest way to get x2.56 without floating point numbers is to multiply by 256 (0x100) and then divide by 100 (0x64).
Applying this operation precisely matches the observed hex results. Hooray. Well this has been a fun challenge .

scale = int((scale * 0x100) / 0x64)

The result of building a compilation of all of the above ROMs in the original builder Vs my script:

SNESAdvance build script complete! Preservation goal achieved.

 

[DanTheManMS]

Excellent to see! Only suggestion I have is to add Snezziboy support. I personally liked Snezziboy more than SNES Advance, mostly because of the vertical scaling and the fact that you could configure the graphics settings on the fly, rather than relying on the builder and hardcoded SuperDAT values.

Snezziboy was also better at handling freezes caused by the game reaching out to the nonexistent sound core, so a lot of the patches in the Superdat in the format of "address=EAEA" (EA being NOP, No Operation) became unnecessary. Patches in the form of "address=42xx" or "address=DBxx" are speedhacks and are *mostly* universal and shared between the two emulators, but not always. For example, there are a few Snezziboy-specific patches we found for some of the most popular games:

https://web.archive.org/web/20090430142302/wiki.pocketheaven.com/Snezzi_dat

I've attached as much sourcecode as I have for the last v0.26 version of Snezziboy. It was downloaded straight from their SourceForge [EDIT: I mean SubVersion] page, before it went down.

Also just because it's relevant, here's me using the Snezziboy debugger to find speedhacks for games. You'll see the SNESAdvance.org homepage there somewhere, where the technotes.txt file once resided.

 

[patters]

Well this was quite a tricky challenge, but I've done it

The C code for the builder wouldn't compile on my modern MacBook so I ported it all to Python 3.
The builder for the Snezziboy emulator is a much more complex beast than the others. It builds a large memory map lookup table, and it also has to do some quite specific checks on the SNES ROM to determine HiROM versus LoROM which took some understanding. A bit of a hassle as Python doesn't have all the integer types that C has, but it was a well bounded problem and now I know way more about Python than I did, which was why I embarked on this in the first place.

Can you test it? It's only on the repo as code at the moment, I will wait for feedback before I publish a new zip file 'release'. Enjoy!
https://github.com/patters-syno/gba-emu-compilation-builders/blob/main/snezziboy_compile.py

What are the most impressive games to try other than the ones mentioned in the readme - any surprises? I see from an old post of yours that there used to be a published Snezziboy compatibility list, with optimal layer and BG settings for each game. Has any of that survived the great Pocketheaven cull?

EDIT - just fixed a major issue at 19:06 BST, which broke the compilations when passing in more than one romfile argument. If you downloaded it before then, please freshen your copy.

 

[DanTheManMS]

What are the most impressive games to try other than the ones mentioned in the readme - any surprises? I see from an old post of yours that there used to be a published Snezziboy compatibility list, with optimal layer and BG settings for each game. Has any of that survived the great Pocketheaven cull?

Yep, it was actually at the bottom of the Snezzi_dat page I already posted: https://web.archive.org/web/20090508192702/http://wiki.pocketheaven.com/Snezziboy_Compatibility_List

As for my personal recommendations, man this is getting nostalgic. I wrote the vast majority of those game-specific notes. We tolerated a lot more manual fiddling in-game at the time it seems, heh.

Super Mario World is going to be the first recommendation. Very good speed, no freezes, fully playable. One of the only games to ever need the "backdrop" option set, as noted in the compatibility list.

Other good ones to look into are Aladdin, Acrobat Mission, the first DKC, Mario Paint patched for controller input, and U.N. Squadron.

Unfortunately I'm going to be busy this whole weekend so it may be a bit before I get the chance to test.

I'm truly impressed at what you've done. Bravo sir.

 

[patters]

Cool. I've added the wiki and the compatibility list links to the Emulator Tips section at the bottom of the repo markdown.

 

[Takokeshi]

The C code for the builder wouldn't compile on my modern MacBook so I ported it all to Python 3.

Lmao dude. That's dedication

 

[patters]

Lmao dude. That's dedication

Lol. It's the best way to learn a new programming language. Gotta have a clear objective, ideally one that presents a decent challenge. Each emulation builder was slightly more difficult than the last, so this was perfect.

 

[hippy dave]

Thanks for these! Have you seen/considered HVCA at all? Another nes/famicom emulator that has FDS support. https://www.gamebrew.org/wiki/HVCA_GBA

 

[patters]

Just been having a look at HVCA - I can't even build a working .GBA file. If I use the merge.bat script with ROMs from archive.org I just get an Err.22 after the FDS splash screen. I've tried removing spaces from the ROM filenames, several different BIOS revisions, selecting a cart reset routine, ignoring the cart reset routine, using only a single game, multiple games, etc. I've tried the resulting binary in VisualBoy Advance (in Windows XP) and in OpenEmu (in macOS) - so it seems not to be a GBA emulation accuracy issue. I'm stumped. Any pointers @hippy dave ?

 

[hippy dave]

Just been having a look at HVCA - I can't even build a working .GBA file. If I use the merge.bat script with ROMs from archive.org I just get an Err.22 after the FDS splash screen. I've tried removing spaces from the ROM filenames, several different BIOS revisions, selecting a cart reset routine, ignoring the cart reset routine, using only a single game, multiple games, etc. I've tried the resulting binary in VisualBoy Advance (in Windows XP) and in OpenEmu (in macOS) - so it seems not to be a GBA emulation accuracy issue. I'm stumped. Any pointers @hippy dave ?

Honestly I hadn't tried it yet, but will try to mess with it next time I get Windows fired up - will let you know if I get anywhere. Thanks for looking!

 

[patters]

I've also tried the previous version of HVCA - v1.28. Still doesn't work. NES roms work fine on both that and the newer version v1.40, but I get Err.22 with any FDS image I try. I do see the animated FDS BIOS screen:

• On HVCA v1.40 it displays "Please Push,A.Button"
• On HVCA v1.28 it displays "Please Set Disk Card" (which apparently you should see when no floppy is inserted). OpenEmu on macOS shows this briefly before the game autoboots.

I was using a Windows XP VM. Maybe it needs Windows 98. I wonder whether this emulator may have been dependent on an improper dump of the FDS BIOS ROM which might no longer be circulated: https://web.archive.org/web/2009121...aven.com/boards/viewtopic.php?t=4709&start=30

Post automatically merged: Apr 10, 2023

Any ideas @DanTheManMS ? I can see you used to mirror files for HVCA back in the day: https://web.archive.org/web/20091219222921/http://www.pocketheaven.com/boards/viewtopic.php?t=4052

 

[TechieSaru]

Try adding the following 16-byte header to each of your FDS roms:

46 44 53 1A nn 00 00 00 00 00 00 00 00 00 00 00

nn - Number of sides to FDS disk (e.g. 1 side = 01)

 

[patters]

Try adding the following 16-byte header to each of your FDS roms:

46 44 53 1A nn 00 00 00 00 00 00 00 00 00 00 00

nn - Number of sides to FDS disk (e.g. 1 side = 01)

Brilliant, that's it - working now! Of course, back in those days most ROMs were headered. How these things have changed...

 

[hippy dave]

Go team!