CSE2 - The Cave Story decompilation project

Why would you have to use numbers? Just program it to accept a string for a data structure.

Oh right. Cuz C can do anything. I apologize, my brain too small and am still old fashioned at custom TSC. No seriously, how the hell did i forget that's already a thing? <NAM uses strings in TSC!

 

I know. That's why I had that strikethrough included. <NAM was made entirely with ASM, too!

<NAM - NAMebox (<NAMstring$)
Creates a namebox to be used alongside only <MSG (and technically, <MS4). The character count for the string can be however long, as long as it can fit in the game screen. Use <NAM$ for removing it.

 

You can't give names to variables in <VAR. Only 008-123.

I was thinking of improving it so that it could have names assigned.

Doesn't it also use flag memory?

That can probably just have its own memory in CSE2, but you'd still have to use numbers, sadly

Correct-ish, there's no reason why it would have to use flag memory (that I can think of), but it also doesn't need to use numbers. Where'd you get that idea?

actually, one reason: saving.
so... I'd have to change how the game saves, too.

gonna go look at how that works, one sec
edit:
predictably, it's in "profile.cpp"

here's the save object structure:

Spoiler

struct PROFILE
{
	char code[8];
	int stage;
	MusicID music;
	int x;
	int y;
	int direct;
	short max_life;
	short star;
	short life;
	short a;
	int select_arms;
	int select_item;
	int equip;
	int unit;
	int counter;
	ARMS arms[8];
	ITEM items[32];
	PERMIT_STAGE permitstage[8];
	signed char permit_mapping[0x80];
	char FLAG[4];
	unsigned char flags[1000];
};

i'm sure it would be easy enough (even for me) to load in the JSON library and change the structure to

Spoiler

struct PROFILE
{
	char code[8];
	json variables;
	int stage;
	MusicID music;
	int x;
	int y;
	int direct;
	short max_life;
	short star;
	short life;
	short a;
	int select_arms;
	int select_item;
	int equip;
	int unit;
	int counter;
	ARMS arms[8];
	ITEM items[32];
	PERMIT_STAGE permitstage[8];
	signed char permit_mapping[0x80];
	char FLAG[4];
	unsigned char flags[1000];
};

here's the code for saving a profile:

Spoiler

BOOL SaveProfile(const char *name)
{
	PROFILE profile;
	FILE *fp;
	const char *FLAG = "FLAG";
	char path[MAX_PATH];

	// Get path
	if (name)
		sprintf(path, "%s/%s", gModulePath, name);
	else
		sprintf(path, "%s/%s", gModulePath, gDefaultName);

	// Open file
	fp = fopen(path, "wb");
	if (fp == NULL)
		return FALSE;

	// Set up profile
	memset(&profile, 0, sizeof(PROFILE));
	memcpy(profile.code, gProfileCode, sizeof(profile.code));
	memcpy(profile.FLAG, FLAG, sizeof(profile.FLAG));
	profile.stage = gStageNo;
	profile.music = gMusicNo;
	profile.x = gMC.x;
	profile.y = gMC.y;
	profile.direct = gMC.direct;
	profile.max_life = gMC.max_life;
	profile.life = gMC.life;
	profile.star = gMC.star;
	profile.select_arms = gSelectedArms;
	profile.select_item = gSelectedItem;
	profile.equip = gMC.equip;
	profile.unit = gMC.unit;
	profile.counter = gCounter;
	memcpy(profile.arms, gArmsData, sizeof(profile.arms));
	memcpy(profile.items, gItemData, sizeof(profile.items));
	memcpy(profile.permitstage, gPermitStage, sizeof(profile.permitstage));
	memcpy(profile.permit_mapping, gMapping, sizeof(profile.permit_mapping));
	memcpy(profile.flags, gFlagNPC, sizeof(profile.flags));

	// Write to file
	fwrite(profile.code, 8, 1, fp);
	File_WriteLE32(profile.stage, fp);
	File_WriteLE32(profile.music, fp);
	File_WriteLE32(profile.x, fp);
	File_WriteLE32(profile.y, fp);
	File_WriteLE32(profile.direct, fp);
	File_WriteLE16(profile.max_life, fp);
	File_WriteLE16(profile.star, fp);
	File_WriteLE16(profile.life, fp);
	File_WriteLE16(profile.a, fp);
	File_WriteLE32(profile.select_arms, fp);
	File_WriteLE32(profile.select_item, fp);
	File_WriteLE32(profile.equip, fp);
	File_WriteLE32(profile.unit, fp);
	File_WriteLE32(profile.counter, fp);
	for (int arm = 0; arm < 8; arm++)
	{
		File_WriteLE32(profile.arms[arm].code, fp);
		File_WriteLE32(profile.arms[arm].level, fp);
		File_WriteLE32(profile.arms[arm].exp, fp);
		File_WriteLE32(profile.arms[arm].max_num, fp);
		File_WriteLE32(profile.arms[arm].num, fp);
	}
	for (int item = 0; item < 32; item++)
		File_WriteLE32(profile.items[item].code, fp);
	for (int stage = 0; stage < 8; stage++)
	{
		File_WriteLE32(profile.permitstage[stage].index, fp);
		File_WriteLE32(profile.permitstage[stage].event, fp);
	}
	fwrite(profile.permit_mapping, 0x80, 1, fp);
	fwrite(FLAG, 4, 1, fp);
	fwrite(profile.flags, 1000, 1, fp);

	fclose(fp);
	return TRUE;
}

maybe change it to:

Spoiler

BOOL SaveProfile(const char *name)
{
	PROFILE profile;
	FILE *fp;
	const char *FLAG = "FLAG";
	char path[MAX_PATH];

	// Get path
	if (name)
		sprintf(path, "%s/%s", gModulePath, name);
	else
		sprintf(path, "%s/%s", gModulePath, gDefaultName);

	// Open file
	fp = fopen(path, "wb");
	if (fp == NULL)
		return FALSE;

	// Set up profile
	memset(&profile, 0, sizeof(PROFILE));
	memcpy(profile.code, gProfileCode, sizeof(profile.code));
	memcpy(profile.FLAG, FLAG, sizeof(profile.FLAG));
	profile.stage = gStageNo;
	profile.music = gMusicNo;
	profile.x = gMC.x;
	profile.y = gMC.y;
	profile.direct = gMC.direct;
	profile.max_life = gMC.max_life;
	profile.life = gMC.life;
	profile.star = gMC.star;
	profile.select_arms = gSelectedArms;
	profile.select_item = gSelectedItem;
	profile.equip = gMC.equip;
	profile.unit = gMC.unit;
	profile.counter = gCounter;
	profile.variables = JSON.stringify(variables); // that's javascript syntax but you get the point
	memcpy(profile.arms, gArmsData, sizeof(profile.arms));
	memcpy(profile.items, gItemData, sizeof(profile.items));
	memcpy(profile.permitstage, gPermitStage, sizeof(profile.permitstage));
	memcpy(profile.permit_mapping, gMapping, sizeof(profile.permit_mapping));
	memcpy(profile.flags, gFlagNPC, sizeof(profile.flags));

	// Write to file
	fwrite(profile.code, 8, 1, fp);
	File_WriteLE32(profile.stage, fp);
	File_WriteLE32(profile.music, fp);
	File_WriteLE32(profile.x, fp);
	File_WriteLE32(profile.y, fp);
	File_WriteLE32(profile.direct, fp);
	File_WriteLE16(profile.max_life, fp);
	File_WriteLE16(profile.star, fp);
	File_WriteLE16(profile.life, fp);
	File_WriteLE16(profile.a, fp);
	File_WriteLE32(profile.select_arms, fp);
	File_WriteLE32(profile.select_item, fp);
	File_WriteLE32(profile.equip, fp);
	File_WriteLE32(profile.unit, fp);
	File_WriteLE32(profile.counter, fp);
	for (int arm = 0; arm < 8; arm++)
	{
		File_WriteLE32(profile.arms[arm].code, fp);
		File_WriteLE32(profile.arms[arm].level, fp);
		File_WriteLE32(profile.arms[arm].exp, fp);
		File_WriteLE32(profile.arms[arm].max_num, fp);
		File_WriteLE32(profile.arms[arm].num, fp);
	}
	for (int item = 0; item < 32; item++)
		File_WriteLE32(profile.items[item].code, fp);
	for (int stage = 0; stage < 8; stage++)
	{
		File_WriteLE32(profile.permitstage[stage].index, fp);
		File_WriteLE32(profile.permitstage[stage].event, fp);
	}
	fwrite(profile.permit_mapping, 0x80, 1, fp);
	fwrite(FLAG, 4, 1, fp);
	fwrite(profile.flags, 1000, 1, fp);
	

	fwrite(profile.variables, sizeOf(JSON.stringify(profile.variables)), 1, fp);

	fclose(fp);
	return TRUE;
}

That's the string reading part. The harder part is making a data structure system with indexes that are accessible via string... so that you can use it once.

that's why I'm thinking JSON, it seems perfect for the part
keys can be whatever you want, with as many as you want
values can be anything that can be serialized

edit: found a JSON library
github nlohmann/json

any thoughts on all of this?

 

Using JSON in assembly?! Why I ought to smack you!

I don't see any assembly here.
Does TSC count as assembly?
if so, I wasn't thinking of using JSON in it,
I was thinking of using JSON to store the "variables" & their data internally (in C++)

although, I was thinking of making it possible to make arrays & stuff in TSC

<ARD (array define)

<ARDNAMEOFARRAY<ARD

<ARS (array set)

<ARSNAMEOFARRAY[INDEX]<ARSdata<ARS

<ARG (array get; name may need to be changed as this may already exist)

<ARGNAMEOFARRAY[INDEX]<ARG

<ARL (array length; returns length of array. mainly used for getting the latest index of an array.)

<ARLNAMEOFARRAY<ARL

for example,
say you kill a Mimiga (dark example, but memorable)

<ARDkilledMimigas<ARD
<MSGKilled Jack!<ARSkilledMimigas[<ARLkilledMimigas<ARL]<ARSJack<ARS<NOD
You have killed <ARLkilledMimigas<ARL mimigas!<NOD<CLO<END

<ILT (is less than)
<IGT (is greater than)

maybe a <MTH (math) command?
<MTH:ADD(or MULTIPLY, or DIVIDE, or SUBTRACT, or SQRT, or ROUND, or FLOOR, i don't know)<MTHTOADD1<MTHTOADD2<MTH


also thinking of adding a FOR loop command (to loop through the arrays) but maybe that's not a good idea?

it would probably go like this
syntax: <FOR: incrementing variable (goes up by one at end of each cycle)<FOR conditional (needs to be true; if it's false, it breaks the loop)<FOR (code to repeat)<FOE
for example,

<FOR:<VAR:INCREMENTINGVAR<VAR0<VAR<FOR<ILT:<GVRINCREMENTINGVAR<GVR<FOR<ARLkilledMimigas<ARL<FOR 
     <MSGYou have killed <ARG:killedMimigas[<GVR (get var)INCREMENTINGVAR<GVR]!<NOD
<FOE(for end)

you could use it for repetitive stuff, too:
<MSG100 bottles of beer on the wall, 100 bottles of beer!<NOD
Take one down, pass it around,<NOD
<FOR:<VARasdf<VAR0<VAR<ILT:<GVRasdf<GVR:99<FOR
    <MTHSUBTRACT<MTH0<MTH<GVRasdf<GVR<MTH bottles of beer on the wall!<NOD
    <MTHSUBTRACT<MTH0<MTH<GVRasdf<GVR<MTH bottles of beer on the wall, <MTHSUBTRACT<MTH0<MTH<GVRasdf<GVR<MTH bottles of beer!<NOD
    Take one down, pass it around, 
<FOE
0 bottles of beer on the wall!<NOD<CLO<END

that's all for now, will edit this with new ideas later

am I being too ambitious?

 

Looking into the plans for the new <VAR system multiple times so I can get a grip of how it'd work, I can see this being more practical than classic <VAR. Hell, it looks like most stuff from <VAR is already taken care of from your new command list! I don't think though, that I understand how <ARS would convert "Jack" to a single value like 1 and add it to the killedMimigas string's total. Unless you plan on using universal defines for a separate .tsc file for all maps to read. Maybe an explanation will be needed...

 

Why make so many operations? Why not just have an <SCR that lets you program things normally?

I don't know. In Javascript, eval() is a lot less efficient.
I can see the point, though - it's just that I don't know how to do that yet.

Looking into the plans for the new <VAR system multiple times so I can get a grip of how it'd work, I can see this being more practical than classic <VAR. Hell, it looks like most stuff from <VAR is already taken care of from your new command list! I don't think though, that I understand how <ARS would convert "Jack" to a single value like 1 and add it to the killedMimigas string's total. Unless you plan on using universal defines for a separate .tsc file for all maps to read. Maybe an explanation will be needed...

I haven't seen the classic <VAR - do you have a link to the thread?

also,
<ARS is "array set"
so

<ARSkilledMimigas[<ARLkilledMimigas<ARL]<ARSJack<ARS

would set the array index[array's length] to "Jack"
basically, it appends "Jack" to the array, because arrays start at 0

equivalent of (in Javascript) killedMimigas.push("Jack")

so if killedMimigas was already
["Sue", "Toroko", "King"]
then <ARSkilledMimigas[<ARLkilledMimigas<ARL]<ARSJack<ARS would change it into
["Sue", "Toroko", "King", "Jack"]

 

I haven't seen the classic <VAR - do you have a link to the thread?

There is no thread for <VAR, unfortunately. Most custom ASM don't exactly have its own thread, unless theyre being released in bulk or contributed to a thread to dump them in for collecting. However, all the information on it is in my earlier post in this thread and in the Hackinator guide in Booster's Lab.

You may also need to just fiddle around with it in a modded version of vanilla for testing purposes.

 

There is no thread for <VAR, unfortunately. Most custom ASM don't exactly have its own thread, unless theyre being released in bulk or contributed to a thread to dump them in for collecting. However, all the information on it is in my earlier post in this thread and in the Hackinator guide in Booster's Lab.

You may also need to just fiddle around with it in a modded version of vanilla for testing purposes.

OK, thanks


Edit:
Because my free time right now is at an all-time low, I can't work on this (at this time).
If I release anything it will be in at least a month.
If someone else wants to work on it, that's welcome, but I can't spare the time right now.

 

Just downloaded the Oct 16th release of enhanced 2.0, and it runs without a framerate limit? If this isn't isolated to me, then newcomers might stop there.

Also curiously, I couldn't run it on my ImDisk ramdisk; which can run most things, except something like Windows Edge browser, or Left4Dead 2 which I guess requires a more complete emulation of a drive. For example it doesn't show up in disk management with this software. I used to have to use another ramdisk tool to get that working... I assume it'll work on a usb stick, but didn't test.
The error btw was "Couldn't read general purpose files". Tried copying over older versions for testing, and I guess it's always been that way. Loads and exits. Just noting.

 

Well that was embarrassing. So not only did CSE2E v2.0 ship with a broken Makefile, but it also had a broken frame-limiter.

Well, here's v2.0.1. Usually, for a hotfix release like this, it'd be v2.0.0.1 instead, but this update has an extra feature: since v2.0, I've gone through all the game's code and neatened it up, improving the formatting without affecting ASM-accuracy. Oh, speaking of, there are some ASM-accuracy improvements in this release too - apparently some inaccuracies slipped through the cracks in the last release.

Here are the GitHub releases:
CSE2 - Accurate
CSE2 - Portable
CSE2 - Enhanced

 

On this latest release my game froze right after the Curly fight. Only once though, worked on a retry.

e: also, upon reaching the end of the waterway section(before Ironhead), my screen goes black and I got this message:

https://imgur.com/sGfz6r4

Trying again I got "You find nothing of interest" instead. I can continue past that dialogue but then I'm just in total blackness. I can hear myself moving around but I can't do anything and it's seemingly a softlock.

 

Darn. Sorry about that: I usually test on Linux, where this issue doesn't happen. It's a really weird bug, actually: there's a bugged command in the Waterway's TSC file, '<FLJ850:0111', which causes the game to read from an invalid place in memory. If what it happens to read is set to 1, it branches to an invalid TSC event, causing the bug you see. This has never been an issue before, because the memory it read was always 0, but this time, for this specific version compiled with Visual Studio 2019, it's 1.

Here are fixed versions:
v2.0.1.1 - Portable
v2.0.1.1 - Enhanced

 

Thanks to Aar, some more bugs were found in the enhanced branch. They've been fixed in this update:
v2.0.1.2 - Enhanced

 

As per a user suggestion, I've made 60FPS optional in CSE2E. So if you like widescreen, but don't want to lose the original floaty physics, there you go.
v2.0.2 - Enhanced

 

Geez-Louise, has it really been that long since the last update?

Scope-creep is a real pain. This update just kind of kept accumulating features, meaning the earlier ones have been kept hidden all this time. Anyway, here's CSE2 v2.1.

CSE2 v2.1

Binaries:
CSE2 - Portable
CSE2 - Enhanced

Perhaps the biggest thing in this update is the port to the Wii U - yep, both the portable and the enhanced branches have been brought to the Wii U. And, unlike Cucky's Wii port, this one is built straight from unmodified CSE2 sources, so it should be much easier to maintain and keep up-to-date. The magic of mainlining.

To play CSE2 on the Wii U, your console will need to be soft-modded, with the Homebrew Launcher installed.

The next biggest change is probably the enhanced branch's new in-game settings menu: now, instead of relying on the DoConfig tool, you can change the settings from within the game itself. Unfortunately, a few options don't take effect until the game is restarted.

The settings available in this menu include...
* Keyboard input rebinding
* Controller input rebinding
* Resolution
* Framerate
* V-sync
* Smooth sprite scrolling
* Soundtrack

Notably, you can now perform full input rebinding. No need for a hacky old hardcoded WASD layout like the DLL loader had.

You might have also noticed the soundtrack option. Like the DLL loader, there is now a user-friendly way to switch between the game's original Organya soundtrack, and various others - namely Nicalis's, and Vince94's SNES soundtrack.

The addition of a soundtrack option marks a change in the direction of the enhanced branch: for all this time, the enhanced branch was intended as an improved version of the engine for modders, yet many people merely use it as a way to play an improved version of Cave Story. Now the branch has been repurposed to suit this unintended audience, while a new branch, enhanced-lite, has been introduced to fulfil the original goal of meeting the needs of modders.

Supporting multiple soundtracks complicates work for modders, who would either have to create multiple versions of each custom song, or disable soundtracks they are not interested in supporting. Because of this, support for multiple soundtracks is not present in the enhanced-lite branch. Currently, this is the only difference between the two branches, but in the future this will likely change.

Playing alternate soundtracks in-game requires copying each soundtrack's .ogg files to the `data/Soundtracks` folder. Each soundtrack gets its own subfolder.

Now for the more boring technical changes...

CSE2 no longer has a hard-dependency on SDL2. This is what enabled the Wii U port to be made. Much like CSE2's various 'render-backends', all platform-specific logic has been split-off to its own selection of backends. Notably, on PC, developers now have the option of using the much-lighter GLFW3 in lieu of SDL2.

Additionally, a new renderer has been added: OpenGLES2. This is handy for the Raspberry Pi, particularly the 3B+, where the SDLTexture backend would struggle to meet 60FPS at 2x internal resolution. This could also be handy in the future for a mobile port.

Modders might be interested to know that, thanks to GabrielRavier, the <PHY TSC command has been added. Likewise, <MIM has been fixed, so that it persists across reboots (previously it was not being saved to Profile.dat). It appears I never documented the new TSC commands in any of these update posts. Whoops. But yeah, currently <PHY, <MIM, and <MS4 are available.

Since the last update, I've also fixed an issue with the audio mixer, which caused a nasty persistent hanging problem on everyone's PC but mine. Sorry about that.

And... I think that's it. This update was long overdue, but there's still so much left that I want to get done in the future. For now, I hope you enjoy what's here.