Answered this question in a few threads and so far it's been the same... I had a problem just like that.
It's the sound or something related to it, at least in mine and other people's cases. To solve this problem, go to :
Control Panel ->
Sounds, Speech, and Audio Devices -> Sounds and Audio Devices -> Audio Tab -> Sound playback Advanced -> Performance Tab -> Hardware acceleration
Lower that all the way or like one up.
Hit OK or apply and if all goes well, [don't think you have to restart]
It Might just stop the crashing. You might have to crank it back up after you are done with Cave Story though, so don't forget about that
[assuming it fixes it]
My problem was I would be playing Cave Story on my comp with XP and I'd hear these random sound malifunctions or whatnot... and then randomly it'd just... FREEZE crash and burn
It worked for me, hope it helps!