for future reference, cavestory defaults to you first controller. i have a psx pad that i need to plug in by itself, becase if i plug in my 360 controller it takes priority as joystick 1.
also, if you still have problems, you should get a program like key2joy and just map it over the keyboard.