Alright~ Well I got Steam Overlay working with Cave Story.
But, it will likely depend on your videocard and O/S, if it supports the software that I discovered.
It's cool though. I'm able to play Cave Story in 1680x1050 without switching resolutions, with steam overlay. (It also fixed some poorly nearest neighbor scaled issues I had with a RPGmaker game.)
But, it might have some glitches. I've tested a few mods, and the one issue I found was the blade's lvl 3 attack looks glitched. As if it isn't properly cutting just the part of the image it needs. But, I didn't test this in other mods/original to confirm.
Could probably get these issues fixed as it is an active project with SVN... So maybe community effort? Haven't checked.
Anyway! Here it is! DXGL :
DXGL is a project to create a DirectDraw/Direct3D version 1 to 7 implementation that runs on OpenGL 2.x. It is intended to alleviate some of the graphics glitches inherent with using legacy DirectX interfaces on modern video cards. The API will be 100% binary compatible with the system ddraw.dll file.
DXGL is currently in an alpha stage, but several applications and games already work.
https://www.williamfeely.info/wiki/DXGL#Downloads
Requirements :
Windows XP, Vista, 7, 8, or 8.1 (May work under recent builds of Wine)
OpenGL 2.0 or higher compatible video card (3.x or higher recommended), with hardware accelerated non-power-of-two size textures
Visual C++ 2010 x86 runtime, available at
http://www.microsoft.com/en-us/download/details.aspx?id=5555 (will be installed if not present)
To get it working, after installing, open up DXGL Config, Add... the game.
Then I had to choose:
Texture filter: Nearest or Bilinear
Video mode: Aspect corrected stretch (couldn't get full-screen working)
You can play with the other options too, though most won't make a visual difference (maybe antialiasing...)
There's even "post-processing shader", which I have not looked at yet... Might be amusing if I can run some other third-party injectors to change the look of CS... lol
To add Steam Overlay, load steam, and Add a non-steam game... add it, and if you have DXGL setup, it should work.
EDIT: Here's some screenshots :
With overlay
http://cloud-2.steampowered.com/ugc/571148671380357210/CBEE459FCE1FBBEF53F1CA8C010AD25B6A4114DB/
Bilinear
http://cloud-4.steampowered.com/ugc/571148671380353206/93C7000769481A878F65093B03AB69B2F7A821E3/
Nearest Neighbor
http://cloud-4.steampowered.com/ugc/571148671380358242/31942916B84FAC7AEF6943521BC0B7856CF0FF75/