Hmm, physical media, physical media... How about SRAM cartridges?
If you get enough, you should be able to get decent quality.
Battery-back them so that you don't lose all the data, and then you could have music cartridges!
That could be fun. As for playing them back, that may be more difficult...