I never actually thought of that, that's honestly a very plausible theory. Maybe Mr. Little was out getting food, and maybe one thing they eat is mushrooms, or some other vegetation from the graveyard.
Then again, when you pick up Mr. Little, it's not like he says "Hey come on, I'm busy...