That bothered me too, yes. But it stops being like that after GrassTown, once you gather Jenka's dogs. That's where Quote comes in and things get awesome.
But wasn't it the crown, not Ballos, that had control over Misery and Balrog? I'm assuming it hadn't been restored yet.
Yeah, that's true...