I've always thought the Labyrinth was somewhere near the center of the island and to the left, considering that you move right through most of it, which brings you to the Core, which I would assume would be near the center of the Island.
Then again, it also makes sense that the Seal Chamber...