The metaphor is apt for Geographical reasons in addition to those economic, social, and political. At least according to my Eighth-grade Pennsylvania History class.
Consider the positions of the Thirteen Original Colonies, how they stretch along the Atlantic. If you imagine an arch stretching from Massachusetts (part of which became Maine) to Georgia, why then, Pennsylvania is the middle of the arch.
Okay, so you're looking at a map, and clearly, any arch shaped after our Thirteen Original Colonies would not hold much weight. But, Pennsylvania is in the middle if you were counting each state as evenly sized bricks...
You see, the Keystone is in the middle. That's why Altoona, PA can call itself the Keystone City of the Keystone State (purely on the geographical in this instance, I must confess). It's in the middle of the state.
Pretty damn clever, huh.