The expression "on the other hand" means "considering the opposite side of the issue." If you want to discuss two sides of an issue in juxtaposition, you can introduce one with "on one hand..." and the other with "... but on the other, she is only an earwig."

An amusing variation, "on the gripping hand, was invented by Larry Niven / Jerry Pournelle in their novel, The Gripping Hand. The Moties are an alien race with two human-like hands on one side of their body and a large, powerful arm for gross manipulation on the other. They reason in three parts. For example:

On one hand, I should get up and go to work.
But on the other, I'm really tired and want to stay in bed.
On the gripping hand, I blew the place up yesterday afternoon. So none of that matters.