The most familiar kind of periscope is that used in submarines to allow their crews to look around in the open air without actually coming to the surface, typically in order to spot targets for torpedoing. Its most basic form is a tube with a mirror at each end, something like this:
____
|/__--------\______/
| |
| |
...
| |
__| |
O--___/|
Light comes in at the top of the tube, then is reflected downward by one angled mirror, caught by another at the bottom of the tube, and sent out the viewing end. In cheap toy periscopes children use to play at looking around corners without being seen, that's really all there is. For a practical instrument of war, however, you need various other optical elements like lenses or curved mirrors to get useful viewing out of this sort of contrivance, since you'd otherwise lose all your field of view looking down such a narrow tube.
A big drawback of having a look around with a periscope is that you may well be spotted by your prey (or, worse yet, by a Catalina or its modern-day equivalent). This is even more of a problem when the enemy has radar, as they have for the last fifty years or so. As a result, submarine captains like to avoid coming up to periscope depth if at all possible. Modern submaries mitigate this by using various stealth measures on the periscopes themselves, as well as a wide variety of sensors like active and passive sonar arrays to see better underwater and avoid the need to come up to the surface. Modern periscopes also tend to have fancy electronics in addition to or instead of the fancy optics, granting the viewer access to things like night vision, heads-up displays, and multiple views.
Periscopes also appear anywhere else you have the need to look around from inside something and don't want to put a window in for whatever reason: tanks and other armored vehicles, bunkers, parade floats, and so on.