What is it?
A gyroscope is really just a spinning wheel. What makes a spinning wheel interesting is the fact that it resists changes in its orientation along the roll and yaw axis (counting the pitch axis as the spinning motion of the wheel). Because of this, a gyroscope can be used as a stable reference point to measure the change of orientation of something else.
If you were to take a bicycle wheel off of the frame and give it a good spin, holding it by the axle, you would have a pretty good gyroscope going. A bicycle wheel has some decent mass to it, and the fact that most of it is located around the circumference gives it good rotational inertia. It is this rotational inertia that creates the gyroscope effect.
If you were to take the bicycle wheel at rest and rotate it along any axis, it would move around pretty easily. All you are fighting is the inertia of the wheel's mass, which isn't much. But give it a good spin and suddenly you've granted it a large amount of rotational inertia. Rotational inertia is a measure of how difficult it is to change the way something is spinning.
What does it do?
Intuitively, the faster you spin the wheel and the more massive the wheel is, the more difficult it would be to stop the wheel from spinning. Somewhat less intuitively, the greater the percentage of the wheel's mass is located around the circumference, rather than near the axle, the more difficult it would be to stop the wheel from spinning (consider how much easier it is to twirl a large sledgehammer while holding the head rather than the handle).
Where the gyroscope baffles intuition, however, is the fact that its rotational inertia not only makes it difficult to stop the rotation, but also makes it difficult to change what plane it is rotating in. If you hold the spinning bicycle wheel vertically and try to angle it so that it is spinning in the horizontal plane, you will feel the wheel resisting your efforts. Likewise, if you hold it facing North and South, it will resist being moved to rotate facing East and West.
A classroom physics demonstration I once heard about involves two briefcases, one with an iron weight in it and one with a spinning gyroscope. Because of the iron weight, both briefcases weighed the same. There was no apparent difference between them when picked up. However, if two students are instructed to each pick up one briefcase, walk to the end of the desk, and then turn a corner, the results can be almost slapstick in nature. The normal weighted briefcase will offer little resistance to the sudden right-angle change in direction, but the gyroscope briefcase will continue in a straight line, seemingly of its own volition, while the student turns the corner, causing him to lose his balance and stumble.
So how do we explain this behavior?
We can simplify the case by looking at a spinning baton, which is a rod weighted on both ends rather than a continuous circle. The weights move in a circular motion as the baton is spun. As the baton is being spun, we can apply a force to the weighted ends in opposite directions, such that it would twist the baton out of its plane of rotation. Say weight A is pushed left and weight B is pushed right.
180 degrees later, the weights are on opposite sides of the circle from where they were before. Weight A is now in the location experiencing a force to the right. Weight A carries its leftward force to this side, canceling out the rightward force being applied on this side. Likewise, the rightward force of weight B is carried to where the leftward force is being applied, and cancels it out. The baton just canceled out its own imbalanced forces because of its rotation, maintaining the original plane of rotation.
The continuous wheel of the gyroscope behaves in a similar manner, except the forces are continuous along the entire wheel, which gives it even more resistance to change.
Well this is all rather nifty, but what can we use it for?
The gyroscope effect is present whenever a wheel with significant mass is spun. A bicycle will of course benefit from it, since the example we've been using all along was a bicycle wheel. In the bicycle's case, the gyroscope effect of the wheels will help keep the bicycle upright and resist falling over, because falling over would be a change in the rotational plane of the wheels. It also helps the bicycle maintain a course roughly straight ahead, because turning would also be a change in the rotational plane of the wheels, which allows a confident rider to take his hands off the handlebars. We can see, however, that a change in pitch is not resisted if we attain a significant rate of speed and suddenly apply only the front brakes.
The gyroscope effect is likewise present in toys such as the yo-yo and the Frisbee. In the case of a yo-yo, any change in the plane of rotation would prevent the yo-yo from traveling up and down the string smoothly, so the spinning motion of the yo-yo helps it. In the case of the Frisbee, the spinning motion keeps it parallel to the ground so that it flies in a straight line. The more spin you give a Frisbee, the better it will fly. There also exists a wrist strengthening ball which has a gyroscope inside of it, and fighting the gyroscope by rotating the ball by moving your hand around gives the wrist quite a workout.
In more practical applications, the gyroscope is used as a reference point to measure orientation changes in aircraft, submarines, and missiles. This is possible by mounting the gyroscope on gimbals (low friction mounts which can be freely moved around, allowing the gyroscope to remain in its original orientation as the frame changes orientation around it). Several gyroscopes are used in order to measure all the axis of rotation, because a single gyroscope cannot measure pitch, after all that's the direction it's spinning in. Satellites and space stations contain multiple gyroscopes, because in free fall they are the only reference point available. Some of these gyroscopes will be redundant, so a single failure will not be catastrophic for the multi-billion dollar project.