Contrary to what
Inkoate said in his writeup, the Geocaching community frowns on buried caches, and tries strongly to discourage people from disturbing
nature. Burying things involves digging holes, potentially killing local
plant life or disturbing local
ecosystems (beaches, believe it or not, have very intricate ecosystems that are easily disturbed by digging in the sand). Furthermore, many caches are placed in
state parks, where it would be illegal to bury them.
The most common form of geocache is a Tupperware or Rubbermaid plastic water-tight container holding various items. It's usually a good idea to write the name and coordinates of the cache on the outside of the container in permanent marker. It's also a good idea to enclose the items inside the cache in Zip-loc bags in case water gets inside the container.
Caches should be hidden in places away from well-used paths or parks. The last thing you want is for a non-geocacher to stumble across your cache and plunder it. Even so, you want to avoid putting your cache in a "No Trespassing" area, or else the locals will become angry and the unfortunate geocachers who try to find your cache may inadvertently get in trouble.
Many geocaches require you to solve puzzles or complete treasure hunts before finding the final cache. Some caches are "virtual caches", meaning there is no actual cache, but instead the coordinates lead you to a nice place to watch the stars, a hilltop with a good view, a good bird-watching area, etc.
There are special items called travel bugs which are intended to be transported between caches. When a geocacher finds a travel bug (which can be any item with a small travel bug tag attached to it), the cacher enters the unique number on the tag into the geocaching website and logs that it was picked up. That cacher then leaves the bug at another cache and logs where he left it, and the site tracks the bug's progress. Many travel bugs have goals, such as getting from the East Coast of the U.S. to the West Coast, or being placed in the cache at the highest altitude.
Geocaching is so popular now that some participants don't even use GPS devices to find the caches. It's entirely possible to use a good map to locate the cache coordinates, and then find the cache via careful searching of the area.
I highly recommend geocaching; it's like finding pirate treasure without having to deal with pirates. It's also an excellent way to find cool new hiking trails and get some exercise after a week of sitting in front of a computer.