Routing on the
Internet is
destination based. This means that
every machine on the Internet has a
routing table which tells it the next
machine on the
path to every
IP address. They don't know the
entire path, just one
hop.
Imagine that no one knows how to get somewhere. However, everyone has a notion of which direction it is. That's basically what happens. You send out a packet, it goes to a router that knows a little more than you, but still not much. It looks up in the table to whom it should forward the packet in the routing table, and so on. This continues, hop by hop, until the destination is actually reached.
Of course, the IP addresses are compressed into groups called CIDR blocks for storage in this table, as otherwise the table would have to have 4 billion entries, one for each IP address.
The problem is how to compute this table. There are a variety of routing protocols, including BGP, OSPF, EGP, and RIP. All of these put entries into the table based on the knowledge the protocol gives them. However, like the speed of light limit, once something occurs to disrupt the network, it isn't known by everyone instantaneously; the knowledge of this change has to work its way through the routers, like gossip.
If I send a packet to Seattle from Houston, I may send it through San Francisco, which may be having power problems that I haven't heard about yet. Of course, as it gets closer to SanFran, they routers there will say "Ah, the router's down in San Francisco, but there's a router in Houston that has a path!" It then becomes a game of "hot potato" until the time to live field kicks in.
Another common cause of routing loops, especially within corporate networks is route aggregation.
Route aggregation occurs when I announce something like that I know how to get to all houses between 100 and 200 as well as between 400 and 500 on Main Street. Someone then looks and says "Well, no one else knows about Main Street, so instead of making two announcements (one for 100-200 and one for 400-500), I'll announce a route for Main Street." This simplifies the routing table, but I probably have designated someone the default route, which is the route I send to if I can't find a better match in my routing table. So, a packet is sent for 235 Main Street. It gets forwarded to me, I send it back to my default route, and the two of us play "pass the buck" until we run out of "money" (that pesky time to live again).