The first Madison Square Garden was constructed in 1871 as Barnum's Monster Classical and Geological Hippodrome at 26th Street and Madison Avenue (4th avenue). Architect Stanford White bought this open-air venue of fire eaters and chariot races and changed its name to "Madison Square Garden" in 1879.

The second Madison Square Garden was designed by White and built on the site of the first in 1889.  In 1906, White was shot dead in the Garden's rooftop cabaret by a man he had allegedly cuckolded.  The New York Life Company owned the land it was on, and in 1925 they decided the land was a good place for their headquarters.

The third Madison Square Garden was built in 1925 on Eighth Avenue between 49th and 50th Streets.  The building was eventually torn down, and is now the site of World Wide Plaza.

The fourth Madison Square Garden (the current one) was built between 1963 and 1968 on top of a new Pennsylvania Station on Seventh Avenue between 31st and 33rd Streets.

Pieced together from
www.brittanica.com
www.explorenyc.com
www.radiocity.com/b3d.html
plus a tidbit or two from perdedor.