There are many men who carried the title Lord Elgin. The most famous outside Canada is described here.
Sir Thomas Bruce was the seventh Earl of Elgin and lived from 1766 to 1841. Elgin is a town in the north of Scotland. This particular Lord Elgin is, justifyably or not, blamed for the metaphorical rape and very real theft of works of art that once adorned the Parthenon. The Elgin Marbles are not, in fact, marbles, but things made of marble that had belonged to the Parthenon until they were taken to Scotland and eventually sold to the British Museum when Lord Elgin ran out of money.
At one time, Elgin was happy, as the British Ambassador in Constantinople. He had just been married. His architect, Thomas Harrison, was building for his wife a Grecian paradise in Scotland to which he would eventually return, which he didn't originally intend to adorn with actual ancient Greek artifacts.
Then he was convinced that, rather than imitating the things, he might as well take them with him before anyone blows them up or burns them to make lime for whitewash. So he, while not in attendance and with governmental approval, had the team he had originally intended to make casts and drawings begin hauling things away.
After a few years the marbles were assembled in England. Elgin spent some time imprisoned in France, and returned to a mixed reception and the added sting of a devastated diplomatic career. His wife also decided that he wasn't the best fish in the sea. He eventually lost all kinds of money selling the marbles to the British government, and ended his life marble-less, Grecian paradise-less, and wife-less.