The Holy See is the see of the bishop of Rome, i.e. the Pope, also called the pontiff, pontificate, or holy father. But this usage of "see" is archaic and obsolete, you say. Why, yes it is. A see, derived from Latin "sedes", meaning "seat", is the official city or other center from which an authority figure may wield their authority. The OED states that a see is any "seat of dignity or authority; esp. a royal seat, throne. Hence the rank or position symbolized by a throne." The throne or seat itself symbolizes the office and all the power inherent in it, but is equated with the rest of the city. The city then becomes the tangible symbol of the power of that figure. Capital cities in general can be called seats or sees, but so can smaller towns with some local power: for instance, some medium town in the middle of Iowa can be a county seat.
In this particular case, the see is associated with religious rank and power as opposed to pure political power; thus it receives the qualifier "holy" and is seen as the see of holiness for the Roman Catholic church as a whole. It is the focus and creation point of papal authority. Physically, however, the Holy See is the principality of Vatican City, within the city of Rome. This particular facet is interesting, as Vatican City is a country in its own right. It is not only a city: the usage of "Holy See" to mean the country (even though it is a very small country) implies that the Pope has more power than many other see-wielding figures. And then the conflation of Rome and Vatican City is interesting as well: does the Pope have two seats of power? And how much political power comes along with the religious power? The Pope is the ruler of Vatican City politically, but not of Rome; he is the spiritual ruler of the Catholic church, however, which is certainly predominant in Rome. Very interesting indeed.