This conflates several kinds of prefixes. I am not being thorough here; this is just a bit of clarification. There are basically four sorts that precede surnames:
  • of or from a place
    examples are French, Spanish, and Portuguese de, Italian di, German von, Dutch van, etc. Some of these have spread, so you get Hungarian names with de or von. There is no living prefix of this kind in English, but a form of the word at occurs in Alan a-Dale and Thomas à Beckett.

    This may occur with the article 'the' in the place name, as in de La Rochefoucauld, Dutch van den...; and may combine with the article: Portuguese dos Santos, Italian del, della..., French, des, etc.

  • son of or (rarely) daughter of
    Irish and Scottish mac, Welsh ab or ap, as in ap Rhys. In some common surnames the B or P has fused, as in Bowen, Pritchard, Price, Powell (from Howell).

    Irish also has Ó and its feminine counterpart . These are used in living Irish: a woman called O'Brennan in English is called ní Bhraonain in Irish (Enya's surname). Actually, see Irish Gaelic family names by Ubiquity for a much better and more accurate explanation.

    Old French fitz (modern French fils).

    Arabic ibn or bin, Hebrew ben. There are corresponding female forms Arabic bint, Hebrew bat, but I'm not sure how commonly they are actually used.

    Arabic has an oddity in that when you become a parent you may be known as Abu... 'father of' or Umm... 'mother of'. I'm not aware of any other language (not even Hebrew) that does this.

  • the expressing a quality
    French Leblanc = (the) White. Often fused with the name, as here. Italian La Guardia. But the same prefix may also be part of the place name in a toponymnic.

    Arabic al-Khwarizmi = the Khwarizmian, the person from Khwarizm. The L assimilates to certain consonants, but European transcriptions don't always show this: so you may see Anwar as-Sadat or Anwar al-Sadat, or plain Anwar Sadat). (And the al- is also seen as el-.)

  • titles
    The Islamic title Hadji and the Christian Papa (priest) both occur in Greek names.
Such things also occur further afield. In Japanese no is used in some older names, though not modern ones; in Kenya we have (in which language I don't know) President Daniel arap Moi called President Moi, so arap is some such element.

A different matter is prefixes within place names, and people then taking their surname from that: as in Saint and all its forms (Hungarian Szent, Romanian Sfîntu, etc. etc.).