Romany Word for a Non-Romany Person
"Mashkar le gadjende leski shib si le Romeski zor"
"Surrounded by the Gadje, the Rom's only defence is his tongue"
Pronounced "gah-jay" or "gay-djuh", the word Gadje (or Gadjo) is a Romany ("gypsy") word used to describe someone who is either not of Romany extraction, or one who has left the Romany life and is living outside the community and culture. The ethical code and philosophy of the Romany is fairly complex, and embraces spiritual ways, religion and cultural standards. One example of this is that they decline to excrete where they eat, hence will construct latrines outside the camp, and view anyone who uses an indoor toilet facility as unclean.
In the same way that the Amish describe their non-Amish neighbours as "English", it is used with regard to anyone outside their own culture. However, it means more than just "different"; there's a connotation of uncleanness and impurity, hence it is a denigrating term.
European Use and Etymology
As with so many "European" words, this has been borrowed in various parts of the world. In Scotland, the Borders, and parts of Northeast England, the word gadgie may be used, often to describe chavs (another word of Romany extraction) or in the sense of "mate, pal, friend". Gorgio is sometimes heard in parts of England; gajo is used with a similar meaning in Portugal, and gadjo is not uncommon in parts of France.
The etymology is uncertain, possibly coming from a word meaning "peasant", or one who works the land. For a nomadic people, the thought of anyone dwelling in a fixed house or community would be at best odd and at worst, totally alien, so this strikes me as being possible.
Another possibility is connected with the persecution and subsequent expulsion from of the Romani ("Romany") people from lands around Afghanistan by a Moslem ruler named Mahmud Ghazni (which was rolled into the Roma language as gadje. Yet a third tale comes from Hungary, where the Romany people were persecuted by one Pasha Machmut (another Moslem), who was headquartered in the city of Gadzso. You pays your money and you takes your choice. In any case, if you're described as "gadje", it's not a Good Thing.
Encyclopædia Britannica
Wikipedia
Urban Dictionary
A Roma Etymology