A very Irish exclamation. Irish (the language, not the nationality, and don't call it Gaelic!) is highly visual and metaphorical, making up for what it lacks in vocabulary with rich constructions of image and sound. It has been theorized that this is why there have been so many world-famous English-language writers from Ireland. When the Irish way of thinking and constructing sentences finds its way into English, the results can be very colourful.
One of the classic Irish constructions is adding emphasis by building a list. So, for example, the Irish for "Hello" is "Dia dhuit", which means "God be with you". The proper reply to this is "Dia is Muire dhuit" - "God and Mary be with you". Now, it may happen that someone is especially pleased to see you, and so greets you right off the bat with "Dia is Mhuire dhuit!", in which case, you should reply, "Dia is Muire dhuit is Pádraig!", or "God and Mary and Patrick be with you!" That would be Saint Patrick, the patron saint of Ireland, naturally. In theory you can go on adding saints' names indefinitely, although in practice there is a limit to how pleased you can be to see anyone, even Enya.
So we come to "Jesus, Mary and Joseph!", something you will very frequently hear still in Ireland, especially outside of the increasingly cosmopolitan city of Dublin. Obviously, a common exclamation or curse would be "Jesus!". This can be emphasized using the same additive method as above, to result in "Jesus and Mary!", which is quite common, and "Jesus, Mary and Joseph!" which for some reason is more common than "Jesus and Mary!" — possibly it is thought that Mary would be lonely in the presence of the godhead without her husband. The addition of holy names may, and often does, continue, particularly among members of the older generations. I have heard older people exclaim (in Irish or English), such things as "Jesus, Mary and Joseph and the Holy Spirit and all the angels and saints!" or "Jesus, Mary and Joseph and the three shepherds and three wise kings and the fuckin' ox and the ass too!" Until you've heard a native Irish speaker curse in Irish, you don't realize how much artistry can go into a stream of words resulting from hitting one's finger with a hammer.
Pavlovna says re Jesus, Mary and Joseph: Must be a Catholic thing as Polish speakers do something very similar. So much so that when I was a child I was convinced Jezusmaria was one word because that's what it sounds like (and I think it's spelt as one word too)