Expression used as a semi-polite coverup for cursing -- you know it's not really French, the hearer knows it's not French, but you can acknowledge that what you're saying might be rude without actually admitting that it's rude. Presumably originates in apologizing for using foreign phrases in front of people who don't speak the language in question.