"Jove" is the ablative form of the name of Jupiter, head of the Roman pantheon. Like most Latin words, it comes into English and other languages mainly through the ablative case. (Another example would be the ablative of amor, amoris, m., "love" -- its ablative form, amore, can be found in several of the Romance languages.)

It is interesting to note that Latin once did not contain the letter J, and thus the name was spelled "Iove." Also, the Latin letter V was once used interchangeably with the letter U, and was pronounced as such, sounding like the English letter W. Thus "Jove" was pronounced more like "Yoh-weh."

When you see how much that resembles the Hebrew "YHWH," it's hard to avoid making snide little comments.