I've heard people (mainly popular science writers) talk about "Quantum Weirdness." What they mean by that expression is the various counter-intuitive predictions of quantum mechanics. Effects such as tunneling, entanglement (and spooky action at a distance), uncertainty, and the ability of a particle to interfere with itself are common examples of what people find "weird" about quantum mechanics. Basically what this boils down to is that on the microscopic scale of atoms, objects do not behave in the way we're used to see macroscopic objects behaving. I borrow this expression to coin a new term for something I haven't heard people talk about quite as much, but I feel is quite a bit weirder. At least it is weirder insofar that it is more likely to lead me into circular thinking and give me headaches whenever I consider it. Examples of "Set Theory Weirdness" are issues about the axiom of choice, the undecidability of Cantor's continuum hypothesis, etc. The basic question I can't seem to find an answer to is, are sets real? Is set theory simply all the consequences of its axioms, or is it trying to describe some actual concept?
Number theory, for example, is a theory about something very concrete - the natural numbers. Though Gödel has shown that even in number theory there are undecidable propositions, that simply means that the commonly accepted axioms of Peano Arithmetic are not a full description of number theory, and that in fact no formal system derived from a collection of axioms ever will be a full description of the theory of numbers. That is a statement about formal systems, not a statement about number theory. For example, Goodstein's theorem is known to be correct, but does not follow from the axioms of Peano Arithmetic.
The same cannot be said about set theory. If the real numbers actually existed, then they would either have or not have a subset that is neither countable nor of the same cardinality as the continuum. The fact that we have no way to decide, other than by asthetic or intuitive arguments, whether we should accept or reject the continuum hypothesis shows why sets should not be thought of as actual things. We cannot even gain insight experimentally on whether or not to accept it, as we could for Goodstein's theorem!
The other alternative, then, is to simply accept set theory as a purely formal theory. Sets are nothing but whatever is described by the collection of all consequences of the axioms of set theory. Very well then. If that is the case and set theory does not describe our world, then when we are proving something, say Goodstein's theorem, using set theory, why should we believe the result? Why do you expect me to take set theory on the basis of common sense, when it gives such nonsensical results, such as the undecidability of the continuum hypothesis?
This is usually the point in my thinking where I start to get a headache, which is just as well since it seems to me to be an impasse. Today I have avoided that headache, probably because I am putting my thoughts to writing, which is usually a good way of clarifying one's thoughts to one's self. Note, please, that I am not saying that set theory is bull; just that it is extremely weird, much weirder than quantum mechanics, and is too seldom called upon to answer for its weirdness. Of course, that's a physicist's point of view, and a mathematician is welcome to lambaste me for poor understanding of set theory and to expose my naive view of the matter.