According to my
brother, "the
difference between
mathematics and
physics is that mathematics deals with
reality."
Think about it.
What mathematics tells you is objective truth. Not all mathematical theorems may be applicable in all situations (for instance, if you're trying to navigate on the surface of the Earth, clearly spherical trigonometry and spherical geometry are relevant, while hyperbolic geometry is not). But all theorems are true whenever applicable. Mathematics deals with reality.
What physics tells you is (usually) empirical truth. When you get down to the bottom of matters, physics only generalises measurements performed on the real world. And these measurements could always be wrong, or worse, be measurements of the wrong thing. For instance, verification of Newtonian mechanics didn't deal with high velocities or very small scales, because of an incorrect homogeneity assumption. And relativistic verification always seems to assume anisotropy (of space), for very good reasons. Physics deals with perceived reality.
Of course, a side-effect of all this is that physics can say a lot more about the world than mathematics. It goes out on a limb, and this is the payoff.
NOTE
Physics isn't really all that "bad". Some great ideas in physics have a distinctly mathematical character. Schrödinger is said to have written down the wave equation out of the blue (say, inventing it by intuition alone). In other words, the wave equation is the only way for the state of a (quantum mechanical) particle to evolve, so it is "unavoidable". But we still prefer the wave equation to other equations because of empirical verification, not just because of its mathematical elegance. And we could be wrong, even if no errors have been made in the derivation of quantum mechanics, simply because the world doesn't work in the way we thought it did!