"Mary is confined to a black-and-white room, is educated through black-and-white books and through lectures relayed on black-and-white television. In this way she learns everything there is to know about the physical nature of the world. She knows all the physical facts about us and our environment, in a wide sense of 'physical' which includes everything in completed physics, chemistry, and neurophysiology, and all there is to know about the causal and relational fact consequent upon all this, including of course functional roles. If physicalism is true, she knows all thereis to know. ... It seems, however, that Mary does not know all there is to know. For when she is let out of the black-and-white room or given a color television, she will learn what it is like to see something red, say."
Frank Jackson, "What Mary Didn't Know", 1986
When Mary is shown something red, is this a new experience?
Most people would answer yes - they would agree Mary would be surprised by the red thing. So it seems that knowledge of a fact isn't equivalent to the experience of the fact.
According to Jackson, this is a counterexample to the philosophy of materialism (the belief that the world is defined by res extensa). Materialists, of course, disagree.
Nagel argues that consciousness has an irreducibly subjective aspect. Even thought we may know everything about bats, we don't know what it's like to be a bat. To him, this shows that because theories of the mind treat the mind as a functional/intentional/computational system, it seems inevitable that such theories would be unable to accomodate "points of view" / the subjective character of experience. If so, then we can't depend on these theories to explain the mind.
Edit: I found this cool quotation of David Hume
on this topic, in "An Enquiry Concerning Human Understanding"
I believe it will readily be allowed, that the several distinct ideas of colour, which enter by the eye, or those of sound, which are conveyed by the ear, are really different from each other; though, at the same time, resembling. Now if this be true of different colours, it must be no less so of the different shades of the same colour; and each shade produces a distinct idea, independent of the rest. For if this should be denied, it is possible, by the continual gradation of shades, to run a colour insensibly into what is most remote from it; and if you will not allow any of the means to be different, you cannot, without absurdity, deny the extremes to be the same. Suppose, therefore, a person to have enjoyed his sight for thirty years, and to have become perfectly acquainted with colours of all kinds except one particular shade of blue, for instance, which it never has been his fortune to meet with. Let all the different shades of that colour, except that single one, be placed before him, descending gradually from the deepest to the lightest; it is plain that he will perceive a blank, where that shade is wanting, and will be sensible that there is a greater distance in that place between the contiguous colour than in any other. Now I ask, whether it be possible for him, from his own imagination, to supply this deficiency, and raise up to himself the idea of that particular shade, though it had never been conveyed to him by his senses? I believe there are few but will be of opinion that he can: and this may serve as a proof that the simple ideas are not always, in every instance, derived from the correspondent impressions; though this instance is so singular, that it is scarcely worth our observing, and does not merit that for it alone we should alter our general maxim.
So it seems the idea of Mary is older than I had thought. I don't think Hume's example is nearly as rigorous as Jackson's, since imagining "a greener blue" (even if you've never seen it) is no harder than imagining a more virtuous horse. But the framework for "Mary" is there, sort of.