In his book Principia Ethica G.E. Moore presented the idea that moral terms such as 'Good' are indefinable.
To give an example of another indefinable concept, he uses 'yellow'. The perception of the color yellow produces a mental state which he claims cannot be defined to a person who has never experienced it -- that is, you cannot describe color to a blind person. The same goes, he claims, for concepts like Good, Right, and Ought.
His main defense of this is that the sentence "I know it's Good, but does it produce pleasure?" makes perfect sense, while (For example) "I know he's a bachelor, but is he an unmarried adult male?" does not. Surely, if we had the correct definition, it would be senseless for us to say, "I know it's good, but is it {Correct Definition}?"
It seems that no matter what we put into the second half of that question, the sentence will make sense.
"I know it's good, but does is it in accordance with nature?"
"I know it's good, but does it produce positive utility?"
"I know it's good, but is it soy?"
None of these sound like pointless questions, at least not in the way "I know it's a square, but does it have four equal sides?" does. *
This idea that good can only be understood as good was quite strong for a long while. Moore published Principia Ethica in 1903. I don't know when the naturalistic fallacy was overturned, but in it's time it was compared to Hume's famous "You can't get Ought from Is" argument (Hume's Maxim).
The Counter Argument
Moore seems to be confusing meaning and reference. Meaning is what you mean, while reference is what the word picks out. (I know that didn't help. Next up, a demonstration!)
An oft-used example to demonstrate the difference is the morning star and the evening star, both of which are Venus, seen at different times of the day.
When I say 'evening star' and 'morning star', I mean two different things. (By evening star, I mean that bright star I see in evening, and by morning star I mean that bright star I see in the morning.) But they both refer to Venus. (When I talk about the morning star, I am referring to Venus, even if I don't mean to).
So when I say, "I know it's the morning star, but is it that bright star I see in the morning?", it makes no sense -- but when you say, "I know it's the morning star, but is it the evening star?", this is a useful and relevant astronomical question.
Statements like "I know it's good, but does it produce pleasure?" make sense because, even while they may refer to the same thing, they may mean different things. Statements like "I know it's a square, but does it have four equal sides?" both refer to and mean the same thing.
* Confused? If so, here is some more explanation. If not, skip this.
If you know what a square is, and you know what 'four equal sides' means, then you know that if it is a square, then it does indeed have four equal sides. When asking "I know it's a square, but does it have four equal sides?", you will not be getting any useful information when you get the answer.
Moore is saying that if you were to have a good definition of the word 'good', then you will be able to formulate a question of the form "I know it's good, but is it X?" that is redundant. He also thinks that there is no question such that it starts out "I know it's good, but is it....", and such that the second part is redundant. People do indeed come up with complex counter examples to show that things that 'make you happy', 'produces the most overall utility', 'is in accordance with nature', etc., etc. are not always definitely good.