There are many writeups on E2 describing the “traditional” rules governing the relative positioning of commas and quotation marks, describing the alternative ruleset known as “logical punctuation”, and explaining how the latter is superior to the former. Herewith, I present an apology for the traditional style; not because I prefer it, but because I enjoy defending unpopular causes, and because, more seriously, unpopular causes sometimes look less unpopular once you understand them properly. It should make an interesting counterpoint to Gritchka's writeup, though it wasn't written as a rebuttal thereto. Rather, it was composed as an email (hence its uncompromisingly terse style) in response to a friend who asks:
Why should the comma come inside the quotation marks when it's not part of the quoted speech? Why does there have to be a comma at all, when it separates the verb from its object?

You'll observe that the sentence
“I'm going,” he said.
does not conform to a normal English word order if you think that ‘“I'm going”’ is the object of ‘said’. In fact it conforms to an abnormal English word order which lays great stress on the object, as in:
The penguins I at least asked for help. The pot plants I merely glanced at in despair.
It might even be permissible to use a comma after the object in the above sentences. But there's a much better interpretation, especially when you consider that not only is your original sentence not thought of as an unusual sentence permutation, but neither is:
“I'm going,” he said, “to have tea.”
To wit, the expression ‘he said’ is parenthetical in all direct speech, and for this reason is set off in commas. The main verb of the sentence quoting the direct speech is "am [going]"—inside the inverted commas. (This is not to say that ‘to say’ cannot take a direct object; consider:
She said no.
They said Mass.)
This theory also explains conveniently why one might (in the context of a dialogue in which it is clear who is speaking) present alone as a complete sentence:
“I'm going.”
And it justifies the convention in some popular European languages of not closing the inverted commas merely to indicate the speaker:
„Je m'en vais, a-t-il dit, pour que je prenne le thé.”
However, it leaves one unhappy with the sentence (on whose validity I'd be interested in an opinion):
“I'm going,” he said, and went.
But I'd be prepared to consider allowing
“I'm going,” he said, and he went.
because, once one has allowed that ‘“I'm going”’ is a sentence, what more natural than to concatenate it with another using ‘and’? Unfortunately this opens the door to the unnatural-seeming:
“I'm going,” and he went.
The positioning of the commas is more complicated to explain. You will note that there is a convention, more æsthetic than grammatical, that the comma, colon, semicolon and full stop are closely allied to the preceding word (in a way that quotation marks, for example, are not) and come as close to it as possible. Thus the comma after ‘he said’ certainly comes before the open quotes, because the open quotes come after the space and the comma comes before it, and to position them otherwise is intolerable by any known standards.

Nevertheless, the idea that the comma should come after the close quotes comes from your mistaken parsing of the sentence as:

XXXXXXX, he said.
where XXXXXXX is the object of the sentence, and any internal structure or meaning it may appear to have is an illusion and to be ignored. If, instead, you read XXXXXXX as the main clause as I have been advocating, it instantly becomes living, breathing language, and after its last word becomes a fine place to put a comma owing to the æsthetic considerations I describe above. To put it another way, the comma must come immediately after a word, and once you stop treating XXXXXXX, including its quotation marks, as a word for grammatical purposes, it is no longer valid to put a comma after the quotes.
How 'm I noding? 1-800-MSG-APOT