The diegesis, simply put, is the world of the story. It is the narrative within which the events of the plot occur and the characters exist. Diegetic sounds in a film, then, are the sounds which exist within the story world and are or could be heard by the characters.
Diegetic sound can be confused by some people with what is properly known as production sound; that is, the live sound recorded on the set which is intended to be synchronized with the picture being recorded simultaneously by the film or video camera. Production sound is contrasted with all the various sound elements created in post production such as special sound effects, foleys, Automated Dialogue Replacement, music (score and/or soundtrack and/or music heard by characters within the film), etc.
Whether or not a particular sound in the film is diegetic or non-diegetic has nothing to do with when in the production process it was created or recorded. It is probable that dialogue recorded on set will be intended to be understood by the audience as being spoken and heard by the characters in the movie, and therefore will be diegetic sound, but this is not always the case. If dialogue recorded on set were repurposed and used as voiceover1 intended only for the audience's ears, then that sound would be properly understood as non-diegetic. Conversely, it is usually the case that music being listened to in the film by the characters is not actually playing on the set during filming. This music is added later in the sound design or mixing process, but is nevertheless diegetic sound because it is understood by the audience as being heard by the characters; i.e. it's coming out of Little Billy's boombox. It is rare that musical score intended for the audience is intended to be understood as audible to the characters, no matter how ham-handedly telegraphic the score might be, and as such score is almost always non-diegetic.
You never realized that filmmaking could get so pointy-headedly philisophical, did you?
1. Voiceover is not always non-diegetic. Voiceover by a character in the film certainly is diegetic (see Sunset Boulevard or The Man Who Wasn't There) even though other characters can't "hear" it. However, voiceover provided solely by a narrator who does not otherwise appear in the film (e.g. Alec Baldwin's voiceover in The Royal Tenenbaums) probably cannot be considered diegetic.