Symptoms of an allergic reaction to Penicillin can include the following:

Rashes
Hives
Itchy eyes
Swollen lips, tongue, or face

Rarely a Penicillin allergy can lead to an anaphylactic reaction, usually occurring within an hour of taking Penicillin. Some symptoms may be difficulty breathing, hives, wheezing, dizziness, loss of consciousness, rapid or weak pulse, drop in blood pressure, swelling of the throat or tongue, skin turning blue, diarrhea, nausea, and vomiting.

Chances of an allergic reaction are greater if you have had a positive skin test for a Penicillin allergy, had hives that showed up soon after taking Penicillin, or if you have previously had an anaphylactic reaction to Penicillin.

Usually during emergency treatment for an emergency allergic Penicillin reaction an epinephrine shot is given to the patient. Also antihistamines and corticosteroids may be given intravenously.

For less severe allergic reactions a person may want to take over the counter antihistamine(s).

A significant number of people who think they have a Penicillin allergy may not have one. Their sensitivity to Penicillin may have changed from what it was in the past. Adverse reactions to Penicillin may me mistaken for an allergic reaction. Some frequent adverse reaction symptoms may be:

Diarrhea
Hypersensitivity (an undesired reaction caused by the normal immune system)
Nausea
Rash
Neurotoxicity (exposure to natural or artificial toxic substances, neurotoxins, that alter the normal activity of the nervous system in such a way as to cause damage to nervous tissue): limb weakness/numbness, loss of memory, delusions, cognitive and behavioral problems, and headache.
Hives
Superinfection (when a cell previously infected by one virus is co-infected with a different strain of the virus or another virus, at a later date).

Less frequent adverse reactions include:

Fever
Vomiting
Erythema (redness of the skin caused by an increase in blood flow to different tissues in the body)
Dermatitis (inflammation of the skin)
Seizures
Angioedema (rapid swelling of the dermis, subcutaneous tissue, mucosa and submucosal tissues)
Pseudomembranous colitis (inflammation of the colon)


Sources:
http://www.webmd.com/allergies/guide/penicillin-allergy-topic-overview
http://www.mayoclinic.com/health/penicillin-allergy/DS00620/DSECTION=symptoms
http://en.wikipedia.org/wiki/Penicillin_allergy