To call Utena bizarre without meaning is to not fully understand the series. Everything you need to know what's going on is present if you're willing to work for it; and that is what most fans find so satisfying. Much of this writeup could contain spoilers.
The following are some examples of symbolism that run rampant in Revolutionary Girl Utena. It is a very complex series and I for one am under the impression that every second of it has a greater purpose, though I guess no one but the director knows for sure. And he's not the telling type; in a recent interview his reply to an enquiry on the outfits was simply, "I thought cosplay girls would look cute in them."
The French Influence: Utena is an homage to two other animes, both of them older classics. The first one is Osamu Tezuka's Ribon no Kishi (Princess Knight/Knight of Ribbon) which was the first shoujo ever. The basic storyline follows a young girl who is raised to act like the prince of her kingdom, though she secretly wishes to be a princess. The second series is Riyoko Ikeda's Rose of Versailles, and herein lies the French motif. It is present in the puffy sleeves of the girl uniforms and the overall appearance of the Student Council. Anthy's pets all seem to have French names (as indeed her name is; the Japanese don't have a 'thy' sound).
The Roses: Rose of Versailles is about a soldier named Oscar who was raised as a man and must guard Marie Antoinette. The rose of the title refers to Marie and it can be inferred that Anthy's position as the Rose Bride plays a delicious irony on this motif. In addition, roses flood every scene of Utena and her name itself translates to the calyx of a flower. In addition to being a good visual device, its usage probably began as a reference to the older anime.
Color: It's a very simple concept, but if you know your color wheel the main relations of the Utena characters become immediately more significant. Character pairs that use this method include Anthy/Nanami, Touga/Saiyonji, and Miki/Juri (although Juri/Ruka is also the same pairing, obviously). The colors of roses, clothing, balloons, and most anything in the series seem to be clues pointing to specific characters and should be noted.
Eggs and Revolution: Utena purposely does not explain what Revolution means at the beginning of the series, and the nearest hint it gives you is the chick speech, which is really no help at all. It becomes apparent that the idea of being born is important, however. The egg motif is most evident in Episode 27: Nanami's Egg and is dealt with little elsewhere. Revolution and birth seem to correlate as nearly the same idea in most of the series.
Death: Opposite of being born, of course, is dying. The Academy resembles a funeral mound, after all, and coffins become almost as obvious as roses as the story develops. I would spoil much of the series if I embellished on this too much, but suffice to say that each character's search for something eternal probably deals with their desire to make sense of death; they wish to be shown proof that living is not for nothing, each in their own way. Utena, interestingly enough, is not looking for something eternal because she found it already when she was little.
Sex: From the opening animation onward you should have thought there might be sex in Utena. And there is, but most of it is so subtlely mentioned it could easily be missed. The really sexual scenes are very late into the series, beginning around episode thirty. I believe that cars are used to indicate sexual feelings or deeds around this point; the Akio car becomes an important part of each duel and it appears in several questionable scenes. The movie Adolescence of Utena also features cars prominently, but the purpose of them seems more friendly, less sexual.
Letting Down the Hair: This device is also used excessively in Utena, as Juri, Anthy, and Nanami all loosen their hairstyles in relation to certain plot points. The basic meaning seems to be that a character with their hair down is being true to themselves, or showing who they really are. For the movie, several characters were given longer hair, indicating a difference in the movie character's persona, however slight.
There are a million and a half other clues that could be noted, such as the pointing hand in the Black Rose Saga, the divine justice elephants, and Anthy's glasses, but I'm not yet an expert on the series so this is all I have to offer. If you find Utena too bizarre for you right now, rest assured that with enough patience every question you may have will be answered. In episode three you find out Anthy is afraid of crowds; you won't know why until episode 34. However, the wait is well worth it, as it is for most of the mysteries of Utena.