A couple of other things should be kept in mind about synths, including what type of synth, the brand, how it is played, and the interface.
Type of synth
Contrary to popular belief, there are different kinds of synths. Pick up a copy of Keyboard Magazine or a similar rag. Flip to the review section and take a gander. There are keyboards, tabletop synths, beatboxes, softsynths, organs, electric pianos, rackmounts... the list is endless.
The beginner would likely want to go for a keyboard or softsynth. A keyboard is a synthesizer with a built in keyboard, usually non-weighted keys (except in the larger, much heavier, and more expensive models). The reason why is because in order to use the synth, one must have a way to control it. If you already have a midi guitar or already own a midi keyboard, you can get another kind of synth (like a rackmount or tabletop synth), and it would most likely be more economical.
Keep in mind your price range and needs, however. For the sub-$2000 range, you can't expect anything higher than a 76-key keyboard with non-weighted keys (a keyboard with 61-72 non-weighted keys adds about $500 dollars to the price of the synth, while 88-key weighted tends to add $1000 or more).
Softsynths are software synthesizers. They are extremely inexpensive because the hardware is not included. However, if you already have a relatively good computer (350 mHz for Macs, > 600 mHz for Windows boxes) and midi and audio in/out, you could be on your way to aural bliss. Some great softsynths include Propellerheads Reason (a beautiful, stable rackmount synthesizer), Propellerheads Rebirth (303, 808, and 909 drum/bass synthesizer), Max/MSP (a very interesting sound creation package that imitates electrical circuits but has a learning curve like an acute angle), and anything by Native Instruments (sound synthesis). Beginners would want to start with Rebirth, as it is less expensive and has much less knobs to get in the way of the music. Reason is the way to go for almost everyone else, having midi support, a build in sequencer, infinite expandability, and rock-hard stability. I would avoid Unity DS-1, however, as it seems to be unstable and not that useful to the professional musician.
Beatboxes would be a good investment for fledgeling DJ's. They are like drum machines but have more lead and accompaniment sounds and so make it possible to jam without a background band, and with midi support can also play back and sequence sounds. They are usually dirt-cheap at the sub-$500 range (dirt-cheap in the music hardware sense, anyway).
Tabletop synths are synthesizers that resemble beatboxes or perhaps drum machines of twenty years ago. In fact, many work as the aforementioned. However, most (like the Roland SH-32) have much to offer in the synthesis area, with knobs and red LED's galore. These things also tend to be very cheap, and are sometimes rackmountable. They also look like something from Star Wars, but usually have surprisingly good sound for their price.
Rackmounts are synthesizers that are meant to be placed on a rack — a musician's shelf of signal processors, mixers, and synthesizers. Therefore, they have holes for screws, are very compact, and are shaped somewhat like squat VCR's sans tape deck. As such, they tend to have very small front faces with diminuitive LCD screens, few knobs and buttons, and usually require a keyboard or other form of control in order to output sound. They are usually less expensive than their keyboard counterparts.
Electric Pianos and Organs are the synthesizers of yore — they have no midi, no audio output, need to be tuned, and sound cheesy, but people love them, and they have spawned many digital emulators. They fetch high prices and can be found only rarely. Still, modern keyboards (like the Nord Electro) can emulate the sound of these old gems, and there are even softsynths that do likewise.
Brand of synth
Brand is actually quite important when buying a synth and can even determine the relative price range/quality/type of the synth. For example, Yamaha tends to make less expensive, performance-driven synths with cheap keyboards but decent sounds at a sub-$1500 price range. At this price, Yamaha makes great keyboards, but be sure to check out Roland as well, which, although the sound quality is weaker, the hardware is better and the software support is better too — so weigh the alternatives.
If you have the cash to blow, however, I would get a Korg Triton. Called the Silver Beast, it is large, heavy, and one of the best synths out there. It has a touch screen, a ribbon (a trackpad-like control device), great keys, and sweet synth sounds. This will push you back upwards of $2500 though, and that is just for the 71-key version. If budget is an issue, check out the Triton LE, the Trinity, or the übercool Karma, all by Korg. Avoid the Yamaha Motif, which has very similar sounds to its other synths but weaker keys than others in this price range and not as good of features as the Triton. I have not had any experience with any Roland keyboards in this range, however.
Roland also makes great hardware interfaces, but be sure to check out Mark of the Unicorn and MidiMan, especially if you own a Mac (which is considered the de facto music computer by most professionals anyway).
If you don't have the mony to shell out, get a controller keyboard by Roland, which will set you back less than a thou, or get an Oxygen-8 (a mini-keyboard with USB for use with softsynths), and then use a softsynth in conjunction with it or get an inexpensive tabletop or rackmount synthesizer, like the Roland SH-32.
The interface
There are many ways to control your music. The first is using audio cables. These can include the manically stupid Mic cables (three-prong cables that it seems nobody supports), Instrument cables (the kind used for electric guitars), 1/8" headphone cables (a blessing for software musicians), optical I/O, mLAN, Ethernet, RCA jacks (like used on TV's), and many others. I will not go into great details into these, but just keep in mind that for the casual musician a Griffin iMic ($35 USB in/out, but only one stereo connection) or similar should keep those in need of a simple solution happy with CD-quality sound. Interfaces by MidiMan would be the next better, followed by the firewire and PCI-card interfaces that can cost over a grand being churned out by Mark of the Unicorn and others. All of these will support the cables that are important to at least eighty percent of musicians. If you need more information about these cables and more, I suggest you check out Keyboard Magazine or the Internet.
Midi. Since the seventies, it has been used on many a synth and computer. Most interfaces are Serial or USB, as midi does not require the high speeds of a firewire connection. However midi is very versatile, transmitting data (not actual sound data, mind you, which is why it is so versatile) to synths, computers, and even lighting systems. Unfortunately, glitches abound and many systems have trouble communicating to certain synths and certain synths are equally finicky. Thankfully, other standards are coming out, like the firewire-utilizing mLan (which transfers data and audio), ethernet, and even wireless data transfer. Thus, getting a system that uses these might be good in the long run, but wait until your synth supports them before going on a buying spree.
Nearly all synths support midi, so that should not be a concern. However, getting a midi connection for your computer should be top priority. I reccomend getting one from MidiMan, who make inexpensive USB interfaces, or Edirol, who are even less expensive, but tend to have relatively bad support. Mark of the Unicorn also makes some reportably good interfaces, but I have had little experience with them.
Coda
I will put more information in here as soon as I have more time. However, this should get one more prepared to deal with the harsh world of buying a synth. Also, don't worry about getting them used, it can be a very good idea, especially B-stock items, which are items used for demo-ing but little else. They can be up to two-thirds off the normal price but be as good as new. So always ask the staff at your music store if they have an item B-stock. It can make your journey into the world of electronic music much, much cheaper.