Voice over IP (or IP telephony) is an alternative to the conventional Public Switched Telephone Network (PSTN), which has been in use for more than 100 years.
PSTN uses the concept of circuit switched networks to directly connect two users with a physical connection through a series of circuit switches. This path is setup and held for the duration of the call.
Voice over IP uses packet switching networks to deliver the data, which is split into packets and individually sent out over a network or series of networks to the receiving user. IP telephony relies on the concept of routing for each individual packet to traverse the networks between the two users. Routing means that the packets will not necessarily travel the same paths to the destination (but will hopefully get there in the right order!), and individual switches only make connections long enough for a packet to pass through, so that resources are not tied up for the duration of the call as with the PSTN.
The main aim of the Voice over IP technique is that it should simulate the existing phone system (PSTN) and interface with it where necessary. Much of the global-scale communications rely on packet switching networks.
The two systems can reach each other through gateways, such as an SS7-to-IP gate way which will convert data from a SS7 (the North American standard) format to an IP format, namely H.323 or SIP, and vice versa.
Since circuit switched networks require resources allocated for the duration of a call, much of the possible bandwidth is wasted by only one or neither user transmitting usable data (i.e. one user is listening, or periods of silence by both users). This is the main inefficiency of the technique. VoIP is able to use the bandwidth much more effectively by eliminating this silence and by using compression of the data, whilst still providing near or better than the voice quality of the permanent connection of the circuit switching network.