The word dielectric is completely synonymous with the word electrical insulator. Typically dielectric is used when the electric polarization effect in a material is of interest, while insulator is used when the high resistance to current is important. I will speak about the polarization effect since there's not much interesting about a resistor of infinite resistance*.
* Well maybe there are some interesting issues. If an insulator gets too thin (~1nm), it no longer blocks current because of a quantum mechanical process called tunneling. This is a real-life problem--it effectively sets a minimum on the gate-insulator thickness of a MOSFET. Another less interesting but technologically important issue is the fact that insulators can break down over time, eventually allowing current to flow through relatively small resistance. This also puts a minimum on the gate-insulator thickness.
When an electric field (i.e. a voltage) is applied to a dielectric, it acts on the protons and electrons in the dielectric through the Lorentz force--F = qE. If the material is made of polar molecules, then the force causes those molecules to rotate such that the vector from their positive to negative ends points in the direction opposite that of the applied field. Even if the material is made of nonpolar molecules (i.e. a covalently-bonded crystal), the electric field causes the electrons in the atoms to move in the opposite direction to the positively-charged nucleii. If the electric field is of a reasonable magnitude, it won't break the atoms apart--it just distorts their charge distributions. As in the polar-molecule case, the dipole moment (proportional to the vector mentioned above) will point in the direction opposite to the electric field. In either case the dielectric effectively reduces the electric field, since the redistributed charge in the dielectric creates an electric field that subtracts from the applied field.
A standard "parallel plate" capacitor
|
+ |
|
------------ Conductor/dielectric interface
| --- /|\ Redistributed charge
|Eapp |
V | | Epolar
| |
\|/ +++ | Redistributed charge
----------- Conductor/dielectric interface
|
- |
|
Actually the charge distribution is distorted in every dielectric atom, not just the ones near the interface. However, the distorted charges in the middle of the dielectric cancel eachother out--for every region of negative charge there's an adjacent region of positive charge.
The reduction of electric field is equivalent to an increase in capacitance. Capacitance is (basically) defined as charge stored on the conductor/dielectric interfaces divided by applied voltage. The voltage across a capacitor is (basically) EL, where E is the electric field inside and L is the length between interfaces. Since the dielectric polarization subtracts from applied electric field, it takes more charge to create an electric field in a dielectric than to create it in vacuum. Moreover, a capacitor with a dielectric between its plates has higher capacitance than one with air between its plates.
Every dielectric material is characterized by its dielectric constant ε, which quantitatively describes how big the polarization effect is. Capacitance is always proportional to the dielectric constant of the insulating material. Sometimes, the dielectric constant is considered to be a complex number. In this case, the real part dictates the polarization effect and the imaginary part dictates how conductive the insulator is. Unless the "dielectric" is a semiconductor (in which case both the polarization effect and electrical conduction can be signficant), the real part of the dielectric constant contains all the useful information. The dielectric constant is a strong function of frequency (so I guess it's not all that constant). Often the term refers to the DC (frequency = 0Hz) real part of the dielectric constant.
The most important dielectric materials for integrated circuit applications are silicon dioxide and silicon nitride. The biggest reason that silicon is the semiconductor of choice for modern integrated circuits is that a high-quality dielectric--silicon dioxide--is easily grown from it. Semiconductor companies and university researchers would love to find a replacement for silicon dioxide with a higher dielectric constant. This is because large gate-capacitance is a very desirable quality of a MOSFET. While there are tons of insulators with much higher dielectric constants, none have proven suitable for integrated circuits applications. Such "high-k dielectrics" are a hot area of research (k is the ratio of the dielectric constant of a material to that of vacuum).
Low-k dielectrics to replace silicon dioxide as insulation material between wires (interconnects) are also being heavily investigated. Interconnect capacitance is highly undesirable. Unfortunately, using air as the insulator between interconnect wires is not a feasible option because of mechanical instability.