The security dilemma is one way that international relations theorists explain the outbreak of war. It is held to be especially pernicious because it can cause war even when neither party to the conflict desires it. The dilemma is that certain policies pursued by states which they believe will make them more secure actually make them less secure in the long run because they increase the chance of the state suffering armed attack. This simple theory is also helpful in understanding the outbreak of civil wars and ethnic conflicts.
According to the security dilemma, there is only so much security to go around. Any state that makes itself more secure by increasing its military capacity only does so at the expense of neighbouring states, whose own militaries are less impressive as a result. The country that is arming itself might have purely defensive intentions, but it will find it hard to convince its neighbours that this is the case and so risks sparking an arms race. Unable to trust one another, tensions between the countries will rise and one may launch a pre-emptive strike.
This is why rising great powers almost inevitably become involved in a major war during their ascent, as other countries expand their militaries and enter into defensive agreements to counter-balance the threat they perceive from the rising power. The dilemma is that while the rising power believed it was making itself more secure by building up its military, it may find itself in a war as a result of the fear other countries have for it. Aware of this problem, China has so far gone to great lengths to market the concept of "China's peaceful rise", protesting that its ascent to the big-boy league is not a threat to anyone else. By so doing, China is trying to avoid the security dilemma landing it in a major war with the declining power, which history strongly suggests will accompany its rise.
The security dilemma is operative because of the problem of trust between states. States fear each other because any one of them could theoretically go to war with any other, and at any time. They exist as if they were in Thomas Hobbes' state of nature, where they have no guarantee of each other's good will. Within individual societies, we avoid the security dilemma by surrendering the right to use violence almost entirely to the state. We do not have armed groups roaming the country engaged in blood feuds because the state can overpower them; avoiding the security dilemma is the benefit we get for surrendering the right to kill.
When the state becomes too weak, civil wars can break out. Ethnic conflicts are a particularly brutal example of the security dilemma because of the very low level of trust that often exists between various ethnic groups and the impact of racism or religious hatred. We saw this happen recently in Iraq, where the United States destroyed the power of the Iraqi central authorities and sparked a massive arms race between Sunnis and Shi'as, who given the history of the last thirty years have every reason to fear one another.
The two groups were in competition for finite resources - political power, natural resources, and land - and each believed the other would resort to violence to get them. Fearing massacres at the hands of the other group, they sought to make themselves more secure by forming militias; but each group's militia merely made it appear more threatening to the other. In this way, a civil war is a microcosm of the international system, where states compete for finite things - power, resources, land - and fear each other because there is no central authority to overawe them and make them co-operate and share.
Avoiding the security dilemma means recognizing the legitimate security demands of other states. Europe's history in the twentieth century provides a perfect example of how the security dilemma can break down if this rule is not observed. Europe used to have what was called a "balance of power" system, where security was supposed to be provided by the correct distribution of fear, with never too much and never too little fear experienced by any given state; the bipolar system during the Cold War was similar. The idea was that no state had to fear another or could be allowed to inspire fear in another because all kept their ambitions and militaries in check, so that a rough balance existed between forces; because the balance existed, the cost of going to war was always so high that it was presumed no-one would attempt it.
Unfortunately, this system amounted to nothing more than a vague attempt to control the security dilemma rather than transcending it, and Europe's security broke down when a rising power - Prussia - broke away from these polite norms so it could attain what it saw as its rightful place in Europe. It turned out that Prussia (later Germany) viewed its "rightful place" as being in control of vast parts of Europe, which understandably inspired fear across the Continent. World War I was the result. Then, when Germany was defeated, the victorious powers tried to completely cripple Germany by imposing the most onerous surrender treaty on her, forcing her to demilitarize and hand over large tracts of land. This later led to a harsh reaction from Germany and contributed significantly to the rise of Hitler; by trying to achieve absolute security at the prize of Germany's complete insecurity, the rest of the western world became a classic victim of the security dilemma.
After World War II, European countries resorted to all sorts of trust-building mechanisms between one another and formed the European Union, drawing closer together and demilitarizing. The last war had been so horrible that they had finally realized that the only way to defuse the security dilemma was to create an area in which military competition was no longer necessary, effectively ceding parts of their sovereignty to a central authority in exchange for peace and a seat at the table from which Europe's shared resources were distributed.
This co-operation was possible because of Europe's shared history and cultural affinity, and anyone who imagines the experience might be repeated on a global scale by the creation of a world state is deluding themselves. Countries cannot resolve the security dilemma simply by talking unless they already trust one another, and a world state counting, for instance, the United States and Iran as citizens would just as soon find itself in civil war as the existing system would result in ordinary war. The security dilemma is far from resolved yet.