There are several reasons for this..

1. The scent of intelligence hangs heavy in the air. (Natural selection, anyone?)
2. Many people go to libraries to study, and often when one is studying at least a portion of one's mind is seeking a distraction from studying.
3. Books are very sensual, organic things - delicate yet powerful - and the way someone handles other peoples' books reveals a lot about them as a lover.
4. Libraries are full of half-secret corners, like dead ends in a maze. All those narrow, dimly-lit stacks...
5. All the whispering creates a subtle atmosphere of illicitness.
   
6. Sex is nearly opposite of what you're supposed to think of in the library - rationality, fact, and logic versus passion.