From sound synthesis terminology, self-resonance is a useful property of some analog filters that can be (somewhat ineffectively, due to the lack of internal noise) modeled in digital ones. A filter without any resonance at all cuts off all sound below a certain threshold, set by the user. Like so, in a low-pass filter:

 D |-----.         
 b |      \        
   |       .       
   |       |       

In a resonant filter, the frequencies immediately preceding the cutoff frequency are amplified instead of cut, sort of a feedback loop in that part of the sound spectrum. The graph looks like this:

   |      .-.      
   |     /   \     
 D |---~'     \    
 b |           .   
   |           |   
   |           |   

Now, self-resonance occurs when you take that "boost" one step further, pushing the amount of feedback up to (or over) the amount that is damped by the filter's circuitry. The feedback is then self sustaining (or self amplifying) and essentially drowns out the frequency input to the filter:

   |        .-.    
   |       /   \   
 D |      /     .  
 b |     /      |  
   |    /       `  
   |--~'        |  

Not only is the resonance self sustaining, but it can even get itself going from random noise in the filter without any input at all, if the resonance is set high enough. A heavily resonant filter can thus be used as a dirty, evil sounding oscillator if the need arises. In non-damped (read: old fashioned) designs, the resonance could get far enough out of hand to burn out the filter's components. Even in modern filters self-resonant signal can become so hot that it ruins mixers, speakers, ears, etc.

Note: The illustrations were partially stolen from mingux's writeup under resonance.