The full quote is actually "The Walls have Mice, and the Mice have Ears". The earliest reference to this line that I've found is from the third-century Palestinian sage Rabbi Levi, although it is also claimed to be Persian in origin