Have you ever noticed that in "The Matrix", the phones that are designated hard lines are never used as actual phones?
Cellular phones (and regular wired ones) are for communication with the "real world" outside the Matrix -- and apparently, anyone's cell phone can be used, not just the ones Neo and Company bring with them. But a hard line is apparently designed to do one thing: bring people in and out of the Matrix through a secret pirate signal.
Possibly they're all part of an old physical telephone network which is no longer in common use; this would explain why the Agents can cut the line feeding them from outside the building. But without variation the hard lines are all situated in places where they'd rarely be used by ordinary people. Probably if they tried to pick one up, they'd get no dial tone, put the handset back down in frustration, and look for another phone somewhere else. It's all part of the clever camouflage.
It would have been nice if the script had taken a second to explain that, I suppose. But then again, if they went around explaining everything, it wouldn't have been much of an action movie.