2's complement is simply arithmetic modulo 2n, where n is the number of bits you have. You can take any consecutive 2n numbers and claim your bits represent them, so just pick the numbers starting at -2n-1. This explains why addition and and multiplication continue to work.

sockpuppet's fun example with the diverging infinite series is just an example of how natural p-adic numbers really are; here p=2.