It is true that destruction of the machine and media is necessary for true protection. This also means that the data will be lost, but if the data is that incriminating then it would have to be destroyed to save you from punishment.
The quickest way to destroy all media is not to take the machine apart and systematically damage everything beyond repair, but to take everything out in one go.
Note: some of these methods are dangerous, and are not recommended, these are here purely as an example of a method of achieving total security.
- Rig the machine to self destroy – set up explosives around the machine (the charge doesn't need to be large to destroy all chance of recovery) and all related media with some form of remote detonator (such as one you can use when you realise that the FBI are on their way – garage door opener perhaps). Have your machine in a place likely to contain dangerous household chemicals - petrol, gas canisters, fertilisers etc. - like a shed or detached garage, then these chemicals will catch fire (assuming you set up to), aiding the damage and helping it to look like an accident. The resulting explosion and fire might look suspicious, however they would not be able to prove anything.
Keep a second machine in the house for normal use. Connect it to the internet and use it to buy things from Amazon. Have an email account (for purely innocent uses) and make sure the machine is used for various other things. This will make the real machine which has just been destroyed look like an old machine which was not used.
Do not connect your real machine to the internet, or any other form of outside connection, this means that the need for security on the machine is far less, something such as PGP would do, since the idea is to stop anyone getting at the machine in the first place.
Have some way of getting to your real machine which doesn't involve you being seen (underground is an obvious method). This avoids suspicious movements around.
As has already been said, do not draw attention to yourself, this can help to avoid any more drastic action. Being seen as a good member of the community helps, so hold a job which carries respect (law enforcement, fire department etc). Involve yourself in community events so there are people to speak out for you should you be caught.
Have a good home security system, proximity devices, guard dogs, guards if they can be afforded. You are less likely to be surprised this way. Do not have a partner, or hired help, they might confess, or stumble upon something they shouldn't.
Know your rights, do not allow them access unless they have the appropriate orders. Hire a very good lawyer.
Obviously this is all a bit far fetched, but it is very difficult to hide what you might be doing completely.