Almost every modern processor can implement virtual machines in hardware. If you're using an operating system with memory protection1, each process is allocated its own virtual machine, with a virtual memory space separate from the other processes. When a process tries to access an area of memory in the virtual machine which hasn't been allocated by the OS yet, it causes a GPF/segfault2 (in most operating systems) and is immediately killed, as there is no sane way to recover from this error.

In an OS that doesn't use virtual machines (windows 3.1, for instance), everthing runs on the same processor and memory space, and programs that run amok can potentially overwrite other programs, or even the OS (like in corewars). Windows 9X is a bit of a special case, as it uses one virtual machine for each win32 program, a single virtual machine that all the win16 programs share, and allows win16 and dos programs to do stuff in real mode (and thereby hose the OS)3

Hardware virtual machines can also be used to emulate other operating systems, for example NT's Virtual DOS Machines, or VMware

1 - Which would be any modern operating system, including windows NT, mac os X, linux, etc. etc. It does not include mac OS < 9, Windows 9X, windows CE, or almost any other consumer OS.
2 - Any access of virtual memory which is not mapped to physical memory causes a page fault, and the operating system is then responsible for either (if the page does exist, but has been swapped out) restoring the page from disk, or (if the page does not exist) terminating the process (GPF/segfault).
3 - see Windows 9x does not have true memory protection