A field of mathematics heavily used in 3-D calculations. Since every point in 3-D space has three values, you can represent rotations, movements, and other actions within 3-D space as matrix computations; solving these matrix activities and doing a final projection into 2-D gives you a representation of a 3-D world and objects in 2-D. This is how OpenGL works, why Doom and Quake look like they do, and represents much of the work going on in computer graphics. See SIGGRAPH.

No, Doom does not use 3d matrix maths for rotations and the like. This is because it is not really 3d. It uses a technique called raycasting. Raycasting works by traversing a tree of all the walls in the level for each column of pixels, and drawing the floor leading up to that wall, and then the wall itself, and finally the ceiling. None of this involves 3d matrix mathematics.