When you have light reflected off some surface; there is some part of the light that is scattered equally in all directions. This is the reason that we can see matte surfaces like paper, cloth, brick and so on, even if we are not directly in front of it, or seeing a direct reflection of the light source lighting the surface. The fact that it's scattered in all directions roughly equally is the thing that makes it diffuse. This distinguishes it from specular reflection, or the type of reflection you get from a mirror, which reflects light only in one particular direction, or unequally in some directions.

Diffuse reflection basically depends on the angle betwee the light source and the surface. If the surface faces the light source, then it will be bright. If the angle between the light source and the surface is great, it's dimmer. The relationship is described mathematically as Lambert's Law.