The minicomputer was a small - deskside to room-size - mainframe-style system that popped up in the late 60's and early 70's. Minicomputers began a cycle that was repeated in the industry several times later: they provided a substantial proportion of what a mainframe did while being vastly cheaper. The first truly popular minicomputers were probably the PDP-8 and the PDP-11 from DEC, although a wave of other systems arrived around the same time - IBM's System/3 line and its successors, the Data General Nova, HP's 2100 and 3000 lines, and dozens of others. These machines were cheap enough (a few tens of thousands of dollars) for many businesses to start operating them as their first computers.
These days, minicomputers have been marginalized. The big UNIX systems are considered "midrange systems," a separate category from minicomputers; the only two mini operating systems left are IBM's OS/400 and HP's VMS. OS/400 is a descendant of IBM's old System/3 line and its successors, and still has a fairly large customer base (IBM claims 100,000) running on the same Power Systems hardware that IBM's UNIX system does. HP's VMS - a very distant derivative of the RSX-11 operating system on the PDP-11 - is still developed and sold, and runs on Itanium hardware. It has a dwindling but significant customer base, including many government and military customers. The HP 3000 line, which ran HP's MPE operating system, survived until a few years ago, but was gradually phased out after HP switched to Itanium hardware.
Unlike the mainframe, minicomputers probably don't have much of a future. IBM can still claim impressive numbers of customers for their OS/400 systems, but those numbers are a shadow of where they were ten years ago. There's relatively little place these days for a strange proprietary platform programmed in strange proprietary languages running strange proprietary software, even if those platforms can claim an impressive historical lineage. OS/400 and VMS will survive at least another few years, but at some point, they'll probably follow the other minicomputer systems into the attic of history.