Very shortly after Krupp, MAN, Henschel and Daimler-Benz began production of the Panzerkampfwagen I, the Wehrmacht ordered these companies to go back to the drawing board to produce a 10-ton tank to support the 5-ton PzK I. By 1935, the earliest forms of the PzK II, known as Ausf a1, a2, and a3, rolled off the production line, sporting beam-riding suspension like the PzK I. The beam-riding suspension contained three articulated pairs of roadwheels connected on the outside by a girder.

In 1935, the Wehrmacht had decided to leave the PzK II under the same classification as the PzK I, a "Landwirtschaftlicher", or "agricultural tractor". These first variants of the PzK II were used for training with larger, heavier vehicles. In 1937, the last development model of the PzK II, the Ausf c, was mass-produced into Ausfs A, B, and C rolled out sporting the new modified Christie suspension, with five pairs of roadwheels instead of three.

Further revisions of the PzK II occurred, but it was clear from field experience that the PzK II was still inadequate, even for most reconaissance missions. Even in later models, the PzK II's armor was too thin (20mm) and its armament was too modest (20mm KwK 38) to damage a significant portion of the Allied armor. However, development efforts on the PzK II assisted in the creation of the Panzerkampfwagen III and IV, and the Marder II.

The PzK II was used as the main tank in Poland up until 1942, and saw moderate action under Erwin Rommel in North Africa. In 1942, after being replaced, most PzK II Ausf F series tanks were mounted with dual flame-throwers mounted on trackguards, and renamed as Flammpanzer II Flamingos.