The Pak 38 was first used by the German forces during the Second World War in April 1941. When the Germans faced Soviet tanks in 1941 during Operation Barbarossa, the Pak 38 was one of the few early guns capable of penetrating the 45 mm (1.8 in) sloped armor of the T-34's hull at close range.
After the Spanish Civil War, the German authorities started to think that a new anti-tank gun would be needed, even though the 3.7 cm Pak 36 had proven to be very successful.
They asked Rheinmetall-Borsig to produce a new and more capable AT-gun. They first designed the Pak 37 in 1935, but the German authorities did not approve it because of its low capabilities. Rheinmetall-Borsig were forced to create a new gun under the designation Pak 38, which fitted a new and longer L/60 barrel and was approved for mass production in 1939.
The gun was also equipped with Panzergranate 40 APCR shots with a hard tungsten core, in an attempt to penetrate the armor of the heavier KV-1 tank.
Although it was replaced by more powerful weapons, it remained a useful weapon and remained in service with the Wehrmacht until the end of the war.
The Pak 38 carriage was also used for the 7.5 cm Pak 97/38 and the 7.5 cm Pak 50 guns.
Romania imported 110 Pak 38s in March 1943. The guns remained in service with the Romanian Armed Forces until 1954, when the 57 mm anti-tank gun M1943 (ZiS-2) replaced them.