TASS said the four dead included the gunman. Local prosecutors were sent to the scene to open an investigation.
Footage circulating on social media showed a man firing as he walked down a street. At the end of the video he fires from close range at someone lying on the ground. Reuters was unable to verify the footage.
Krymsk is a small city in Russia's southern Krasnodar region near the Crimean peninsula, which Russia unilaterally annexed from Ukraine in 2014.
(Reporting by Reuters; Editing by Kevin Liffey)