Recommended reading:
Armed and Dangerous: The Rise of the Survivalist Right

Published in 1988, this presents a very good history of survivalism and its eventual alliance and merger with the crazy right. It didn’t start out that way. Survivalism as such originated during the initial stage of the Cold War nuclear standoff in the 50s. Many people realized that the government’s own civil defense measures were wholly inadequate, if not absurd, and started to research the prospect of surviving a widespread catastrophe on their own.
This received a mighty boost from the Cuban missile crisis of 1962 and the brief fad for building individual fallout shelters. People who were interested in this tended to be fairly conservative politically but there was no real political component to it until later in the 1960s.

Right wing extremism, especially conspiracism, developed along a separate track at first. The founding of the Birch Society in 1958 and the passage of comprehensive civil rights legislation in the mid 60s were major milestones. The former provided the conspiracist narrative, while the latter solidified the idea among even casual racists that the feds were The Enemy. By a not very gradual process, the federal government replaced the Soviets as the source of the threat and ravening black mobs replaced nuclear warheads as the primary tactical threat. The rise of televangelism in the 70s, and its ability to popularize the notion of literal prophecy, added a new element, providing a religious rationale for the extremist school of the survival culture.

Today, there are still “old-school” survivalists, that is, the ones who really understand the subject and who are not conspiracist wingnuts, but they naturally tend to keep a very low profile.
By the late 80s, the extreme right had largely reached its current form, a witch’s brew of Cold War anxiety, the backlash against social change, and the media-driven revival of fundamentalist religion.