As far as I understand it,
speculative fiction was a term first used by writers in the second half of the 1960's who wanted to distinguish their work both from more conventional 'hard' SF, and the turgid
Science Fantasy genre then gathering pace. They quite consciously retained the magic initials '
SF' in order to acknowledge their 'heritage', but disparaged the un-literary and superficial tales of either the "whizz-bang", rocket-wars end of the genre or the sub-Lovecraftian "ew-splatt" variety. The writers who had worked within, but transcended, the traditional SF forms were their starting point and inspiration (though many claimed inspirations more within the traditional literary canon, such as
James Joyce and
Franz Kafka.)
A transatlantic alliance of new writers formed, including Michael Moorcock, Norman Spinrad, John Sladek, Donald Barthelme - who at first could only find an outlet for their more unusual pieces in the Moorcock-edited New Worlds magazine, which would publish works using unconventional and experimental narrative forms and styles, and was more interested in talking about people and societies than about technologies or weird slimy creatures.
Typical of this were the sequence of Jerry Cornelius stories published there. Moorcock invented the character, and a way of writing fiction - purposely otiose, using obscure references without stating them, parodic of 60's culture, blending newspaper headlines and current news items with the 'action' - and several New Worlds writers took up the baton and ran with it, writing their own Cornelius stories and paying little regard to any superficial consistency between their respective efforts.
The genre 'arrived' in the States in 1972 with the publication of Harlan Ellison's Dangerous Visions anthology, and its sequel Again Dangerous Visions, containing stories from many 'new wave' American writers such as David Gerrold (for such he was, then) and Carol Emshwiller, as well as contributions from 'older wave' writers, such as Kurt Vonnegut (The Big Space Fuck) and Ray Bradbury (Christ, Old Student in a New School) trying their hand at the 'new thing'. Philip K. Dick, somewhere in between these two generations, contributed the outstanding Faith of our Fathers, which caused a furore, and had a damaging effect on Dick's career, when Ellison claimed in the accompanying preamble that it had been written under the influence of LSD (which PKD always denied).
As things have turned out, SF has evolved into something of a continuum, between straightahead adventure writing set in the future, and the more social and literary concerns of today's speculative fictioneers, but it's unlikely the best contemporary writers operating between the two (I would name Greg Egan leaning towards one side and Iain M. Banks towards the other) would be creating works with the depth and relevance that they do, without those pioneering efforts.