Don't Deride the Masses
It's a tired trope. Our protagonists are the only ones who understand and accept the truth. The mass of people must be cajoled and carefully guided into doing the right thing.
A little while ago, I was reading another Golden Age sci-fi novel: City At World's End, by Edmond Moore Hamilton. It was a fun fast-paced read - a survival story turning into a political story, of an entire city - a contemporary American small city called Middletown - being isekai'd (transported) into the distant future when Earth is almost uninhabitable. But, the dominant theme of the book surprised me: the one-dimensional blinkered provincialism of almost all the townsfolk. Our protagonist, a scientist at the local defense plant, is one of the few people who really understands the situation and sees that they need to take extreme measures to survive this barely-habitable world (and then, later, to relate to the future humans when they do arrive). Virtually everyone else needs to be shepherded and nudged along to be willing to do anything; even the town's mayor must be coaxed and cajoled into doing the obvious, and even our protagonist's own wife figuratively drags her feet at every step.
As I considered this, I realized I shouldn't have been surprised. I'd seen this before. City at World's End is an extreme example, but this's a frequent trope I've seen a lot in twentieth-century science fiction. Frequently, our protagonists are the only ones, or among only a few people, who understand and accept some significant truth. The mass of people who don't understand must be carefully guided into doing the right things. Perhaps they don't believe it when they hear, or more frequently it's hidden from them lest they panic. (Here, our protagonist and his fellow scientists hide that no one else is probably living on Earth, lest people despair.) Either way, it's for their own good.
Consider, for instance, the famous (1968) 2001: A Space Odyssey. After the mysterious but obviously alien monolith is discovered on the moon, its existence is promptly concealed from virtually everyone. Not even Dave Bowman and his fellow astronauts are told - even though (in the book) their mission is changed from Jupiter to Saturn to investigate why the monolith sent a radio signal there. At least Dave Bowman finds out before arriving at Saturn; the public stays ignorant longer. In the film, in fact, this enforced deceit is why the computer HAL9000 goes mad and kills most of the astronauts.
This theme is played out in far more detail in Isaac Asimov's Foundation trilogy (published in serial form between 1942-1950). There, amid a dying galactic empire, Hari Seldon founded the Encyclopedia Galactica Foundation ostensibly to shorten the dark age after the empire's fall. But, it quickly turns out, he in fact planned them to be the nucleus of a new empire - and what's more, he predicted and designed their growth through his new science of "psychohistory." The dead Seldon deceives and shepherds his Foundation, which in turn deceives and shepherds the people of the petty states succeeding the dead Empire. Seldon has set the Foundation on a plan to build a new Empire without telling them what it is; they in turn create an artificial religion of "Scientism" so they can make the neighboring countries dependent on them for technology without teaching them real science.
These are just two prominent examples; it's a very common trope across many science-fiction books, and others as well.
Why do writers do this?
One reason to do this is a simple trick to bring about an obvious conflict. Our protagonist is right; the people around him are wrong; he must do something that is obviously right - perhaps even for their sake - but they oppose him. Perhaps it's because they don't like what he's doing; perhaps it's because they simply don't know his goal. Either way, a plot obviously follows. City at World's End milked this in high measure, and Foundation also uses it several times.
In addition, this can showcase your protagonist as a more special and sympathetic person, because he is one of the special few who understand the hidden truth. The Foundation trilogy uses this throughout the first two volumes; and City at World's End does this too. (In the third volume - perhaps deciding turnabout was fair play - Asimov chooses his protagonists from among the people in the dark. Their apparent heroism turns out not to achieve what they're hoping for, but they're still shown as special, and we the readers learn in the end that even their misunderstandings serve Seldon's plan. However, most other novels don't go here.)
In part, I think, this's a historical relic. In the 1950's when many of these books were being written, there really were a lot of towns somewhat like fictional Middletown. They might not have insisted on staying in their houses if Earth was freezing around them, but they weren't participating in the new growing global culture and modern technology. Robert Heinlein makes use of this apt juxtaposition in Have Space Suit, Will Travel where our protagonist returns from his journey through outer space to a small-town soda fountain where people don't care where he's been.
And also, it's part a historical relic in another sense, where the government was keeping more secrets in the 50's and 60's before the modern age of transparency in government. They did have some reason for this, of course - the Cold War. Today, keeping the 2001 Monolith secret would be impossible, even if the government wanted to try.
But also in part, this's a power fantasy where our author writes his single protagonist (with, maybe, a few supporting characters) doing everything or almost everything. This shows him as heroic, but at the expense of everyone else. Interactions and inter-character dynamics become far simpler. In City at World's End, whenever I saw town's mayor, or our protagonist's wife, I knew ahead of time what notes were going to be hit and where the interaction was going to go; and I could make a fair guess what people's feelings were going to be about it afterwards.
On top of this, I think this can give rise to an unhealthy sense of superiority for the readers. Many science fiction fans already have a sense of being a small subculture - even now to some extent, and that was far more the case in the 50's and 60's when speculative fiction wasn't yet a bestseller. So, when novel after novel invites its readers to sympathize with the protagonist who's fighting against the ignorance and shortsightedness of the masses while guiding them to a better day - I think fans would, if subconsciously, draw the parallels. When a novel blatantly sets the science-fiction fandom against the outside world (such as Niven's Fallen Angels, with insertions of some actual real-life fans) it can be treated as an overdone joke. But when novel after novel invites people to draw the parallels - I think that can subconsciously get through what might be dismissed consciously.
Eric Flint takes aim at this trend in the afterword to his (2000) novel 1632:
More generally, the American characters who populate 1632 are all figments of my imagination. But I like to believe they are a faithful portrait of the American people. Part of the reason I chose to write this novel is because I am more than a little sick and tired of two characteristics of most modern fiction, including science fiction.
The first is that the common folk who built this country and keep it running—blue-collar workers, schoolteachers, farmers, and the like—hardly ever appear. If they figure at all, it is usually as spear carriers—or, more often than not, as a bastion of ignorance and bigotry.
...
If there is one human characteristic which truly recognizes neither border, breed nor birth, it is the courage to face life squarely.
Flint is largely talking about social class here, but I think his critique is accurate to how many authors treat "common folk" of every social class in general.
In contrast, Flint praises many of the "common folk" in his 1632, both in the original novel, and the series following from it. You can say that's an artistic decision, and he certainly chooses a premise and plot that allows them to shine. When a West Virginian small town is transported into the middle of the Thirty Years' War, average West Virginians are going to have a chance to shine. And, that's the favorite aspect of the series in my mind: all the side stories about common people.
But also, another author could've told that story very differently. In several ways, 1632 can be aptly contrasted with City at World's End: both involve an American town transported to a distant and unfriendly time, where the townsfolk can't keep up their normal lives and need to take extreme measures to survive the conditions and the local people. However, the townsfolk and their leaders respond to that premise differently. In 1632, their leaders frankly tell their constituents everything, inspire them to take action, be resourceful, and "start the American Revolution" in their surroundings. Even before the challenge is given, they're living up to it. But in City at World's End, the townsfolk are hanging back, blowing through limited resources (like food and coal), and not taking any helpful action. Their leaders don't inspire them, but cajole them, and hide almost everything that can be hidden - and up to the end, the townsfolk contribute almost nothing to the eventual triumph.
If Hamilton (the author of City at World's End) had written 1632 using the same tropes he used in his actual novel, we would have protagonist Mike Stearns negotiating with the "downtime" European leaders one-on-one, and those would be almost the only scenes that change the course of the story - most other Americans would huddle at home in Grantville without talking to "downtimers." If Flint had written City at World's End, everyone would've been inventing new tricks to survive on the new world, our protagonist's role would've been an information clearing-house rather than information generator, and they would've inspired the natives of that time on the page by talking with them.
Both these stories could have been fun to read, but (at least in my opinion) Flint's route would've been more likely to be a better story. It gives a more multidimensional view of more characters, and reminds us that they are full people and there're many other stories that could be going on in the background. Better worldbuilding, including this, makes for a better story.
This's a tired trope that makes for simpler and easier stories. However, so much richness and dynamics between characters are sacrificed. They can still be fun with it, but they would be even more fun without it.
This seems related to https://tvtropes.org/pmwiki/pmwiki.php/Main/TheOnlyOne and https://tvtropes.org/pmwiki/pmwiki.php/Main/TheMainCharactersDoEverything.
I think there's a natural tendency in fiction to make all important plot developments happen as the result of the protagonists or villain's efforts. It just doesn't make sense to waste time on side characters that the audience won't remember anyway, and besides, the audience is here to see your main character do cool stuff, which isn't going to happen if everyone else has all the ideas before they can.
I don't think this is limited to SF either, because the forces that cause it apply to *all* fiction. It's just more noticeable in SF because ideas are more important.
I think one reason this crops up so much in (especially earlier) sf is a kind of revenge. Growing up in a community that is uninterested or actually hostile to intellectual pursuits (as some communities in the US definitely have been) the young writer builds up a chip on his shoulder that later shows up in these stories. If expect it less from younger writers who are growing up in the post-80s culture that provides outlets for nerds, and that is less hostile to sf generally.