|You know it makes sense|
Star Trek was described, in its older incarnations, as "utopian". Gene Roddenberry wanted to portray a society which had solved its basic problems, and was looking outward. It is post-history. Star Trek TOS was made at a time of great social conflict, not to mention the Vietnam War, but imagined the possibility of change. This vision is even more subversive now. Contemporary ideology insists that there is no alternative, that inequality, violence, and oppression are inevitable and we might as well get used to it, but Star Trek refuses to accept this. That's not to say Star Trek necessarily presented a consistent or worked-out Utopia. In TOS things often altered according to the need of the current story. Quite a lot of things still seem to go wrong, but they are aberrations. Kodos the Executioner carries out atrocities, but is stopped by the Federation ("The Conscience of the King", TOS); Lt. Stiles is racially prejudiced, but this is not acceptable to his commander ("Balance of Terror", TOS); etc. The idea of a better society was fairly constant. (In TNG, where there was a more self-conscious utopianism, the presentation was not always successful: sometimes Captain Picard and others tell you about their society's superiority rather than show you.)
It is curious how much hostility this attracts. There is a desire, even among those who are supposed to be working within its framework, to somehow make it more like everything else. Ira Steven Behr, the producer and writer from Star Trek: Deep Space Nine, has been quoted that he thought DS9 was more realistic than TOS and TNG, which he found implausible. But supposing Utopia was the point? As Jane Austen might put it, "Much more realistic, I dare say, but not near so much like Star Trek."
Perhaps there is a psychological parallel to the way some American discourse seems to regard it as a grievance that Europeans take more holidays. How dare they make their society more pleasant?
In DS9 most of the novelty comes from the setting, involving sustained interaction with other cultures, and from the unusual degree of long story arcs, and the "realism" can be defended as showing the Federation coping with crisis. However, it also introduced Section 31, a ruthless secret organization, which seems to be running an endless dirty war on behalf of the Federation. Ira Behr, the executive director of Deep Space Nine, has been quoted as saying that perhaps the Federation can be a paradise because someone does its dirty work. It's "very complicated". Of course. It's always "complicated". Incidentally, let's stop to think about the idea that the Federation could be a paradise because its security is taken care of by Section 31. This idea is tempting for about 0.68 seconds, after which you notice the implicit assumption that the good society is basically about security. In fact Section 31 is quite irrelevant to, for example, the economy in which material things no longer motivate people. Even now, you would not explain Sweden's successful welfare system by its secret service.
Imperfections, including some violations of rights, do not invalidate the Star Trek utopia, especially when the violations are seen to be challenged (e.g. "Measure of a Man" TNG). "Realism" in the sense of dealing with the society's issues is not a problem. Similarly, the details of Roddenberry's original ideas, such as his hostility to religion, may be modified or questioned. Section 31, however, implies that the whole thing is a fraud. The future society has not really solved the problems of the present. It has just got better at killing its enemies.
All this was an unnerving anticipation of the threats to previously agreed democratic standards in the 21st century. Perhaps if the programme-makers had been making the same decisions now, with so many examples of authorities who have little hesitation in breaking the rules, they might have been less inclined to go in that direction.
There is no denying that there is a large part of the fan base that likes the "realistic" approach. It can produce good stories. But it is abandoning the reason Star Trek was created in the first place. Television and movies are full of "dark", "realistic", visions. Star Trek offered something different. You might think that this would make it interesting as an exception, but no, the desire is to close off the alternative. There must be no alternatives.
Although the portrayal of the future as like the present (except with warp drive) is presented as "realistic", in fact historians will tell you that the world changes. Our attitude to slavery would seem implausible, to the point of incomprehensibility, to an ancient Roman. It is the "realistic" parts of Star Trek which are sometimes the least plausible in the long run, as for instance with the original series' half-heartedness on changed gender roles. Another interesting case on a smaller scale is that Nicholas Meyer wanted to add a "No Smoking" sign to the bridge, and asked why people would stop smoking in the future after doing it for so long. That was in 1982. (Smoking was absent in TOS.) Similarly, in reading premodern proto-science fiction, it is not the daring predictions in books such as The Reign of King George VI (written in 1763) which now seem absurd but the "realistic" parts such as the projection of eighteenth-century politics into the twentieth century. For a classic exploration of this point, see Poul Anderson's 1956 story "The Man Who Came Early," where a twentieth-century man in tenth-century Iceland finds that apparent contradictions and implausibilities are spotted in his account of modern life.
It has to be said that Star Trek is not alone in this. Mission Impossible was a successful TV series which was based on a highly unusual premise. Each week the team would achieve some very difficult task, such as a rescue from the Eastern Bloc or preventing a coup, by means of an ingenious, super-elaborate (and sometimes high-tech) fraud. It was a deadly practical joke; a mind game. The point was that (with a few exceptions) nothing went wrong. It had ritualistic sequences which were deeply satisfying (everyone remembers "This tape will self-destruct in five seconds").
But then, there was a film of Mission Impossible. The film-makers either did not understand the original or did not care, and produced an ordinary spy action film. As a spy action film it might be considered good or bad, but it wasn't Mission Impossible. Why take the name and identity of something highly distinctive and throw away what made it interesting? Greg Morris, who played Barney in the TV series, is reported to have walked out of the film and described it as "an abomination". (Like Uhura, Barney was an anti-stereotypical character for the time, being an African-American playing the brilliant and assured technical expert, while a white actor played his muscular companion.)
Postscript. At the time of writing I have only seen the first season of Discovery, but it seems—despite the rather dark subject matter, starting a war in the first episode—to be stressing the positive values of the Federation again. Perhaps now that we are in a world where the threats to democracy and civil society are coming out into the open, there is a renewed recognition of why it's important for Star Trek to provide an image of a better possibility. I hope so.