When one says something like “most scholars think x” or “the theory of y has not convinced many experts”, how is that actually determined? Are there polls conducted regarding different theories?
In modern media it pretty much just means they found two people who think that. If they want to get “official” they can arrange for polls to be done but those are very easily crafted to get the results they want.
It’s just a saying™️
Systematic reviews and meta-analysis combine information from multiple studies. In the former, these studies are interpreted together to see what the overall conclusion is. In the latter, data from multiple studies is actually reanalyzed to get an objective overall outcome. Some of these studies combine information from 10s or 100s of studies. Generally, most scientists believe the outcomes of these review papers to be the status quo of the field.
I don’t think it’s an objective metric. Based on my experience, they talk amongst each other at research institutions, conferences, and through journal articles. If someone claims “most experts think x” when in reality most experts do not, then most experts hearing it will probably speak up about how wrong it is, shoot it down during peer review, or publish scathing critiques in response to it.
A “most experts” proclamation that aligns with reality will also cite several prior publications that have also been read and cited widely, which shows the idea has kinda stood the test of time.
Source: I been in the game a while, despite several attempts to escape. I do wonder if other fields have more objective approaches.
