I was listening to Star Talk the other day (Scientists Who Were Persecuted for Being Right, with Matt Kaplan). They were discussing this topic and noted that science done well is very messy. There should be point and counterpoint discussion/argument, as that is what makes the method work. However, to an outsider it looks like no one knows what they are doing. Unfortunately, since humans are involved, there are mistakes, misinterpretation, misunderstanding, poor judgement and falsification. This leads the outsider into further distrusting the process.
In my experience in the chemical industry, taking data and analyzing it statistically often resulted in answers way smaller than the error bound. I always look skeptically at study results that claim fantastic accuracy about things that can only be measured very inaccurately. They always explain that they corrected for the things that made it inaccurate and I think oh-oh you threw out the inconvenient data. The scientific method is hard to do well and you have to be good at knowing when to give up on an idea because the answer you got was way smaller than the error bound.
When the research summary says it was a meta analysis my alarm bells really ring. I don't understand how any meta analysis can possibly understand the actual error bound on data taken by many different people in many parts of the world. Particularly when the studies they analyse often were attempting to answer very different questions. Seems like a method to ensure finding causation where none really exists.
The facetious rule of thumb the StarTalk folks quoted was: "If you need statistics to get the answer it is not really science." There is an element of truth in that statement.
I recently finished college (and am devouring your substack's archive, sorry, not feeling quite full yet), and it's so funny admiring science before trying to actually make it and then seeing it from the inside and noticing it's essentially "orderly infighting". Nonetheless, it only made studying science even wilder, like: "Wow we actually got somewhere by doing this!".
Writing through this comment I noticed that it only reinforces your past point that science is a strong-link problem, and the proof is that we actually got where we are almost "despite" the way it works.
This reminds me — a deep dive on the etymologies of “belief” and “worship” is instructive.
Belief and love come from the same PIE source, *lewbʰ — “to care, desire, love.” And worship is what you turn to, over and over. So “what you hold to be epistemically true” actually has a lot to do with “what you hold dear” — which is why no amount of facts will dissuade.
And as in your Eddington example, what you turn to, over and over, curves your back and restricts what you’re able to see. For the man with a hammer, everything is a nail.
Yes! And “Science” is just one form of knowledge production, one way of understanding, producing, and categorizing what we count as knowledge, and subsequently how we shape our societies. The equation of numerical data to truth is a very modern invention that shaped modern knowledge into the data-obsessed culture the west has now.
Shapin and Schaffer’s “leviathan and the air pump” and Mitchell’s “rule of experts” are great works in history of science that discuss this, in addition to the literature you use
Sure, but that’s like saying using cream is the only way to make butter. It is, but there’s lots of other types of milk products (knowledge) that are just as legitimate, but the process of making butter (science) can only ever produce one of its types, butter (scientific knowledge). And so it’s able to create good but very specific kinds of knowledge, while many other equally legitimate forms of knowledge are not only labeled as illegitimate but cannot even be explored or known because they do not fit into the kinds of structures of processes and inquiry that the scientific method imposes.
Wrong. That's pomo happy talk. If you want to design a bridge, you need the science of mechanics and vectors. Nothing else will do and nothing else is "just as legitimate".
Thanks for this. Nice examples of 20-20 hindsight. "Science advancing vis funerals" and the exceptions. Nothing is ever simple..... Anyway, very good post! Thank you
I was listening to Star Talk the other day (Scientists Who Were Persecuted for Being Right, with Matt Kaplan). They were discussing this topic and noted that science done well is very messy. There should be point and counterpoint discussion/argument, as that is what makes the method work. However, to an outsider it looks like no one knows what they are doing. Unfortunately, since humans are involved, there are mistakes, misinterpretation, misunderstanding, poor judgement and falsification. This leads the outsider into further distrusting the process.
In my experience in the chemical industry, taking data and analyzing it statistically often resulted in answers way smaller than the error bound. I always look skeptically at study results that claim fantastic accuracy about things that can only be measured very inaccurately. They always explain that they corrected for the things that made it inaccurate and I think oh-oh you threw out the inconvenient data. The scientific method is hard to do well and you have to be good at knowing when to give up on an idea because the answer you got was way smaller than the error bound.
When the research summary says it was a meta analysis my alarm bells really ring. I don't understand how any meta analysis can possibly understand the actual error bound on data taken by many different people in many parts of the world. Particularly when the studies they analyse often were attempting to answer very different questions. Seems like a method to ensure finding causation where none really exists.
The facetious rule of thumb the StarTalk folks quoted was: "If you need statistics to get the answer it is not really science." There is an element of truth in that statement.
Your work is always enlightening.
I recently finished college (and am devouring your substack's archive, sorry, not feeling quite full yet), and it's so funny admiring science before trying to actually make it and then seeing it from the inside and noticing it's essentially "orderly infighting". Nonetheless, it only made studying science even wilder, like: "Wow we actually got somewhere by doing this!".
Writing through this comment I noticed that it only reinforces your past point that science is a strong-link problem, and the proof is that we actually got where we are almost "despite" the way it works.
This reminds me — a deep dive on the etymologies of “belief” and “worship” is instructive.
Belief and love come from the same PIE source, *lewbʰ — “to care, desire, love.” And worship is what you turn to, over and over. So “what you hold to be epistemically true” actually has a lot to do with “what you hold dear” — which is why no amount of facts will dissuade.
And as in your Eddington example, what you turn to, over and over, curves your back and restricts what you’re able to see. For the man with a hammer, everything is a nail.
Sometimes the hardest thing is not proving that something no longer works.
It is admitting that it is no longer worth organizing your life around it.
Yes! And “Science” is just one form of knowledge production, one way of understanding, producing, and categorizing what we count as knowledge, and subsequently how we shape our societies. The equation of numerical data to truth is a very modern invention that shaped modern knowledge into the data-obsessed culture the west has now.
Shapin and Schaffer’s “leviathan and the air pump” and Mitchell’s “rule of experts” are great works in history of science that discuss this, in addition to the literature you use
The Scientific Method (and proof in mathematics) is the ONLY way to produce science and scientific knowledge.
Sure, but that’s like saying using cream is the only way to make butter. It is, but there’s lots of other types of milk products (knowledge) that are just as legitimate, but the process of making butter (science) can only ever produce one of its types, butter (scientific knowledge). And so it’s able to create good but very specific kinds of knowledge, while many other equally legitimate forms of knowledge are not only labeled as illegitimate but cannot even be explored or known because they do not fit into the kinds of structures of processes and inquiry that the scientific method imposes.
What other forms of knowledge and what is it exactly that makes different forms of knowledge: a) different and b) equal (or unequal)?
Exactly.
While I am indeed skeptical of Anna Lytvynova's claim, I would like to make it clear that my question is intended sincerely and not rhetorical.
"that are just as legitimate"
Wrong. That's pomo happy talk. If you want to design a bridge, you need the science of mechanics and vectors. Nothing else will do and nothing else is "just as legitimate".
Thanks for this. Nice examples of 20-20 hindsight. "Science advancing vis funerals" and the exceptions. Nothing is ever simple..... Anyway, very good post! Thank you