Dealing with untested claims made

JudoJosh

JudoJosh

Pro Virili Parte
Awards
3
  • RockStar
  • Legend!
  • Established
In a previous post of mine I touched on the concept of sciences (in)ability to "prove" things.Int his one I am going to touch on a little more science philopshy but this time try make it more usable for us. First let me quickly recap the basic idea of the other post and maybe expand on it slightly more by addressing another misconception about scientific evidence.

Another common misunderstanding when it comes to science is the idea that evidence in support of a theory means that theory is proven. Hopefully from the last post, http://anabolicminds.com/forum/supplements/262453-does-science-prove.html, we know this is not the case. Remember, science does not prove, it disproves. So what does that mean for the theory? It means, that until something can be disproven via empirical research, that theory will be the theory that is most favored at that time. Let me try and explain this in something relateable. A while back it was believed that eating 6 times a day was optimal for body composition. This was the prevailing theory. During this time we had some evidence that supported this theory, but does that mean that theory was proven? No, it is NEVER proven. It simply is the prevailing theory until it is DISPROVEN. What we see now in a lot of nutrition research is studies that are questioning this. This is the nature of science. It attempts to disprove in order to expand our understanding about something. So back to what evidence means for a theory, basically what we do is test to find if something is valid. If we can’t disprove it, then it remains valid. Now, if it is disproven then new theory will take its place. So how about now, would the new theory be proven? No. It is simply the current understanding of something. In science, there is no final say. Everything is continually being questioned. This is how science works. I believe this is a hard concept for people to grasp. Through our academic life we are used to be told this is how something is and that is the final say. In science this concept doesn’t really exist. We are constantly trying to disprove things or parts of things in order to get a better understanding of how something works. Research never proves anything. All it can do is provide support for a theory.

What about things that have not yet been tested?
Just because something that has not been tested yet, does not mean it is not valid. This might seem controversial to some but it shouldn’t really be. Saying there has been no research to support what you are saying is not the same as saying it won’t work. Often times in our industry especially, practical application of an idea is often going to ahead of science. And this is rightfully so. When there is a new periodization being tested in a exercise lab, where did this new periodization come from? It came from coaches in the gym who have been using it. Our experiences in the gym and kitchen is what lays down the foundation for hypothesis to be created and tested. Just because something isn’t tested doesn’t mean it is invalid and not useful. It simply means it is untested and we are unsure about it.

So how do we approach things that aren’t tested?
We should always be skeptical over any claims that are made regardless over the data or lack of that supports it. This does not change. However, when it comes to things that aren’t tested it is a good idea to find out the track record of the idea. Often time the next latest and greatest diet, training routine, supplement, or what have you will also not be tested yet. In these cases, when something is new we should sit back and see how it unfolds because chances are it is just a fad and it will fizzle out. But for the things that are tried over and over throughout the years with anecdotal success. We should not dismiss them simply because they haven’t been tested. Chances are it can be true. Can’t immediately throw something in the garbage because it hasn’t
been tested. Instead we should give it time to prove itself.

It is unrealistic to require everything that we do or take to have a meta-analysis done to support it. This is true especially in our field because of all the nuances that exist on top of the already nuanced nature of science. Instead, what we should do is examine the claim being made against our current body of knowledge. What this means is we check to see if the claims being made our scientifically founded. For example, if a coach presents a new twist on an exercise and makes a claim that it can improve X, it would be silly if I reject this claim solely because the movement hasn’t been studied specifically with X as the outcome measure. Instead what I should do is examine the movement and see if our current body of knowledge supports the general principle of the movement. So to add some real world context to this train of thought let’s use the following claim as an example, 4 to 8 sets x 60 seconds at 90% intensity with 4 minutes of recovery will improve a MMA fighters stamina. Do we need a specific study done using this work:rest ratio and looking at MMA performance as the outcome to accept this claim? While a study would be nice, it is not necessary. We can pit this claim against what we know about the energy systems used during a MMA fight and the adaptation effects training variables have on energy systems. The claim made utilizes ATP-PCr Anaerobic glycolysis and the typical training adaptations that result from training this system are an increased available of muscle glycogen and increased glycolytic enzymes. This leads to increased ATP production and power output during short high intensity efforts, which is the environment an MMA fighter will often find themselves in. Another example would be if the claim is made that exercise Y improved rotational speed. If exercise Y is a rotational exercise done fast, based off the principle of specificity, it makes sense that exercise Y can help improve rotary speed. In both of these examples, a specific study is not required to support the claims because the claims made are based off principles that have repeated shown to be true. The caveat here is, how the claims are presented. If the either claim stated their approach was the best approach or better than alternatives, then we would need empirical evidence presented in order to validate that claim. This is because the claim itself has changed. It is no longer about IF it is effective, it is now proposing itself as some special missing link that in absolutely and universally superior. I really like this quote by Hitchens, “Extraordinary claims require extraordinary evidence.” Applied here it means the initial claims made align with our current body of knowledge so has a low evidence requirement threshold as opposed the second versions which would go against our current body of knowledge. If the claim being made seems far-fetched, unrealistic, unreasonable, etc then it requires substantial evidence in order for us to accept it.

So I caution against simply dismissing claims that aren't currently specifically supported. Instead of checking if those claims are “scientifically proven”, what I propose is we critically evaluate those claims and see if they are scientifically founded. You know the saying, don’t throw the baby out with the bathwater. Same deal here. We can’t be super dismissive with things, especially in this field, just because a specific study doesn't exist to support it.
 
Driven2lift

Driven2lift

AnabolicMinds Site Rep
Awards
0
I'm glad you take the time to write and share these Josh
 
thebigt

thebigt

Legend
Awards
6
  • Best Answer
  • The BigT Award
  • Established
  • Legend!
  • RockStar
  • First Up Vote
josh i am truly impressed....not only for your skills, but for the passion you bring to what you do.

keep doing what you are doing...you are a great asset to this forum!!!
 
JudoJosh

JudoJosh

Pro Virili Parte
Awards
3
  • RockStar
  • Legend!
  • Established
Awe.. you make me blush :blush:
 
thebigt

thebigt

Legend
Awards
6
  • Best Answer
  • The BigT Award
  • Established
  • Legend!
  • RockStar
  • First Up Vote
Awe.. you make me blush :blush:
i have been hearing some really interesting[scary]stuff about GMO foods[seeds]and the dirty tricks monsanto has played on the american public...any thoughts?
 

Similar threads


Top