Summary of Lightning Tester data

custom

custom

Well-known member
Had 1-2 guys who could not complete the logs, but other than that, everyone did a fantastic job of keeping nice, detailed logs.

Here is the final breakdown:

Thunder + Lightning

Invalid Link Removed


Highest Bench Gain: 50 Pounds
Highest Squat Gain: 100 Pounds
Highest Bodyweight Gain: 7 Pounds
Most Weight Lost: 5 Pounds




Lightning Only:

Invalid Link Removed



Highest Bench Gain: 30 Pounds
Highest Squat Gain: 65 Pounds
Highest Bodyweight Gain: 6 Pounds
Most Weight Lost: 3.5 Pounds
 
Some suggestions: To make any sense of this information you need a placebo group to determine if there is any statistical significant increases in poundage that can be attributed to your product. Also, I think some outliers need to be removed. A 100lbs squat increase? How long did they use the product? With all due respect, I am fairly confident that this increase is not a result of your product but of many confounding variables.
It is nice to finally see some hard data, congrats on that. Too many new products are accompanied by tag lines and "results" that seem born from the minds of muscle tech "scientists".
cc
 
You are right about the other variables that could be taken into consideration. However, you must understand that to run a proper test in which most variables could be controlled would be very difficult for these companies.

I would understand your point if this chart was all Custom showed in way of gains made without any explanation (much in the way a company like muscletech would). However, there is a sense of explanation in the logs that have been kept.

Sure a 100 lb. squat increase is extreme. Hell it was me that did it. I know that that was not 100 % due to the product. The explanation for this is in my log. I am somewhat new to doing legs so I will have somewhat newb gains in my legs. However even with newb gains 100 lbs is outstanding and therefore may say something about the product. Furthermore the averages taken try and eliminate those extreme outliers.

I completely understand your point of view, but without unlimited funds I think Custom did a great job with this test.
 
Dont misinterpret my comments, I greatly respect his work and testing practices but I was offering suggestions to improve his compilations of the results.
I am of the mind that supplements should be taken by those who have many years of training and who need the little extra boost to give them the gains they desire. That being said, I would have more closely selected testers based on that criteria, thus uncommon squat gains (as experienced by yourself) would not skew the data collected.
I also dont buy the 'not enough money' bit as it costs little to nothing to send out an equal number of placebo pills. Obviously, he already owns excel and thus software purchasing is a non-issue. Am I missing any hidden costs here?
It is also essential that he includes the sample size when presenting his data, especially if the desire is to work with average. There are also much better statistical methods to determe if outliers exist than averages.
cc
 
Doing a "real" test with placebos and double-blind controls and everything is super crazy hard to do...it is not something I could do myslef but would have to hire a facilitating company to do. But I do see your points and they are well taken.
 
I have been involved in a few clinical trials so I am quite aware of the difficulties... especially if you have very little funding and support staff. Again I commend you on the work that you have done as I have grown tired of ads suggesting that I can put 5" on my arms in 30 seconds.
cc
 
canadian champ said:
Dont misinterpret my comments, I greatly respect his work and testing practices but I was offering suggestions to improve his compilations of the results.
I am of the mind that supplements should be taken by those who have many years of training and who need the little extra boost to give them the gains they desire. That being said, I would have more closely selected testers based on that criteria, thus uncommon squat gains (as experienced by yourself) would not skew the data collected.
I also dont buy the 'not enough money' bit as it costs little to nothing to send out an equal number of placebo pills. Obviously, he already owns excel and thus software purchasing is a non-issue. Am I missing any hidden costs here?
It is also essential that he includes the sample size when presenting his data, especially if the desire is to work with average. There are also much better statistical methods to determe if outliers exist than averages.
cc
Click to expand...


You are right of course, but to the layman none of that makes a difference. Statistical significance means jack to the people custom is trying to target. The sample sizes were small, the test wasnt controlled (wasnt double blind, wasnt placebo controlled, confounding vars werent taken into account, etc) but I think that much analysis is overkill here.

All Matt is doing is compiling the feedback from the log testers of his product, not making claims. Just simply putting forth the information.

Multiple testers didnt even report results.

I think for marketing purposes, these graphs simply put forth the information from the testers in a concise form that is clear and easy to understand. Remember, the key is that Matt isnt saying "Gain 100lbs on your squat in 4 weeks!" hes saying I had a tester who gained 100lbs on his squat. Not that anyone and everyone will.

But of course, you are right here and it would be lovely to have a controlled study. Its just really really hard to do, like you said.
 
You must log in or register to reply here.
Back
Top