by Gary S. Aumiller, Ph.D.\u00a0 ABPP<\/span><\/p>\n \u00a0<\/p>\n So, my 11-year old daughter had to do a science project earlier this year and she decided because her mother was constantly saying that playing on the IPad was hurting her, she would test out whether IPad play really had an impact on her attention span.\u00a0 So, we set it up with three alternate forms of a neurological test, the Trail Making Test, a connect the dots type of activity, (okay I helped a little), one before, one in the middle and one at the end.\u00a0 There was an hour and a half of Minecraft in-between each of the trials.\u00a0 We got her friends over and measured the change from trial to trial, with both time and number of errors as the variables.\u00a0 The results\u2026.well, I\u2019ll get to that.<\/span><\/p>\n We start training kids from an early age on the rigors of scientific method and how to make a scientific study.\u00a0 We train them how to test a theory and how to make a hypothesis.\u00a0 We train them that science requires experiments to answer questions and learn more about our world.\u00a0 What I think we forget to train them in is that one study does not provide a definitive answer but only a suggestion.\u00a0 The ignorance of the statement that it is only a suggestion is how we come up with a bunch of \u201cFake Science\u201d being reported and guiding our way of life.<\/span><\/p>\n Okay, I borrowed \u201cFake Science\u201d from \u201cFake News\u201d \u2018cause it would make a good weekly TV show.\u00a0 That is sort of the problem also \u2014 good studies make better entertainment or as one minister once told me \u201cdon\u2019t let the truth interfere with a good story.\u201d\u00a0 In a study by Brian Nosek called the Reproducibility Project, Dr. Nosek and a team of 269 co-authors started trying to reproduce studies from three prestigious journals in psychology.\u00a0 In the Reproducibility Project, as it has come to be known, only 39% of 100 studies from 98 papers in the journals were reproducible.\u00a0 Meaning when they redid 100 studies only 39 got the same results in the same direction.\u00a0 That\u2019s less than a third of a chance that the study you read is correct.\u00a0 Meanwhile, people were relying on these studies and making decisions on the results of these studies.\u00a0 But, how can we even be sure this replication study is for real. A Stanford epidemiologist, which is a person that studies disease origins, says that 39% number is high.\u00a0 In fact the number should be closer to 80% of studies are not reproducible, in other words, you have 1 in 5 chance of having accurate information.\u00a0 He said the Reproducibility Project tested studies only from distinguished journals.\u00a0 If you move to all\u00a0 journals, the data is even more favorably skewed, and further the amount of fraud and misapplied statistics is ridiculous.\u00a0 There are major incentives for studies to find results in a certain direction.\u00a0 For one, you must get it published.\u00a0 Second, the news channels will cover it if it is profound, and thirdly, funding comes to those that get results.\u00a0 In all, only 3% of research money goes to replicate that which is already reported.\u00a0 Once it is published, people forget about caring whether it is accurate.\u00a0 So much for double checking your work that we are all taught in elementary school.<\/span><\/p>\n Now for the really bad news! \u00a0This seems to apply to more sciences than just psychology.\u00a0 In cancer biology, one analysis showed that only 6 of 53 studies are replicable.\u00a0 THAT\u2019S A WHOPPING 11%!!\u00a0 In other words, if your doctor is depending on the current journal articles to select a treatment for your cancer, 89% percent of the time they will give you the wrong treatment.\u00a0 An overall study of medical treatments out early this year said from The Journal of the American Medical Association, The Lancet, and The New England Journal of Medicine (three of the world\u2019s most prestigious medical journals) only 34% of the treatments that were found to be efficacious in original studies were actually efficacious in replication.\u00a0 You got a one third chance of it working.\u00a0 That\u2019s a good batting average, but not the odds I am looking for from my doctor.<\/span><\/p>\n Where does this lead us.\u00a0 Hell, I don\u2019t know.\u00a0 You are sort of at a physician\u2019s mercy, I guess.\u00a0 Read up when you have something wrong with you.\u00a0 Find the things that have worked and been replicated.\u00a0 When something\u2019s gone awry, try something else.\u00a0 There are some major individual differences in people and remember that when you pick treatments.\u00a0 What works for you may not be what works for other people.\u00a0 Look at studies with a little more caution.\u00a0 Don\u2019t get sold on one technique until you see a ton of replication and definitely check the connection of the researcher to the treatment.\u00a0 And finally keep in mind \u2014 replicate, replicate, replicate.<\/span><\/p>\n So, what did we find with my daughter\u2019s science project?\u00a0 Between the first and second test there was the beginning of deterioration suggesting time on the IPad could be bad news.\u00a0 The scores on the second test were longer in time it took to complete the test and slightly higher in errors.\u00a0 Right before the final test though, pizzas had arrived, so there were tons of errors, but they were definitely much faster completing the tests.\u00a0 So, I would guess a scientist would say the IPad increases mental processing, but you are not as efficient.\u00a0 Or something like that!\u00a0 I suspect someone might get a different response in replication.\u00a0 :}<\/span><\/p>\n \u00a0<\/p>\n
\n<\/span><\/p>\n