Flawed data led to findings of a connection between time spent on devices and mental health problems, Sewall, 2021

https://theconversation.com/flawed-...nd-mental-health-problems-new-research-162585

Craig Sewall is a Postdoctoral Scholar of Child and Adolescent Mental Health at the University of Pittsburgh.

He explains that studies on the association between mental health outcomes and time spent on digital devices suffered from their reliance on inaccurate self-report measures.

Absent from the discussion about the putative harms of digital tech is the fact that practically all academic studies in this area have used highly flawed self-report measures. These measures typically ask people to give their best guesses about how often they used digital technologies over the past week or month or even year. The problem is that people are terrible at estimating their digital technology use, and there’s evidence that people who are psychologically distressed are even worse at it. This is understandable because it’s very hard to pay attention to and accurately recall something that you do frequently and habitually.

Researchers have recently begun to expose the discrepancy between self-reported and actual technology use, including for Facebook, smartphones and the internet. My colleagues and I carried out a systematic review and meta-analysis of discrepancies between actual and self-reported digital media use and found that self-reported use is rarely an accurate reflection of actual use.​

Sewall worked on re-assessing these outcomes using objective measures of time spent on devices (automatic on-device time tracking) and found no association between them.

In my own research as a doctoral student in social work, I found that the link between digital technology use and mental health was stronger when self-report measures were used than when objective measures were used. An example of an objective measure is Apple’s “Screen Time” application, which automatically tracks device use. And when I used these objective measures to track digital technology use among young adults over time, I found that increased use was not associated with increased depression, anxiety or suicidal thoughts. In fact, those who used their smartphones more frequently reported lower levels of depression and anxiety.
​

He calls for using reliable measures in scientific research, noting how self-report measures are widespread in his field of research and beyond.

This has enormous implications. Although measurement isn’t a sexy topic, it forms the foundation of scientific research. Simply put, to make conclusions – and subsequent recommendations – about something you’re studying, you must ensure you’re measuring the thing you’re intending to measure. If your measures are defective, then your data is untrustworthy. And if the measures are more inaccurate for certain people – like young people or those with depression – then the data is even more untrustworthy. This is the case for the majority of research into the effects of technology use over the past 15 years.

Imagine that everything known about the COVID-19 pandemic was based on people giving their best guesses about whether they have the virus, instead of highly reliable medical tests. Now imagine that people who actually have the virus are more likely to misdiagnose themselves. The consequences of relying on this unreliable measure would be far-reaching. The health effects of the virus, how it’s spreading, how to combat it – practically every bit of information gathered about the virus would be tainted. And the resources expended based on this flawed information would be largely wasted.

The uncomfortable truth is that shoddy measurement, as well as other methodological issues including inconsistent ways of conceiving of different types of digital tech use and research design that falls short of establishing a causal connection, is widespread. This means that the putative link between digital technology and psychological distress remains inconclusive.​

 

As an experienced parent of teenagers I know what is right for them and this kind of objective meddling (probably funded by the games industry) threatens to undermine my authority and the family unit

 

Science without reliable measurements is just clowning around until the real work begins, basically Mythbusters. Whatever it is, it's not science until then and basically indistinguishable from politics.

First learn how to measure. Then learn. You can't skip the first step, we know for a fact it doesn't work. Can people working in scientific professions get this now? It's not just some optional nice to have where judgment substitutes perfectly. It literally doesn't.

 

Actually, Mythbusters was all about measurement and put the likes of the PACE trial to shame! BPS research is more at the Sesame Street level

Using proper measurements just does not seem to stick probably because it is cheaper to do things badly and you can get away with shoddy research in these days of universities valuing quantity of research papers over quality.

Food studies have to be done in special settings where food is weighed before a meal and then afterwards not by asking people what they ate last week.

They always said that a shower used less water than a bath until they fit meters to measure how long people actually spent in the shower. it turns out most people spend about 8 minutes but round this down to 5 when asked.