1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 20th November 2023 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Flawed data led to findings of a connection between time spent on devices and mental health problems, Sewall, 2021

Discussion in 'Other health news and research' started by cassava7, Aug 10, 2021.

  1. cassava7

    cassava7 Senior Member (Voting Rights)


    Craig Sewall is a Postdoctoral Scholar of Child and Adolescent Mental Health at the University of Pittsburgh.

    He explains that studies on the association between mental health outcomes and time spent on digital devices suffered from their reliance on inaccurate self-report measures.

    Absent from the discussion about the putative harms of digital tech is the fact that practically all academic studies in this area have used highly flawed self-report measures. These measures typically ask people to give their best guesses about how often they used digital technologies over the past week or month or even year. The problem is that people are terrible at estimating their digital technology use, and there’s evidence that people who are psychologically distressed are even worse at it. This is understandable because it’s very hard to pay attention to and accurately recall something that you do frequently and habitually.

    Researchers have recently begun to expose the discrepancy between self-reported and actual technology use, including for Facebook, smartphones and the internet. My colleagues and I carried out a systematic review and meta-analysis of discrepancies between actual and self-reported digital media use and found that self-reported use is rarely an accurate reflection of actual use.​

    Sewall worked on re-assessing these outcomes using objective measures of time spent on devices (automatic on-device time tracking) and found no association between them.

    In my own research as a doctoral student in social work, I found that the link between digital technology use and mental health was stronger when self-report measures were used than when objective measures were used. An example of an objective measure is Apple’s “Screen Time” application, which automatically tracks device use. And when I used these objective measures to track digital technology use among young adults over time, I found that increased use was not associated with increased depression, anxiety or suicidal thoughts. In fact, those who used their smartphones more frequently reported lower levels of depression and anxiety.
    He calls for using reliable measures in scientific research, noting how self-report measures are widespread in his field of research and beyond.

    This has enormous implications. Although measurement isn’t a sexy topic, it forms the foundation of scientific research. Simply put, to make conclusions – and subsequent recommendations – about something you’re studying, you must ensure you’re measuring the thing you’re intending to measure. If your measures are defective, then your data is untrustworthy. And if the measures are more inaccurate for certain people – like young people or those with depression – then the data is even more untrustworthy. This is the case for the majority of research into the effects of technology use over the past 15 years.

    Imagine that everything known about the COVID-19 pandemic was based on people giving their best guesses about whether they have the virus, instead of highly reliable medical tests. Now imagine that people who actually have the virus are more likely to misdiagnose themselves. The consequences of relying on this unreliable measure would be far-reaching. The health effects of the virus, how it’s spreading, how to combat it – practically every bit of information gathered about the virus would be tainted. And the resources expended based on this flawed information would be largely wasted.

    The uncomfortable truth is that shoddy measurement, as well as other methodological issues including inconsistent ways of conceiving of different types of digital tech use and research design that falls short of establishing a causal connection, is widespread. This means that the putative link between digital technology and psychological distress remains inconclusive.​
    Last edited: Aug 10, 2021
  2. Sean

    Sean Moderator Staff Member

    Louie41, Hutan, alktipping and 5 others like this.
  3. Remain in Light

    Remain in Light Established Member (Voting Rights)

    As an experienced parent of teenagers I know what is right for them and this kind of objective meddling (probably funded by the games industry) threatens to undermine my authority and the family unit;)
    Louie41, Hutan, Michelle and 3 others like this.
  4. rvallee

    rvallee Senior Member (Voting Rights)

    Science without reliable measurements is just clowning around until the real work begins, basically Mythbusters. Whatever it is, it's not science until then and basically indistinguishable from politics.

    First learn how to measure. Then learn. You can't skip the first step, we know for a fact it doesn't work. Can people working in scientific professions get this now? It's not just some optional nice to have where judgment substitutes perfectly. It literally doesn't.
    Louie41, Hutan, alktipping and 3 others like this.
  5. Mithriel

    Mithriel Senior Member (Voting Rights)

    Actually, Mythbusters was all about measurement and put the likes of the PACE trial to shame! BPS research is more at the Sesame Street level :(

    Using proper measurements just does not seem to stick probably because it is cheaper to do things badly and you can get away with shoddy research in these days of universities valuing quantity of research papers over quality.

    Food studies have to be done in special settings where food is weighed before a meal and then afterwards not by asking people what they ate last week.

    They always said that a shower used less water than a bath until they fit meters to measure how long people actually spent in the shower. it turns out most people spend about 8 minutes but round this down to 5 when asked.
    EzzieD, Wyva, Louie41 and 4 others like this.

Share This Page