University researchers have completed the biggest systematic review and meta-analysis to date of how people’s perceptions of self reported screen time compare with what they do in practice, finding estimates of usage were only accurate in about five percent of research.
The international team say that this casts doubt on the validity of research on the impact of self reported screen time on psychological health, and its influences on government policy, as the huge majority rely on participants to self report screen time how long they spend on electronic devices, as opposed to logs of actual usage, or tracked time.
“For decades, researchers have relied on estimates of how we use various technologies to study how people use digital media and the potential outcomes this behavior can lead to. Our findings suggest that much of this work may be on unstable footing,” said lead researcher Dr Doug Parry at Stellenbosch University.
“The screen time discrepancies highlight that we simply do not know enough yet about the actual effects (both positive and negative) of our media use. Researchers, journalists, members of the public, and crucially policy makers need to question the quality of evidence when they consider research on media uses and effects. We can no longer simply take claims of harmful effects at face value.”
The researchers investigated whether questionnaires and scales addressing ‘debatable’ media use, such as excessive or so-called ‘addictive‘ self reported screen time, were suitable substitutes for logged utilization. They discovered an even smaller association with usage logs for these measures.
Published in Nature Human Behavior, the study identified every existing analysis that compares logged or tracked media usage measures with equal self reported screen time. They screened over 12,000 articles for inclusion and found 47 research that included both types of measures. From here they were able to recognize and extract 106 comparisons, according to 50,000 people, to tackle the question of how closely self-report quotes relate to logs of real use.
“These highly flawed studies are over-inflating the relationships between digital media use and typically negative outcomes, such as mental health symptoms and cognitive impairments, which of course explains the pervading view that smartphones among other technologies are bad for us,” said Dr Brit Davidson from the University of Bath’s School of Management.”
“Media and technology use takes the blame for everything from increases in teenage depression and suicide to higher incidence of Attention Deficit Hyperactivity Disorder (ADHD) and violence. If we want to properly investigate harms, we must first tackle assumptions about screen time and disentangle how people are actually using their phones or other technologies of interest.”
“Importantly, these questionable studies are also being used to influence policy. The UK and Canada both have forms of screen time guidelines based on poorly conducted research, which is clearly worrying and hard to reverse.”
The research group also included Dr Craig Sewall at The University of Pittsburgh; Dr Jacob Fisher in the University of Illinois Urbana-Champaign; Hannah Mieczkowski in Stanford University; and Dr Daniel Quintana at the University of Oslo.
The researchers expect that their research will lead to a change in measurement practices regarding technologies, alongside beginning to fix the story about technology and society. They state that only by better understanding what individuals actually do with their technologies, can we start to really understand the effect of these on society and people.
A systematic overview and meta-analysis of discrepancies between logged and self reported screen time is published by Nature Individual Behaviour.