Learning from ‘failed’ research through new journal
Health psychologists can now also publish their ‘failed’ research in a new online journal: Health Psychology Bulletin. The condition is that they provide complete open access to their data, analyses and choices. According to founders Gerjo Kok, professor of Applied Psychology at Maastricht University and Gjalt-Jorn Peters, lecturer in Methodology and Statistics at the Open University, in-depth research will be rewarded and not, as it happens now, sensational findings. “With the competition model in science, you can hardly afford to find nothing.”
According to editors in chief Gerjo Kok and Gjalt-Jorn Peters, the cases of fraud–though not only in psychology–forced them to face the facts. They point out that a lot of research cannot be repeated successfully, which seems to suggest that much more published research falls short. Everyone also knows the cause—the system of publish or perish. Kok: “You can hardly afford to find nothing.” Peters: “Research that produces no outcomes is flawed. Research that produces clear outcomes makes everyone so happy that people forget about the details. That’s how they think at journals, too.”
Aggressive meat eaters
Is this a typical problem for psychology? Both psychologists differ in opinion on this. Kok does not think so and sees the same thing in medicine, for example. Peters believes it is a pitfall for every researcher. “But psychology is more vulnerable because it’s about people and is more susceptible to sensationalism.” He gives an example of a ‘fancy’ finding in social psychology that meat lovers should be more aggressive. How it should be, he believes can be seen in nanotechnology. “There, they publish everything. If you don’t get results in a study, you just write exactly what was done.”
The new online journal Health Psychology Bulletin (HPB) of Kok and Peters is a response to this. It has been established to allow the publication of research that does not produce results or has been executed poorly. Kok: “In fact, we allow everything under the condition that it’s clearly stated how the research was conducted. The requirement is therefore that full disclosure is given regarding all data, materials and analyses. That’s also the real mistake of poor research. Openness allows research to be repeated, which can detect errors, and it’s a bit harder to manipulate data. This way, we want to stimulate more transparency and integrity.”
Collaboration model
The significance of this is great. Kok: “It’s substantially important to the discipline, because for years we’ve believed in something that’s not right. Applications arise from a wrong theory. Bad science is bad for humanity.” But there is also a methodological importance. Publishing faulty research leads to improvement. A common mistake, for example, is that conclusions are drawn based on an insufficient number of subjects. Peters: “You should always make an analysis beforehand of the minimum number of subjects that are needed. This often happens only rudimentarily.” That is why the HPB forms a platform for discussion about what has gone wrong with research, what else could have been done and what lessons can be learned from it.
The HPB is also an initial step in shifting from the prevailing competition model in science to a research model that focuses on collaboration. Peters: “Scientists now conduct research from their own island and see each other as competitors. But for science, it’s much more efficient to work together.” Such openness should also have a different reward system. Peters: “Instead of the impact factor of journals, you could also reward the disclosure of data sets by a researcher. Now, the system rewards sensational findings and not in-depth research.”
Citation scores
Both psychologists realise that the competition model is almost impossible to change. Peters: “Everyone has been indoctrinated.” Kok: “It’s a catch-22. Everyone knows that the current system is flawed, but during the next visitation we’ll just be back to looking at citation scores. The citations will then again be rewarded by the universities. Those who score high, get more resources. People also want to make a career.” In any case, their initiative could not have been possible five years ago. “The awareness is there, and in ten years our approach will be normal.”
One improvement would be if the power of the professional journals is reduced. The founders of the HPB advocate for a worldwide network of cooperating universities that makes it possible to have a platform of publications. Kok: “That’s also necessary; there’s no more keeping up. In the past, I read a journal on the train and was caught up again. That’s now unimaginable. But here, too, you can see the duality. If I have a hit in a nice journal, I'll broadcast it around.”
Stone in the pond
“In our bulletin, an article will almost never be rejected”, emphasises Kok, “if they just clearly write down what you can and can’t conclude. Errors aren’t bad if you’re open about them. And if people don’t want to participate, let them explain why. There’s no harm in throwing a stone in the pond to see the ripple effect.” Peters: “If our bulletin becomes a wasteland of failed research, that's fine. Our goal is to learn from it. That’s what we’re looking for, how it can be useful, with an emphasis not only on the outcomes but also on what you can learn from the processes.”
Gerjo Kok (1948) is professor of Psychology at Maastricht University. His research focuses on applying psychological theories on behavioural change to reduce social problems. He also engages in the development of models of behavioural change for health promotion and disease prevention, energy saving, traffic safety and discrimination.
Gjalt-Jorn Peters (1981) is university lecturer in Methodology and Statistics at the Open University. His research focuses on methodology, statistics, behavioural change and health issues related to nightlife.
Also read
-
FPN’s prof. dr. Rainer Goebel has been awarded an ERC Advanced Grant of € 2,5M for his research project Reading the Mind’s Eye: AI inspired personalised brain models of mental imagery. Goebel is among 255 researchers (out of 1829 applications in all domains) in Europe to receive the grant, and he is...
-
When nitrous oxide (laughing gas) is used recreationally, its presence remains detectable in the breath and bloodstream for at least 60 minutes after inhalation, and the development of an instrument to measure it is technically feasible. These were among the findings of a study at Maastricht...
-
What happens in the brain of someone with a psychiatric disorder? Eleonora Broggi, an alumna of the bachelor Biomedical Sciences at Maastricht University, is investigating brain patterns in people with autism spectrum disorder at King’s College London. Read how Eleonora uses Biomedical Sciences for...
