The importance of Research

Why you should care if your treatment or healthcare professional is evidence based.

The aim for clinical research is to improve healthcare and treatments. It is used to find out if treatments work and if so in what context? For what demographic? For what age group? For how long? Does it have any unwanted side effects? Does it cause more harm? And the list goes on and on!

Here are some examples of treatments that were adopted in medicine without quality research and

ended up being harmful:

• Treatments used in schizophrenia, see reviews by Soare et al., 1996, Thornley & Adams 1998.
  

• Cocaine was marketed to treat depression, sinusitis, lethargy, and many others. Later they found about the harms of this drug and that it likely contributed to several of the issues it claimed to treat.
  

• Fen-phen, a weight loss drug which was approved by the FDA which was later found to cause serious cardiac issues.
  

• Heroin has previously been marketed for children to assist with cold and flu symptoms as a nonaddictive opioid.
  

• Numerous mental health treatments (lobotomy, bloodletting, insulin coma therapy, metrazol therapy).
  

• You may have recently seen the Netflix series ‘painkiller’ around Oxycodone and the Sackler family? This story also has a fascinating book by Patrick Keefe ‘Empire of Pain: The Secret History of the Sackler Dynasty’
  

• Many, many other examples exist, even of treatments still used today. You can find a lot of them on the website https://quackwatch.org and https://www.painscience.com/index-pain-treatments.php
  

You can see why quality research is vital!

We can also not trust a healthcare professional's opinion or experience without supporting evidence. A healthcare professional (and/or your family member, partner, parents, friend, neighbor) may claim that a certain treatment or product is extremely effective. However, without supporting research, the effectiveness can actually be for a number of factors that are not specific to the treatment (Hartman 2009). Some include:

• The placebo effect. This effect can not be underestimated. For example immediate pain relief following different manual therapies can be partially accounted for by the placebo effect: 74%-87% (Ezzatavar et al., 2024). To read more please read our blog on placebo (link to blog).
  

• Natural history. The body has the amazing ability to heal itself (Cuijpers & Cristea 2016). Several conditions, illnesses, signs and symptoms do improve with time and often without treatment (Cuijpers & Cristea 2016). However, healthcare professionals may take credit for this natural history and believe it is due to their treatment.
  

• Regression to the mean. This often applies for more persistent conditions such as persistent pain. People living with persistent pain often experience fluctuations in their pain levels. These people often seek healthcare when their pain is at its worst. The pain may have naturally reduced to their average pain level with time however, because they received treatment at this time, they may attribute this improvement to the treatment (Cuijpers & Cristea 2016).
  

• Confirmation bias, practitioners have spent numerous hours studying and learning about different treatments. Often spending huge money in the process. With all these sacrifices they can be more inclined to see larger results then really occur (Cuijpers & Cristea 2016). To account for this, practitioners should use reliable and valid outcomes measures that are as objective as possible. Healthcare professionals often overestimate their ability to accurately measure things using visual inspection. Some healthcare professionals are very sensitive to cognitive dissonance.
  

• Financial incentives
  

• They don’t know what happened to the people that did not return for a follow up appointment. These are often the people that had either an amazing experience and did not feel the need fora follow up, they might have forgotten their appointment, they may have moved towns, become ill, etc. Another explanation is that they actually had a negative experience, an increase in symptoms, felt invalidated, etc. The healthcare professional would not be aware of this and continue to only see people that have positive experiences. This may make them believe that it is far more effective then it really is.
  

• Without follow ups they don’t know if the improvements continued in the longer term.
  

• They are not using any outcome measures.

Mark Crislip “The three most dangerous words in medicine: in my experience.”

Preston Long “ The fact that patients swear by us does not mean we are actually helping them.

Satisfaction is not the same thing as effectiveness.”

So if it works does it really matter if it is supported by evidence?

• Possibly the biggest problem with undergoing treatment that has no supporting evidence is that informed consent becomes an issue. There is a great debate around this topic (Alfano 2015, Schwartz 2015, Stoessl 2020, Schulman & Richman 2022, Rees 2023, Pallocci et al 2023). In my opinion not disclosing a treatment is most likely due to non-specific factors is not allowing the person to give informed consent. Especially if the practitioner sells the treatment with no supporting high quality evidence as being effective, risk free and being able to “fix” your pain.
  

• If you have not been provided with informed consent and that the treatment is more experimental then evidence based. Would you still be happy to pay money? There is a quote that goes something like "It seems to me that placebo treatments ought to be paid with a placebo payment.”
  

• How do we know the possible side effects from treatments that have not been rigorously studied? Would you undergo a treatment that provides you with short term pain relief however there is a tiny chance you may suffer from a stroke?
  

• The last issue is the false narratives that surround these ineffective treatments continue to promote a biomedical view on pain which is a complex biopsychosocial problem. This again highlights why quality research is important.

So what does good quality research include?

• The new treatment or product being studied is not being studied by the developers or other stakeholders that could benefit from it being successful. Results are shown to be significantly more favorable when the authors have an invested interest (Munder et al., 2011)(Munder et al.,2012).
  

• We need to account for the placebo effect. Placebo effects are present in most treatments, to see if a treatment is more effective than the placebo effect there should be a separate group that undergoes a sham treatment.
  

• Randomisation, if participants are not randomized to different groups there may be a large difference in baseline and features between the groups participants. Researchers may be inclined to choose participants that may respond better to the treatment. When randomisation is concealed, effect size appears to be more modest (Cuijpers et al., 2010).
  

• Blinding of researchers to what group the participant (sham vs real) is assigned to is also important. When researchers are not blinded they may treat participants differently to enhance results (Cuijpers et al., 2010).
  

• The number of participants pre and post trial are reported. An explanation should be provided for any missing data or drop-outs (Cuijpers & Cristea 2016).
  

• It includes outcome measures that are validated and reliable in measuring what the study claims to measure.
  

• A large enough group of participants. Small samples may not give a true representation of the population studied. Cruijpers & Cristea (2016) provide a great example by comparing it to a coin toss. In a coin toss there is 50% chance of getting tails and 50% chance of getting heads. If you toss it 100 times there is a higher chance of getting 50/50 vs if you toss it a 10 times. Treatments should be considered useless until proven effective - Paul Ingraham from https://www.painscience.com.

References

1. Alfano M. (2015). Placebo Effects and Informed Consent. The American journal of bioethics : AJOB, 15(10), 3–12. https://doi.org/10.1080/15265161.2015.1074302

2. Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size. Psychological medicine, 40(2), 211–223. https://doi.org/10.1017/S0033291709006114

3. Cuijpers, P., & Cristea, I. A. (2016). How to prove that your therapy is effective, even when it is not: a guideline. Epidemiology and psychiatric sciences, 25(5), 428–435. https://doi.org/10.1017/S2045796015000864

4. Ezzatvar, Y., Dueñas, L., Balasch-Bernat, M., Lluch-Girbés, E., & Rossettini, G. (2024). Which portion of physiotherapy treatments' effect is attributable to contextual effects in people with musculoskeletal pain?: A meta-analysis of randomised placebo-controlled trials. The Journal of orthopaedic and sports physical therapy, 1–28. Advance online publication. https://doi.org/10.2519/jospt.2024.12126

5. Hartman S. E. (2009). Why do ineffective treatments seem helpful? A brief review. Chiropractic & osteopathy, 17, 10. https://doi.org/10.1186/1746-1340-17-10

6. Michal, A. L., Zhong, Y., & Shah, P. (2021). When and why do people act on flawed science? Effects of anecdotes and prior beliefs on evidence-based decision-making. Cognitive research: principles and implications, 6(1), 28. https://doi.org/10.1186/s41235-021-00293-2

7. Munder, T., Gerger, H., Trelle, S., & Barth, J. (2011). Testing the allegiance bias hypothesis: a meta-analysis. Psychotherapy research : journal of the Society for Psychotherapy Research, 21(6), 670–684. https://doi.org/10.1080/10503307.2011.602752

8. Munder, T., Flückiger, C., Gerger, H., Wampold, B. E., & Barth, J. (2012). Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis. Journal of counseling psychology, 59(4), 631–637. https://doi.org/10.1037/a0029571

9. Pallocci, M., Treglia, M., Passalacqua, P., Tittarelli, R., Zanovello, C., De Luca, L., Caparrelli, V., De Luna, V., Cisterna, A. M., Quintavalle, G., & Marsella, L. T. (2023). Informed Consent: Legal Obligation or Cornerstone of the Care Relationship?. International journal of environmental research and public health, 20(3), 2118. https://doi.org/10.3390/ijerph20032118

10. Rees M. (2023). Patient autonomy and withholding information. Bioethics, 37(3), 256–264. https://doi.org/10.1111/bioe.131304

11. Schulman, K., & Richman, B. (2022). Informed Consent as a Means of Acknowledging and Avoiding Financial Toxicity as Iatrogenic Harm. AMA journal of ethics, 24(11), E1063–E1068. https://doi.org/10.1001/amajethics.2022.1063

12. Schwartz P. H. (2015). Placebos, Full Disclosure, and Trust: The Risks and Benefits of Disclosing Risks and Benefits. The American journal of bioethics : AJOB, 15(10), 13–14. https://doi.org/10.1080/15265161.2015.1074315

13. Soares, K., McGrath, J., & Adams, C. (1996). Evidence and tardive dyskinesia. Lancet (London, England), 347(9016), 1696–1697. https://doi.org/10.1016/s0140-6736(96)91525-1

14. Stoessl A. J. (2020). Deception and the ethics of placebo. International review of neurobiology, 153, 147–163. https://doi.org/10.1016/bs.irn.2020.03.030

15. Thornley, B., & Adams, C. (1998). Content and quality of 2000 controlled trials in schizophrenia over 50 years. BMJ (Clinical research ed.), 317(7167), 1181–1184. https://doi.org/10.1136/bmj.317.7167.1181

This blog was written by Samuel Bulten