P values and NNTs: What they are and their roles in critical assessment

Richard B. Evans, PhD

Editor-in-Chief, Royal College of Veterinary Surgeons (RCVS) Knowledge Veterinary Evidence; University of Missouri

Proceedings Paper (PDF) Slides (PDF) Screencast (video)
 
Download proceedings paper (PDF) Download slide deck (PDF) Stream presentation video
 

Please note: Some proceedings papers or presentation slide decks were not made available by the speaker. Also, selected screencast recordings from the webinar may be viewed only by active EBVMA members after logging into their member portal.

Abstract

The elevator pitch for a canine orthopedics research paper might go something like this: “We did a blinded RCT on 100 dogs comparing TTA to Tightrope and got statistical significance on PVF.”

That single-sentence summary reports two design features, the study design and sample size, and it gives two results, the statistical significance on an outcome variable. Understanding statistical significance (P values) and making outcomes meaningful (NNTs) are the topics of this talk.

P values, which are used to identify real results, are ubiquitous in veterinary research, but their basic interpretation (especially P>0.05) is often misunderstood and incorrectly applied. Moreover, researchers often P hack their results, teasing out a few P<0.05 among a forest of P>0.05, giving the impression that results are real when they probably aren’t.

It’s the job of critical appraisers, and a central part of this talk, to understand what P values mean in the larger context of determining if the results in an article are real or due to chance.

P values are one side of the outcomes coin. The other side is the clinical importance of the results. For example, are group differences large enough to effect changes in practice patterns among clinicians? Often results are described in terms of means of outcome variable: the mean peak vertical force, the mean body condition score, and so on. But is a change of 0.5 in BCS meaningful? That depends.

Numbers needed to treat (NNTs) are way of summarizing outcomes in a way that is clinically useful. I’ll describe NNTs and give examples when they are needed and how to calculate them.

Keywords

evidence-based veterinary medicine (EBVM), Critically Appraised Topic (CAT), p-value, Number Needed to Treat/Harm (NNT/NNH), critical assessment


All content from "Best Evidence, Better Care", the 7th EBVMA Symposium held Friday, 11 November 2015 is © 2015 by the EBVMA (publication) and the author (content) under a Creative Commons Attribution, Non-Commercial, Share-alike, 4.0, International License. Some content may be restricted to active EBVMA members. See our membership page to join or renew or send us an email to membership@ebvma.org. If you're interested in speaking or participating in our AAVSB/RACE accredited continuing education programs, contact us at conference@ebvma.org.

Series Navigation<< Bias in Evidence-based MedicineVeterinary All Trials Initiative (VetAllTrials) >>