Self-Reported Adherence: 95% Say Yes, 38% Real

In the FEM-PrEP trial, women reported taking their pills on at least 6 of the previous 7 days at 94.7% of visits. When investigators checked plasma for tenofovir, that near-perfect report matched detectable drug only 38% of the time. Same women, same visits, two numbers that describe different realities.

Participants overreport their dosing, and the error runs in one direction. The inflated number reads like good news until the efficacy analysis comes back null, at which point the cost has already been paid: diluted treatment effect, an underpowered study, and an endpoint you cannot defend.

The 95/38 gap, unpacked

FEM-PrEP tested oral tenofovir-based PrEP in women at risk of HIV. The trial collected adherence three ways, scored as four measures, and compared each against drug found in the body. The analysis by Agot and colleagues scored every measure on positive predictive value: when a participant looked adherent by that method, how often was drug actually present?

Adherence measure	Positive predictive value
Self-report: took pills 6+ of past 7 days	38.0%
Self-report: took pills 1+ of past 7 days	42.2%
Pill count: 1 or fewer days missed	26.2%
Self-report: “usually/always” in past 4 weeks	28.7%

The drug criterion differs by row: the two 7-day self-report measures use plasma tenofovir at different thresholds, while the pill-count and 4-week measures use a combined plasma plus intracellular (tenofovir-diphosphate) standard.

All four participant-dependent measures landed under 45% positive predictive value against objective drug levels. The pill count, which many teams trust more than a diary because it involves counting physical tablets, scored worst at 26.2%. Read that PPV as a bet: when a FEM-PrEP participant’s pill count said she was adherent, drug turned up in her body about one time in four. That is the ceiling on how much you can trust a returned diary or a bottle handed back at a site visit.

One caveat the number deserves. PPV depends on the underlying prevalence, and FEM-PrEP was an extreme case: true adherence in that population ran near 36 to 41%. Run the same diary in a population that actually takes its pills and the PPV climbs. So treat 38% as proof that self-report collapses exactly where adherence is worst, which is where you most need to know, rather than as a constant you can apply to any protocol. The authors add a second caveat that cuts against their own measure: because plasma tenofovir has a short half-life, a participant who swallowed one pill shortly before her clinic visit could show concentrations resembling someone who dosed faithfully all month.

Why it is not just PrEP

You might read FEM-PrEP as a story about one hard-to-study population in one HIV trial. The upward bias shows up elsewhere. A meta-analysis of MEMS electronic monitoring versus self-report pooled 11 studies covering 1,684 patients across therapeutic areas. Self-report averaged 84.0% adherence; electronic monitoring of the same patients averaged 74.9%, a 9-point gap in the same direction FEM-PrEP found. The two methods correlated at r=0.45.

Read those authors honestly, because they land somewhere gentler than we do: they call the correlation “at least moderate” and conclude that questionnaires “give a good estimate of medication adherence.” They also concede that “neither the MEMs nor SRQs can replace each other.” Take the concession seriously. A 9-point average gap tolerable for a chronic-care clinic is not the same thing as a 9-point gap in the arm of a registrational trial, where the effect size you are trying to detect may itself be smaller than the measurement error. Your protocol carries a burden of proof a clinic does not.

Coordinators are not to blame for this. Asking people to report their own behavior produces upward bias as a structural property of the question. The mechanics of why pill counts drift the same way, from dumping to timing blindness, are covered in our companion post on why pill counts overstate adherence.

The hidden cost: soft adherence data blows your statistical power

Here is where the measurement problem turns into a money and power problem. When participants in the treatment arm skip doses, the treated group behaves more like the control group. The observed difference between arms shrinks. Penn State’s STAT 509 course states the mechanism directly: nonadherence dilutes the treatment effect and lowers study power.

Statisticians correct for this by inflating the planned sample size. The adjustment for drop-out and drop-in is:

N* = N / (1 - R_O - R_I)²

The squared denominator matters. With a 10% drop-out rate and a 20% drop-in rate, the required sample size roughly doubles. Nonadherence does not add cost linearly. It compounds. And that formula only helps if you know your true nonadherence rate. If your adherence data is self-reported, you are feeding the correction a number that is inflated by roughly the FEM-PrEP margin, so you under-inflate the sample size and enroll a trial that is quietly under-powered from the first patient.

Put numbers on it

A systematic evaluation of adherence patterns on sample size and power modeled what happens as adherence falls from 100% to 50% across drug profiles. For a long half-life drug with rapid onset, the required sample size doubled from 100 to 202 and power fell from 90% to 63%. For a long half-life drug with delayed onset, the sample size rose 3.5-fold from 105 to 351 and power collapsed from 88% to 41%.

The worst case is a short half-life drug. The same paper concludes that for drugs with short half-lives, nonadherence patterns can dilute the evidence for efficacy until it is “indistinguishable from the response to placebo.” A drug that works can read as a drug that does not, because of dosing nobody measured. You pay tens of millions of dollars to manufacture that type II error, and the trial gives you no signal that it happened.

These are simulations rather than observed trials, so read them as the shape of the risk instead of a forecast for your protocol. The direction holds regardless: the less you know about real dosing, the more of your power you are spending blind.

The part that should scare sponsors

You can recover from the budget hit by enrolling more patients. The endpoint hit has no such remedy, and self-report will not warn you it is coming. A study of 134 MEMS-monitored heart-failure patients tracked whether adherence predicted event-free survival, meaning who stayed alive and out of the hospital. Two of the three objective measures predicted it: dose-count P=.004 and dose-day P=.008, while dose-time did not (P=.224). Self-reported adherence from the same patients predicted nothing at all: P=.402.

Sit with the number underneath that result. Among the 21 patients whose measured adherence fell below 60%, 76.2% reported taking their medication most or all of the time. The patients at highest risk were the ones whose self-report looked cleanest. If a diary cannot sort survival in an observational cohort where the stakes are hospitalization and death, it will not sort dose response in your trial.

The dose data you cannot verify is the dose data most likely to be wrong, and it biases the analysis toward hiding a real effect. Nonadherence concentrates among the participants who report best, so a larger sample does not average the error away. It bends the treatment arm toward the control arm where you have the least visibility, and you learn about it at unblinding, when the number is already fixed.

What objective dosing evidence actually looks like

The fix is to replace a claim with an observation. Four options do that, at different cost and rigor:

• Drug-level assays (PK). Blood or urine testing proves the drug was in the body. Strong evidence for a single moment, blind between draws, and defeatable by dosing right before a scheduled visit.
• Electronic monitoring (MEMS). Smart caps timestamp each bottle opening. Good timeliness, but an opening is not a swallow, and dedicated participants can curate openings.
• Directly observed therapy (DOT). A clinician watches the dose. Strong evidence, expensive, and hard to scale beyond a clinic visit.
• Video-verified dosing. The participant records a short timestamped clip of the ingestion, which software verifies. It captures the swallow itself and writes it into an audit-grade record.

The behavioral levers that raise the rate you then measure are a separate workstream, covered in our guide on strategies to improve trial adherence. On the measurement side, AI video dose verification paired with tamper-evident, hash-chained dosing records gives you objective per-dose evidence without a site-visit bottleneck or a clinician watching every dose live. It approximates DOT at the scale of a decentralized trial. It does not replace PK for biochemical confirmation, and it depends on capture quality. It does close the ingestion-evidence gap that diaries, pill counts, and even MEMS leave wide open.

Bottom line for clin-ops

Treat objective dosing evidence as endpoint insurance. If your adherence data is self-reported, assume it runs high, assume your drop-in and drop-out corrections inherited that inflation, and assume your study has less power than the protocol claims. How much less depends on your population, and you cannot know without measuring. That is the whole problem.

The heart-failure data says self-report will stay quiet until the endpoint reads null. The power modeling says a working drug can look like placebo when unmeasured dosing runs against you. Decide up front which adherence evidence you would put in front of a regulator, then capture that evidence per dose. Objective participant compliance monitoring exists to make that the default.

---

Want to see video-verified dosing and the tamper-evident audit trail on your own protocol? See video-verified dosing in a demo.