An illustration of a doctor holding a clipboard and a patient who is pulling apart a curtain on the doctors back to see...
Illustration by Bénédicte Muller
When we peer into our patient portals, we don’t always see ourselves more clearly.

One afternoon not long ago, I sat entering notes into a patient’s medical record. She was in her forties, and her labs showed anemia. The causes of anemia range from menstruation to cancer, and so pinpointing the correct underlying diagnosis is critical. Physicians are trained to formulate a full roster of possibilities, known as the differential diagnosis, and then to work down the list systematically. We’re taught to cast a wide net—celiac disease, parasitic infections, thalassemia, lead poisoning, liver disease, B12 deficiency, myeloma, sickle-cell disease, G6PD deficiency—because you’ll never make a diagnosis if you haven’t included it in your differential.

But I hesitated before entering my differential into the computer system. Should I include the more serious possibilities, even though they were much less likely? In the past, I wouldn’t have thought twice about it, as the chart served primarily as a tool for the medical team to communicate among ourselves. But a new law, the 21st Century Cures Act, had recently been fully implemented, making medical records open to patients by default, in real time, including doctors’ notes. My in-box was already jammed with panicked messages from people convinced that they had catastrophic illnesses, based on minuscule lab discrepancies and panic-inducing Google searches. How would my patient react to seeing my ruminations about possible colon cancer or duodenal ulcer in the note?

One of the earliest known medical records—an account of wound treatment—was written on papyrus in Egypt, around 1600 B.C. Its intended audience seems to have been fellow-healers, who could learn from it. More than a thousand years later, Hippocrates helped develop the concept of the medical-case history, which was likely geared toward educating and informing other practitioners. This contributed to a physician-centric view of the medical record, which has endured. Into the nineteen-seventies, patients in most U.S. states could obtain their medical records only with the help of a lawyer. In 2000, the hipaa Privacy Rule—which clarified how the health-care industry should implement the Health Insurance Portability and Accountability Act of 1996—required organizations to disclose most kinds of medical information to patients upon request. But health-care organizations were under no obligation to eliminate the paperwork, fees, or surly clerks that stood between patients and their records.

As medical records inched from paper to computer during the nineteen-nineties, a few institutions began experimenting with patient portals—Web sites for patients to access their records. These sites, which began appearing more widely in the two-thousands, were clunky to use, and there wasn’t much in there beyond appointments and some test results. But, in 2009, as part of its attempt to drag the U.S. economy out of recession, the federal government offered incentive payments to medical organizations that could demonstrate “meaningful use” of electronic medical records. Requirements for the funding came to include the ability for patients to “view, download, and transmit” their medical information. With money now on the table, patient portals proliferated.

In 2010, Jan Walker and Tom Delbanco, who have backgrounds in nursing and internal medicine, respectively, founded a research collaborative called OpenNotes, based at Beth Israel Deaconess Medical Center in Boston. This initiative went a step further and experimented with allowing patients to read the actual notes that their doctors were writing. Patients in the initial studies were overwhelmingly enthusiastic about the idea. For physicians, this was a radical change: lab values and radiology reports were neutral information, but notes felt personal. The Cures Act went into full effect in the spring of 2021, granting patients access to their records by default, so that they didn’t have to request it. In the brief lull between covid-19’s soul-crushing winter and the start of the deadly Delta wave, the medical profession abruptly found itself staring into a two-way mirror.

Transparency has always been seen as a hallmark of honesty and integrity. The logic is familiar: democracies aspire to be transparent, but dictatorships are opaque; faithful spouses are guileless, while philanderers lie; reputable businesses operate in the open, but shady operations literally draw the shades. Collectively, we’ve embraced Louis Brandeis’s dictum that sunlight is the best disinfectant. We demand transparency in government, charitable institutions, nutrition labels, and middle-school grading rubrics. The medical record should be no different.

And yet, in writing the medical note for my patient with anemia, I felt an awkwardness creeping in. There was something disquieting about knowing that my every word might be scrutinized. The academic purist in me pressed for a thorough differential and a full accounting of my thought process—to avoid error, but also to accurately inform the next physician about the current state of affairs. I worried, however, about downstream angst, both for me and for the patient. She had been transparent with me, sharing her story, her body, her blood chemistry. She’d answered personal questions about her medical history, sexual partners, drug and alcohol use, and dietary habits. Still, I hesitated when it came time for me to be fully transparent with her.

In the end, I dashed off something like “anemia eval” —a pathetic skeleton of a note that I wouldn’t have tolerated from a medical student—and tore off the exam-table paper in preparation for the next patient. I felt rushed, annoyed, and frankly disgusted at my dumbing down of the chart. And I couldn’t tamp a rising coil of unease. Why had a system designed to encourage transparency resulted in my drawing the curtain?

In 2008, Ethan Bernstein, an associate professor at Harvard Business School, travelled with a team of students to a vast mobile-phone-manufacturing plant in southern China. Managers scanned stadium-sized production floors, surveying entire assembly lines at a glance. Ubiquitous digital monitors provided real-time data for all to see. A rainbow palette of hats and uniforms instantly pegged the position and rank of fourteen thousand employees, among whom Bernstein’s undergrads were discreetly embedded.

The students, who’d been born in China, lived, worked, ate, and socialized with their colleagues. They found that, as in any high-pressure workplace, employees were constantly fashioning more agile strategies for getting the job done, often bypassing procedures handed down by bureaucrats from on high. What amazed Bernstein was the lengths to which workers went to conceal their innovations. One told him, “It’s most efficient to hide it now and discuss it later.” Ironically, the plant’s transparency enabled this subterfuge: the colored hats that allowed management to identify each worker’s position also let workers spot managers from across the floor, giving them time to camouflage their improvisations. As the managers made their rounds, they became loci of decreased productivity.

Bernstein’s team hung a curtain in the factory, preparing to conduct an experiment that involved observing assembly lines on one side, and then comparing them with the control lines on the other. But when a worker offhandedly mused about how much better it would be if the curtain completely closed them in on all sides, Bernstein went with her idea, instead. In just a week, the mere shielding of these assembly lines increased productivity by as much as ten to fifteen per cent—a change that persisted for the five months that Bernstein was able to continue observing. Instead of hiding their various tweaks and improvements, the workers behind the curtains could share them with their colleagues without fear of drawing unwanted attention; they even sometimes cross-trained one another, making it easier for people who worked different positions to cover for one another. “When you get watched, you lose autonomy,” Bernstein told me. The result is what he calls the transparency paradox: well-intentioned measures meant to increase transparency and promote productivity, creativity, and compliance often accomplish the opposite. Observation is not a neutral process; human beings react.

Transparency in government, to the extent that it exists, has been hard-won. For most of its history, Congress could be witnessed only by the few members of the public lucky or connected enough to snag seats in the gallery. From time to time, radio broadcasts captured Congress at work; in 1947, a single live-television broadcast of the opening session of the House of Representatives served as the first salvo in the fight for fuller public access. It was followed by spotty televised coverage of the occasional Presidential speech, and then by an outright ban on television cameras in Congressional sessions and committee hearings, in 1952. In the Watergate-chastened nineteen-seventies, calls for transparency led to a brief test of closed-circuit television, in 1977.

That year, a journalist named Brian Lamb submitted a proposal to the governing body of the newly emerging cable-television industry suggesting the creation of a public-affairs channel to show gavel-to-gavel coverage of congressional proceedings. The Cable-Satellite Public Affairs Network (C-span) launched in 1979. It touted itself as “your unfiltered view of government”; the comedian Elayne Boosler has referred to it as “Wild Kingdom but with congressmen.” C-span was hailed as a win for transparency and accountability. Representative Al Gore, who was the very first lawmaker to appear on C-span, argued that it offered “a solution for the lack of confidence in government.” Because it does not filter or edit coverage, C-span is generally considered unbiased—a transparent view.

Along with other lawmakers, Newt Gingrich—who arrived in Congress shortly after C-span did—quickly realized the potential of the medium. Members of Congress delivered more pugnacious speeches at the end of the day, which appeared to stun their opponents into silence—sometimes because nearly everyone had gone home, a fact that the cameras didn’t reveal. (Later, cameras started panning the room.) The researchers Gloria Gennaro and Elliott Ash have examined the so-called C-span effect, demonstrating a correlation between the “emotionality” of lawmakers’ rhetoric and the levels of C-span viewership in their districts. They also found that exposure on C-span has no measurable effect on legislative efforts—perhaps because such efforts often require detailed deal-making and uncomfortable trade-offs that are easier managed in quiet backrooms. C-span, therefore, might not be showing us lawmakers in the wild so much as encouraging them to grandstand to the camera in ways that can be antithetical to the actual work of governing. (By contrast, increased coverage by local news does appear to correlate with legislative progress, though without impact on the emotionality of lawmakers’ oratory.)

These counterintuitive effects sit awkwardly alongside our valorization of transparency. David Pozen, a soft-spoken law professor at Columbia University, told me point-blank that “transparency has a proven track record of failing to deliver on its ostensible benefits.” Open-file discovery, for example, is a transparency effort intended to level the playing field during litigation; it requires prosecutors to share the results of their investigations with defense teams, so that defendants can make more informed decisions about plea bargains. These disclosures also protect defendants from being surprised during trial by witnesses or evidence for which they haven’t prepared. But analyses have found that these laws haven’t always resulted in meaningful improvements for defendants. This might be because most defendants are poor, and their overworked, underpaid public defenders can’t effectively utilize the additional information. But police and prosecutors may also change their behavior once they know that whatever they uncover will be shared with the defense.

Transparency, Pozen told me, “invites conceptual confusion about whether it’s a first-order good that we’re trying to pursue for its own sake, or a second-order good that we’re trying to use instrumentally to achieve other goods.” In the first case, we might feel that transparency is an ideal always worth embracing, whatever the costs. In the second, we might ask ourselves what it’s accomplishing, and how it compares with other routes to the same end.

“There is a standard view that transparency is all good—the more transparency, the better,” the philosopher C. Thi Nguyen, an associate professor at the University of Utah, told me. But “you have a completely different experience of transparency when you are the subject.” In a previous position, Nguyen had been part of a department that had to provide evidence that it was using state funding to satisfactorily educate its students. Philosophers, he told me, would want to describe their students’ growing reflectiveness, curiosity, and “intellectual humility,” but knew that this kind of talk would likely befuddle or bore legislators; they had to focus instead on concrete numbers, such as graduation rates and income after graduation. Nguyen and his colleagues surely want their students to graduate and earn a living wage, but such stats hardly sum up what it means to be a successful philosopher.

In Nguyen’s view, this illustrates a problem with transparency. “In any scheme of transparency in which you have experts being transparent to nonexperts, you’re going to get a significant amount of information loss,” he said. What’s meaningful in a philosophy department can be largely incomprehensible to non-philosophers, so the information must be recast in simplified terms. Furthermore, simplified metrics frequently distort incentives. If graduation rates are the metric by which funding is determined, then a school might do whatever it takes to bolster them. Although some of these efforts might add value to students’ learning, it’s also possible to game the system in ways that are counterproductive to actual education.

Transparency is often portrayed as objective, but, like a camera, it is subject to manipulation even as it appears to be relaying reality. Ida Koivisto, a legal scholar at the University of Helsinki, has studied the trade-offs that flow from who holds that camera. She finds that when an authority—a government agency, a business, a public figure—elects to be transparent, people respond positively, concluding that the willingness to be open reflects integrity, and thus confers legitimacy. Since the authority has initiated this transparency, however, it naturally chooses to be transparent in areas where it looks good. Voluntary transparency sacrifices a degree of truth. On the other hand, when transparency is initiated by outside forces—mandates, audits, investigations—both the good and the bad are revealed. Such involuntary transparency is more truthful, but it often makes its subject appear flawed and dishonest, and so less legitimate. There’s a trade-off, Koivisto concludes, between “legitimacy” and “the ‘naked truth.’ ”

Finn Janning, a Danish philosopher based in Barcelona, studies the gap between self-knowledge and self-deception. Under the guise of transparency, he told me, we can deceive ourselves, reasoning that we are good because we are open about what we are doing. (Modern-day autocrats often present themselves as “transparent.”) He also noted that transparency is not necessarily a positive motivating force. “Transparency does not lead to you acting better,” Janning said. “On the contrary, you try to fit into the norm. You act in a predictable way.”

From my end of the stethoscope, it’s always seemed obvious that patients should own their medical records, and be able to see them. But openness can be challenging in practice. When our hospital initially rolled out its patient portal, a few of my older patients asked that I remove references to erectile dysfunction from their medical records. Their adult children handled the household tech, they explained, and my patients preferred to keep their Viagra prescriptions private. In other cases, multiple family members can access a patient’s chart, messaging me about test results and treatment plans, and it can be complicated to deduce the hierarchy of responsibility. Advocates have raised increasing concerns about the ease with which abusers can gain access to victims’ medical records; health-care settings have traditionally been secure places for people experiencing domestic violence, elder abuse, and human trafficking, but, with medical records becoming more accessible, patients may feel less certain that their words are safe.

At the same time, there are remarkable upsides to the transparent medical record. Patients are able to review their diagnoses, medications, and treatments at their own pace. They are able to share information with family members if they choose. They can spot errors—medications they are no longer taking, medical history that’s missing, allergies that aren’t noted. They can prepare for their next visit, and perhaps feel on more equal footing with their doctor. And they can obtain test results without the interminable wait for the doctor’s call.

Medicine is a highly specialized field, and like philosophy it involves the challenge of bridging the gap between experts and nonexperts. Regulations coming out of the Cures Act prohibit “information blocking,” and in effect require that test results and doctors’ notes be released immediately to patient portals. In practice, this means that patients often see results before their doctors do and are presented with a fire hose of raw data, shorn of context. And yet on many patient portals, because of algorithms that offer only a binary distinction between normal and abnormal, lab values that stray a meaningless half percentage point out of range are labelled as “abnormal” along with results that have grave portent. Doctors’ ubiquitous use of multicomponent lab panels, which bundle together many different tests, virtually guarantees that every patient receives at least one “abnormal,” which, in our hospital’s system, is written in blood-red letters with garish yellow highlighting, plus an exclamation point, just in case the takeaway hasn’t been fully conveyed. Even seemingly straightforward yes-or-no results, such as those from H.I.V. or covid tests, can be easily misinterpreted, since false positives and false negatives occur depending on the course of the disease and the operating characteristics of the test. Meanwhile, results such as CT scans and MRIs often contain paragraphs’ worth of information that looks alarming but isn’t. Try reading the CT report of your lumbar spine and not coming away convinced that you’ll be paralyzed for life.

Patients turn from the portal to Google, then flood doctors’ in-boxes with messages, panicked by the possibility that they have cancer or multiple sclerosis or any of an assortment of inglorious diseases that appear on the Internet grossly out of proportion to their actual prevalence. But some abnormal test results will indeed be real, and some will be seriously concerning. In our current transparent system, tests showing brain atrophy or liver masses are released to the portal in the same manner as cholesterol levels and blood-pressure readings. For some patients, discovering these results on their own can be empowering; for others, it can be catastrophic. In one devastating stretch of twenty-four hours, two of my patients learned of their cancers’ metastatic reappearance by way of the portal. Their in-boxes pinged with new test results; they read them before either their oncologist or I had even seen the scans, let alone called.

Each morning, I log into our medical-record system with a certain amount of dread. Which will I tackle first—the torrent of test results or the deluge of responses it’s inspired? Some patients will have to be talked down from a ledge over a minor lab abnormality. Far worse is the serious result to which I haven’t yet had time to formulate a response: a biopsy result for which I’ve not yet tracked down the appropriate specialist to sketch a road map, or an adrenal mass on a CT scan for which I’ve not yet orchestrated the intricate endocrine evaluation. In the past, I’d do the legwork before I called the patient. Now that buffer is gone, and I am pressured to act immediately: the patient has seen the result, and further delay would be unconscionable. This timbre of rush imperils thoughtful analysis, and I worry incessantly about missteps.

Historically, the medical profession has had little use for transparency. Grave diagnoses were routinely withheld, on the assumption that they would further patient suffering. The Black men who participated in the infamous syphilis study at Tuskegee, in 1932, were not told that the trial aimed to study untreated syphilis, nor were they made aware of—or offered—penicillin, which became widely available the following decade. Generations of patients with mental illness were often institutionalized with little or no information released to them or their families.

There are strong ethical reasons, therefore, to pursue transparency in the medical record. But, as Pozen points out, we should not be lulled into treating transparency as a first-order good, like compassion, respect, avoiding harm, or putting the patient first. In a recent survey of more than eight thousand patients conducted by OpenNotes, nearly all the respondents said that they preferred immediate access to their test results, even if their doctors hadn’t yet reviewed those results. This was true even for the vast majority of people who said that they’d experienced increased worry in the face of results that were abnormal. It’s an understandable preference—one that every patient has the right to hold. But simply throwing open the medical record and calling it a day allows us to rest on our laurels without doing the hard work of fixing what’s inside. Police departments often point to body cameras as evidence of accountability without actually addressing the problem of police violence. Lawmakers can laud themselves for their transparency via C-span without having to engage in the gritty compromise needed to move legislation forward. Transparency might better be viewed as one possible means to desirable ends—not an end in and of itself.

It might also be wise to recognize that there are times when patients themselves may want to draw the curtain. Some years ago, I cared for an elderly man who’d been admitted to the hospital with weight loss and a swollen abdomen. A CT scan revealed an aggressive cancer that had already spread to multiple organs. It was clear that he was at a terminal stage. The patient’s son insisted that I not tell his father anything. It would crush his spirit, I remember him saying. The son explained that, in his culture, families didn’t tell elders that kind of news.

On a slate-gray winter afternoon, I pulled a chair up to my patient’s bedside and asked him what he understood thus far. He told me he had a stomach illness that was preventing him from eating. I explained that we now had more information. Ordinarily, this would be where I’d gently work my way into what is by far the hardest conversation a doctor can have with a patient. This time, though, I started with a question: “How much would you like to know?” I told him that he could have as much or as little detail as he wanted, or none at all. I assured him that I could keep his answer—and even how much he chose to know—confidential from his family.

He was silent for a few moments. Our gazes drifted out the window, toward the barges trundling down the East River. Then he replied that he wasn’t interested in the names of the medications we were prescribing or the details of the treatments. He didn’t want to know the results of the CT scan or the biopsy.

I took his hand and reassured him that we would take care of his stomach illness. The son handled the paperwork, the phone calls, the medications, and the necessary arrangements. The father watched television and napped. Several weeks later, he died at home, by all accounts peacefully. ♦