In this episode, we discuss:
- Dale’s background in molecular diagnostics
- Test performance in a controlled environment vs. in the real world
- The four types of COVID-19 tests
- Why there are false-negative results but rarely, if ever, false-positive results
- Asymptomatic antigen testing
- Making an informed decision around holiday plans
- The future of testing
Show notes:
- Covid Updates newsletter on substack.com
- The Business of Life Science on Apple Podcasts
Hey, everyone, Chris Kresser here. Welcome to another episode of Revolution Health Radio. This week, we’re going to be talking all about COVID-19 testing and what the currently available testing can and cannot tell us.
There are a lot of misconceptions out there right now about testing, whether we’re talking about [polymerase chain reaction] (PCR) tests, rapid [loop-mediated isothermal amplification] (LAMP) tests, or rapid antigen tests. And I wanted to record this episode because I know we’re approaching Christmas and the holidays, and a lot of folks are considering traveling and thinking about what role testing might play in determining if traveling to see relatives is safe. There is just a lot of misinformation out there. I have been confused about it myself. But I invited Dale Harrison as a guest to discuss this. And I hope that this episode will clarify a lot of the misconceptions and help you make a more informed decision.
So Dale Harrison is a senior executive with more than 20 years of experience in the biotech industry. He’s held multiple VP and C-level positions with companies in the research tools and molecular diagnostic sectors, with direct experience in PCR, next-generation sequencing, [clustered regularly interspaced short palindromic repeats] (CRISPR) diagnostic platforms, and assay development. His original background was in research physics and later technology development around robotics and automation. He transitioned into the biotech market in the mid-1990s, originally working for a [venture capital-backed] startup as CTO developing robotic DNA and protein synthesizers, and high-throughput automated production lines for DNA synthesis.
Since then, he’s gone on to hold CEO, CTO, and VP of Commercial Development roles in multiple companies in the biotech space. So Dale is super knowledgeable, not only about the underlying scientific methodologies that are used by these different testing platforms but also on how the performance of these tests differs in the real world, versus the studies that are done on them in highly controlled environments. And that’s going to be a big theme in the show, as you’ll see because it really changes things. If the test doesn’t perform in the real world as well as it did in the lab or in the validation studies, that’s information that people really need to know about, whether you’re an individual or you’re a clinician [who’s] running these tests.
So [I] hope you get a lot out of the episode. It’s a little bit technical in parts, so bear with us, but there’s some really solid info in here. And again, my hope is that it helps you to make a more informed decision about your gatherings over this winter. All right, so without further delay, I bring you, Dale Harrison.
Chris Kresser: Dale, it’s such a pleasure to have you on the show. And thanks for agreeing to join us on such late notice.
Dale Harrison: Well, thanks for inviting me.
Dale’s background in molecular diagnostics
Chris Kresser: So let’s start with your background in molecular diagnostics. We’re going to be talking all about diagnostics and testing for COVID[-19] today, and I’d love to give people a sense of where you’re coming from.
Dale Harrison: So I’ve been in the biotech industry for about 20 to 25 years now, since the ‘90s, and mostly around various aspects of molecular diagnostics. So I’ve worked in what’s called oligo manufacturing, where you manufacture the synthetic DNA that goes into some of these molecular tests. I’ve also run a genetic sequencing lab. In more recent years, I’ve focused much more [on] the area of clinical diagnostics, where I’ve worked in areas like next-generation sequencing as a diagnostic platform, as well as PCR and LAMP. And I’m currently involved in a project using CRISPR technology to do viral diagnostics. And I have some experience on the [U.S. Food and Drug Administration] (FDA) regulatory side.
Chris Kresser: Great. Those are all very relevant to the conversation today. We’re going to be talking about PCR, LAMP, and a new CRISPR test that is not available yet, but there was a study just published on it, and then some of the finer points of these different types of tests.
So I want to start with a 30,000-foot view, which is the difference between test performance in a laboratory or perhaps in a very well-controlled scientific study, and test performance in the real world when real people are out there getting these tests. This is something that you’ve mentioned in a couple of your articles on your Substack blog—which we’ll talk about at the end of the show—and in an email correspondence back and forth. And I think it’s really important for people to understand. So tell us a little bit about that.
Test Performance in a Controlled Environment vs in the Real World
Dale Harrison: So when a test is approved by the FDA, there [is] a set of documents. There’s a key document called an IFU, an instruction for use document. And in that, it usually contains some information about the performance of the test in terms of false negatives and false-positive rates. Then you move into the real world. The real world tends to be a great deal messier. But one of the things to understand about these sorts of tests, in general, is that certainly, for infectious disease testing, what you’re looking for is some molecular fingerprint of a particular pathogen, a particular virus or bacteria, or there’s a number of other types of pathogens that you can test for. And so generally, what you’re looking for is either the presence of a highly targeted protein molecule or a highly targeted DNA molecule. And that would only be present if the virus or the bacteria was actively replicating inside a patient.
So one of the problems with testing, in general, is that it’s not entirely a clear yes/no answer. So you can have a test and get a positive result. And that positive result can be in error. So you can come out with a positive even when there’s no infection. It can also come back with a negative when there is an infection. With most testing that’s out there at the moment, there are many more issues around the false negatives. The false-positive rate is extraordinarily low. And again, if you look at these FDA documents, what you see is typically 99 plus percent specificity, meaning that it will reliably give you a positive answer if the pathogen is present. So the tests are really quite sensitive and quite specific. The problem comes in when you actually try to deploy this in the real world because the validation studies that go into the FDA approval are typically done in pretty ideal conditions. You’re selecting the particular patients that you’re going to test, you’re selecting the conditions that the patient is in, and you also typically have pretty highly skilled people performing the test, gathering the samples, and running the instruments.
Chris Kresser: Right. You’re in a controlled environment, right? They’re not standing in a parking lot reaching into someone’s car and swiping their nose.
Dale Harrison: So when you move into the real world, a lot of things can go wrong. And what you end up seeing is that almost everything that goes wrong will ultimately result in what’s called a false negative, meaning that the test returns a negative [result] when in fact you’re infected. And the false negatives are really [a] serious issue with these tests. So one important factor is that the tests tend to be very sensitive to how well the technician taking the sample does their job. It requires real skill and precision. So these technicians are generally very well trained. But, if you’re taking a lot of samples in a short period of time, again, there’s a lot of things that can go wrong if the person taking the sample just doesn’t quite do it right.
The other thing that happens with something like COVID[-19], an upper respiratory disease, is that the virus will typically colonize the nose and upper throat first, and then as the infection progresses, it’ll tend to migrate down lower into the lungs. So if you’re pulling a sample just from a nasal swab, or a throat swab, the virus may have already migrated lower and you may simply miss it. The other thing that happens is there’s a process called “mosaicing” where the virus isn’t necessarily spread evenly across the entire nasal cavity or the entire back of the throat, [and] it may appear as though it exists in patches. And so, if you just randomly hit the wrong spot, you end up getting, in some cases, too little virus in the sample to be able to detect.
Chris Kresser: And this has been an issue with [Helicobacter] pylori testing, as well, and other pathogens that have that same patchy distribution.
Dale Harrison: Yeah. And the other issue is that once the sample is collected, it’s got to be properly handled, properly packaged, properly shipped, and properly labeled. And so there are a number of things that can go wrong. So one of the things that happens, especially for viruses, is that most of the viruses that we’re looking for, when you take the sample, with that sample comes other enzymes that are present in the nose or the throat, and their job is to break down RNA. This is especially an issue with RNA viruses. So [the] flu and COVID[-19] are major examples.
So these enzymes will very rapidly break the RNA down. And so the sample has to be transferred into what’s called a viral transport , which is a special reagent that basically slows down that enzymatic process, and slows down the degradation of the RNA. The tube needs to be properly labeled and it needs to be shipped—hopefully, all [of] that goes well. And then, once you get it at a testing lab, typically you’re processing hundreds or thousands of samples at a time in large batches. And most of this is done robotically. And so there are additional areas where errors can occur just in the robotic handling.
Now, all of the things I’ve talked about so far are really things that will cause you to miss getting enough sample to detect. So you’re going to get a false negative. The one area where you actually see a very, very low rate of false positives, and we’re talking one sample in [10,000] to 100,000, is basically error handling by the robots, where you’re mixing reagents on a plate. So typically, you do these things on what are called “96-well plates” with about 92 samples, and about four controls on the plate. And the robot has to add reagent, mix the reagent, and you can get cross-splashing if the robot gets a little too aggressive with it. And there was actually a case back in May, or early June, where one of the major tests was seeing false positives [at] the rate of about one out of, again, between 1 in 10,000 and 1 in 100,000 samples. And it was traced back to cross-splashing between the wells because the robot had too aggressive a mixing protocol.
The FDA issued a notice, there were corrections made to the software, and the problem went away. But the risk of getting a false positive is very, very low. It generally means that you’ve either mislabeled a tube, or you’ve accidentally had contamination from one sample to another sample when it’s on the robot.
Chris Kresser: Right.
Dale Harrison: And so there’s a lot of nonsense talk about how you can’t trust the test because there are a lot of false positives. And the fact is, there are no false positives. I mean, they’re effectively zero.
Chris Kresser: Right, right. With the exception of maybe antibody testing, which we can talk about a little bit more later.
Dale Harrison: Right.
Chris Kresser: But, if we’re talking about PCR and testing for the actual pathogen, the specificity is so high that effectively, as you’re saying, false positives are not really the thing we need to be concerned about. It’s much more about false negatives, which, of course, [carry] more risk, too. The risk of a false positive [is] that someone unnecessarily has to self-isolate, and that does have some significant downsides, but not to the extent that a false negative does when someone is actually positive, and then believes that they’re negative, and maybe isn’t as careful with distancing, or wearing a mask, or other measures that they might take.
And from what you’ve described, it’s pretty clear that there would be a significant difference [between] carefully controlled test environments and the real world. It’s like I said before, there’s somebody standing in a parking lot outside of the hospital. It’s cold, they’re standing there for a long period of time, [and] they are probably very well trained, but people are human. They’re having to do so many swabs. There’s a lineup of cars. And they’re having to do that over and over again while standing outside for a long period of time. It seems almost impossible that there wouldn’t be some errors introduced in that process.
Dale Harrison: Right. So there’s this difference between the official false-negative [and] false-positive rates, the official specificity and sensitivity rates, and then, essentially, your effective false positives and false negatives are in the real world. So again, things like the robotic handling errors I was talking about [are] something that really has nothing to do with the test; [they have] to do with how you handle the sample.
Chris Kresser: Sure.
Dale Harrison: The errors around sample collection, or just the randomness, [have] nothing to do with the core test. So the thing that people need to understand—and even a lot of people in the clinical environment fail to really understand these distinctions—is that the information from the FDA that comes from the test validation process really represents the absolute ideal performance of the test. And then everything that happens from that moment forward is going to be worse than that level of performance.
So one of the analogies I make is, you go to the car dealership, and the dealer tag on the car says that you should get 45 miles to the gallon. You buy the car, but the reality is, you never see more than 28 miles to the gallon.
Chris Kresser: Right, right.
Dale Harrison: And it’s to the point where the TV ads all end with, “and your mileage may vary.” And again, because what’s happening there is in the regulatory environment, how miles per gallon is measured for a vehicle, is in an extraordinarily idealized environment.
Chris Kresser: In a wind tunnel or something. Inside a factory somewhere.
Dale Harrison: Right. And it represents the sort of the ultimate upper ideal performance of that engine. But it doesn’t really represent what you see in the real world.
Chris Kresser: This is so important. I have a number of patients, family members, friends, and practitioners [who] we train in our training programs reaching out about this. They go to Abbott’s website, for example, and they look at the specificity and sensitivity numbers for, well, let’s just focus on sensitivity, because as we established, the specificity is not really much of a consideration. But they look at the sensitivity numbers for the Abbott ID Now or the Abbott BinaxNOW, which we’ll talk about in a moment, and they say, “Hey, these look pretty good. This seems to be a pretty reliable test.” But what you’re saying here is that we really can’t look at those numbers and assume we’re going to get that kind of performance in the real world. And the difference is not insubstantial. We’re not talking about like 1 or 2 percent; we’re talking about north of 10 percent, maybe even 20 percent difference in the numbers.
Dale Harrison: Or even greater in [some] cases.
As we approach the upcoming holidays, we must make informed decisions around the safety of gatherings. In this episode of RHR, I talk with Dale Harrison about the misconceptions of COVID-19 testing, and how you may not be able to rely on test results to keep you and your family safe. #covid19testing #chriskresser
The four types of COVID-19 tests
Chris Kresser: Right. So let’s now switch gears and talk a little bit about the currently available testing options. So if we can kind of break it into three broad categories: PCR, rapid testing, and antibody testing. So give us an overview of each of these different types of tests.
Dale Harrison: Well, and you probably should break it into four large categories: PCR, rapid test, antigen test, which is a different type of rapid test, and then the antibody test. And so the question is, what are you testing? You have tests that answer the question, “Do I have a current active infection?” Meaning, “is there a virus that’s replicating in the patient?” And then there’s a second type of test that answers the question, “Was I previously infected?” And so, let’s start with the, “Was I previously infected” test. These are the antibody tests. And what they’re looking for, as the name implies, are antibodies in the bloodstream.
So what characterizes them is that you’re doing a blood sample, not a nasal swab, or a nasopharyngeal sample. You’re either doing a full blood draw, or you’re doing a pinprick. And so you’re looking for protein molecules that your body’s immune system has manufactured to attack the virus. And these generally won’t begin to show up at detectable levels until about two weeks after the onset of symptoms. So this is not something that’s useful during the time that you’re sick, but it’s useful to look back retrospectively to know whether or not you’ve been exposed.
The other types of tests are really looking at, “Is there some sort of an active infection.” And so [there are] three big categories. [There is] the PCR test, which is when you go into a clinic and they have the footlong Q-tip that they shove down your nose. They’re actually gathering a sample from the upper back portion of the throat that can’t be reached from the mouth. And so they’re actually just going through the nose to get to the throat.
Chris Kresser: Yes, they’re not pleasant.
Dale Harrison: Yes. Well, there are even more unpleasant versions that have been pioneered in China.
Chris Kresser: Oh, wow.
Dale Harrison: It goes down the throat into the upper portion of the lungs, and provides much more reliable results, and much lower false-negative rates. But it’s sort of like a cross between sword swallowing and using a Q-tip.
Chris Kresser: Right. It might be hard to get people to come [to] get tested once the word gets out about that test.
Dale Harrison: Right. Well, that’s why it works in China.
Chris Kresser: Right, no choice. Right. So, okay, [we’ve] got PCR; that’s one of the three.
Dale Harrison: And that has to be sent off to a lab under ideal conditions. You might get the result back in a day. Under realistic conditions, it’s going to be more like three days to maybe a week. If things are really bad out there, it can stretch to as much as 10 days or two weeks, simply because the labs are backed up. Then there’s another category of tests that are just generally called “rapid tests.” And the classic one is the Abbott ID Now. This is the one that was famously used by the White House and helped give us the Rose Garden super-spreader event.
Chris Kresser: Right. Was the NBA also using that, too, I think in Florida?
Dale Harrison: Yes.
Chris Kresser: Yeah.
Dale Harrison: Yeah. And so this is a test that’s done on-site. So it’s a so-called “point-of-care” (POC) test, meaning that the instrument is right there where the sample is gathered, you prep and place the sample directly on the instrument, and you get an answer back quickly. Quickly meaning 20 to 30, maybe 40 minutes at the most. And then the ftegory is called the “antigen test.” So these first two tests are looking for genomic RNA from the virus, part of the virus’s genomic code. The antigen test is not. The antigen tests tend to be even faster. These are five-minute, 10-minute tests. They usually use what’s called a “lateral flow strip,” which is a little disposable plastic thing that looks like a pregnancy test. As a matter of fact, a pregnancy test is a type of antigen lateral flow strip test. So these antigen tests are not looking for RNA, what they’re looking for are proteins. And specifically, everybody’s seen a picture of the virus with the big ball with the spikes on it.
Chris Kresser: The spikes, yeah.
Dale Harrison: So inside that ball is another ball, or more like a bag. So there’s a bag made out of proteins called the nucleocapsid. And that bag contains all the RNA. And so the nucleocapsid tends to be very unique to a specific virus, where a lot of the proteins on the outer coat can often be hijacked [by] other proteins the cell is manufacturing, and so you have to be careful about trying to test for what’s on the outer coat, or the outer envelope. What you need to look for is the nucleocapsid envelope inside. And so these are just proteins. And so these tests are looking for the presence of these proteins.
So the advantage of these is that they run very fast. Again, many of these will provide you [with] an answer in about five minutes. The downside is that they tend to have a very, very high false-negative rate. And so in molecular testing in general, there’s this idea of “no free lunch.” You can have a test that’s fast, cheap, or has high specificity and sensitivity, meaning, very low false negatives and very low false positives. So basically, you get one of the three. And if you give up one or more, you give up one of those items, you can then gain in the other areas. So, with the rapid test, the 20-minute test, what you’re essentially doing there is, it’s still a relatively expensive test, but it’s a fast test. And so that speed is paid for by having less ability to detect the pathogen that you’re looking for.
The antigen tests are both fast and cheap. And so [there are] things like the Abbott BinaxNOW test, which is a very inexpensive $5 to $7, one-use disposable test. And you can run the whole thing in 10 minutes. So there, you’ve got fast and cheap, but you pay double for that in terms of very, very high false-negative rates.
Why there are false-negative results but rarely, if ever, false-positive results
Chris Kresser: Right. So let’s actually bring that into more focus here now, since we’ve given a kind of overview of each of these tests, and we’re going to do a little bit of a deeper dive into each of them in a moment. But what can people generally expect for false-negative rates with these tests in the real world? Not in carefully controlled conditions, but in the real world. So PCR, the rapid test, the LAMP, and the antigen test. What kind of false-negative rates might we see?
Dale Harrison: So there have actually been some studies that have made it into the peer-reviewed literature, looking specifically at this question. And again, there are a lot of variables … Are you doing it [in] a drive-thru line versus in a major medical center? But in general, the laboratory-run PCR, which is sort of the standard test that everyone thinks of that gets sent off to a lab and takes a few days to get the results back, is going to typically run about 20 percent false negatives in actual clinical usage. The rapid test that you do right at [the] point-of-care is higher than that. It’s typically going to be 30 or 40 percent false negatives. And I’ll give you a specific anecdote on that. I have a relative who is an air ambulance medical technician. And so they have one of these Abbott rapid tests onboard the helicopter. Because are they going to do a cardiac patient? Are they going to do a car wreck? And they need to rapidly know [if they] should don all of their protective gear.
Chris Kresser: [Personal protective equipment (PPE)]. Yeah.
Dale Harrison: And what they were finding was that repeatedly, they would pick up a patient unrelated to COVID[-19]—[a] car wreck, a motorcycle accident, a cardiac event—they test negative in the helicopter, they get into the ER, they’d be able to run a better test, typically a full PCR test, because a lot of hospitals will have a lab.
Chris Kresser: Have their own equipment there.
Dale Harrison: Yeah, for internal use. And so they can turn these things around in an hour or so.
Chris Kresser: Right.
Dale Harrison: And they were repeatedly finding patients that were testing negative in the helicopter are now positive when in the ER. And what that led to was them basically abandoning the use of the rapid test, and just always flying in full PPE.
Chris Kresser: Right, just assuming that everyone is positive, basically.
Dale Harrison: Right. Because that was the only safe way to transport patients was to just assume everyone’s a positive.
Chris Kresser: Right. So that’s pretty high, 30 to 40 percent false negative. But it’s even worse for the antigen, right?
Dale Harrison: Right. And so the reality of these antigen tests is that they’re about a 50 percent, sometimes more than 50 percent, false-negative rate. They can be quite high. And again, there’s a lot of reasons for that. Some of it has to do with sample collection, some of it has to do with processing the test, especially on the tests that are designed for a non-trained person to be able to run. They don’t have any that are legally allowed for home use yet, but they’re getting ready for home use. You’ve got a lot more opportunities to get things incorrect there. So there are a number of issues with that. But there’s a more fundamental issue with these tests.
So if you drop back for a second, and ask the question, “what are these tests actually doing inside the machine?” The way these tests work is that you’ve got a very specific target molecule. So for the PCR test, it’s generally a very short, maybe 50-nucleotide-long RNA sequence that is highly unique to that virus and won’t be found in any other organism on the planet. And this allows them to be very specific, to be able to make sure that if they detect that thing, that thing will only have come from the virus that they’re targeting in the test. But the way these things work is through a process called “molecular amplification.” So it’s essentially a chemical process that allows you to make billions of copies of a particular molecular sequence that’s found in the sample.
And so a laboratory PCR test will typically do about 40 cycles. Each cycle doubles the number of copies of the targeted sequence that was in the original sample, which means at the end of this, you can get between a billion- to a trillion-fold amplification, but you still have to have some material to start with. So if you don’t have any of the targeted viral RNA in the original sample, you could run this through 500 cycles and you’re going to get nothing.
Chris Kresser: A billion or a trillion times zero is still zero, right?
Dale Harrison: And that is why these tests have such a low false-positive rate. They’re able to go in there and take very, very tiny amounts of sample and amplify it enough that they can detect it at a microscopic level. What they’re actually doing is, as they amplify these things, they’re attaching fluorescent molecules. And then you basically shine a light on the test chamber that contains the liquid that’s going through this amplification process. And those fluorescent molecules will then light up, and you can detect that in a very specific photodetector. So you’re looking for a very specific wavelength of light shining off of the molecules. And once you’ve got a large enough signal for your photodetectors to see, then you know you’ve got a positive and you can actually stop the run at that point because you’ve got a positive.
The LAMP process, or these rapid, 20-minute tests, still use molecular amplification, because again, that’s the only way to really get a sensitive test. But part of what makes them rapid is that they’re using both a less efficient amplification technology, and they’re running it for a shorter period of time. So again, “no free lunch.”
Chris Kresser: Right.
Dale Harrison: So, in the end, what happens there is that you’ve got [a] 1,000-, maybe 10,000-fold lower level of amplification at the end of the test than you would have with the regular PCR test. And so you simply have less sample to be able to detect, and a greater likelihood of missing what you’re looking for. Therefore, [a] higher false-negative rate.
Chris Kresser: Right.
Dale Harrison: The antigen tests are unique because they’re not looking for DNA or RNA; what they’re looking for is a protein. And the problem is, we know how to amplify DNA, but we do not know how to amplify proteins. There are ways to amplify proteins, but it’s something that looks more like a long-term fermentation process where we’re actually growing organisms over long periods of time. So there’s no rapid chemical molecular amplification technology for proteins, the way there is for DNA. As a little side note, the COVID[-19] virus is actually an RNA virus. We also don’t know how to amplify RNA, but we do know how to amplify DNA. And so these tests are often referred to as RT-PCR test, [and] the RT stands for “reverse transcription.” So on the very first cycle in the test, you essentially turn the RNA into a matching DNA sequence, because you can amplify the DNA, but you can’t amplify the RNA. This is called “reverse transcription” because, in a normal cell, the DNA in the cell transcribes to RNA, which then translates into proteins. This is sort of how the blueprint gets executed from DNA into proteins. So reverse transcription is reversing that process in the opposite direction from what normally happens in a cell, going from RNA to DNA.
With the antigen test, because we’re looking for protein molecules, there’s no amplification technology for proteins. They can work on a pure chemical basis, rapidly, within a test. And so you rely on a different sort of amplification process where you’re essentially binding antibodies to the antigen and then flowing that complex down a paper strip, typically, and having that antibody trigger a whole series of molecules that are attached to the surface of the paper, [which] will cause a color change. And so there is a type of amplification, but you’re not actually amplifying the original protein molecule. But you are amplifying the effect of it as you flow it down a strip of paper. And again, this is how a home pregnancy test works.
Chris Kresser: Right.
Dale Harrison: So the end result there is you have something that’s very fast that once you put the sample on the little lateral flow strip, you’ll have an answer in five minutes or less. But you’ve got a very low level of amplification, and therefore, you need a lot of virus in your sample in order to be able to detect anything.
Asymptomatic Antigen Testing
Chris Kresser: Right. This is probably a good segue into another weakness of the antigen test, [which] is that [it’s] not approved or validated for people without symptoms. And that I think the one study that I sent you or you sent me showed up to a 70 percent false-negative rate in people with no symptoms.
Dale Harrison: Right, right. So right now, the tests that have what’s called “emergency use authorization (EUA) approval from the FDA have only been validated on patients that are symptomatic, and are approved for patients that are symptomatic. The original idea behind the antigen test was that you’ve already essentially got a medical diagnosis. If you’re a physician in a hospital, you’ve got the patient in front of you, you’ve got all of their symptoms, maybe you have lung imaging, you’re seeing clear indications of COVID[-19], [and] you need to confirm your diagnosis because you need to either send that person to the COVID[-19] ward or to the pneumonia ward. And you don’t really want to make a mistake there. But you[’ve] got to make a decision now. You’ve got all the indicators there. And you want to confirm that diagnosis with a very quick test because you have to make an immediate decision.
Chris Kresser: Very different from a screening test of people who are asymptomatic.
Dale Harrison: Right. So when you start using the test on asymptomatic [people], it’s not universally true, but most people who are asymptomatic are in fact what [is] called “pre-symptomatic,” meaning they will eventually develop symptoms, but the virus hasn’t replicated to the point that the immune system recognizes the virus.
Chris Kresser: So it’s not producing those antibodies that will bind to the antigen and show a positive result?
Dale Harrison: Well, no, because again, this is measuring proteins from the virus itself, not antibodies.
Chris Kresser: Right.
Dale Harrison: But what happens is you just don’t have enough virus, you don’t have a high enough viral load at that point to [diagnose]. Because again, there are some people that appear to have a very high viral load, are very infectious, but never show symptoms. Then there are other people that are pre-symptomatic, which is more common, where you’ve got a high enough viral load that you’re infecting people, but things haven’t progressed to the point that the immune system is aware that there’s an infection and has started to kick in. And so, as the virus replicates more and more, it finally reaches a point where there’s so much virus present, that the immune system sees it.
And so the antigen test is useful when there’s a very high viral load, because again, ideally, you need a lot of virus collected on that sample in order to have enough viral proteins for this fairly insensitive test to be able to see anything. Now, what that means is, if that antigen test returns a positive, you can take that to the bank. That’s absolutely guaranteed 100 percent positive. And this is one of the things that’s interesting with the antigen test. There are a lot of handling mistakes that can be made with a PCR or even a rapid test, but because the antigen tests are so simple, it’s not going to get left in the FedEx truck in the sun for two days, [and] it’s not going to get mislabeled. So a lot of those sort of handling mistakes that are issues with the other tests aren’t really an issue with the antigen test, because it’s immediate; it’s right there in front of you.
So if you get a positive [result] on an antigen test, you can absolutely take that to the bank. It’s absolutely for sure that you’re infected. If you get a negative, then you pretty much have to assume that you’re a presumptive positive until you can get a better test to show a negative.
Chris Kresser: Right. So a positive is [a] positive, a negative is a maybe.
Dale Harrison: Yeah, a negative is a presumed positive.
Chris Kresser: Right, yeah. Even stronger.
Dale Harrison: At least with the antigen test.
Chris Kresser: Right. So let’s also talk about test timing, because that’s another issue even with PCR and the LAMP testing. Let’s say, you’re going to visit your family or something and you get exposed on an airplane. And right when you arrive at your destination, you get a PCR test. Is that likely to return a positive result even if you’ve been infected?
Dale Harrison: No, it won’t. And so, typically, if you assume a sort of normal progression for COVID[-19] from exposure to [the] onset of symptoms, to essentially clearing the virus from your system, for most people—and again, there’s a lot of variability here, but for most people—that’s about a 10-day cycle. [For] half of it, you’re pre-symptomatic, [and for] half of it, you’re symptomatic. So, on average, most people will take about five days between when they [get] infected and when they start to show symptoms; they’ll start to run a fever for instance, or have a cough.
And then it actually only takes about five days after you start developing symptoms before the immune system has effectively cleared the virus from your system. And so what happens is that you need the virus to replicate, to grow for a day or two days after you’ve been infected before a test will reliably pick up a positive. So if you get infected this morning and you get tested this afternoon, it’s probably going to be negative. If you got infected two days ago and you got tested today, then you’ve got a pretty good chance it’s going to show up [as] positive.
Chris Kresser: What is the correlation between infectiousness and the likelihood of testing positive? In other words, some people have said, “Well, hey, if you test negative, that means there’s not enough virus replicating to show a positive result. But it also means that you’re likely not contagious.” Do you think that that can be accurately said or not?
Dale Harrison: No. There’s absolutely no reliable evidence. And as an aside, I have clients that have a number of companies in molecular diagnostics. I can tell you, every single one of them would love to be able to produce an infectiousness test. But no one has any idea how to do that. What these tests do is they answer the question, “Is there replicating virus in you?” But there are a lot of unknowns around what causes one person to be infectious versus another person.
One thing we know is that almost all people who are infected infect none or maybe one other person. And a very small number, maybe 15 percent, maybe 20 percent of the outside, are responsible for 80 or 85 percent of infections.
Chris Kresser: The super spreaders.
Dale Harrison: These are the super spreaders. And no one can tell the difference between a super spreader and a non-spreader, in terms of anything you can measure, look at, and test. They don’t look any different. But that super spreader might be passing the virus on to 30, 50, or 100 other people [whereas] most of the people with COVID[-19] may pass it on to none, or to one person, maybe a couple of people.
Chris Kresser: That’s the real wild card here. We have no way of distinguishing between someone who’s very unlikely to spread the virus and someone who is a super spreader. So again, it’s kind of similar to a negative is a presumed positive. I guess you kind of have to presume that somebody is a super spreader unless you have evidence that they’re not. And at this point, we don’t have any way of showing that they’re not, right?
Dale Harrison: Now, there is some assumption that they must be expelling a larger amount of virus. It doesn’t mean they have a larger amount of virus, they just may be more efficient at expelling it. There may be something unique about the super spreaders where every time they breathe out, they’re somehow moving much more of what would otherwise be the same amount of virus, into those droplets, or aerosol, that they’re breathing out. So, the problem is, we just don’t know what that is. And if you don’t know what it is, and you can’t characterize it, you can’t develop a test for it.
The other issue is there’s so much variability in the sampling. So you could take the same technician and the same patient and do 10 swabs in a row. And you’re going to see quite different quantitative results in the test run. So again, this is this nonsense that’s been going around for some time about lowering the cycle threshold. So this is the number of amplification cycles that the PCR process is going through to amplify the original RNA from the sample. And the problem is you can’t say that 30 cycles mean that you’re not infectious to others, but 40 does. Because you could see huge variances in the amount of virus that gets collected in a sample, again, with the same technician using the same technique on the same patient multiple times. And again, I’ve looked at some of the data from the validation studies that are done as a part of submitting the FDA application for approval. And there’s this huge variability. And that’s why these tests are generally binary tests; they give you a yes/no answer. There’s not useful information in the cycle count, for the most part.
Chris Kresser: Right, right.
Dale Harrison: Certainly not something, not useful at a diagnostic level.
Chris Kresser: Not to the point that we can rely on it and make decisions based on that information. So let me just summarize what we’ve talked about so far, then I want to make this a little more real by talking about a scenario of someone going to visit their family. And then I want to finish up by quickly talking about new test methodologies that are either in development or are maybe even close to being released, that might be able to help address this quagmire that we’re in here.
So far, the available testing can tell us if there’s an active infection. So if it’s a positive, it’s almost certainly a positive. But they cannot tell us whether we’re contagious to others, they’re not very useful if you get tested right after you have been exposed within a day or two or five days after the onset of symptoms. They can’t tell you if you’re going to become symptomatic or remain asymptomatic, and they cannot tell you whether it’s safe to assume that you’re negative and visit other people and feel certain that you’re not going to infect anybody else, particularly if you’re not wearing a mask or doing the typical precautions. Is that accurate?
Dale Harrison: Oh, absolutely. Yes.
Chris Kresser: Okay, so let’s play this out a little bit. This is quite unrealistic, but I just want to do it as a thought experiment. So let’s say somebody had access to PCR testing, and they could get results within a few hours. Is there a way that testing could be used to generate a fairly high level of safety or certainty that you’re not infected? Let’s say you get tested a couple [of[ days before you leave, you drive to your destination, you get tested again when you arrive, two days after the first test, and both of those tests are negative. What would this sort of level of confidence be in your mind if both of those tests were negative?
Dale Harrison: I mean, part of it depends on if you think you’ve been exposed. So there’s kind of a bayesian statistic element here, this notion of priors. Have you had contact with someone who is now sick? Have you done something that is high risk, like go hang out at a bar all night? If so, then you’ve got prior information that would indicate that you’re higher risk, and therefore, you should have less confidence in those negative test results. But I’ll tell you something else that I have seen and heard multiple times now, which is this idea of “test shopping.”
So there was a Facebook thread in a private Facebook group that I followed back over the summer of these two wedding planners. So they run a professional wedding [planning] business. And one of the bridesmaids tested positive, so they went on the forum asking if people knew of other places they could get tested because they needed to show a negative test so that [they were] allowed to go to the wedding. And so they already have a positive test and they want[ed] to keep taking tests until they [got] a negative, and then that’s what they were going to show to be able to justify being in the wedding. And I’ve heard of this idea of test shopping a lot. The deal is, as soon as you get one positive, you’re positive. I’ll throw one other thing in here, too, another one of the endless conspiracy theories that seem to go around. This notion that “you don’t really have the virus, you just have leftover components. You have viral debris.”
Chris Kresser: You’re positive, but you’re not contagious. That’s some of what people say.
Dale Harrison: Well, yeah, some people say, “there’s a positive and not contagious, but then there’s also a positive.” This is the whole “casedemic” nonsense that all of those positives aren’t really measuring an actual live virus.
Chris Kresser: Just fragments.
Dale Harrison: Yeah, just fragments. The deal is, one thing that is well understood about RNA viruses is that living organisms do not like to see RNA outside of a cell. RNA has one place it’s supposed to be, it’s inside a cell, and it shouldn’t be hanging around for more than 10 minutes. It exists for very brief periods of time, as the DNA is transcribed into the RNA, which is then immediately used, and merely translated into proteins. And so the body is flooded with these enzymes called “RNases,” whose sole job is to take any random RNA molecule, and within minutes to an hour or so, slice these things into nuts and bolts.
And so the idea that you’re going to have stray RNA laying around on top of your throat for a month that’s unrelated to the live virus is really nonsense. If you’ve got RNA, the chances are that whatever produced that RNA was replicating within the last two or three hours. Now, one of the things that is suspected is that people are able to maintain long-term, low-grade latent infection, meaning that there’s a small patch that’s continuing to replicate virus at a very low level, but not quite at the level that is going to draw the attention of the immune system. Especially with upper respiratory viruses, the immune system has a hard time effectively reaching the upper respiratory area. And so there’s this idea of the virus still being there, and still replicating. But if you get a positive on one of these tests, you’ve almost surely had actively replicating virus in you within the last few hours. So this idea that a positive test doesn’t really indicate that you have the virus is nonsense. You’ve got the virus.
Making an Informed Decision Around Holiday Plans
Chris Kresser: Right. Let’s break this up because we’re coming up to the end of the time here. And I wanted to get this out before the holidays. I mean, we missed Thanksgiving, but hopefully, we can get this out before the Christmas holiday so that people can make more informed decisions. Certainly, different people have different levels of risk tolerance; different people are at different levels of risk with COVID[-19] due to their age, whether they have pre-existing conditions, etc., and people are going to make choices based on that. But I’ve heard from quite a few people, friends, colleagues, etc., who were planning to use testing in a way that is not really supported by the evidence.
So, for example, a colleague of mine has a school pod and they were going to try to purchase an Abbott BinaxNOW test to screen all of the parents and kids every day, or every other day, or every third day, or whatever, even when they’re asymptomatic. But as you’ve made abundantly clear, that’s not going to be an effective strategy because you have up to 70 percent of false negatives with people who are not symptomatic using the antigen test. So I wanted people to get a better sense of the shortcomings of these tests so that it doesn’t give them false confidence. And so they don’t make a plan to go see their family and use one of these rapid tests as a way of ruling out infection and then find that they were positive and that they actually ended up infecting a family member, particularly one who was at high risk. I really don’t want that to happen. And so that’s the impetus for putting this out there as soon as we can.
What are the testing methodologies that are either in development or maybe close to being released, that could help us to address some of these shortcomings and actually provide useful actionable data?
Dale Harrison: I mean, I think in general, it’s dangerous to rely on testing as the cornerstone of a process of trying to remain safe. And there [are] four words that encapsulate that, “White House Rose Garden.” There’s no population in this nation that has been more intensively tested than the White House staff and the visitors to the White House. They were running these machines nonstop; they were using the Abbott ID Now rapid test. Every single person that entered the White House got tested; they weren’t allowed to see anybody until the test results had come back. Every person in the White House was tested every day when they came in. Nowhere on the planet has there been a more intensive testing process. And because they relied entirely on testing, so no masking, no distancing, no sort of mitigation other than the testing, you’ve seen now multiple super spreader events. I mean, we’re watching one right now with Trump’s legal team. And I think that this is probably the single best example.
Whatever testing program you’re going to dream up is going to be a fraction of what the White House was doing, and it has not prevented the White House from seeing multiple super spreader events. So you simply cannot rely on the testing. And I think you’re far better off [relying] on this idea of “priors.” Have prior knowledge about the people you’ve been around, and will be around, and their activities. Again, has someone been exposed to a sick person? If they’re running a fever and coughing, you may think it’s a cold, but you have to assume that it’s COVID[-19].
Chris Kresser: Right.
Dale Harrison: Have you engaged in activities that are high risk? Do you go to restaurants and bars on a regular basis, or to the gym on a regular basis? Whether you have a mask on or not, you’re now in a significantly higher risk category. And so, a lot of this is what sort of behavior you have engaged in and what sort of people you’ve been exposed to, [which] will give you an indication of your general risk level. But what you see in events, and again, the White House Rose Garden event is a classic example, there was probably only a single super spreader at that event [who] ended up passing it to quite a number of people. And the problem is that the entire group is only as safe as the least safe member. If you’ve got 40 people who wear a mask, never go to a store, never go to a restaurant, never go to a bar, and never leave their house, but you[’ve] got one person who hits the bars regularly, that entire group is now at a risk level equivalent to that one person [who] has engaged in the riskiest behavior.
Chris Kresser: Especially if that person happens to be a super spreader, right? And is spreading more virus than someone else.
Dale Harrison: Right. And you may luck out. Maybe they’re infected, and they’re not going to infect anybody. But you don’t know.
Chris Kresser: Right.
Dale Harrison: You always have to assume that the person you’re looking at is potentially infected, and potentially a super spreader. So again, I mean, I think the assumption has to be that people are infected, and you have to take measures. You have to do mitigation. And there are a lot of clinics around the country that have done a really good job with this. So these walk- in clinics, emergency care clinics, psychiatric service clinics, these [places] that aren’t handling [COVID-19] cases on a regular basis, but have managed to have very good track records of not having infection within the facility. I have a close friend who’s a senior executive in a company that runs a series of site clinics on the East Coast. And every person there takes it seriously. Everyone gets tested once a week, everyone wears a mask, [and] everyone follows very specific protocols because their assumption is that absolutely everyone is infected. If we behave like they are, that’d be our best shot at no one getting infected. And so that sort of paranoid mindset is useful. If you think there’s a risk, isolate or quarantine; if you’ve got symptoms, isolate. If you engage in high-risk activity, let people know. And know who you’re inviting to events.
The future of testing
Dale Harrison: Now, you asked about future technology. So there’s a number of really interesting changes that are underway in the testing environment and new technologies that are coming out. One of the issues that we’ve seen is just the sheer volume of testing is historically unprecedented. Never in medical history have we run this many tests for a single infectious disease, every day, across the country. And so, part of what we’ve seen is a real backlog in the labs in terms of just all of the manual work that needs to be done to set up, prepare the samples, get them on the machines, and get the test run. So there’s some really interesting stuff [coming] with what’s called, “next-generation sequencing.” These are genetic sequencing machines that can sequence an entire genome in a matter of a few hours. And so Illumina is the big company that produces these. Their flagship product is called an Illumina NovaSeq, and it allows you to do genetic sequencing on massive amounts of DNA in one shot.
So there are companies that are working on using that technology to [do] massively parallel testing [of] tens of thousands of samples in a run. So the idea is you could put [25,000] to maybe 50,000 samples on a single machine, and run this thing in a matter of six hours to be able to get a result. And each sample is essentially molecularly tagged with things called DNA barcoding, where you basically attach little, tiny molecular sequences that barcode the molecules from each patient sample so that you can then separate everything out at the end in software. It’s a very clever idea and it is a potential way to massively increase the volume of testing that can be done.
Chris Kresser: So would that have roughly the same sensitivity and specificity of PCR, but would just be faster, and would test more samples at a time?
Dale Harrison: It would potentially have much lower false-negative rates because instead of looking for, say, a 50-nucleotide piece, you could be looking for multiple different segments. You’re essentially sequencing everything.
Chris Kresser: The whole thing. Right, right.
Dale Harrison: There’s also another interesting area of development called “metagenomic-based next-generation testing.” And so the idea here is you take a sample, and you basically sequence all the DNA in the sample or all the RNA in the sample, you push the data into software, and then you subtract out everything that’s human. So you throw away all the human sequences, and you look only at what’s leftover. And then you start looking these sequences up in a master reference database of all known sequences. The advantage of this is that you could potentially detect a whole panel of different diseases in a single test run. So you could test for every possible strain of cold, flu, all the other major upper respiratory viruses, and COVID[-19]. And so you could come back with a single sample and single test, and you could see someone has a fever and a cough. Well, what is it? This thing would then come back and tell you exactly what it is. If you’ve got more than one infection, it could see that. It could even see different genotypes, so different genetic mutations of the same virus.
So one of the clients I’ve done a fair amount of work with in Cambridge is developing this technology for the human transplant market, to be able to screen transplant patients for various types of viral infections, and be able to basically see everything in the sample.
But the most interesting stuff is based on CRISPR technology. So this is the gene-editing technology that’s been in the news a lot. And in fact, the two inventors of this won the Nobel Prize in Medicine last month. I’m actually doing work with the company that those two Nobel laureates founded a couple [of] years ago. And one of the things they’re working on [is] CRISPR-based test platforms. CRISPR is this enzyme that does two things: it identifies a genetic sequence, and then it has molecular scissors that will then slice that sequence apart. And you can adapt those functions to do diagnostic testing. So you engineer the enzymes so that they will detect a specific sequence from, say, the [severe acute respiratory syndrome] virus. But instead of clipping the genetic sequence from the virus, it reaches over and clips an unrelated DNA sequence that releases a fluorescent molecule into a solution. And once it’s clipped enough of those fluorescent molecules, and you actually get this sort of lawnmower effect where a single DNA sequence from the virus triggers the enzyme, then the enzyme undergoes what’s called, “trans cleavage,” which means it just starts slicing everything it sees within reach.
As it starts slicing these reporter molecules that are surrounding it, again, sort of like mowing the grass, you could then have tens of thousands of fluorescent molecules released into the solution that can then be detected with the photosensors. So it’s a very interesting technology. And again, a completely different way to do amplification. In this case, you’re essentially amplifying the number of detector molecules that are present, and that are available to be seen from a single detection. But the advantage with CRISPR is that it has [the] potential to be extremely fast, meaning potentially a five-minute test, very inexpensive, and with both very, very high sensitivity and specificity. So it can be very, very specific, but it can be extraordinarily sensitive. So potentially, you could have something with these CRISPR tests that is significantly better than the best PCR test.
Chris Kresser: But could be done at home and quickly.
Dale Harrison: And could be done at home and inexpensively. And this is a very emerging area. The earliest work on this is only two or three years old.
Chris Kresser: Right.
Dale Harrison: But the CRISPR diagnostics has the ability to basically break the “no free lunch” rule, where you can have something that’s cheap, fast, very sensitive, and very specific. And so there are products that are developed right now that are very interesting that would be a single-use disposable over-the-counter product. There [are] already been a number of press releases done by companies that are working on this. So GlaxoSmithKline[’s] homecare division is very involved in this. There are a number of companies that are CRISPR specialists that are working on these sorts of tests. But the goal would be to have a cheap, disposable, five-minute, over-the-counter test that you go down to CVS and buy, and be able to pay 10 or 20 bucks for, and have the company still be able to do these profitably. I don’t know if we can get there yet. There’s both a lot of engineering challenges and a lot of biochemistry challenges that still have to be worked out. But I think it’s [a] very promising technology, and it has the ability to potentially supplant all known diagnostic testing technology.
Chris Kresser: Wow. I mean, yeah, that’s pretty phenomenal. And like you said, possibly breaking the free lunch rule. And that sounds like a test that could make a significant impact on the pandemic if it actually is possible to realize that in time.
Dale Harrison: Yeah. And we’re probably six months out from the first of these going to market. It’s hard to know. There’s a lot of money being poured into this. So there’s sort of no shortage of resources and talent working on it, but these things take time. But one of the things I think you’re going to see is that in 10 years from now, there will be very little in the way of PCR in the clinical environment. And you’re probably going to see two different types of tests. You’re going to see these CRISPR tests that are going to be your sort of every day, almost go-to point-of-care. You’ll be able to have a small device that you can run immediately, wherever the patient is. And then you’ll see these metagenomic next-generation sequencing tests that are able to basically take one sample and find everything, all pathogens, within the sample, which is a very powerful technique. There’s nothing quite like that currently on the market.
Chris Kresser: No, as a clinician, I can tell you that I will be waiting with bated breath for that day to come. Dale, I’m so sorry, you’ll have to wrap it up now. We’ve got to move on. But thank you so much for doing this. It’s been incredibly helpful. And where can people find out more about your work? You have an excellent Substack that I follow, and then I understand you’ve recently launched your own podcast.
Dale Harrison: Yes. So I have a newsletter on Substack that’s called “COVID Updates.” And the way to reach it is just DaleWHarrison.substack.com.
Chris Kresser: We’ll put a link to that in the show notes as well as to the podcast.
Dale Harrison: Okay.
Chris Kresser: And where do people find your podcast?
Dale Harrison: So the podcast, which I’ve just launched and there’s only one episode up, but there’ll be more shortly, is called The Business of Life Science. And it’s a much more specialized podcast that is basically looking at the commercial development side of the biotech industry. I do a lot of work outside of the lab that is important for these biotech companies to be able to know what products to create, and then how to bring those products to market. So it’s called The Business of Life Science and there [are] new episodes coming soon.
Chris Kresser: Yeah, we’ll put a link to that, as well, in the show notes for anybody that might be interested. So thank you, again, Dale. And yeah, we’ll have you back in maybe six, nine months’ time, if the CRISPR technology is getting closer to being realized, and we can talk a little bit more about that. But I really appreciate you taking the time to do this, and I think it’s going to help a lot of folks who are thinking about the holidays and how to plan for that.
Dale Harrison: Well, thanks, I really appreciate the opportunity to come on and talk. It’s obviously a very interesting subject with a lot of detail. And I think there’s a lot of misunderstanding as to how these things work, and certainly how they work in the real world. And unfortunately, I think a lot of people are making incorrect decisions.
Chris Kresser: Uninformed decisions.
Dale Harrison: Uninformed decisions, somewhat, because they just don’t understand the details and the complexity involved in this area.
Chris Kresser: Yeah, the limitations. All right. Well, thanks, everybody, for listening. Keep sending your questions in [to] ChrisKresser.com/podcastquestion and we’ll see you next time.