By Amy Robinson, Jim Waldo
On Dec. 14, the United States administered the first doses of a coronavirus vaccine to health care workers. The U.S. government has set an aggressive timeline aimed at vaccinating the majority of Americans by June 2021. The end is in sight, but we still have a long way to go. Controlling the spread of the pandemic in the intervening months remains an urgent priority. Currently, many states are still chasing the promise of coronavirus exposure notification apps. Last week, California rolled out CA Notify, joining 18 other states as well as Guam and the District of Columbia in deploying the Exposure Notification System developed by Apple and Google (Apple | Google ENS). But it is essential to recognize that the inherent technical limitations mean that contact-tracing apps, at best, play a relatively small public health role and, at worst, risk doing more harm than good.
Effective exposure notification tools must minimize both false positives and false negatives.
In mid-October, thousands of English and Welsh citizens using a National Health Service (NHS) app built with the Apple | Google ENS received phantom alerts that they had potentially been exposed to the coronavirus. While the NHS later fixed the bug, the incident caused wide-scale confusion, fear and frustration. These sorts of false positives are not harmless. They can overburden limited testing capacity, as concerned citizens seek unnecessary tests. And false positives increase the likelihood users will ignore future exposure warnings and thus fail to obtain needed testing.
False negatives are also problematic. A false negative occurs when a person who was exposed to the coronavirus does not receive a notification. If asymptomatic and unaware of a possible infection, the individual could spread the virus further. Medical experts have dubbed such oblivious asymptomatic transmission “the Achilles’ heel” of the pandemic, especially as the holiday season strains social distancing compliance. Contact-tracing apps that do not alert people of actual exposure are not assisting in the larger public health response and may create a false sense of security among individuals who use them.
A digital contact-tracing tool that doesn’t sufficiently minimize false positives and false negatives poses real risks. As more and more states implement this technology in the final months of the pandemic—a period public health experts warn will be the most difficult and deadly yet—we must understand the accuracy of the underlying technology. Unfortunately, Bluetooth technology simply cannot provide location information that is sufficiently granular or consistent to produce reliable digital contact-tracing apps.
Most digital contact-tracing apps rely on Bluetooth low energy, rather than GPS, technology. Early in the pandemic, location tracking based on smartphones’ GPS technology was proposed as a potential aid to contact tracing. For example, projects showing the possible spread of the virus by tracking the movements of college students during spring break gave the impression that individual locations could be tracked using the same GPS technology. But while GPS tracking on phones can track students dispersing after spring break, it is not accurate enough for the precise measurements needed in contact tracing. Contact tracing requires a resolution that can identify co-location within 6 feet (approximately 2 meters) for a minimal duration of time (currently 15 minutes). But GPS doesn’t function at all in some buildings, including the kinds of indoor spaces where contacts are the most likely to cause viral spread. And the resolution of even optimally functioning GPS in a smartphone is only 7 to 13 meters in an urban environment. GPS accuracy is also affected by factors such as time of day, time of year and WiFi signal strength. Ultimately, phone GPS accuracy is two to three times lower than what would be needed for effective contact tracing.
Most mobile contact-tracing apps have attempted to avoid the shortcoming of GPS by using the Bluetooth networking present in phones. Bluetooth is a near-field networking technology and thus would seem more appropriate for the finer-grained location information necessary for this kind of application. The problem is that even Bluetooth likely doesn’t provide the granularity required by digital contact tracing to estimate distance.
To begin, Bluetooth does not measure distance directly. Instead, the Apple | Google ENS uses the simple idea that a signal becomes weaker the farther away it is. Therefore, one can use the attenuation, or reduction in signal, to infer distance. A weak received signal strength indicator (RSSI), as approximated by the power measurement of dB, would mean that you are farther away; a strong RSSI indicates you are closer. While conceptually simple, this method is far from simple in practice.
RSSI does not provide a clear and consistent measure of distance for a variety of reasons. It naturally fluctuates by as much as 5 dB even in controlled settings. It also fluctuates by device; a weak RSSI could be caused by the phone emitting a weak signal, or the other phone not having a sensitive receiver. In tuning their recently released TraceTogether app, Singapore researchers found that devices varied by as much as 20 dB in highly controlled settings due to different Bluetooth hardware, antenna layout, and even operating system configurations such as battery saving features. A difference of 20 dB translates roughly to a factor of 10 in terms of distance measured. In practice, that could be the difference between measuring 1 meter and 10 meters.
Apple | Google ENS can tune the algorithm based on these known variations of supported devices. Apple and Google explained that they can even extend calibrations to currently unsupported devices based on averaging associated devices. However, the companies themselves note that this is “a very coarse method of calibration” intended to serve as a “stopgap” until data from more devices is available. Apple and Google provide a confidence level of low, medium or high for each calibration. Of the nearly 12,000 devices in the most recent calibration file, 90 percent are listed as low confidence.
Most importantly, RSSI fluctuates significantly depending on real-world situations. Bluetooth low energy signals, which are a type of radio wave, can absorb into or reflect off of various surroundings. For instance, a person’s body can absorb Bluetooth radio signals, making the signal look much weaker and therefore that the devices are much farther away than they are. Therefore, a person simply rotating their body can alter the signal strength by as much as 20 dB. Changing the position of a phone in a purse just 1 meter away from another phone can also alter the signal strength by 10-20 dB. Even the presence of carpet and furniture could make the phones seem farther apart than they are. Signal absorption can therefore increase the level of false negatives since the ENS would not register that a person came in contact with an infected person.
These real-world fluctuations can also increase the incidence of false positives. Especially in indoor settings, signals can reflect off of metal and other reflective surfaces, making them appear much stronger and therefore much closer. Signals could even reflect off of wet pavement. In replicating a subway car, researchers found that the RSSI increased as the phone moved from 2 meters to 4 meters away. Therefore, the ENS would register that the people were getting closer when in reality they were moving farther apart.
Bluetooth signals can also propagate easily through certain types of walls. While signals have a difficult time passing through blockwork or cement walls, they easily pass through stud walls. Stud walls are generally used to separate rooms, whereas blockwork is generally used to separate adjacent houses and apartments. Therefore, the risk of a false positive may be low among neighbors, but much higher for individuals in office spaces. RSSI cannot capture the real-world conditions, such as a dividing wall, that drastically change the risk of infection.
Taking into account the fluctuations and real-world scenarios, RSSI is a difficult and inconsistent measurement. In trying to determine if two people came within 1.5 meters of one another, one study ultimately concluded that RSSI yields an error rate of around 50 percent, even when the two people were next to each other for 10 minutes. Apple and Google acknowledge in their documentation that “Attenuation is a very noisy proxy of distance.” If a contact-tracing app cannot reliably determine whether you were within 1 meter or 10 meters of an infected person, it cannot be anything more than a marginal public health tool.
On top of the difficult task of approximating distance from Bluetooth, there is the challenge of configuring the app. The Apple | Google ENS takes into account several different factors, such as distance and time, in calculating someone’s exposure risk value (ERV) as measured by meaningful exposure minutes (MEMs). The ENS then determines what level of MEMs triggers a notification. Newer versions of the app allow for state public health authorities to modify the exposure thresholds based on things like exposure to an infected individual on the first day of symptoms versus 14 days after the onset of symptoms. But these thresholds necessarily incorporate distance and whether the contact was immediate, near, medium or other.
Apple’s example defines “immediate” as less than 40 dB, “near” as between 40 dB and 53 dB, “medium” as between 53 dB and 60 dB, and “other” (that is, far away) as greater than 60 dB.
Even though state public health authorities can customize threshold Bluetooth attenuations, Apple’s example reveals the granularity necessary for reliably calculating someone’s exposure risk. Apple uses intervals of just 7 dB to set critical distance categories. Only 20 dB separates a serious “immediate” exposure from the “other” exposure that is supposedly far enough away so as not to be factored into the calculation. However, something as minor as the positioning of a phone could result in a change of 20 dB.
The benefit of the customization model is that it allows state authorities to decide whether to err on the side of false negatives or false positives. A number of epidemiologists believe setting conservative estimates—and thus reducing the incidence of false positives, though increasing the incidence of false negatives—is the better approach. In discussing the Swiss app—set to notify someone within 2 meters of an infected person for at least 15 minutes—epidemiologist Marcel Salanthé commented that “if somebody gets an exposure notification, we will feel damn sure it’s actually been a contact.” But considering the fluctuations of Bluetooth’s precision as a distance proxy in real-world scenarios, even that statement appears overly confident.
It is tempting to presume that any measures to improve contact tracing are worth pursuing. After all, if even a few asymptomatic individuals are alerted to their risk of infection, that’s a good thing. But in a resource-constrained environment, it is critical to evaluate the entire picture, which includes the kind of false positive episodes that occurred in the U.K. Even if well intentioned, measures that risk increasing public confusion and government distrust should be approached with extreme caution. Unfortunately, a digital notification system that presents a high number of false positives or false negatives threatens to do just that. As Bluetooth technology currently stands, digital contact-tracing tools risk undermining rather than enhancing current contact-tracing efforts. Without more precise location data, it is best that we stick to the traditional tools of contact tracing for now.
No comments:
Post a Comment