Data and source code at https://framagit.org/saji_nh/jp_pcr
The source code, data, and images may be freely used without permission. However, the original data for PCR tests and related statistics were compiled by Toyo Keizai, and their efforts should be acknowledged by citing their data repository https://github.com/kaz-ogiwara/covid19.
Questions and comments may be addressed to Saji N. Hameed by email (saji'at-mark'u-aizu.ac.jp)

Saliva based PCR tests

Remarks - A tiny sample size, combined with the apparent mild nature of the sickness makes this study a shaky premise for switching the PCR testing policy

Background

  • The Japanese Ministry of Health approved saliva based PCR tests on June 2, on the condition that it be applied within 9 days of the appearance of symptoms. (Article in the Asahi Shimbun)

  • Till this date, people had to be screened for fever exceeding 37.5C for 3 consecutive days. PCR tests on nasal swabs were conducted only on people who passed this screening (for comparison, this is a best practice for other situations like an HIV diagnosis). (Ref1) (Ref2).

  • On July 17, the ministry lifted the restriction that the saliva based PCR test can only be done on symptomatic people. Thus, the tests can now be applied on anyone regardless of their health status. (Article in the Asahi Shimbun)

Premises for the first decision

The first decision was based on a study conducted at the Self Defense Force Central hospital. It was carried out under the auspices of the Emerging/re-emerging infectious disease and vaccination policy research project. This report is available online, and here

A short summary and discussion is given below.

Methods

  1. The researchers collected 88 saliva samples from a number of people (number not specified) confirmed positive by a PCR test. No further information is available on the status of their health during the time of diagnosis. Nor is there any information on the symptoms experienced or the severity of the symptoms.

  2. The saliva samples were collected within 14 days of onset of symptoms and kept frozen till testing.

  3. These patients had been tested with the PCR-based procedure using samples collected with nasal swabs. It is not clear whether the nasal swabs were collected on each day and the PCR repeated daily, or whether the nasal swab based PCR was conducted only once.

  4. The saliva samples were tested on two standard PCR machines, 3 Direct PCR machines, and a different methodology called LoopampEXIA.

Findings

On each day after onset of symptoms, saliva samples were tested on the 6 different machines mentioned above. The positive matches between the two kind of samples (nasal swabs vs saliva) were checked.

Based on these tests, the investigators claim that saliva-based testing produced identical results to that of nasal swabs, if the tests were conducted within 9 days of onset of symptoms.

Saliva expt1

Discussion

There are two major problems with the experiment. Both are significant.

A tiny sample size

The number of samples is too little for the experimental results to be of any value in public health policy.

  1. The number of samples for each day of analysis ranges from one (1) on day two and day 13 to seventeen (17) on day 6. This is quite a departure from standard scientific practice. Given the apparent gravity of the situation, one expects sample numbers to be of the order of 100s if not 1000s.

  2. There are no controls in this experiment to evaluate the accuracy of the tests, in particular the fraction of false positives and false negatives. As noted in the discussion below, many of the patients seem to be mildly sick and thus have a low viral load. PCR tests are known to be unreliable in these cases and likely create false positives (Ref1) (Ref2). Further, at low viral loads, PCR machines operate at the higher limit of amplification (Ct cycle greater than 32 = 2 Billion times amplification), increasing the likelihood of unreliable results.

  3. The sample sizes on each day suggest that these samples are from mildly sick people. For example, there are only two samples on day 1, and one on day 2. The figure shown above records the number of samples entering and leaving the tests each day. On day 2, it is seen that the sample number is reduced by 1 – Did one person get better the second day of the experiment and leave the hospital? Or did this person not test PCR positive on day 2? We cannot know because such details are not reported.

  4. On day 3, four more samples are analysed, while on day 4 the sample number reduced to three. We interpret this as three new patients being tested on day 3, but one probably discharged on day 4.
    The number of samples are the largest (seventeen, 17) on day 6, and drops to eleven (11) on the next day. This may mean that most people recovered on the 6th day, which in turn implies that all these people were mildly sick.

No controls to determine false positives

It is apparent that the tests are carried out on mildly ill people, most of whom are seen to leave the experiment on or before day 7 (a week after onset of symptoms). According to viral infection theory, mildly sick people have a lower viral load, since the innate immune system could stop viral replication and prevent sick conditions from exarcerbating. This further implies that the PCR machines are operating at high amplification cycles (Ct > 32), thereby producing invalid results, possibly false-positives. Thus, from the details reported on the experiment, these people need not necessarily be infected with the presumed SARS-CoV2 virus; they could have well been infected by one of the 'regular', known human coronaviruses.

Recommendations

Given the apparent gravity of the situation, it is highly recommended that the relevant Japanese authorities revert to the original scheme for COVID diagnosis – namely, screening for associated symptoms, followed by PCR on nasopharyngeal swabs.

We further recommend that experiments be conducted to evaluate false-positives associated with this diagnostic with a sample size that is statistically significant.