Since specificity/AAA reflects the ability to accurately identify negative controls more prevalent than patient samples, the 95% CI of these metrics tends to be narrower than for sensitivity/PFA, indicating the proportion of positive cases a test may have. Nor is it possible to determine, on the basis of these statistics, that one test is better than another. Recently, a British national newspaper published an article about a PCR test developed by Public Health England and the fact that it did not agree with a new commercial test in 35 samples out of 1144 (3%). For many journalists, this was obviously proof that the PHE test was inaccurate. There is no way to know which test in any of these 35 disagreements is right and which one is wrong. We simply do not know the actual state of the subject in compliance studies. Only by further examining these discrepancies would it be possible to identify the reason for these discrepancies. The clinical relevance of a test depends on the prevalence of the detected disease. The confidence one can have in a positive or negative result in a given clinical population is quantified by the positive predictive value (APP) or negative predictive value (APN) of a test. PpV and NPV of a trial with a robust and specific sensitivity/PFA/NPA change depending on the prevalence of the population. CLSI EP12: User Protocol for Evaluation of Qualitative Test Performance Protocol describes the terms Positive Percent Agreement (AAE) and Negative Percent Agreement (APN). If you need to compare two binary diagnoses, you can use an agreement study to calculate these statistics. In this scenario, ground Truth positive patients and Ground Truth negative patients are also likely to be misclassified by the comparator.
(A) Compparator without misclassification, which perfectly represents ground truth for 100 negative and 100 positive patients. (B) Apparent performance of the diagnostic test based on the comparator`s poor ranking rate. The error bars describe 95% empirical confidence intervals by median, calculated over 100 simulation cycles. The actual test power is displayed when fp and FN rates are 0%. The notions of sensitivity and specificity are appropriate in the absence of a misclassification in the comparator (FP rate = FN rate = 0%). The terms Positive Percent Agreement (AAA) and Negative Percent Agreement (AAE) should replace sensitivity or specificity when the comparator is known to contain uncertainty. The FDA has issued nine COVID-19 antibody tests for Emergency Clearance (AEE). The IFU (Instructions for Use) document for each test states its sensitivity and specificity in the form of a positive percentage of compliance (PFA) or a negative percentage of compliance (NPA) with an RT-PCR (Transcription Polymerase Chain Reaction) test and 95% confidence intervals (CI) for each value. Since there is not yet such a standard for COVID-19, serological test designers have reported sensitivity and specificity as a positive predictive match (PFA) or negative predictive coincidence (APN) with RT-PCR tests on patients` nasal smears. . . .