Bias and privacy in AI's cough-based COVID-19 recognition
Bibliographic record
Abstract
We read with interest the Comment by Coppock and colleagues,1Coppock H Jones L Kiskin I Schuller B COVID-19 detection from audio: seven grains of salt.Lancet Digit Health. 2021; 3: e537-38Summary Full Text Full Text PDF PubMed Scopus (21) Google Scholar in which the authors express their thoughtful opinion about several simultaneous works by independent research groups worldwide (eg, Massachusetts Institute of Technology, National Research Council of Canada, University of Cambridge, and Swiss Federal Institute of Technology Lausanne). One of these works was our own; a pioneering, multicentre, international study2Andreu-Perez J Perez-Espinosa H Timonet E et al.A generic deep learning based cough analysis system from clinically validated samples for point-of-need COVID-19 test and severity levels.IEEE Trans Serv Comput. 2021; (published online Feb 23.)https://doi.org/10.1109/TSC.2021.3061402Crossref Scopus (42) Google Scholar with a clinically validated dataset of forced coughs alongside quantitative RT-PCR from participants who physically attended a test centre. Participant control was performed on site by health personnel at the partner health centres that contributed to this study.2Andreu-Perez J Perez-Espinosa H Timonet E et al.A generic deep learning based cough analysis system from clinically validated samples for point-of-need COVID-19 test and severity levels.IEEE Trans Serv Comput. 2021; (published online Feb 23.)https://doi.org/10.1109/TSC.2021.3061402Crossref Scopus (42) Google Scholar The Comment1Coppock H Jones L Kiskin I Schuller B COVID-19 detection from audio: seven grains of salt.Lancet Digit Health. 2021; 3: e537-38Summary Full Text Full Text PDF PubMed Scopus (21) Google Scholar focuses on human audio biometrics in general, albeit eight out of ten referenced works made use of coughs as their audio source. The use of cough sounds to detect respiratory system abnormalities had been investigated for at least 3 years before the COVID-19 pandemic.3Botha GHR Theron G Warren RM et al.Detection of tuberculosis by automatic cough sound analysis.Physiol Meas. 2018; 39045005Crossref PubMed Scopus (57) Google Scholar Cough analysis is a particular form of audio biometrics, and other forms of audio biometrics cannot be discussed interchangeably. Although person authentication via speech has been achieved to some degree; to date, recognising an individual with ease in a large database solely by the sound of their cough is inconclusive. The same applies to inferring emotional traits. The prospect of re-identifying patients from cough sounds raises several concerns. Will participants whose data is used in developing these pre-screening systems be able to benefit from it in the future in an unbiased manner? Additionally, how can personal biometric data be made public, ensuring that it always remains non-identifiable? On the basis of the current health context, official calls have been made not to trivialise the privacy and protection of patient data.4WHOJoint statement on data protection and privacy in the COVID-19 response.https://www.who.int/news/item/19-11-2020-joint-statement-on-data-protection-and-privacy-in-the-covid-19-responseDate: Nov 19, 2020Date accessed: August 2, 2021Google Scholar, 5Pierucci A Walter J-P Joint Statement on the right to data protection in the context of the COVID-19 pandemic.https://www.coe.int/en/web/kyiv/-/joint-statement-on-the-right-to-data-protection-in-the-context-of-the-covid-19-pandemicDate: May 14, 2020Date accessed: August 2, 2021Google Scholar Subsequent research initiatives from public bodies must now help to enable inclusive research clusters and secure collaborative infrastructures in the domain of audio biometrics. Our training, development (validation), and holdout (test) sets do not contain data from the same participant (ie, they are participant-independent) to avoid spurious discerning patterns that could compromise classification scores.2Andreu-Perez J Perez-Espinosa H Timonet E et al.A generic deep learning based cough analysis system from clinically validated samples for point-of-need COVID-19 test and severity levels.IEEE Trans Serv Comput. 2021; (published online Feb 23.)https://doi.org/10.1109/TSC.2021.3061402Crossref Scopus (42) Google Scholar However, a general assumption is that a competent biometrics classifier should maximise the distinction between divergent patterns within participants, while minimising that of different participants within the same class. The inclusion of divergent observations (negative and positive) from the same participant, in the training set only, could help to satisfy this assumption; however, this effect requires further study. To conclude, there is hope that rapid, point-of-need pre-screening for COVID-19 via forced cough sounds captured from smartphones or portable devices could be feasible in the short term. Online interventional studies are now necessary to explore the potential of this novel technology and to evaluate its real-life performance, health impact assessment, end-user utility, and acceptability. We declare no competing interests. COVID-19 detection from audio: seven grains of saltDigital mass testing for COVID-19 via a mobile phone application could be made possible through machine learning and its ability to identify patterns in data. COVID-19 appears to confer unique features in the audio produced by infected individuals,1 and machine learning COVID-19 detection from breath, cough, and speech audio recordings has yielded promising results.2–4 In this critique, we present seven major issues with this research and argue that further investigation is needed before conclusions about the detectability of COVID-19 from audio can be made. Full-Text PDF Open AccessBias and privacy in AI's cough-based COVID-19 recognition – Authors' replyWe thank Humberto Perez-Espinosa and colleagues for their constructive points regarding our Comment,1 which raised concerns over the work on COVID-19 detection from bioacoustic recordings. We take this opportunity to note that the study by Perez-Espinosa and colleagues2 represented one of the superior COVID-19 audio datasets that were collected. Although the study was not completely free from the "seven grains of salt" detailed in our Comment,1 it was large scale, validated by quantitative RT-PCR, and the participants were blinded. Full-Text PDF Open Access
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".