Automated facial analysis of infant pain expressions: progress and future directions

PainChek Infant, a mobile application (app) based on automated facial evaluation and analysis for assessing procedural pain in infants, is a welcome addition to recently developed automated tools for coding infant facial expressions. In The Lancet Digital Health, Kreshnik Hoti and colleagues document the strong psychometric properties and performance of PainChek Infant compared with the widely used manual Neonatal Facial Coding System Revised (NFCS-R).1Hoti K Chivers PT Hughes JD Assessing procedural pain in infants: a feasibility study evaluating a point-of-care mobile solution based on automated facial analysis.Lancet Digit Health. 2021; (published online Sept 1.)https://doi.org/10.1016/S2589-7500(21)00129-1Summary Full Text Full Text PDF PubMed Google Scholar, 2Grunau R E Craig KD Neonatal facial coding system revised: training manual. University of British Columbia, Vancouver, Canada2010Google Scholar Both assessments are based on identifying a small number of facial muscle actions (six in PainChek Infant, five in NFCS-R) present in each of four 10 s video segments of infants undergoing an inoculation procedure (baseline, preparation, during, and recovery). PainChek Infant scores were highly correlated with NFCS-R scores and with the Observer Administered Visual Analogue Scale (ObsVAS). The reported results of the study are impressive, but additional work is needed to address a number of issues, discussed below. One strength of PainChek Infant is the use of the Baby FACS facial action coding system for infants and young children,3Oster H. Baby FACS: facial action coding system for infants and young children. Monograph and coding manual. 2010. New York, NY, USA.Google Scholar a fine-grained, anatomically based manual coding system for identifying the facial muscle action units (AUs) underlying infant facial expressions. Baby FACS coders can precisely and objectively specify the component AUs and AU configurations and their intensity, making it possible for other researchers to compare, replicate, and expand on their findings. Because infants' distress expressions are complex and labile, potentially involving as many as 12 co-occurring AUs and rapid changes in facial configurations, comprehensive manual coding with Baby FACS is painstaking and time consuming. Like most automated facial expression analysis tools, PainChek Infant is designed for rapid identification of pain expressions based on detection of a small number of facial muscle actions.4Zamzmi G Kasturi R Goldgof D Zhi R Ashmeade T Sun Y A review of automated pain assessment in infants: features, classification tasks, and databases.IEEE Rev Biomed Eng. 2018; 11: 77-96Crossref PubMed Scopus (29) Google Scholar A potential limitation is that the authors do not report full-face expressions of co-occurring AUs, which are crucial for research on the temporal dynamics of infant facial expressions, as shown in a 2009 study5O'Neill MC Ahola Kohut S Pillai Riddell R Oster H Age-related differences in the acute pain facial expression during infancy.Eur J Pain. 2019; 23: 1596-1607Crossref PubMed Scopus (7) Google Scholar on changes in AU configurations over the first minute following inoculation and age changes from 2 to 12 months. PainChek's reliance on a small number of AUs could result in both false negative and false positive errors, because three of the six AUs PainChek is trained to detect can occur in both hedonically positive and negative expressions, depending on the other AUs present. Such errors are unlikely in the first 10–15 s following inoculation but could occur later in the minute following inoculation or in older infants, reflecting efforts to regulate negative emotion.5O'Neill MC Ahola Kohut S Pillai Riddell R Oster H Age-related differences in the acute pain facial expression during infancy.Eur J Pain. 2019; 23: 1596-1607Crossref PubMed Scopus (7) Google Scholar A 2018 review4Zamzmi G Kasturi R Goldgof D Zhi R Ashmeade T Sun Y A review of automated pain assessment in infants: features, classification tasks, and databases.IEEE Rev Biomed Eng. 2018; 11: 77-96Crossref PubMed Scopus (29) Google Scholar emphasises the importance of coding pain intensity for assessing the severity of injury or illness and differences in pain responses related to age, ethnicity, or medical conditions. PainChek Infant does not directly assess the intensity of infants' pain elicited distress. The authors instead report a higher number of individual target AUs and higher pain scores (average 5–6 out of 6), in the 10 s inoculation segment than in the other three segments, suggesting greater distress. However, this approach treats all target AUs as interchangeable, potentially ignoring key features of the most and least intense distress expressions and clinically relevant individual differences illustrated in the 2009 study on changes in AU configurations.5O'Neill MC Ahola Kohut S Pillai Riddell R Oster H Age-related differences in the acute pain facial expression during infancy.Eur J Pain. 2019; 23: 1596-1607Crossref PubMed Scopus (7) Google Scholar PainChek Infant developers acknowledge the absence of convincing evidence for the specificity of pain-elicited facial expressions distinct from cry faces shown in non-pain-eliciting contexts, as reviewed by a previous study.6Ahola Kohut S Pillai Riddell R Does the neonatal facial coding system differentiate between infants experiencing pain-related and non-pain-related distress?.J Pain. 2009; 10: 214-220Summary Full Text Full Text PDF PubMed Scopus (33) Google Scholar As acknowledged in the paper use of PainChek Infant should be viewed in the context of verifying and quantifying pain when a source of pain is known or suspected. This absence of evidence is not because infants are generally incapable of showing differentiated facial expressions, as shown in a study of infants aged 2 h and their differential facial responses to sweet, sour, salty, and bitter tastes.7Rosenstein D Oster H Differential facial responses to four basic tastes in newborns.Child Dev. 1988; 59: 1555-1568Crossref PubMed Google Scholar A study involving both Baby FACS coding and automatic detection of nine AUs8Hammal Z Chu WS Cohn JF Heike C Speltz ML automatic action unit detection in infants using convolutional neural network.Int Conf Affect Comput Intell Interact Workshops. 2017; 2017: 216-221PubMed Google Scholar found that infants showed clearly differentiated pleasure and frustration expressions. However, comprehensive Baby FACS coding in a 2007 study9Camras LA Oster H Bakeman R Meng Z Ujiie T Campos J Do infants show distinct negative facial expressions for fear and anger? Emotional expression in 11-month-old European-American, Chinese, and Japanese infants.Infancy. 2007; 11: 131-155Crossref Scopus (41) Google Scholar failed to show the specificity of infants' distress expressions in fear versus frustration-eliciting situations. Other researchers have also failed to show differentiated facial expressions of specific negative emotions. One possible explanation for the apparent absence of specific, differentiated facial expressions of physical pain and intense distress from other causes is that intense cry faces and vocalisations are evolutionary adaptations that serve as general alarm calls, demanding immediate attention from caregivers to identify and remove the source of pain, threat, or anxiety. This view is supported by evidence that intense physical and psychological pain are represented in the same brain area, the dorsal subdivision of the anterior cingulate cortex, in infants and adults.10Eisenberger NI Lieberman MD Why rejection hurts: a common neural alarm system for physical and social pain.Trends Cogn Sci. 2004; 8: 294-300Summary Full Text Full Text PDF PubMed Scopus (771) Google Scholar In summary, PainChek Infant can be a valuable, time saving tool for assessing procedural pain in infants, especially in the first 10–15 s following inoculation. Addressing the issues noted above will require detailed observation by human eyes of variations in pain-elicited responses and changes over time, as well as sophisticated automated facial evaluation and analysis instruments. I declare no competing interests. Assessing procedural pain in infants: a feasibility study evaluating a point-of-care mobile solution based on automated facial analysisPainChek Infant's use of automated facial expression analysis could offer a valid and reliable means of assessing and monitoring procedural pain in infants. Its clinical utility in clinical practice requires further research. Full-Text PDF Open Access

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.