Uncovering Concerns of Citizens Through Machine Learning and Social Network Sentiment Analysis
Bibliographic record
Abstract
Artificial Intelligence and Machine Learning (AI/ML) as analytical tools can be applied across multiple social domains. Thus, these tools are being deployed in several ways to address societal issues and concerns for “social good”. For instance, AI/ML has applicable use cases for crisis response, economic empowerment, educational demands, environmental challenges, equality and inclusion, health and hunger, and security and justice. In this work, we seek to explore the power and capability of AI/ML in understanding citizens’ engagement, which can improve governance and smart city deployment. Specifically, we studied the views expressed by online users about the city of Saskatoon in Canada. The analyzed views have become a value chain that community leaders can use to improve the governance structure of the city. In the study, we extracted 114,390 comments from Reddit (i.e., Saskatoon subreddit posts) between January 1, 2019, and September 20, 2023, to discover topics to highlight citizens’ concerns. We compare the performance of three major topic models, namely, Latent Dirichlet Allocation (LDA), Non-negative Matrix Factorization (NMF), and BERTopic with a K-means clustering algorithm in the discovery of topics from the collected Reddit comments. The BERTopic with the K-means clustering algorithm achieved the highest coherence score of approximately 0.64 in the extraction of 25 topics from the dataset. Our findings showed that BERTopic can discover coherent and diverse topics compared to LDA and NMF. We found 12 underlying themes by merging related topics. Also, we leveraged SiEBERT (a pre-trained transformer model), 4 supervised ML models, and VADER (a lexical sentiment analysis classifier) to identify the sentiments expressed in each theme. The SiEBERT model outperformed the other sentiment classifiers with an accuracy of 89% in the prediction of sentiments. The research discovered factors for smart city engagement such as Housing and Facilities, Education, Downtown Development, Tourism and Entertainment, Policing, Healthcare, Online Community, and Cost.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.007 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.005 | 0.003 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".