Big Data Analytics: Methodology Evaluation and Development for Gene Set Analysis & Spatial Cluster Detection in Administrative Health Data

This study focuses on two key domains of big data analytics: high-dimensional gene expression data analysis and population-based administrative health data analysis. Analyzing large-scale biomedical datasets is increasingly common in modern biomedical research, where the exponential growth of data generation creates both outstanding opportunities for scientific discovery and significant analytical challenges. In the genomic analysis domain, this research focuses on evaluating and enhancing statistical gene set analysis (GSA) methods for gene expression data. This investigation involves a comprehensive evaluation of the Linear Combination Test (LCT) performance using different covariance matrix estimators, including ridge, graphical lasso, and adaptive lasso methods. This investigation is particularly crucial for high-dimensional gene expression datasets where the number of variables (genes) vastly exceeds the number of observations (samples), making the calculation of covariance matrices computationally challenging and often resulting in singular or ill-conditioned matrices. The study methodologically assesses LCT stability and accuracy across these various estimators, with special attention to unbalanced study designs that frequently occur in biomedical research due to practical constraints in patient recruitment and sample collection. The findings demonstrated that while shrinkage estimation performs optimally in balanced designs across all correlation levels, ridge estimation provides superior performance in unbalanced scenarios with better Type I error control compared to shrinkage's inflated error rates under unbalanced conditions. Additionally, within the genomic analysis domain, the research develops a novel GSA approach called GSHAPA for single-cell RNA sequencing (scRNA-seq) data. This method strategically combines Random Forest (RF) algorithms with SHapley Additive exPlanations (SHAP) values to address critical limitations in GSA analysis: traditional statistical methods fail to capture non-linear gene interactions in sparse scRNA-seq data, while existing machine learning approaches provide high performance but lack the explainable artificial intelligence integration needed to understand which specific genes drive pathway activity in individual cells. GSHAPA demonstrated superior precision, computational efficiency (5× faster), and significantly lower false positive rates compared with Gene Set Enrichment Analysis (GSEA) as a baseline method. The method also uniquely provides pathway scores for individual samples. Unlike current approaches that force researchers to choose between interpretable but poorly-performing statistical methods or high-performing but black-box machine learning models, GSHAPA bridges this gap by delivering both effective non-linear modeling capability and interpretable insights. The second research domain focuses on the application and comparison of existing spatial scan methods using population-based administrative health data to detect geographic clusters of croup cases. Cases were identified using emergency department (ED) visit and physician claims data from April 2017 to March 2023 in Alberta, Canada. Yearly analyses were conducted on each data source separately (univariate) and for both data sources simultaneously (multivariate) to detect overlapping or distinct geographic patterns. This methodological comparison demonstrated the complementary value of univariate and multivariate approaches for spatial cluster detection, particularly when analyzing multiple healthcare data sources. Multivariate spatial scan statistics identified areas with consistently higher risk across both data sources, while also detecting unique clusters where only one data source showed significant patterns. The analysis revealed consistent geographic clustering across Alberta, with distinct utilization patterns between ED visits and physician claims data reflecting different healthcare-seeking behaviours. This research explores big data analytics through methodological evaluation and development in genomic analysis, and through application and comparison of advanced statistical methods in population health data analysis. The key findings develop novel approaches for genomic analysis and provide a methodological comparison of spatial cluster detection methods, contributing to analytical capabilities for large-scale biomedical data analysis.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.