Ivor Cribben and Anastasiou Andreas’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

We extend our congratulations to the authors for their innovative contribution. Here, we address three key points: change-point labelling, handling imbalanced datasets, and computational complexity. Change-point labelling in datasets has many flaws and challenges. The primary flaw is the subjectivity inherent in the task as it often relies on human judgement. This subjectivity introduces biases and inconsistencies. One challenge is the lack of a universally accepted standard for change-point labelling unlike other machine learning classification problems. Hence, it is difficult to compare results across studies, hindering reproducibility and reliability. A further issue is that change-point labelling can be a time-consuming and labour-intensive task, especially for large and complex time series datasets. This process often requires domain expertise and can be impractical for real-time or high-frequency data analysis. Imbalanced class distributions in datasets are another issue. Change-points are often rarer than normal instances, but imbalanced datasets can lead to skewed evaluation results, with methods prioritizing the majority class and failing to effectively detect true change-points or managing an excessive rate of false positives. We wonder whether the authors explored examples of imbalanced data, specifically those involving significant changes in approximately half of the dataset (N/2). It is important to underscore that addressing the labelling and imbalance challenges is pivotal for change-point methods that rely on training neural networks. The sample size used for training neural networks plays a crucial role in determining the model’s performance and generalizability. An excessively large sample size might lead to increased computational costs and training time without significant gains in performance after a certain point. In the first step of the proposed algorithm, there is the necessity of training a neural network using a considerable sample size. A discussion on this topic could convince the reader that the proposed method can be extended to an online framework (as discussed in Section 7). Apart from accuracy, and especially in high-frequency data, online change-point detection methods have to be very computationally efficient in order for users to act promptly. The computational complexity in the simple univariate setting is also crucial to understand extensions to practically meaningful adaptations of the algorithm to multivariate, possibly high-dimensional frameworks. Furthermore, an expansion of the method to the multiple change-point framework is discussed through an idea similar to that of moving sum (MOSUM; Eichinger & Kirch, 2018). It would be beneficial to the reader for the authors to justify this choice; is it due to MOSUM’s low computational complexity?

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.