Bibliographic record
Abstract
Video data include a significant amount of useful information that can be exploited by dierent organizations to gain more insights into running their business. The data are large and growing at a high rate because cameras, nowadays, are installed everywhere, gathering information all the time. Therefore, the data need large storage to be preserved and large computing power to be processed. Dierent technologies such as Apache Spark, Apache Storm, and Apache Hadoop have been widely used to perform big data processing on computer clusters. This thesis introduces general solutions and algorithms that can be used with dierent technologies, such as Apache Spark, Apache Storm, and Apache Hadoop, to improve the performance of processing big video data on computer clusters. However, the thesis focuses on using Apache Hadoop to provide an empirical evaluation of the proposed algorithms. Apache Hadoop is selected since it has been designed to work on commodity hardware. This thesis has been investigating dierent approaches to improve the performance of video processing on Hadoop clusters. These approaches are based on using the Hadoop MapReduce programming model to distribute the processing of big video data, and dierent sampling methods to avoid unnecessary computation while processing the video data. Change detection is used to detect the changes based on which the proposed algorithms sample the frames. The proposed algorithms can support video processing on faulty systems as well. The thesis also proposes three novel data placement policies to improve the performance of MapReduce-based video processing.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".