Optical music recognition of square notation using generative adversarial networks

Manually converting Medieval chant manuscripts, written in square notation, to a computer-readable format is costly.Optical Music Recognition (OMR), which automatically performs the conversion from scanned manuscript images, can potentially reduce this cost.Machine learning models can be used to perform OMR, though they often require large amounts of labelled training data.In this work, a generative adversarial network (GAN) is trained to generate images of the individual notes and symbols that make up square notation.A GAN entangles two neural networks in a game, where one is progressively trained to generate increasingly realistic images based on a training set to fool a second network, which iteratively evaluates whether the generated examples appear as real as the training set.The music symbols generated by the GAN are placed on staff lines on synthetic manuscript pages, mimicking the appearance and structure of a real page of square notation.A novel OMR workflow is introduced that includes two sequential machine learning OMR models used in sequence: the first for object detection of musical symbols, the second for determining the vertical staff position of the notes.The baseline OMR workflow experiments are trained with real manuscript images only, and their results are compared against the OMR workflow trained with both real and synthetic manuscript images.Two medieval manuscripts, the Salzinnes Antiphonal and the Einsiedeln Stiftsbibliothek Codex 611(89) are used for the experiments.Comparing against the baseline real data experiments, an increase in the OMR workflow's evaluation metrics demonstrates that the OMR of square notation is improved by training the workflow with both real and synthetic data, assisted by the GAN architecture.Experimental results indicate that the OMR of square notation can be improved by using GAN-synthesized manuscript data.Thank you to all of my colleagues in the Music Technology program at McGill, who inspired me to pursue this work.Specifically, within the Distributed Digital Music Archives and Libraries (DDMAL) Laboratory, I would like to thank Gabriel Vigliensoni for his continued mentoring and introduction to the world of Music Information Retrieval (MIR).Thank you for taking interest in my pursuits, repeatedly editing my thesis proposal, and showing me so much of what the music scene in Montral has to offer.Thank you to Andrew Kam for jogging some ideas about how to envision this

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.