Handling Periodic Signs In American Sign Language Using Synthetic Generation Of Periods
Bibliographic record
Abstract
American Sign Language (or ASL) is the dominant sign language of deaf people in United States and parts of Canada. 500,000 to 2 million people use ASL as their primary language in United States. ASL uses hands, face and body, with constantly changing movements and orientations. Since the language is based on gestures, not a printed alphabet, it gets difficult to know the associated meaning given a video. There are multimedia tools and dictionaries available to view a sign video for a given word but there are no dictionaries available which, given a sign video will respond with corresponding word. This egged on development of video based lookup in ASL dictionary. The vision is to have a system in which a user will be able to lookup the meaning of an ASL sign simply by performing gesture in front of a video camera synced to a computer. The computer will compare the unknown sign with a database of signs to identify the most likely matches. In existing ASL lexicon project a user submits a query sign video and the application finds the most similar signs from the system database. The existing system evaluates the similarity between the query video and every sign video in the video database, using Dynamic Time Warping (DTW) distance. DTW is an algorithm for measuring similarity between two sequences which may vary in time or speed. DTW is a method that allows a computer to find an optimal match between two given sequences (e.g. time series) with certain restrictions. The sequences are warped non- linearly in the time dimension to determine a measure of their similarity independent of certain non-linear variations in the time dimension. The existing ASL lexicon project uses the similarity in hand locations and orientations to lookup a gesture in dictionary of ASL signs. The ability of DTW to cater to temporal misalignments helps us recognize signs differing in time or speed. ASL periodic signs are signs which have repetition of an action. The number of times this action is repeated is signer ‘s discretion. DTW in such cases will still attempt to align input video with the dictionary sign video. The alignment will not be meaningful, if the number of times an action is repeated in input video differs from dictionary sign video. Since DTW in such case results in non-meaningful association, it ultimately results in poor similarity responses. The paper attempts to correct this problem of ‘ Incorrect Period Matching ‘. The paper contributes by defining a protocol for annotating periodic signs and introduces a method for improving system accuracy on such signs. It builds an informative database for periodic video signs on an ASL lexicon dataset of 1113 unique signs. It captures the temporal information (start and end of period) for all the periods executed in each periodic sign video. This paper provides a mechanism to generate periods synthetically. We use the periodic temporal information of the last period to create subsequent periods for the training video. We successfully corrected the problem which spurred while using DTW on periodic signs, by synthetically generating periods.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".