论文笔记-sign language recognition, translation and production
paper list to read:
Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video
Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers andMixture Density Networks
Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives
Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Production
Ben Saunders, Necati Cihan Camgoz, and Richard Bowden
Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Poses
BABEL: Bodies, Action and Behavior with English Labels, CVPR2021
Fingerspelling Detection in American Sign Language, CVPR2021
American Sign Language fingerspelling recognition in the wild. SLT2018
Fingerspelling recognition in the wild with iterative visual attention. ICCV2019
ESD
Two-stage:
- text to pose sequence
- pose sequence to contiguous sign video

Text2pose
using a Mixture Density Network (MDN):

$\alpha_{i}(x_{1:U})$ is the mixture weight of the $i^{th}$ distribution, regarded as a prior probability of the sign pose being generated from this mixture component. $\phi_{i}(y_t|x_{1:U})$ is the conditional density of the sign pose for the $i^{th}$ mixture.
Similar to auto-regressive vae model.
Pose2video
Fingerspelling Detection
手指拼写的作用:
- 专有名词
- 技术术语
- 缩写等没有对应手势的词汇
- 也用于强调和方便
手指拼写占ASL的 12%-35%. 这个比例比大部分手语词汇量都要大。
Fingerspelling is used for multiple purposes, including for words that do not have their own signs (such as many proper nouns, technical terms, and abbreviations) [39] but also sometimes for emphasis or expediency. Fingerspelling accounts for 12% to 35% of ASL, where it is used more than in other sign languages [40].
手指拼写对应的字母与翻译出来的英语是单调对齐的。这有点类似于翻译中的直接音译。
手指拼写的检测对于下游手语识别任务有显著提升作用。
论文笔记-sign language recognition, translation and production
http://www.panxiaoxie.cn/2021/07/11/论文笔记-sign-language-recognition-and-translation/