论文笔记-constrast learning in NLP

paper list:

  • An efficient framework for learning sentence representations.
  • CLEAR: Contrastive Learning for Sentence Representation
  • Declutr: Deep contrastive learn- ing for unsupervised textual representations. arXiv
  • SimCSE: Simple Contrastive Learning of Sentence Embeddings
  • R-Drop: R-Drop: Regularized Dropout for Neural Networks
  • Coco-lm: Correcting and contrasting text sequences for language model pretraining.
  • Learning dense representations of phrases at scale.
  • An unsupervised sentence embedding method by mutual information maximization.
  • Representation degeneration problem in training natural language generation models
  • CosReg: Improving neural language generation with spectrum control.
  • CLAPS: Contrastive Learning with Adversarial Perturbations for Conditional Text Generation
  • LINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding
  • Adversarial PerturbationInterpretable Adversarial Perturbation in Input Embedding Space for Text
  • Contrastive Learning for Many-to-many Multilingual Neural Machine Translation
  • Unsupervised Data Augmentation for Consistency Training
阅读更多

论文笔记-image-based contrastive learning

  • simCLR
  • MoCo
  • BYOL
  • Swin-ssl
  • BraVe
  • What Makes for Good Views for Contrastive Learning?
  • BYOL works even without batch statistics
  • Understanding Self-Supervised Learning Dynamics without Contrastive Pairs
  • Big Self-Supervised Models are Strong Semi-Supervised Learners
  • Understanding contrastive representation learning through alignment and uniformity on the hypersphere.
阅读更多

论文笔记-video transformer

paper list:

  • Training data-efficient image transformers & distillation through attention.
  • An image is worth 16x16 words: Transformers for image recognition at scale.
  • ViViT: A Video Vision Transformer.
  • Is space-time attention all you need for video understanding
  • Video transformer network.
  • Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
  • CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
  • What Makes for Hierarchical Vision Transformer?
  • WiderNet Go Wider Instead of Deeper
阅读更多

论文笔记-dynamic convolution and involution

paper list:

  • CARAFE: Content-Aware ReAssembly of FEatures
  • Involution: Inverting the Inherence of Convolution for Visual Recognition
  • Pay less attention with lightweight and dynamic convolutions
  • ConvBERT: Improving BERT with Span-based Dynamic Convolution
  • Dynamic Region-Aware Convolution
阅读更多

论文笔记-unlikelihood training

paper list:

  • Neural Text Generation with Unlikelihood Training
  • Implicit Unlikelihood Training: Improving Neural Text Generation with Reinforcement Learning
阅读更多