论文笔记-image-based contrastive learning

  • simCLR
  • MoCo
  • BYOL
  • Swin-ssl
  • BraVe
  • What Makes for Good Views for Contrastive Learning?
  • BYOL works even without batch statistics
  • Understanding Self-Supervised Learning Dynamics without Contrastive Pairs
  • Big Self-Supervised Models are Strong Semi-Supervised Learners
  • Understanding contrastive representation learning through alignment and uniformity on the hypersphere.
阅读更多

论文笔记-video transformer

paper list:

  • Training data-efficient image transformers & distillation through attention.
  • An image is worth 16x16 words: Transformers for image recognition at scale.
  • ViViT: A Video Vision Transformer.
  • Is space-time attention all you need for video understanding
  • Video transformer network.
  • Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
  • CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
  • What Makes for Hierarchical Vision Transformer?
  • WiderNet Go Wider Instead of Deeper
  • CoAtNet: Marrying Convolution and Attention for All Data Sizes
阅读更多

论文笔记-dynamic convolution and involution

paper list:

  • CARAFE: Content-Aware ReAssembly of FEatures
  • Involution: Inverting the Inherence of Convolution for Visual Recognition
  • Pay less attention with lightweight and dynamic convolutions
  • ConvBERT: Improving BERT with Span-based Dynamic Convolution
  • Dynamic Region-Aware Convolution
阅读更多

论文笔记-unlikelihood training

paper list:

  • Neural Text Generation with Unlikelihood Training
  • Implicit Unlikelihood Training: Improving Neural Text Generation with Reinforcement Learning
阅读更多