Methods available

Extra flavor

Data

  • Increased data processing speed by up to 100% using Nvidia Dali

  • Asymmetric and symmetric augmentations

Evaluation and logging

  • Online linear evaluation via stop-gradient for easier debugging and prototyping (optionally available for the momentum backbone as well)

  • Normal offline linear evaluation

  • All the perks of PyTorch Lightning (mixed precision, gradient accumulation, clipping, automatic logging and much more)

  • Easy-to-extend modular code structure

  • Custom model logging with a simpler file organization

  • Automatic feature space visualization with UMAP

  • Common metrics and more to come…

Training tricks

  • Multi-cropping dataloading following SwAV:

    • Note: currently, only SimCLR supports this

  • Exclude batchnorm and biases from LARS

  • No LR scheduler for the projection head in SimSiam