Jaisidh Singh

I like to post some things I tinker with occasionally.

• Flax-WeightWatcher
  My own quick WeightWatcher-like tool for FLAX models, since the actual WeightWatcher by CalculatedContent doesn't support JAX/FLAX and XLA. [Tweet]

• pytorch-mixtures
  A minimalist library for popular MoEs & MoD in PyTorch.

• lora-clip
  A library to easily wrap LoRA layer insertion for CLIP.

• Tutorial for "DINo: Continuous PDE forecasting with INRs"
  I've recently made an in-depth tutorial on this cool physics-informed ML paper.

• Auto-DEcoder in JAX
  A quick and easy walkthrough of the auto-decoding process with JAX.

• TokenFormer in PyTorch
  A crisp implementation of the TokenFormer Layer in PyTorch.

• Differential Transformer in PyTorch
  A simple snippet for multi-head differential attention in PyTorch.