I like to post some things I tinker with occasionally.
Flax-WeightWatcher
My own quick WeightWatcher-like tool for FLAX models, since the actual WeightWatcher by CalculatedContent doesn't support JAX/FLAX and XLA. [Tweet]pytorch-mixtures
A minimalist library for popular MoEs & MoD in PyTorch.lora-clip
A library to easily wrap LoRA layer insertion for CLIP.Tutorial for "DINo: Continuous PDE forecasting with INRs"
I've recently made an in-depth tutorial on this cool physics-informed ML paper.Auto-DEcoder in JAX
A quick and easy walkthrough of the auto-decoding process with JAX.TokenFormer in PyTorch
A crisp implementation of the TokenFormer Layer in PyTorch.Differential Transformer in PyTorch
A simple snippet for multi-head differential attention in PyTorch.