Blog

A sparse routing for scalable Transformers

Adversarial generations of discrete sequences

A practical walkthrough for building a tiny ChatGPT

Learning vector fields in continuous and discrete spaces

The backward pass is policy gradient

Understanding the model architecture behind LLMs

Backpropagation of continuous and discrete random variables

Image synthesis via transformers

Diffusion language models for images, languages, and general state spaces

Thoughts on diffusion model alignment

One-step ultra-fast generations for images and videos

A popular tool in finance and stochastic optimal control

A general framework for modeling nonlinear state-space models

A simple empirical Bayesian posterior estimate

Interpretating Bayes’ law via optimal transport for filtering problems

An unbiased Monte Carlo sampler for implicit trace estimation

Can a Transformer represent a Kalman filter?

A Monte Carlo sampler for radial basis function kernels and positional embedding

A standard template for linear state-space models

The pioneering normalizing flows within generative models.

Does a straighter flow always yield more efficient transport?

The optimal penalty can be zero or negative for real-world high dimensional data.

Connections between probability flows, diffusions, and PDEs.

A general coupling technique for characterizing a broad range of diffusions.

An application of Girsanov theorem in parameter estimation.

A framework that unifies ODE, PDE, SDE, stochastic control, optimal transport, and fluid dynamics

An elegant sampler that utilizes Hamiltonian dynamics to propose new states in simulations.

A family of techniques to understand the convergence of random variables.

An inequality that unifies ODE, PDE, SDE, functional analysis, and Riemannian geometry.

Running multiple MCMCs at different temperatures to explore the solution thoroughly.

Negative learning rates help escape local traps.