Research projects
Preprint · 2026Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
Pascal Jutras-Dubé, Patrick Pynadath, Jeremy Lu, Yuan Gao, Ruqi Zhang
A continuous-time framework connecting masked, continuous, and hybrid diffusion.
Read more
Overview
Sticky Jump Diffusions let probability mass leave discrete token embeddings and diffuse through continuous space. The reverse process combines continuous denoising with jumps back to discrete anchors. Denoising Hazard Matching learns the required scores and jump rates from one denoising classifier. The framework recovers masked, continuous, and hybrid diffusion as limiting cases.
Simplex Relaxation for Discrete Diffusion
Jinya Sakurai, Patrick Pynadath, Satoshi Hayakawa, Jaehong Yoon, Xulei Yang, Nancy F. Chen, Xun Xu
Simplax enriches uniform discrete diffusion with auxiliary simplex-valued variables.
Read more
Overview
Simplax augments categorical states with Dirichlet-distributed simplex variables while preserving the original categorical corruption process. This yields a Rao–Blackwellized reverse-bridge objective and a stochastic sampler, retaining categorical inputs to the denoiser. Experiments study text generation on OpenWebText and constrained and unconditional Sudoku generation.
Generative Frontiers: Why Evaluation Matters for Diffusion Language Models
Patrick Pynadath, Jiaxin Shi, Ruqi Zhang
Evaluate diffusion language models through their perplexity–entropy frontiers rather than a single operating point.
Read more
Overview
This note examines dataset choice, likelihood comparisons, and the limitations of generative perplexity. Relating perplexity and entropy to KL divergence motivates generative frontiers: comparing models across their quality–diversity tradeoffs instead of relying on a single decoding configuration.
Why Any-Order Autoregressive Models Need Two-Stream Attention: A Structural-Semantic Tradeoff
Patrick Pynadath, Ruqi Zhang
Two-stream attention helps separate competing semantic and structural demands in any-order generation.
Read more
Overview
Any-order autoregressive models must attend both to semantically useful tokens for prediction and to structurally recent tokens for summarization. Experiments with Decoupled RoPE suggest that separating position from content alone does not resolve this competition, especially at longer sequence lengths.
CANDI: Hybrid Discrete-Continuous Diffusion Models
Patrick Pynadath, Jiaxin Shi, Ruqi Zhang
We figure out why continuous diffusion has struggled on discrete data, and introduce CANDI, a principled solution.
Read more
Overview
Continuous diffusion does extremely well on images, but struggles on discrete data. We introduce token identifiability to study Gaussian noise on discrete data and discover a temporal dissonance between discrete identity corruption and continuous rank degradation. Both are vital for continuous diffusion but become misaligned as the number of categories increases. We introduce CANDI, disentangling the two forms of corruption with an explicit masking schedule to coordinate them and bring continuous diffusion benefits to discrete spaces.
TL;DR
- Token identifiability explains how Gaussian noise corrupts discrete data.
- We find temporal dissonance between identity corruption and rank degradation that hurts continuous diffusion.
- CANDI decouples and coordinates the corruptions to improve discrete diffusion.
VERA: Variational Inference Framework for Jailbreaking Large Language Models
Anamika Lochab*, Lu Yan*, Patrick Pynadath*, Xiangyu Zhang, Ruqi Zhang
* Equal contribution
We use variational inference to introduce a scalable and effective framework for jailbreaking/red-teaming LLMs.
Read more
Overview
Black-box safety testing is crucial as many powerful LLMs are API-only. Existing genetic-algorithm jailbreakers need curated seeds and re-running per test case. VERA treats jailbreak generation as probabilistic inference: we train a small attacker model to learn the distribution of adversarial prompts. Once trained, VERA instantly produces diverse, natural jailbreaks without additional optimization, succeeding across target models.
TL;DR
- Black-box jailbreaking reveals realistic vulnerabilities.
- Genetic algorithms are brittle and costly to rerun.
- VERA learns a distribution of jailbreaks for fast, diverse attacks.
Controlled LLM Decoding via Discrete Auto-regressive Biasing
Patrick Pynadath, Ruqi Zhang
We use gradient-based discrete sampling to enable plug-and-play control over LLM generation.
Read more
Overview
Controlling LLM outputs requires balancing fluency with constraint satisfaction. Energy-based decoding in continuous space struggles with this trade-off. We introduce Discrete Auto-regressive Biasing (DAB), a decoding algorithm that stays in the discrete token domain using gradient-based discrete MCMC within a Langevin-within-Gibbs framework. It defines a joint distribution over text and auxiliary bias sequences to deliver better constraints, fluency, and lower cost.
TL;DR
- Continuous-space control struggles with discrete text constraints.
- DAB keeps sampling discrete while leveraging gradients.
- Achieves better constraint satisfaction and fluency at lower cost.
Gradient-based Discrete Sampling with Automatic Cyclical Scheduling
Patrick Pynadath, Riddhiman Bhattacharya, Arun Hariharan, Ruqi Zhang
We enable discrete gradient-based sampling methods to deal with multi-modal distributions by introducing automatically tuned cyclical schedules.
Read more
Overview
Discrete distributions in deep models are highly multimodal. Gradient-based samplers get trapped in local modes. We propose automatic cyclical scheduling that alternates large exploratory steps with small exploitative steps, combines balanced proposals, and auto-tunes hyperparameters across datasets. This yields efficient multimodal sampling with convergence guarantees and strong empirical performance.
Technical Details
- Multimodal discrete targets trap vanilla gradient samplers.
- Cyclical schedules alternate exploration and exploitation with balanced proposals.
- Automatic tuning adapts across datasets while retaining theoretical guarantees.
