Tri Dao

papers

9,884

total citations

papers (16)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

NEURIPS 2022arXiv

3,551

citations

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

ICLR 2024arXiv

2,224

citations

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

ICML 2024arXiv

1,146

citations

Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers

NEURIPS 2021arXiv

977

citations

HiPPO: Recurrent Memory with Optimal Polynomial Projections

NEURIPS 2020arXiv

838

citations

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

ICML 2024arXiv

549

citations

Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping

ICML 2025arXiv

citations

Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees

NEURIPS 2022

citations

Tri Dao

papers (16)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers

HiPPO: Recurrent Memory with Optimal Polynomial Projections

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Scatterbrain: Unifying Sparse and Low-rank Attention

Decentralized Training of Foundation Models in Heterogeneous Environments

S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces

Long-Context State-Space Video World Models

Rethinking Neural Operations for Diverse Tasks

Transform Once: Efficient Operator Learning in Frequency Domain

Hardware-Efficient Attention for Fast Decoding

Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping

Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees

papers (16)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers

HiPPO: Recurrent Memory with Optimal Polynomial Projections

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Scatterbrain: Unifying Sparse and Low-rank Attention

Decentralized Training of Foundation Models in Heterogeneous Environments

S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces

Long-Context State-Space Video World Models

Rethinking Neural Operations for Diverse Tasks

Transform Once: Efficient Operator Learning in Frequency Domain

Hardware-Efficient Attention for Fast Decoding

Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping

Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees