Mayank Mishra

mayank-mishra

·

https://mayank31398.github.io/

AI & ML interests

Large Language Models, Distributed Training and Inference

Recent Activity

liked a model about 23 hours ago

moonshotai/Kimi-K3

published a dataset about 2 months ago

open-lm-engine/dolma3_mix

liked a model 2 months ago

dots-studio/dots.ocr

View all activity

Organizations

upvoted a collection 5 months ago

M2RNN

Note that the 7B models are MoE with 1.1B active parameters and 400M models are dense models • 14 items • Updated Mar 12 • 7

upvoted a paper 12 months ago

Less Is More: Training-Free Sparse Attention with Global Locality for Efficient Reasoning

Paper • 2508.07101 • Published Aug 9, 2025 • 14

upvoted a paper over 1 year ago

SelfCodeAlign: Self-Alignment for Code Generation

Paper • 2410.24198 • Published Oct 31, 2024 • 25

upvoted a collection over 1 year ago

SmolLM2

State-of-the-art compact LLMs for on-device applications: 1.7B, 360M, 135M • 16 items • Updated May 5, 2025 • 311

upvoted 2 papers almost 2 years ago

The Mamba in the Llama: Distilling and Accelerating Hybrid Models

Paper • 2408.15237 • Published Aug 27, 2024 • 42

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published Aug 23, 2024 • 23

upvoted an article almost 2 years ago

Article

Improving Hugging Face Training Efficiency Through Packing with Flash Attention 2

+4

RQlee, ArthurZ, achikundu, lwtr, rganti, mayank-mishra

•

Aug 21, 2024

• 41

upvoted a collection almost 2 years ago

Power-LM

Dense & MoE LLMs trained with power learning rate scheduler. • 3 items • Updated Mar 2 • 16

upvoted a paper almost 2 years ago

Transformer Explainer: Interactive Learning of Text-Generative Models

Paper • 2408.04619 • Published Aug 8, 2024 • 175

upvoted 5 papers about 2 years ago

OpenDevin: An Open Platform for AI Software Developers as Generalist Agents

Paper • 2407.16741 • Published Jul 23, 2024 • 84

Enhancing Training Efficiency Using Packing with Flash Attention

Paper • 2407.09105 • Published Jul 12, 2024 • 17

Scaling Granite Code Models to 128K Context

Paper • 2407.13739 • Published Jul 18, 2024 • 21

The infrastructure powering IBM's Gen AI model development

Paper • 2407.05467 • Published Jul 7, 2024 • 3

MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention

Paper • 2407.02490 • Published Jul 2, 2024 • 26

upvoted a collection about 2 years ago

Experimental Checkpoints

As name suggests • 0 items • Updated Mar 2 • 1

upvoted an article about 2 years ago

Article

Aligning Large Language Models with BRAIn

gauravpandey1

•

Jun 11, 2024

• 9

upvoted a collection about 2 years ago

Dolomite Engine Sample

This collections contains a sample dataset and model trained via dolomite-engine. Repo: https://github.com/ibm-granite/dolomite-engine/ • 2 items • Updated Jun 30, 2024 • 1

upvoted 3 papers about 2 years ago

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

Paper • 2405.12981 • Published May 21, 2024 • 33

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization

Paper • 2404.03605 • Published Apr 4, 2024 • 1

Granite Code Models: A Family of Open Foundation Models for Code Intelligence

Paper • 2405.04324 • Published May 7, 2024 • 26