d2ar started with a question i could not find a clean answer to: what happens when discrete diffusion language models meet hindi, especially the code-mixed hindi people actually type? i compared eight models across sentiment, named entities, question answering, and summarization, then published the paper and evaluation code.
the benchmark
the data path normalizes devanagari and romanized hindi, keeps hinglish samples visible, and uses matched splits across news, social, formal, and conversational text. every model sees the same task instructions and output schema so a nicer prompt cannot quietly become the result.
classification and extraction use task-specific accuracy and f1. generation gets bleu, rouge-l, and multilingual bertscore, with confidence intervals over the same examples. i kept raw predictions next to the aggregate scores because hindi failures hide inside case marking, gender agreement, compound verbs, and entity boundaries long before one average number admits it.
the models
auto-regressive: gpt-neox, opt, and bloom variants. each token conditions on the prefix and arrives left to right through causal attention.
discrete diffusion: diffusion-lm, mdlm, sedd, and plm-discrete. generation happens through iterative denoising, which gives the model bidirectional context while it repairs a sequence.
what moved
diffusion models stayed closer on summarization and other open generation tasks. the gap widened on structured outputs such as named entities and sentiment, where morphology, ordering, and a strict label schema punish a locally plausible repair. the error sheets were more useful than the leaderboard because they showed where each architecture lost the sentence.
The full paper, dataset splits, and evaluation code are available on GitHub. if you actually read this far you're either a researcher or a stalker. pick a lane.