Unlock: Attention Sinks and Retrieval Decay
Why transformers disproportionately attend to initial tokens (attention sinks), how StreamingLLM exploits this for infinite-length inference, and how retrieval accuracy degrades with distance and position within the context window.
174 Prerequisites0 Mastered0 Working145 Gaps
Prerequisite mastery17%
Recommended probe
Chernoff Bounds is your weakest prerequisite with available questions. You haven't been assessed on this topic yet.
Attention Mechanism TheoryResearch
Not assessed11 questions
Forgetting Transformer (FoX)Research
No quiz
Sign in to track your mastery and see personalized gap analysis.