Attention Sinks and Retrieval Decay

3 selectedDifficulty 5-63 unseenView topic

IntermediateNew

0 answered

3 intermediateAdapts to your performance

Question 1 of 3

120sintermediate (5/10)state theorem

Attention sinks (Xiao et al. 2023) are an empirical phenomenon in transformer language models. What are they?