Why do transformers struggle with long context?
DandyMachine Learning Engineer · HR ScreenLast updated
This question evaluates understanding of transformer architectures, attention mechanisms, and scalability trade-offs, assessing competencies in computational complexity, memory usage, and system-level performance implications.
In a transformer-based model, why is it difficult to process very long input context? Explain the main challenges in terms of computation, memory usage, attention behavior, and training or inference efficiency. Briefly mention common techniques used to reduce these problems.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.