What are you looking for?

Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

AI·Jet-Long is a tuning-free zero-shot method for efficient long-context extension in LLMs that uses dynamic bifocal RoPE with local faithful and adaptively rescaled long-range windows.

Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity

AI·Sparse Delta Memory scales the hidden state of gated linear RNNs to orders of magnitude higher capacity via sparse addressing, improving long-context recall and in-context learning while keeping compute efficient.