Search

What are you looking for?

8 results for compute

AI·Reports of Meta selling excess AI compute and Anthropic signing massive data-center deals prove the notion of AI compute oversupply is a myth; actual deliverable frontier capacity remains scarce.

36氪 AI·36kr.com··2.6Researchaicomputedata-center

Linear attention models allow a fixed state size and a fixed amount of compute per token. However, due to their limited state size, linear attention models fall behind in long-context recall compared to softmax-attention-based transformer architectures. Increasing the state size of linear attention improves recall performance but at the cost of higher FLOPs. In this work, we introduce Sparse Delta Memory (SDM), an architecture that scales the hidden state of gated linear RNNs to orders of magni…

HuggingFace Daily Papers·huggingface.co··0.3paper

In a class of quantum circuits known as peaked circuits, the goal is to predict the most probable bit string at the output of the circuit. Since these circuits are designed to have a sharp peak in their output distribution, in principle it should be possible to simulate them using a truncated state vector with a limited number of terms, or a fraction of the total probability mass. This approximate simulation can be carried out on a classical computer with a sparse representation that stores onl…

HuggingFace Daily Papers·huggingface.co··0.3paper