What are you looking for?

llama.cpp b9966 for sm-tensor

AI·llama.cpp build b9966 fixes regex recompilations for -sm tensor mode, caching patterns per tensor to reduce CPU overhead on decode threads.

产业深度：MaaS，到底给中国云厂商带来了什么？

AI·MaaS shifts Chinese cloud providers from CPU to Token/GPU metrics, with token consumption exploding 16x at Volcano Engine and 8x client growth at Alibaba Cloud, but overall public cloud growth slowed to 8-11%.

Running Qwen3 30B A3B at 50 tok/s on RTX 5060 Ti

AI·Achieves 50-54 tok/s for Qwen3 30B A3B float8 on 16GB RTX 5060 Ti using custom CUDA/C++.

Dual 3060 MoE loading issue.

AI·Identifies dual 3060 MoE loading limit at 12 layers VRAM for Qwen3 35B/122B with CPU offload.

Are EPYC CCDs all you need + benchmarks

AI·Reports EPYC 9374F CCD benchmarks using ik_llama.cpp; finds limited decoding gains vs older 9135 at low thread counts.

Qwen3.6 35B-A3B (Q8_0, no KV quant) single prompt in opencode: "Create a beautiful, relaxing flight simulator in a single html file with mountains, clouds, and endless procedural terrain"

AI·Qwen3.6 35B-A3B Q8_0 model (no KV quant) generates full HTML flight simulator in single prompt and is praised as stronger than Q4_K_M on GPU due to better CPU performance.

A Quantized Native Runtime for On-Device Semantic Audio Generation

Semantic audio applications increasingly require controllable generation on commodity and embedded hardware rather than through framework-heavy datacenter stacks. We present aria, a dependency-free native runtime that runs the complete text-to-music pipeline of Stable Audio~3 (SA3) on ordinary GPUs, CPU-only machines, and a Raspberry~Pi~5, with no Python or deep-learning framework underneath. Our main contribution is a study of quantization: running the model at lower numerical precision to fit…