r/LocalLLaMA

next best jump from qwen 3.6 27b

AI·Community discusses upgrades from Qwen3.6-27B, recommending options like DeepSeek V4 Flash or GLM-5.2 requiring 100-250GB VRAM for noticeable performance gains.

China's DeepSeek developing its own AI chip, sources say

AI·Sources say China's DeepSeek is developing its own AI chip, potentially advancing domestic AI hardware capabilities amid global tensions.

Measuring PCIe transfer under dual GPU with pipeline & tensor llama.cpp

AI·A user shares benchmarks of dual RTX 3090 and Titan RTX cards running llama.cpp with Qwen3.6-27B at 180k context, comparing tensor and pipeline parallel modes for PCIe transfer efficiency.

llama.cpp b9966 for sm-tensor

AI·llama.cpp build b9966 fixes regex recompilations for -sm tensor mode, caching patterns per tensor to reduce CPU overhead on decode threads.

[not mine] costs

AI·Questions costs and viability of aftermarket SXM2 boards for two V100 GPUs, comparing to consumer GPU setups.

Here's your recipes for models for the most popular hardware

AI·The post shares practical recipes and benchmarks for running LLMs on popular consumer hardware, with a new website listing hardware-filtered setups and user-voted usage stats.

Ultra budget 20GB vram with 448GB/s for $100 bucks.

AI·A $100 setup using three NVIDIA P102 cards delivers 20 GB VRAM and 448 GB/s memory bandwidth, sufficient for three concurrent LLM users with high context and better speeds than costlier lower-VRAM cards.

Performance comparison on full compute performance (Anima) and LLM prompt processing of 5090 (600,475 and 400W) vs 6000 PRO MaxQ shunt modded and water cooled (at 300, 400, 475 and 600W), and 6000 PRO WS/SE (600W).

AI·User compares 5090 at 600W full compute and prompt processing vs shunt-modded 6000 PRO MaxQ water-cooled at 300-600W.

I benched quad 5060Tis for code generation with Qwen3.6-27B so you don't have to (it's really good)

AI·The author benches a quad 5060Ti setup for Qwen3.6-27B code generation using MTP, reporting 608 tokens/s prefill and 52.2 tokens/s decode at 256k context on a cost-effective build.

Is it possible to run Qwen 122B in 64GB ram + 24gb vram ? If so, how ?

AI·Discusses feasibility of running Qwen 122B on 64GB RAM + 24GB VRAM and suitable settings.

I built Flaxeo Image a local desktop ui for stable diffusion cpp

AI·Introduces Flaxeo Image, a local desktop UI for Stable Diffusion C++ that exposes model, hardware, and video options.

Upgrading from 2x 3090 - what should I add? (2x A6000/5090/48GB 4090?)

AI·User upgrading from dual RTX 3090 asks for advice on adding cards for 100-110 GB model support, weighing 2x modded 48 GB 4090, 2x A6000, or 2x 5090 in hybrid setup.

CTX: How far can you reasonably go with Qwen 3.6 27B?

AI·Asks how far context window can be stretched with Qwen 3.6 27B at Q8_0 before reliability drops.

Dual 3060 MoE loading issue.

AI·Identifies dual 3060 MoE loading limit at 12 layers VRAM for Qwen3 35B/122B with CPU offload.

I created a super harmful model ! :D (by tweaking it's J-Space!!!)

AI·Shares tool for tweaking models' J-Space using Anthropic Jacobian-Lens to create super harmful behavior.

I feel like I'm not using my hardware efficiently

AI·User with RTX 4090, 3090Ti, and 128GB RAM struggles to run larger models like 122B due to VRAM limits while Qwen3.6-27B runs efficiently without using system RAM.

Follow-up: what I learned scaling a SQLite/FTS5 patent database from 3.5M to 5.36M records

AI·Follow-up post details data-engineering optimizations that scaled a SQLite/FTS5 patent database from 3.5M to 5.36M records while classifying them with Nemotron 9B on an RTX 5090.

Why are MoE models so belittled?

AI·Questions why providers make high-parameter MoE models (e.g. 122B with 10B active) instead of simpler dense equivalents.

Are EPYC CCDs all you need + benchmarks

AI·Reports EPYC 9374F CCD benchmarks using ik_llama.cpp; finds limited decoding gains vs older 9135 at low thread counts.

The U.S. tech industry is increasingly anxious about the rising power and competitive price of open-source AI models from China — and whether the Trump administration will respond with yet another executive order | Politico

Politico: Wall Street’s new obsession: Which CEOs have Trump’s ear?: https://www.politico.com/newsletters/politico-influence/2026/07/10/wall-streets-new-obsession-which-ceos-have-trumps-ear-00993324 submitted by /u/Nunki08 [link] [comments]

Running Qwen3 30B A3B at 50 tok/s on RTX 5060 Ti

AI·Achieves 50-54 tok/s for Qwen3 30B A3B float8 on 16GB RTX 5060 Ti using custom CUDA/C++.

Qwen3.6 35B-A3B (Q8_0, no KV quant) single prompt in opencode: "Create a beautiful, relaxing flight simulator in a single html file with mountains, clouds, and endless procedural terrain"

AI·Qwen3.6 35B-A3B Q8_0 model (no KV quant) generates full HTML flight simulator in single prompt and is praised as stronger than Q4_K_M on GPU due to better CPU performance.

Grok Build CLI uploads your whole repo — full git history + .env secrets — to xAI's cloud, and the opt-out doesn't stop it (wire-captured)

I ran Grok Build CLI (v0.2.93) through mitmproxy. It uploads your entire repo as a git bundle (full history) to xAI's Google Cloud — independent of what you open. With the prompt literally "do not read or open any files," a file I planted came back verbatim when I git clone-d the captured upload. Separately, files it reads (incl. a .env with API_KEY/DB_PASSWORD) go to cli-chat-proxy.grok.com verb…

Training an LLM from scratch on 1800's texts (160GB dataset)

Hi everyone, A year ago I began pre-training language models exclusively on 1800’s London data. Recently I have completed my largest dataset ever, containing 40B tokens or 160GB of 1800-1875 english data from England and the United States. I will soon train a 2B parameter model on it, but for now I’ve trained a 500M parameter evaluation model on a 5B token sample. I have also fine tuned the eval …

Best Local VLMs - July 2026

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detailed as possible in describing your setup (at least hardware and inference engine) nature of your usage (what applications, how much, personal/professional use) tools/frameworks/prompts etc. Rules Onl…