One-click Qwen3.6-27B inference on Windows. 158 tok/s on RTX 5090, 72 tok/s on RTX 3090. Native, no WSL, no Docker, no telemetry.
-
Updated
May 14, 2026 - Python
#
rtx-3090
Here are 20 public repositories matching this topic...
SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI.
memory cuda self-hosted quantization gemma rag awq llm-serving pgvector vllm local-llm llm-inference qwen speculative-decoding kv-cache-quantization tool-calling consumer-gpu rtx-4090 rtx-3090 spec-decode
-
Updated
Jul 22, 2026 - Python
First public benchmark of llama.cpp speculative decoding on Qwen3.6-35B-A3B with a single RTX 3090 (post PR #19493 merge, 2026-04-19). 19 configurations covering ngram-cache, ngram-mod, and classic draft with vocab-matched Qwen3.5-0.8B. Finding: no variant achieves net speedup on Ampere + A3B MoE. Raw JSON, plots, full reproducibility.
benchmark cuda moe ampere mixture-of-experts inference-benchmark llama-cpp ggml local-llm llm-inference qwen speculative-decoding qwen3 rtx-3090
-
Updated
May 16, 2026 - Python
The bare metal in my basement
machine-learning ai deep-learning server gpu unraid hobby-project gigabyte bare-metal inference-server homelab hpc-systems hardware-hacking supermicro cpu-inference oculink cse-825 g292-z20 mcio rtx-3090
-
Updated
Jul 12, 2026
Qwen3.6 vLLM Toolkit — Launcher + Templates, Optimized for 2×24 GB. vLLM launcher with an optimized hybrid chat template drawing from the best community fixes for Qwen 3.6 27B.
-
Updated
Jul 18, 2026 - Jinja
Local agentic coding stack: Hermes Agent + Qwen3.5-27B + GLM-4.7-Flash on dual RTX 3090s. Daily-driver agentic work, no cloud, no metering. Companion to blog.zacharycangemi.com.
glm llama-cpp local-llm qwen agentic-ai tool-calling nousresearch agentic-coding rtx-3090 hermes-agent
-
Updated
May 15, 2026 - Batchfile
Reproducible vLLM recipe for shawnw3i/Huihui-Qwen3.6-27B-abliterated-AWQ-MTP on 2× RTX 3090 in a Proxmox LXC. MTP n=3, 256K context, full vision+tool-calling+reasoning. Silent 24/7 operation at 250W per card. Companion to the base-model recipe.
lxc mtp proxmox homelab uncensored dual-gpu awq vllm llm-inference qwen speculative-decoding rtx-3090 abliterated
-
Updated
May 29, 2026 - Python
poolside Laguna-S-2.1 INT4 + DFlash speculative decoding on 4x RTX 3090: 200K context, 282 tok/s peak decode, gate-proven with a 190K-token prompt. Full levers menu + failure catalog.
-
Updated
Jul 22, 2026 - Python
Local LLM experiments with llama.cpp — RTX 3090 CPU offloading, model benchmarking, and inference optimization notes
-
Updated
May 23, 2026 - Markdown
2.28× faster Claude Code on a local Qwen3.6-27B int4 (RTX 3090) — turbo-64k + long-100k profiles, MTP, tool calling, corruption guards.
-
Updated
Apr 25, 2026 - Python
Reproducible vLLM recipe for Qwen3.6-27B (AWQ-BF16-INT4) on 2× RTX 3090 in a Proxmox LXC. MTP, 256K context, full vision+tool-calling+reasoning. Bare-LXC alternative to Dzombak's Docker recipe. Benchmarks, gotchas, GPU passthrough config included.
-
Updated
May 27, 2026 - Python
Veizik — hardware-aware local AI media runtime. Hardware detection, low-memory execution planning, local licensing and experimental AI-video rendering.
cuda nvidia diffusion image-to-video text-to-video local-runtime ai-video local-ai low-vram rtx-3090
-
Updated
Jul 22, 2026 - Python
100% local voice assistant with Tool Calling, neural TTS, and streaming responses. Runs on RTX 3090 with Ollama + Kokoro TTS + FastAPI. Privacy-first AI.
python text-to-speech voice-assistant fastapi local-ai ollama tool-calling pydantic-ai kokoro-tts rtx-3090
-
Updated
Apr 7, 2026 - Python
Dual RTX 3090 Threadripper Pro AI server with 128 GB RAM — Part 1 of the Local LLM Infrastructure V2 project. Hardware, commissioning, power, and build documentation.
-
Updated
Jul 24, 2026
ElizaOS v1.x agent running Gemma 3 27B locally via Ollama on an RTX 3090, dogfooding @thecolony/elizaos-plugin against The Colony (thecolony.cc).
social-network quantization gemma dogfood ai-agents llm local-inference ollama elizaos elizaos-plugins rtx-3090 thecolony
-
Updated
Jul 16, 2026 - TypeScript
Multi-model vLLM and llama.cpp research on a dual RTX 3090 Threadripper Pro AI server: performance, context scaling, vision, and agentic workloads.
-
Updated
Jul 24, 2026
Benchmark speculative decoding performance for Qwen3.6-35B-A3B on an RTX 3090 GPU using llama.cpp to evaluate model throughput and structural regressions.
benchmark cuda moe ampere mixture-of-experts inference-benchmark llama-cpp ggml local-llm llm-inference qwen speculative-decoding qwen3 rtx-3090
-
Updated
Jul 24, 2026
Improve this page
Add a description, image, and links to the rtx-3090 topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the rtx-3090 topic, visit your repo's landing page and select "manage topics."