Tencent Releases Hy4-Preview 770B Model Weights with 49B Active Parameters

What Happened

Tencent released open weights for Hy4-preview on HuggingFace: a 770B-parameter mixture-of-experts model with 49B active parameters per token. This is the first frontier-scale open-weight release at the 700B+ tier from a major Chinese lab. Weights are available under a permissive license with no per-token API restrictions.

Why It Matters

The release moves the open-weight ceiling from the ~70B dense and ~200B MoE tier into territory previously served only by closed APIs and a handful of proprietary serving stacks. Operators with multi-node GPU capacity now have a self-hosted path to reasoning-heavy workloads that does not route through a Western vendor, an API rate limit, or a terms-of-service review. For organizations under data sovereignty constraints — financial services, healthcare, government adjacent work, and any EU or China-resident deployment — this is the first credible frontier-scale option that can be fine-tuned and served entirely in-house. It also establishes a pricing anchor: if 49B-active MoE inference runs on 8xH100, per-token licensing for equivalent intelligence tiers carries a visible economic ceiling.

Technical Details

Hy4-preview uses a sparse MoE architecture: 770B total parameters, 49B active per forward pass, distributed across expert shards that require multi-node provisioning. Practical serving needs roughly 300GB+ VRAM for active weights plus expert shards — typically 8xH100 (80GB) or 8xH200 across one or two nodes depending on quantization and sequence length. Tencent has optimized the model for Chinese-language tasks, math, and coding benchmarks, where it is competitive with current Western frontier releases. Integration follows standard HuggingFace transformers and vLLM/SGLang serving paths, though expert parallelism and tensor parallelism configuration are non-trivial. Quantized variants are not yet broadly available; unquantized serving carries meaningful memory overhead.

Operational Impact

Evaluation suites must now include Hy4-preview in offline baselines, particularly for Chinese-language, math, and coding workloads — omitting it produces benchmark results that no longer reflect the open-weight frontier. Teams with existing 8xH100 clusters can stand up a 49B-active MoE without new hardware procurement, which was not true six months ago. The cost model shifts from per-token licensing to electricity, rack space, and engineering time; utilization rates become the dominant economic variable. Fine-tuning and continued pretraining on a genuinely large MoE without API constraints is now feasible, which matters for domain adaptation under data residency rules. Teams without hosting capacity face a decision point: build, rent, or wait for quantized and distilled derivatives rather than ignore the release and fall behind on baseline coverage.

What To Watch

Expect quantized (4-bit and 8-bit) and distilled variants within weeks, which will lower the hardware floor and pull this tier into smaller clusters. Watch whether Western labs respond with comparable open-weight releases at the 700B+ tier, or double down on closed APIs and pricing pressure. The adjacent question is tooling: expert-parallel serving stacks, KV-cache compression, and multi-node orchestration remain the bottleneck for broad adoption, and whoever solves that captures the self-hosted frontier segment.