FastPathLabs

Blog and announcements

News from the lab, one experiment at a time. Each announcement names its baseline and its limits, and links to the research note that carries the full record. Results that miss their bar are announced the same way as those that clear it.

Announcements

  1. 14 November 2026

    Experiment 4How our 8-bit state implementation works

    A second post on LeapQuant, written for readers new to linear attention: what is stored, what happens on every token and on every sixteenth token, how the reference core, kernels, state pool and fold scheduler fit inside SGLang, and how the kernels are checked. Nine diagrams.

  2. 14 November 2026

    Experiment 4First serving results for an 8-bit recurrent state

    Our implementation of LeapQuant (Pan et al., arXiv 2609.38166) in SGLang gave about 45% more output tokens per second than SGLang’s fp32 default at 512 concurrent requests, and about 4% more than its 16-bit window state, with a worse p99 time between tokens. One run per arm. Whether the 16-bit state is a fair baseline depends on an accuracy test we have not yet run. This is one of the five lines of our program; the others continue.

Posts

  1. Experiment 4

    14 November 2026

    Inside our LeapQuant implementation: how an 8-bit state is stored, read, folded and served

    The architecture behind the results: the stored residual, compensator tokens and scales; the decode step and the fold; the reference core, kernels, state pool and fold scheduler inside SGLang; the life of a request; and how the kernels are verified.

  2. Experiment 4

    14 November 2026

    ~3× faster linear-attention kernels than packed FP32 at batch 512

    LeapQuant in SGLang with Qwen3.5-9B on one GPU: the serving results against two baselines, why the gain depends on load, where the tail comes from, what the accuracy tests do and do not show, and what we are not claiming.

  3. Experiment 3

    11 November 2026

    From 300 to 3,000 tokens per second on a lab model

    A compiled decode loop, bfloat16, 4-bit weights, a draft head, and a whole speculative round fused into one graph. Now part of research note 3.

  4. Experiment 1

    10 November 2026

    Searching vLLM settings on top of good defaults: 1.14×, not 1.5×

    Thirty combinations of four engine settings on one H100. The best beat the defaults by 1.14× against a bar of 1.5×, so we stopped. Now research note 1.

  5. Experiment 2

    10 November 2026

    KV-aware routing for chat: 16 to 27% less time than round-robin

    Sending each turn to the worker that already holds the conversation, on four GPUs, and why the gain is bounded. Now research note 2.