FastPathLabs

Experiments

Each experiment is written up as a research note: the question, the method, the result, its limits, and the references it rests on. Notes are published whether or not the result cleared its bar, and updated as runs come in. Shorter accounts and announcements are on the blog.

  1. Research note 1

    10 November 2026

    Engine settings on a competent baseline: a pre-registered search

    Thirty combinations of four vLLM settings, measured on one H100. The best raised goodput by a factor of 1.14 against a bar of 1.5, so the experiment stopped. Two settings produced the whole gain.

    Status
    Completed
    Hardware
    1 × H100
    Model
    Qwen3-8B, FP8
  2. Research note 2

    10 November 2026

    Cache-aware routing for multi-turn chat

    A router that returns each turn to the worker holding the conversation, compared with round-robin and least-loaded routing on four GPUs. The gain is real, and bounded by the prefill that would otherwise be repeated.

    Status
    Preliminary
    Hardware
    4 × RTX A6000
    Model
    Qwen2.5-7B
  3. Research note 3

    11 November 2026

    Speculative decoding: tokens per verification and the cost of a round

    On a small lab model, fewer forward passes per token did not become more tokens per second until the whole round was compiled into one graph. The note separates the small-model and large-model regimes and gives the design of the run on an H100.

    Status
    Lab result; GPU run specified
    Model
    16.7M-parameter lab model
  4. Research note 4

    13 November 2026
    Updated 14 November

    Recurrent-state quantization for linear attention: from a first kernel to first serving results

    An implementation of LeapQuant, which stores the recurrent state of linear-attention layers in 8 bits. The first kernel was 7.2 times slower than SGLang’s window kernel; the rewrite is 4.6% slower. In SGLang at 512 concurrent requests it gave about 45% more throughput than the fp32 default and about 4% more than the 16-bit window state, with a worse tail. Whether the 16-bit state is a fair baseline is the next test.

    Status
    In progress
    Hardware
    1 × Hopper-class GPU
    Model
    Qwen3.5-9B
  5. Research note 5

    10 November 2026

    Measuring goodput: calibration and four guards

    How capacity is defined, what calibration showed about throughput and attainment, and four faults in our own measurement that were each larger than the effects under study.

    Status
    In use
    Hardware
    1 × H100
    Model
    Qwen3-8B, FP8