All pull requests

Pull requests

• prefill: let native-layout packs take STRATA_MMQ_RESIDENT_SORT_NE

  #1660·ruibeikaa opened Oct 9, 2026Contributor·

  ·

• prefill gemm: recover the FP16 shapes cuBLAS 12's default algorithm fails on (issue #1650)

  #1659·W1nge opened Oct 9, 2026Contributor·

  ·

• tools tests: align test_setup_risk's WSL mock with install() (the KV-streaming rule test failed inside WSL)

  #1658·chaog992 opened Oct 9, 2026·

  ·

• bench: community report, 2x RX 6900 XT (gfx1030) + Ryzen 5 5600X, IQ3_S at 131K on 0.1.41 - layer split: stock, the two gfx103x switches, the open PRs, and the decode window

  #1657·xjc10 opened Oct 9, 2026Contributor·

  ·
  1

• pipeline-windows: three or more layer-split stages, +14% decode at 4K / +27% at 32K on 4x RX 7900 XT (opt-in); HIP draft-layer scratch race

  #1656·Cass67 opened Oct 9, 2026·

  ·

• bench: community report, 4x RX 7900 XT (gfx1100), Flash-Next IQ3_S at 262K, five opt-ins off vs on, engine 0.1.41

  #1655·Cass67 opened Oct 9, 2026·

  ·

• [WIP] cuda: optional SM75 main-model QSA padded MMA

  #1654·Unmaple opened Oct 9, 2026Contributor·

  ·

• [WIP] cuda: SM75 IQ4_XS four-row reuse and NW3 shape specialization

  #1653·Unmaple opened Oct 9, 2026Contributor·

  ·

• Fix Linux CUDA builds with selectable GCC (#1645)

  #1652·CC-David-CC opened Oct 9, 2026Contributor·

  ·
  1

• prefill: take expert blobs from a helper's VRAM cache when it holds them

  #1649·W1nge opened Oct 9, 2026Contributor·

  ·

• bench: community report - a Tesla V100 (sm_70) confirms the opt-in Volta decode kernels

  #1647·Mitsuasa513 opened Oct 9, 2026Contributor·

  ·

• fix(api): honor chat logit_bias on CUDA and HIP

  #1646·CC-David-CC opened Oct 9, 2026Contributor·

  ·
  1

• sycl: MtpDrafter's queue defaults to nullptr, so a run without --mtp no longer crashes at exit

  #1642·recutita opened Oct 9, 2026·

  ·

• bench: community report, 2x RTX 3080 20 GB, UD-Q4_K_XL, 2 x 262K lanes in a 512K pool, 3.0K tok/s prompt, 86 tok/s decode

  #1640·noon-at-cgn opened Oct 9, 2026·

  ·

• bench: add single R9700 performance report

  #1638·zihaomu opened Oct 9, 2026·

  ·

• batch: the adaptive expert tier keeps adapting in batch windows (--adapt-async 1 beside slots)

  #1637·noon-at-cgn opened Oct 9, 2026·

  ·

• batch-mtp: work with a layer split, +15% two-stream decode, one group under --batch-groups auto

  #1636·noon-at-cgn opened Oct 9, 2026·

  ·

• Add a prometheus metrics endpoint with llama.cpp's names

  #1635·zeevo opened Oct 9, 2026·

  ·

• setup: offer the experimental speed projection for Swift 1.5 too

  #1633·lineape opened Oct 9, 2026·

  ·

• test: pin GPU order in the vision device fixture

  #1632·W1nge opened Oct 8, 2026Contributor·

  ·

• frontend: Add preview of generated HTML and SVG

  #1628·thoives opened Oct 8, 2026·

  ·
  1

• frontend: the tools in a canonical order

  #1624·XinacS opened Oct 8, 2026·

  ·
  1

• fix(serve): make context overflow errors recognizable

  #1623·hulkbig opened Oct 8, 2026Contributor·

  ·
  1

• Add Jetson Orin ARM64 support and shared-memory budgeting

  #1621·calvinvette opened Oct 8, 2026·

  ·

• bench: RTX 5090 community report, unsloth UD-Q4_K_XL and UD-Q5_K_XL with images on 0.1.41

  #1617·CYoung83 opened Oct 8, 2026Contributor·

  ·