fix(xds): carry Lua scripts on the routes with a shared VM per script by zhaohuabing · Pull Request #10056 · envoyproxy/gateway

zhaohuabing · 2026-09-19T00:36:23Z

Lua in an EnvoyExtensionPolicy has been stuck between two bad options: keeping the script in the listener means every edit drains the listener, and moving it onto the routes meant one Lua VM set per route config, which is what took a proxy from 88 MiB to 2.8 GiB in #9355 and led to the revert in #9426.

envoyproxy/envoy#47562 removes the second problem. With shared_vm_id set on LuaPerRoute, Envoy builds one set of Lua VMs per script and shares it across every route carrying that script. The listener keeps one empty, disabled placeholder Lua filter per slot, rounded up to a multiple of ten, so adding, editing or removing a policy is a route configuration update and never rewrites the listener.

The shared VM id is the script's own name, so scripts from different policies never share a VM or its globals; only routes running the same script do.

Compared to the earlier ECDS design, no ordering between RDS and ECDS to get right. The cost is that a policy spanning many routes puts its script text in each route entry, so the RouteConfiguration is larger for that case.

Compatibilitie:

Filter names change from policy-derived to envoy.filters.http.lua/<slot>, EnvoyPatchPolicys and extension servers need to be adjusted accordingly.
A script's VM is now shared across every listener that runs it, where previously each listener had its own. Global variables that a script uses as shared state, such as a cache or a counter, were already shared by all routes of the policy on one listener and are now also shared across listeners. This should rarely matter, but scripts that rely on per-listener globals should be aware of it and use a separate policy per listener if needed, since each policy script gets its own VM.

Fixes: #9355

netlify · 2026-09-19T00:36:28Z

✅ Deploy Preview for cerulean-figolla-1f9435 ready!

Name	Link
🔨 Latest commit	`5a5fe7e`
🔍 Latest deploy log	https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/6abb312ea642e200086dcdde
😎 Deploy Preview	https://deploy-preview-10056--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

codecov · 2026-09-19T00:48:50Z

Codecov Report

❌ Patch coverage is 92.72727% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.43%. Comparing base (f1f4888) to head (5a5fe7e).

Files with missing lines	Patch %	Lines
internal/xds/translator/lua.go	90.62%	3 Missing ⚠️
internal/xds/translator/httpfilters.go	95.65%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10056      +/-   ##
==========================================
+ Coverage   81.41%   81.43%   +0.01%     
==========================================
  Files         266      266              
  Lines       41300    41302       +2     
==========================================
+ Hits        33626    33635       +9     
+ Misses       7673     7666       -7     
  Partials        1        1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

zhaohuabing · 2026-09-21T04:30:02Z

@codex review

A listener now carries one Lua filter per execution slot, holding every script that can run in that slot in its sourceCodes map, and routes select one with LuaPerRoute.name instead of carrying the source. Envoy builds one Lua VM set per distinct script rather than one per route config that embeds source text, and the filter chain no longer grows with the number of Lua policies. Each filter's configuration is served as its own ECDS resource, so adding, editing or removing a script is an extension config update that Envoy applies in place instead of a listener update that drains the listener. Slot counts are rounded up to a multiple of ten to keep the filter list still as policies come and go; an unused slot holds an empty config and builds no VM. Fixes: envoyproxy#9355 Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

Non-TLS Gateway listeners sharing an address and port share one xDS listener, one filter chain and one HTTP connection manager. Naming the Lua slots after the IR listener meant each of them appended a bucket of its own, so three listeners on a port produced thirty Lua filters in a single manager, and thirty ECDS resources, twenty seven of them empty. Scope the slots to the RouteConfiguration the manager serves instead. That is what the listeners sharing a manager have in common, and what routes key their per-filter config against, so they now share one set of slots and one set of Lua VMs. A manager patched by a second listener merges its scripts into the slots already there rather than adding more. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

The extraction pass decided what it owned by parsing the filter name, which a filter added by an EnvoyPatchPolicy or an extension server can match: anything called envoy.filters.http.lua/<something>/<digits> was moved to ECDS, taking its configuration out from under its author. Two such filters in different listeners would also have claimed one ECDS resource name. Record the eligible filters as Envoy Gateway builds each connection manager instead, which happens before the patches and the extension hook run, and have the pass lift only those. Names are no longer parsed. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

Filter types that can appear several times on a listener got their order by adding the index within the type to the type's base order. Lua starts at 13 and filters the ordering does not recognize sit at 50, so a listener with 38 Lua filters sorted its last ones after those, and at 88 they reached ext_proc at 100. The same applies to ext_proc, wasm and dynamic modules. Keep the index as a separate sort key instead, compared after the order and before the name, so a type's filters can never reach the next type's position. With the index out of the order, the gaps at 100, 200, 250 and 300 are no longer headroom for anything, so collapse them and put every type one apart. The order for an unrecognized type and the one for the router are the two that carry meaning, so give them names; an unrecognized filter now runs after everything placed deliberately rather than between Lua and ext_proc, still before the terminal router. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

The new ecds command declared "e" as its alias, which endpoint already had. Cobra sorts subcommands by name before matching, so ecds won and egctl config envoy-proxy e silently changed from endpoints to ECDS. Give ecds "ec" instead and fix its example, which advertised the alias it had taken. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

The ECDS filters were set to apply their default config without warming, so a listener started serving as soon as it was created. Envoy subscribes to ECDS only after it has processed the listener, which left a window on every listener update where the slots were present but empty and the scripts did not run. A script that checks a header or rejects a request was simply absent for that window, and nothing in the config dump showed it. Warm instead, and drop the default config with it. A delivery that fails or is rejected now leaves the slot without a config and Envoy answers with a 500, rather than running the route as if the script had never been configured. Envoy skips a slot disabled by default before it looks for a config, so this reaches only a request whose route enabled the slot: a route with no Lua on it, and the slots no route uses, are unaffected. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

The fixture looks like a near copy of lua.yaml, so record that it covers two listeners sharing one HCM and the slot merging that follows from it. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

Every golden carried an empty EcdsConfigDump, so the loop that fills it from the extension configs had never run under test. A typo in the type URL or a validation failure there would have gone unnoticed, and the symptom is egctl quietly omitting the Lua scripts somebody is trying to debug. Attach a Lua EnvoyExtensionPolicy to the route of the xds fixture, which gives the dump a TypedExtensionConfig with a populated sourceCodes map, and covers the listener and route sides of the same change while it is there. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

…missing script A route names a Lua script by a key inside an ECDS resource, and the two are delivered as separate xDS responses in an order the control plane does not choose. A key added in the same push as the route naming it can arrive after the route, and a key removed in the same push can leave before it. In that window Envoy finds no script under the name and skips the filter silently, which for a denying script is a bypass. Sequence the publish instead of relying on the order. When keys are added, first publish the previous snapshot with the ECDS resources holding both old and new keys, so the scripts exist before anything names them. When keys are removed, publish the new snapshot with the old keys still present and prune them in a further snapshot, so nothing names a script after it is gone. A push that changes no key, which is every script edit, stays a single snapshot as before. The snapshot cache already keeps the last snapshot per IR key, so the previous resources come from there. Snapshots published under one trace share a version, which the cache takes as nothing to send, so a repeated version now gets a suffix. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

Picks up the mirror of envoyproxy/envoy@c5f7278f, which adds shared_vm_id to the Lua filter and its per-route config. A route can then carry its script inline while Envoy shares one VM across every route with the same script and id, which is what lets Lua policies move out of the listener without the per-route VM duplication from envoyproxy#9355. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

zhaohuabing · 2026-09-29T03:17:33Z

@codex review

Serving the scripts over ECDS left a route able to name a script its slot did not hold yet, or no longer held, because the route and the script travel as separate xDS responses in an order the control plane cannot guarantee without tracking each proxy's acknowledgements. Envoy skips the script silently in that window. Put the script back on the route instead, as LuaPerRoute source with a shared_vm_id equal to the script's name. Envoy then builds one set of Lua VMs per script and shares it across every route carrying that script, and across the TCP and QUIC managers of a listener, so the per-route VM duplication from envoyproxy#9355 cannot return. The listener keeps one empty, disabled placeholder filter per slot, so a policy change never rewrites it. Nothing crosses resources any more, so the ECDS extraction, the ordered publish, the snapshot version suffix, the egctl ECDS dump and the per-route route config plumbing all go. Filter names shrink to their slot, since they only have to be unique within a manager. This needs an Envoy build that includes envoyproxy/envoy#47562. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

This comment was marked as outdated.

Sign in to view

zhaohuabing force-pushed the lua-shared-source-codes branch 4 times, most recently from f277143 to fb0cf88 Compare September 21, 2026 03:15

This comment was marked as resolved.

Sign in to view

zhaohuabing force-pushed the lua-shared-source-codes branch 4 times, most recently from 44e2595 to 01c6d30 Compare September 21, 2026 05:13

zhaohuabing mentioned this pull request Sep 21, 2026

fix(xds): keep indexed HTTP filters within their own order band #10063

Closed

zhaohuabing force-pushed the lua-shared-source-codes branch 2 times, most recently from 316c22e to 19fbfac Compare September 21, 2026 06:11

zhaohuabing added this to the v1.10.0-rc.1 Release milestone Sep 21, 2026

zhaohuabing added cherrypick/release-v1.8.5 cherrypick/release-v1.9.2 labels Sep 21, 2026

zhaohuabing force-pushed the lua-shared-source-codes branch 4 times, most recently from 2433302 to 3df7503 Compare September 21, 2026 09:14

zhaohuabing marked this pull request as ready for review September 22, 2026 02:48

zhaohuabing requested a review from a team as a code owner September 22, 2026 02:48

zhaohuabing force-pushed the lua-shared-source-codes branch from 0d19f38 to 2fd3fbe Compare September 22, 2026 02:50

zhaohuabing requested a review from arkodg September 22, 2026 03:03

zhaohuabing force-pushed the lua-shared-source-codes branch 4 times, most recently from 1650031 to a9ee2f4 Compare September 28, 2026 10:10

zhaohuabing changed the title ~~fix(xds): share Lua scripts across routes and deliver them over ECDS~~ fix(lua): per-route config and share vm across multiple routes Sep 28, 2026

zhaohuabing changed the title ~~fix(lua): per-route config and share vm across multiple routes~~ fix(xds): carry Lua scripts on the routes with a shared VM per script Sep 28, 2026

zhaohuabing force-pushed the lua-shared-source-codes branch 2 times, most recently from 13f5731 to 9430c68 Compare September 28, 2026 10:49

zhaohuabing marked this pull request as ready for review September 28, 2026 10:50

zhaohuabing requested review from a team and rudrakhp September 28, 2026 10:50

zhaohuabing force-pushed the lua-shared-source-codes branch 2 times, most recently from 5c710be to c720783 Compare September 28, 2026 12:20

zhaohuabing and others added 10 commits September 29, 2026 02:00

polish

3c6a8cd

Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>

zhaohuabing force-pushed the lua-shared-source-codes branch from c720783 to 0fcd8bf Compare September 29, 2026 02:07

zhaohuabing force-pushed the lua-shared-source-codes branch 2 times, most recently from 7a2d92a to cbebc5b Compare September 29, 2026 03:10

zhaohuabing force-pushed the lua-shared-source-codes branch from cbebc5b to 5a5fe7e Compare September 29, 2026 03:31

Navigation Menu

fix(xds): carry Lua scripts on the routes with a shared VM per script - #10056

fix(xds): carry Lua scripts on the routes with a shared VM per script#10056
zhaohuabing wants to merge 12 commits into
envoyproxy:mainfrom
zhaohuabing:lua-shared-source-codes

zhaohuabing commented Sep 19, 2026 •

edited

Loading

Uh oh!

netlify Bot commented Sep 19, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 19, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as resolved.

zhaohuabing commented Sep 21, 2026

Uh oh!

zhaohuabing commented Sep 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

zhaohuabing commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify Bot commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for cerulean-figolla-1f9435 ready!

Uh oh!

codecov Bot commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as resolved.

zhaohuabing commented Sep 21, 2026

Uh oh!

zhaohuabing commented Sep 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zhaohuabing commented Sep 19, 2026 •

edited

Loading

netlify Bot commented Sep 19, 2026 •

edited

Loading

codecov Bot commented Sep 19, 2026 •

edited

Loading