Align leave-one-out connected-set pruning with the official KSS MATLAB implementation

[ulrichwohak]

Fixes #46.

Summary

This PR aligns leave-one-out connected-set construction with the released Kline–Saggio–Sølvsten (KSS) MATLAB implementation while preserving VCHDFE's public API and subsequent bias-correction code.

The change is deliberately limited to sample selection:

• construct and rank connected components in the firm-transition graph;
• detect articulation workers on deduplicated mover–firm pairs;
• remove each articulation worker's complete raw-observation history;
• repeat component selection and pruning to a fixed point;
• drop one-observation workers only after that fixed point.

find_connected_set, prunning_connected_set, get_leave_one_out_set, their signatures and return fields, and the executable interface remain unchanged. The misspelled exported name prunning_connected_set is retained for compatibility.

Problem

The released MATLAB code and the previous Julia selector used different component objectives:

Stage	Released KSS MATLAB	Previous VCHDFE selector
Initial connected set	Largest component of the firm-transition graph, ranked by firms	Largest component of the full worker–firm bipartite graph, ranked by workers plus firms
Articulation graph	Movers and unique mover–firm pairs	All workers and firms
Component after deletion	Largest mover-induced firm-transition component, ranked by firms	Largest full bipartite component, ranked by workers plus firms

On a common sample, the two articulation graphs identify the same articulation workers. Stayer workers are degree-one leaves. Leaves are never cut vertices, and attaching leaves to firm vertices cannot change whether a worker vertex is a cut vertex. The algorithms can therefore first diverge when selecting a surviving component: MATLAB maximizes the number of firms in the mobility component, whereas the previous Julia routine maximized the total number of worker and firm vertices.

The previous routine also retained a stale worker count across iterations. That state adds isolated phantom vertices and wastes memory, but does not appear to alter articulation workers or component selection. This PR recomputes current graph dimensions in every round without presenting the stale count as a separate correctness bug.

Reproduction

The regression fixture joins:

• a two-worker, three-firm K(2, 3) mover core;
• an articulation worker connecting that core to a fourth firm; and
• six two-observation stayers at the fourth firm.

After removing the articulation worker, the MATLAB rule retains the three-firm mover core (six observations). Total-vertex ranking instead retains the one-firm stayer component (twelve observations). This establishes algorithmic divergence without relying on private data.

Both selectors happen to agree on the public test.csv fixture:

• 71,614 input observations;
• 63,462 observations after the initial firm component;
• 56,044 final observations;
• 28,022 workers;
• 1,684 firms;
• 34,436 matches; and
• 6,414 movers.

Agreement on this fixture is useful for regression coverage, but it does not establish equivalence; the synthetic component-ranking example does.

Implementation

The new selector independently translates the behavior of the released MATLAB routines at commit 8b957ff:

• connected_set.m
• pruning_unbal_v3.m
• build_adj.m
• leave_out_KSS.m

The implementation:

1. preserves the existing worker/observation sorting precondition;
2. operates on raw observation rows and preserves original observation indices;
3. relabels worker and firm identifiers densely after each restriction;
4. raises descriptive ArgumentErrors for tied largest firm components and disappearance of the mover graph; and
5. changes neither exact/JLA estimation nor dependencies.

The articulation graph retains the full current firm dimension, including isolated stayer-only firms, matching the MATLAB construction. Mover–firm edges are deduplicated before articulation detection.

Pruning is always leave-one-worker-out

get_leave_one_out_set always constructs a raw-observation sample that remains connected after removing any one worker's complete history. VCHDFESettings.leave_out_level does not alter sample pruning. Its "match" and "obs" values apply later, inside leave_out_estimation, when computing the bias correction on the selected sample.

The new documentation page makes this separation explicit and records the old and new component rules.

Tests

The new always-run pruning suite covers:

• the synthetic firm-ranking divergence;
• equality of articulation-worker sets with and without degree-one stayers;
• a two-round articulation cascade;
• raw-row singleton counting;
• tied-component and no-mover errors;
• original observation-index preservation and dense relabeling; and
• all public KSS fixture counts above.

Validated locally with Julia 1.8.5:

julia +1.8.5 --project=. -e 'using Pkg; Pkg.test()'
Official KSS pruning | 21 passed

julia +1.8.5 --project=docs docs/make.jl
Documentation build completed

The standalone legacy command julia +1.8.5 --project=. test/test_matrices.jl is not currently runnable on main: it imports undeclared DataFramesMeta, and after bypassing that import it references the removed eff_res helper. This PR does not broaden scope into migrating that stale, non-CI test file; its pruning assertions are represented in the new always-run suite.

Related: #17 concerns parallelization and performance; it is not a duplicate of this component-selection issue.

---

AI Attribution Statement

This work was created with an even blend of human and AI contributions. AI was used to make stylistic edits, such as changes to structure, wording, and clarity. AI was used to make content edits, such as changes to scope, information, and ideas. AI was used to make new content, such as text, images, analysis, and ideas. AI was prompted for its contributions, or AI assistance was enabled. AI-generated content was reviewed and approved. The following model(s) or application(s) were used: GPT 5.6 Sol.

AIA Human-AI blend, Stylistic edits, Content edits, New content, Human-initiated, Reviewed, GPT 5.6 Sol v1.0