Repository navigation
A missing fork or nested commit in the primary is fetched as bootstrap fetches one, statelessly, and a linked worktree's fork checkout is never moved - #831
Conversation
|
Mutation patches for head negative-control: EXIT=101diff --git a/src/llvm.rs b/src/llvm.rs
index 2350a58d8..cb55f7a39 100644
--- a/src/llvm.rs
+++ b/src/llvm.rs
@@ -42,7 +42,7 @@ use std::sync::OnceLock;
use crate::buildlock::{Guard, Keyed};
use crate::compiler::LLVM;
use crate::keystore::{self, Key};
-use crate::sysroot::{clone_tree, git_bytes, git_out, gitlink, recorded_url};
+use crate::sysroot::{clone_tree, git_bytes, git_out, gitlink};
use crate::toolchain::{self, host_triple};
/// What changes how a key's sources become an LLVM and is none of the other
@@ -142,7 +142,9 @@ struct Stamp {
/// the checkout it builds, and writes that checkout's `origin` into every
/// binary: an input no key reads.
fn stamp(fork: &Path) -> Stamp {
- Stamp { revision: gitlink(fork, LLVM), repository: recorded_url(fork, "HEAD", LLVM) }
+ let name = format!("submodule.{LLVM}.url");
+ let url = git_out(fork, &["config", "--blob", "HEAD:.gitmodules", "--get", &name]);
+ Stamp { revision: gitlink(fork, LLVM), repository: url.trim().to_string() }
}
/// [`key`], with what it reads beside `fork`'s committed `src/bootstrap`:
diff --git a/src/sysroot.rs b/src/sysroot.rs
index 83a1d085b..cefab321f 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -520,10 +520,6 @@ fn pinned_fork(root: &Path) -> String {
/// a commit that neither the checkout's `HEAD` nor the pin records there, which
/// only its own `HEAD` may record.
///
-/// A missing commit is never fetched over the network here: the refusal names
-/// the command that fetches it from the URL the pinning tree's `.gitmodules`
-/// records ([`fetch_command`]).
-///
/// Every build in a worktree asks this at once, so the making and the move are
/// each decided and done under the worktree's lock held exclusively
/// ([`Held::act_if`]), which every build that writes or compiles the checkout
@@ -585,9 +581,9 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
Some(primary) => git_run(&fork, &["fetch", path_str(&primary.join("rust")), &pinned]),
None => panic!(
"{} is at {head}, and this tree pins the fork at {pinned}, which it does not hold: \
- `{}` fetches it",
+ `git -C {} fetch origin {pinned}` fetches it",
+ fork.display(),
fork.display(),
- fetch_command(&fork, &recorded_url(root, "", "rust"), &pinned),
),
}
}
@@ -601,17 +597,18 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
}
})
.collect();
- let nested: Vec<(PathBuf, &str, &str)> = recorded
+ let nested: Vec<(PathBuf, String)> = recorded
.iter()
- .map(|(path, commit)| (fork.join(path), path.as_str(), commit.as_str()))
- .filter(|(at, _, commit)| at.join(".git").exists() && git_out(at, &["rev-parse", "HEAD"]).trim() != *commit)
+ .map(|(path, commit)| (fork.join(path), commit.clone()))
+ .filter(|(at, commit)| at.join(".git").exists() && git_out(at, &["rev-parse", "HEAD"]).trim() != commit)
.collect();
- for (at, path, commit) in &nested {
+ for (at, commit) in &nested {
assert!(
holds(at, commit),
- "{} does not hold {commit}, which the fork's {pinned} records there: `{}` fetches it",
+ "{} does not hold {commit}, which the fork's {pinned} records there: \
+ `git -C {} fetch origin {commit}` fetches it",
+ at.display(),
at.display(),
- fetch_command(at, &recorded_url(&fork, &pinned, path), commit),
);
}
// A nested submodule checked out at the commit the pin records there, and nothing more, is no work: a move
@@ -639,7 +636,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
fork.display(),
work.join("\n"),
);
- for (at, _, commit) in &nested {
+ for (at, commit) in &nested {
git_run(at, &["checkout", "--detach", "-q", commit]);
}
// Last, so a move killed before it is asked for again: `HEAD` alone decides.
@@ -655,22 +652,6 @@ fn holds(dir: &Path, commit: &str) -> bool {
git_try(dir, &["cat-file", "-e", &format!("{commit}^{{commit}}")]).is_ok()
}
-/// The URL that `tree`'s `.gitmodules` in `repo` records for the submodule at
-/// `path`, which it names by its path.
-pub(crate) fn recorded_url(repo: &Path, tree: &str, path: &str) -> String {
- let blob = format!("{tree}:.gitmodules");
- git_out(repo, &["config", "--blob", &blob, "--get", &format!("submodule.{path}.url")]).trim().to_string()
-}
-
-/// The command that fetches `commit` into the checkout at `at` from `url`, the
-/// one the pinning tree records: never `origin`, which is whatever URL the
-/// `.gitmodules` of the checkout's making named. A shallow checkout fetches
-/// the commit alone, not the history between it and what it holds.
-fn fetch_command(at: &Path, url: &str, commit: &str) -> String {
- let shallow = git_out(at, &["rev-parse", "--is-shallow-repository"]).trim() == "true";
- format!("git -C {} fetch {}{url} {commit}", at.display(), if shallow { "--depth 1 " } else { "" })
-}
-
/// What a sysroot's [`SOURCES`] says: its key, and the witness of the sources
/// it was built from.
fn sources_text(key: &Key, witness: &str) -> String {
@@ -1739,31 +1720,6 @@ mod tests {
assert_eq!(git(&primary, &["rev-parse", "HEAD"]), superproject, "git ran in the superproject");
}
- /// **A shallow checkout is named a fetch of the commit alone**, which
- /// fetches it.
- #[test]
- fn a_shallow_checkout_is_named_a_fetch_of_the_commit_alone() {
- let base = TempDir::new("fetch-shallow");
- let src = base.join("src");
- write(&src.join("a"), "1\n");
- git(&src, &["init", "-q"]);
- git(&src, &["add", "-A"]);
- git(&src, &["commit", "-qm", "one"]);
- let url = format!("file://{}", src.display());
- let shallow = base.join("shallow");
- git(&base, &["clone", "-q", "--depth", "1", &url, path_str(&shallow)]);
- write(&src.join("a"), "2\n");
- git(&src, &["commit", "-qam", "two"]);
- let two = git(&src, &["rev-parse", "HEAD"]);
-
- let command = fetch_command(&shallow, &url, &two);
-
- assert_eq!(command, format!("git -C {} fetch --depth 1 {url} {two}", shallow.display()));
- let words: Vec<&str> = command.split_whitespace().collect();
- git(&base, &words[1..]);
- assert!(holds(&shallow, &two), "the named fetch did not fetch {two}");
- }
-
/// A compiler in `store`: `rustc` and `rust-lld`, and the C toolchain
/// `src/clang.rs` provisions beside them if `clang`; no cargo.
fn compiler_without_cargo(store: &Path, clang: bool) -> Compiler {mutation-nested-origin: EXIT=101diff --git a/src/sysroot.rs b/src/sysroot.rs
index 83a1d085b..2c5716a21 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -611,7 +611,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
holds(at, commit),
"{} does not hold {commit}, which the fork's {pinned} records there: `{}` fetches it",
at.display(),
- fetch_command(at, &recorded_url(&fork, &pinned, path), commit),
+ format!("git -C {} fetch origin {commit}", at.display()),
);
}
// A nested submodule checked out at the commit the pin records there, and nothing more, is no work: a movemutation-shallow-inverted: EXIT=101diff --git a/src/sysroot.rs b/src/sysroot.rs
index 83a1d085b..017e8af61 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -668,7 +668,7 @@ pub(crate) fn recorded_url(repo: &Path, tree: &str, path: &str) -> String {
/// the commit alone, not the history between it and what it holds.
fn fetch_command(at: &Path, url: &str, commit: &str) -> String {
let shallow = git_out(at, &["rev-parse", "--is-shallow-repository"]).trim() == "true";
- format!("git -C {} fetch {}{url} {commit}", at.display(), if shallow { "--depth 1 " } else { "" })
+ format!("git -C {} fetch {}{url} {commit}", at.display(), if !shallow { "--depth 1 " } else { "" })
}
/// What a sysroot's [`SOURCES`] says: its key, and the witness of the sources |
|
Review of #831 at BLOCKER
NOTE
SEND BACK |
…p fetches one; a move in flight is finished After the fork's LLVM pin moved to ToyOSOrg's repository, the primary's fork checkout refused with `git fetch origin <commit>`, which failed: the owner's `src/llvm-project` was made when `.gitmodules` named rust-lang's, so its `origin` lacks the commit. The build already fetches pinned submodule commits through the admitted `git submodule update` row, and rustc's bootstrap does exactly that on every LLVM build: `git submodule sync`, then `git submodule update --init`, depth 1 where `.gitmodules` says `shallow = true`. So the primary's fork checkout now does the same where it lacks the pinned commit, its own (from the superproject) or a nested submodule's (from the fork). `sync` reads the URL from the moved checkout's `.gitmodules`, so a stale `origin` (one made before the pin's tree named ToyOSOrg's repository) is cured at its root. The primary's refusal arms go. Because git's update reads a nested submodule's URL and commit from the fork's worktree and index, the fork now moves before its nested submodules, and `HEAD` can no longer be the last step. A file in the fork's own git directory (`rev-parse --git-path toyos-fork-move`) says a move is in flight; the next build finishes it, and while it exists a nested submodule's commit alone is no work. The update's own fetch is told not to recurse into the fork's nested submodules: on-demand recursion would fetch each from the `origin` it had before its move, which in the owner's case is rust-lang's llvm-project. A linked worktree runs no `git submodule`: its nested checkout is a git worktree of the primary's, and `git submodule update` there writes its own path as `core.worktree` into the config the primary's shares (measured: git in the primary's `rust/` then fails with "cannot chdir"). A linked worktree lacking a nested commit is still refused, without a command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
b3cf6b5 to
09fd82c
Compare
|
Gate logs at head
|
|
Measurement: where each new
hazard.sh outputfixture.shset -e
S=$1; rm -rf $S; mkdir -p $S; cd $S
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t
g(){ git -c protocol.file.allow=always "$@"; }
mkdir bt && cd bt && git init -q && echo 1 > lib.rs && git add -A && git commit -qm b1 && B1=$(git rev-parse HEAD) && echo 2 > lib.rs && git commit -qam b2 && B2=$(git rev-parse HEAD) && cd ..
mkdir fork && cd fork && git init -q && echo a > x.py && g submodule add -q $S/bt library/backtrace && git -C library/backtrace checkout -q $B1 && git add -A && git commit -qm C1 && C1=$(git rev-parse HEAD) && git -C library/backtrace checkout -q $B2 && echo b > x.py && git commit -qam C2 && C2=$(git rev-parse HEAD) && cd ..
mkdir primary && cd primary && git init -q && echo x > f && g submodule add -q $S/fork rust && git -C rust checkout -q $C1 && git add -A && g submodule update -q --init --recursive && git commit -qm p && cd ..
git -C primary worktree add -q -b wt $S/linked
rmdir linked/rust; git -C primary/rust worktree add -q --detach $S/linked/rust $C2
echo "B1=$B1 B2=$B2 C1=$C1 C2=$C2" > $S/idshazard.sh# Each submodule command the change could run, run where it would run; prints what it wrote to the primary's configs and whether git in the primary's `rust/` still runs.
F=$1; E=$2
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1
g(){ git -c protocol.file.allow=always "$@"; }
snap(){ cp $E/primary/.git/modules/rust/config $E/rc; cp $E/primary/.git/modules/rust/modules/library/backtrace/config $E/bc; }
check(){ diff $E/rc $E/primary/.git/modules/rust/config; diff $E/bc $E/primary/.git/modules/rust/modules/library/backtrace/config; echo "configs unchanged: $([ $? = 0 ] && echo yes || echo NO)"; git -C $E/primary/rust status --porcelain=v2 --ignore-submodules=none >/dev/null 2>$E/err; echo "git status in the primary's rust/: exit $? $(cat $E/err)"; }
for case in linked-sync linked-update primary-superproject primary-fork; do
echo "== $case"; bash $F $E >/dev/null || exit 1; . $E/ids
rmdir $E/linked/rust/library/backtrace; git -C $E/primary/rust/library/backtrace worktree add -q --detach $E/linked/rust/library/backtrace $B2
snap
case $case in
linked-sync) g -C $E/linked/rust submodule sync -- library/backtrace ;;
linked-update) g -C $E/linked/rust submodule update --init --checkout -- library/backtrace ;;
primary-superproject) git -C $E/primary update-index --cacheinfo 160000,$C2,rust; g -C $E/primary submodule sync -- rust; g -C $E/primary submodule update --init --checkout -- rust ;;
primary-fork) git -C $E/primary/rust checkout -q --detach $C2; g -C $E/primary/rust submodule sync -- library/backtrace; g -C $E/primary/rust submodule update --init --checkout -- library/backtrace ;;
esac
echo "command exit: $?"; check
done |
|
Mutations and the negative control at head
m1-url-from-head.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..daa4ed7eb 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -630,6 +630,13 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
fork.display(),
work.join("\n"),
);
+ if primary.is_none() {
+ for (path, _) in gitlinks(&fork, "HEAD") {
+ if fork.join(&path).join(".git").exists() {
+ git_run(&fork, &["submodule", "sync", "--", &path]);
+ }
+ }
+ }
let moving = moving(&fork);
fs::write(&moving, "").unwrap_or_else(|e| panic!("write {}: {e}", moving.display()));
match (&primary, holds(&fork, &pinned)) {
@@ -691,7 +698,9 @@ fn gitlinks(repo: &Path, tree: &str) -> Vec<(String, String)> {
fn submodule_update(repo: &Path, path: &str) {
let shallow = format!("submodule.{path}.shallow");
let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
- git_run(repo, &["submodule", "sync", "--", path]);
+ if path == "rust" {
+ git_run(repo, &["submodule", "sync", "--", path]);
+ }
let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
git_run(repo, &[&update[..], depth, &["--", path]].concat());m2-no-journal-in-predicate.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..a973ca14f 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -580,7 +580,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
.current_dir(&fork)
.status()
.is_ok_and(|s| s.success());
- (head != pinned && !ahead || moving(&fork).exists()).then_some(head)
+ (head != pinned && !ahead).then_some(head)
},
|head| {
if let Some(primary) = &primary {m3-no-in-flight-exemption.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..29f347d8d 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -609,7 +609,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
.lines()
.filter_map(|entry| {
let path = entry.strip_prefix("1 .M SC.. ").and_then(|rest| rest.splitn(6, ' ').nth(5));
- if path.is_some() && in_flight {
+ if path.is_some() && in_flight && false {
return None;
}
let Some((path, commit)) = path.and_then(|path| recorded.iter().find(|(p, _)| p == path)) else {m4-linked-runs-git-submodule.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..83849ab2f 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -590,7 +590,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
}
// The primary's gets a pin it lacks with its move, so nothing it records is known yet.
let recorded = if holds(&fork, &pinned) { gitlinks(&fork, &pinned) } else { Vec::new() };
- if primary.is_some() {
+ if false {
for (path, commit) in &recorded {
let at = fork.join(path);
assert!(
@@ -645,7 +645,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
match (&primary, holds(&at, &commit)) {
(_, true) => git_run(&at, &["checkout", "--detach", "-q", &commit]),
(None, false) => submodule_update(&fork, &path),
- (Some(_), false) => unreachable!("{} lacks {commit}, which was refused above", at.display()),
+ (Some(_), false) => submodule_update(&fork, &path),
}
}
fs::remove_file(&moving).unwrap_or_else(|e| panic!("remove {}: {e}", moving.display()));m5-no-depth.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..3c657b7c6 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -692,7 +692,7 @@ fn submodule_update(repo: &Path, path: &str) {
let shallow = format!("submodule.{path}.shallow");
let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
git_run(repo, &["submodule", "sync", "--", path]);
- let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
+ let depth: &[&str] = if shallow.trim() == "never" { &["--depth", "1"] } else { &[] };
let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
git_run(repo, &[&update[..], depth, &["--", path]].concat());
}m6-no-sync.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..194e78fbb 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -691,7 +691,6 @@ fn gitlinks(repo: &Path, tree: &str) -> Vec<(String, String)> {
fn submodule_update(repo: &Path, path: &str) {
let shallow = format!("submodule.{path}.shallow");
let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
- git_run(repo, &["submodule", "sync", "--", path]);
let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
git_run(repo, &[&update[..], depth, &["--", path]].concat());m7-fetch-recurses.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..c68885190 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -693,7 +693,7 @@ fn submodule_update(repo: &Path, path: &str) {
let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
git_run(repo, &["submodule", "sync", "--", path]);
let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
- let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
+ let update = ["-c", "fetch.recurseSubmodules=on-demand", "submodule", "update", "--init", "--checkout"];
git_run(repo, &[&update[..], depth, &["--", path]].concat());
}
nc-whole-change-reverted.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 0adde5811..8efa09a82 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -502,9 +502,8 @@ fn pinned_fork(root: &Path) -> String {
///
/// The primary's is its own `rust/`, used where it is not initialised yet, which
/// whoever initialises it does at the pin. It is no workspace, so one whose
-/// `HEAD` is not the pin is moved there, refused by name if it has local
-/// changes; a commit missing from it or from a nested submodule is fetched as
-/// rustc's bootstrap fetches one ([`submodule_update`]).
+/// `HEAD` is not the pin is moved there; refused by name if it has local
+/// changes, or does not hold the pinned commit.
///
/// A linked worktree's `rust/` starts as the empty stub `git worktree add`
/// leaves; it is made here, the first time it is needed, as a git worktree of
@@ -517,15 +516,9 @@ fn pinned_fork(root: &Path) -> String {
/// changes rather than moved out from under whoever made them.
///
/// Every nested submodule checked out in it moves with it to the commit the pin
-/// records there, refused by name where it is at a commit that neither the
-/// checkout's `HEAD` nor the pin records there, which only its own `HEAD` may
-/// record — and in a linked worktree where it does not hold that commit, which
-/// [`submodule_update`] may not fetch there.
-///
-/// The checkout moves before its nested submodules, because git's update reads
-/// their URLs and commits from the moved checkout's `.gitmodules` and index; a
-/// move killed between the two is one [`moving`] says is in flight, which the
-/// next finishes.
+/// records there, refused by name where it does not hold that commit, or is at
+/// a commit that neither the checkout's `HEAD` nor the pin records there, which
+/// only its own `HEAD` may record.
///
/// Every build in a worktree asks this at once, so the making and the move are
/// each decided and done under the worktree's lock held exclusively
@@ -580,38 +573,51 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
.current_dir(&fork)
.status()
.is_ok_and(|s| s.success());
- (head != pinned && !ahead || moving(&fork).exists()).then_some(head)
+ (head != pinned && !ahead).then_some(head)
},
|head| {
- if let Some(primary) = &primary {
- if !holds(&fork, &pinned) {
- git_run(&fork, &["fetch", path_str(&primary.join("rust")), &pinned]);
+ if !holds(&fork, &pinned) {
+ match &primary {
+ Some(primary) => git_run(&fork, &["fetch", path_str(&primary.join("rust")), &pinned]),
+ None => panic!(
+ "{} is at {head}, and this tree pins the fork at {pinned}, which it does not hold: \
+ `git -C {} fetch origin {pinned}` fetches it",
+ fork.display(),
+ fork.display(),
+ ),
}
}
- // The primary's gets a pin it lacks with its move, so nothing it records is known yet.
- let recorded = if holds(&fork, &pinned) { gitlinks(&fork, &pinned) } else { Vec::new() };
- if primary.is_some() {
- for (path, commit) in &recorded {
- let at = fork.join(path);
- assert!(
- !at.join(".git").exists() || holds(&at, commit),
- "{} does not hold {commit}, which the fork's {pinned} records there, and a linked worktree's \
- fork checkout gets a commit only from the primary's",
- at.display(),
- );
- }
+ let recorded: Vec<(String, String)> = git_out(&fork, &["ls-tree", "-r", &pinned])
+ .lines()
+ .filter_map(|entry| {
+ let (meta, path) = entry.split_once('\t')?;
+ match meta.split_whitespace().collect::<Vec<_>>().as_slice() {
+ ["160000", "commit", commit] => Some((path.to_string(), commit.to_string())),
+ _ => None,
+ }
+ })
+ .collect();
+ let nested: Vec<(PathBuf, String)> = recorded
+ .iter()
+ .map(|(path, commit)| (fork.join(path), commit.clone()))
+ .filter(|(at, commit)| at.join(".git").exists() && git_out(at, &["rev-parse", "HEAD"]).trim() != commit)
+ .collect();
+ for (at, commit) in &nested {
+ assert!(
+ holds(at, commit),
+ "{} does not hold {commit}, which the fork's {pinned} records there: \
+ `git -C {} fetch origin {commit}` fetches it",
+ at.display(),
+ at.display(),
+ );
}
- // A nested submodule checked out at the commit the pin records there, and nothing more, is no work, nor is
- // one at any commit while a move is in flight. One at any other commit may hold a commit nothing else records.
- let in_flight = moving(&fork).exists();
+ // A nested submodule checked out at the commit the pin records there, and nothing more, is no work: a move
+ // killed before its last step leaves it so. One at any other commit may hold a commit nothing else records.
let status = git_out(&fork, &["status", "--porcelain=v2", "--ignore-submodules=none"]);
let work: Vec<String> = status
.lines()
.filter_map(|entry| {
let path = entry.strip_prefix("1 .M SC.. ").and_then(|rest| rest.splitn(6, ' ').nth(5));
- if path.is_some() && in_flight {
- return None;
- }
let Some((path, commit)) = path.and_then(|path| recorded.iter().find(|(p, _)| p == path)) else {
return Some(entry.to_string());
};
@@ -630,73 +636,17 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
fork.display(),
work.join("\n"),
);
- let moving = moving(&fork);
- fs::write(&moving, "").unwrap_or_else(|e| panic!("write {}: {e}", moving.display()));
- match (&primary, holds(&fork, &pinned)) {
- (_, true) => git_run(&fork, &["checkout", "--detach", "-q", &pinned]),
- (None, false) => submodule_update(root, "rust"),
- (Some(_), false) => unreachable!("a linked worktree's fork checkout fetched {pinned} above"),
+ for (at, commit) in &nested {
+ git_run(at, &["checkout", "--detach", "-q", commit]);
}
- for (path, commit) in gitlinks(&fork, "HEAD") {
- let at = fork.join(&path);
- if !at.join(".git").exists() || git_out(&at, &["rev-parse", "HEAD"]).trim() == commit {
- continue;
- }
- match (&primary, holds(&at, &commit)) {
- (_, true) => git_run(&at, &["checkout", "--detach", "-q", &commit]),
- (None, false) => submodule_update(&fork, &path),
- (Some(_), false) => unreachable!("{} lacks {commit}, which was refused above", at.display()),
- }
- }
- fs::remove_file(&moving).unwrap_or_else(|e| panic!("remove {}: {e}", moving.display()));
+ // Last, so a move killed before it is asked for again: `HEAD` alone decides.
+ git_run(&fork, &["checkout", "--detach", "-q", &pinned]);
eprintln!("{} was at {head}, and this tree pins {pinned}: checked it out", fork.display());
},
);
fork
}
-/// The file in `fork`'s own git directory whose presence says a move of it to
-/// its pin is in flight: the checkout may be at the pin while a nested
-/// submodule is not yet.
-fn moving(fork: &Path) -> PathBuf {
- fork.join(git_out(fork, &["rev-parse", "--git-path", "toyos-fork-move"]).trim())
-}
-
-/// The submodules `tree` in `repo` records, by path, each with its commit.
-fn gitlinks(repo: &Path, tree: &str) -> Vec<(String, String)> {
- git_out(repo, &["ls-tree", "-r", tree])
- .lines()
- .filter_map(|entry| {
- let (meta, path) = entry.split_once('\t')?;
- match meta.split_whitespace().collect::<Vec<_>>().as_slice() {
- ["160000", "commit", commit] => Some((path.to_string(), commit.to_string())),
- _ => None,
- }
- })
- .collect()
-}
-
-/// The submodule at `path` in `repo` checked out at the commit `repo`'s index
-/// records there, fetched as rustc's bootstrap fetches its LLVM: `git submodule
-/// sync`, so its `origin` is the URL `repo`'s `.gitmodules` records, then `git
-/// submodule update --init`, one commit deep where `.gitmodules` declares it
-/// `shallow`. Its own fetch recurses into no submodule of its own, whose
-/// `origin` is synced only once it has moved.
-///
-/// `repo` must be the primary's superproject or its own fork checkout, whose
-/// submodules' git directories are their own: run in a linked worktree's fork
-/// checkout, whose nested one is a git worktree of the primary's, `update`
-/// writes that checkout's path as `core.worktree` into the config the
-/// primary's shares, and git in the primary's fails.
-fn submodule_update(repo: &Path, path: &str) {
- let shallow = format!("submodule.{path}.shallow");
- let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
- git_run(repo, &["submodule", "sync", "--", path]);
- let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
- let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
- git_run(repo, &[&update[..], depth, &["--", path]].concat());
-}
-
/// Whether the repository at `dir` holds `commit`.
fn holds(dir: &Path, commit: &str) -> bool {
git_try(dir, &["cat-file", "-e", &format!("{commit}^{{commit}}")]).is_ok()
@@ -1161,6 +1111,11 @@ fn git_run(dir: &Path, args: &[&str]) {
assert!(ok, "git {args:?} in {} failed", dir.display());
}
+/// Negative-control shim: the name the kept tests read, at the path the change uses.
+fn moving(fork: &Path) -> PathBuf {
+ fork.join(git_out(fork, &["rev-parse", "--git-path", "toyos-fork-move"]).trim())
+}
+
#[cfg(test)]
mod tests {
use super::*; |
|
Review of #831 at Earlier BLOCKERs
On the two additions
BLOCKER
NOTE
SEND BACK |
…out that is ahead of its pin `(head != pinned && !ahead || moving)` parsed as `(head != pinned && !ahead) || moving`, so a marker left by a killed move overrode `ahead`: a commit the agent made in its fork checkout after the kill was checked out from under it to finish the move. The predicate is now `(head != pinned || moving) && !ahead`; `ahead` is false in the primary, whose resume of a killed move is unchanged. a_worktree_pinning_another_fork_commit_gets_its_own_checkout leaves a marker under a checkout ahead of its pin and asserts it stays there; under the old parse it reds, exit 101. The tracker's entry for the pre-existing `git submodule update --init library/backtrace` in a linked worktree's fork checkout carries what git 2.54.0 does there: at that arm's own state (an empty `library/backtrace`) it clones a git directory of the checkout's own and leaves the primary's configuration alone; over a `library/backtrace` that is a git worktree of the primary's nested repository it rewrites the primary's nested `core.worktree`, and git in the primary's `rust/` exits 128. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
Gate logs at head
|
|
Paths scrubbed: the worktree reads Mutation m8, the review's BLOCKER: the move predicate's old parse, Result: m8.patch--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -580,7 +580,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
.current_dir(&fork)
.status()
.is_ok_and(|s| s.success());
- ((head != pinned || moving(&fork).exists()) && !ahead).then_some(head)
+ (head != pinned && !ahead || moving(&fork).exists()).then_some(head)
},
|head| {
if let Some(primary) = &primary {m8.shcd <worktree>
R=<scratch>
cargo test --lib -- sysroot::tests:: > $R/green.log 2>&1; echo EXIT=$? >> $R/green.log
git apply --check $R/m8.patch && git apply $R/m8.patch && { cargo test --lib -- sysroot::tests:: > $R/m8.log 2>&1; echo EXIT=$? >> $R/m8.log; git apply -R $R/m8.patch; }
git diff > $R/restored.diff; echo DONE > $R/donem8.log (mutated), wholegreen.log (fixed), whole |
|
Measurement for the review's NOTE on src/sysroot.rs:568: the pre-existing Result: the command exits 0, clones a git directory of the linked checkout's own ( empty.sh# Line 568's state: the linked fork checkout fresh from `git worktree add`, its library/backtrace an empty directory.
F=$1; E=$2
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1
bash $F $E >/dev/null || exit 1; . $E/ids
cp $E/primary/.git/modules/rust/modules/library/backtrace/config $E/bc
echo "linked library/backtrace entries before: $(ls -A $E/linked/rust/library/backtrace | wc -l)"
git -c protocol.file.allow=always -C $E/linked/rust submodule update --init library/backtrace; echo "command exit: $?"
diff $E/bc $E/primary/.git/modules/rust/modules/library/backtrace/config; echo "nested config unchanged: $([ $? = 0 ] && echo yes || echo NO)"
git -C $E/primary/rust status --porcelain=v2 --ignore-submodules=none >/dev/null 2>$E/err; echo "git status in the primary's rust/: exit $? $(cat $E/err)"output (EXIT is the script's)Where the clone went, read after the run: |
|
Review of #831 at Earlier findings
BLOCKER
NOTENone. SEND BACK |
…ff its pin, and a linked worktree's is never moved The marker said a move was in flight so the next build would finish it, and its exemption let a nested commit through while it existed. Review round 3 found it wrong twice in linked worktrees: `is-ancestor` holds for a checkout at its pin, so a killed move there was never finished, and the empty marker exempted any nested commit, the agent's own included. The primary owns `rust/` and nothing commits in its nested checkouts, so it needs no state: wherever the fork or a checked-out nested submodule is off the commit the pin records, it runs the move again, and a nested submodule whose commit alone differs is no work. A move killed between the fork and its nested submodules is finished by the next build because the nested one is still off its pin. A linked worktree's checkout is never moved. One strictly ahead of its pin (`HEAD` is not the pin and the pin is its ancestor) is used as it stands; any other mismatch of the checkout or a nested submodule with the pin is refused by name, with the one command that moves it, fetching from the URL the tree records where the commit is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
…s a function of its own Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
Round 4 mutations and negative control at
mutate.sh#!/bin/sh
# Applies each checked patch, builds and runs the fork checkout tests, reports the exit, restores.
cd ~/toyos-llvmfetch || exit 2
D=$SCRATCH
F="sysroot::tests::a_worktree sysroot::tests::twelve sysroot::tests::a_linked sysroot::tests::the_primary sysroot::tests::ensure_shallow"
for p in nc-whole-change-reverted m1-no-sync m2-linked-ignores-nested m3-ahead-not-used m4-ahead-not-strict m5-primary-predicate-ignores-nested m6-primary-nested-commit-is-work; do
git apply --check $D/$p.patch || { echo "$p: does not apply"; exit 2; }
git apply $D/$p.patch
cargo test --lib -- $F > $D/$p.log 2>&1; e=$?
git apply -R $D/$p.patch
echo "$p EXIT=$e"
done
git status --porcelain --ignore-submodules=none
cargo test --lib -- $F > $D/fixed.log 2>&1; echo "fixed EXIT=$?"m1-no-sync.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..a6a42b234 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -686,7 +686,6 @@ fn gitlinks(repo: &Path, tree: &str) -> Vec<(String, String)> {
fn submodule_update(repo: &Path, path: &str) {
let shallow = format!("submodule.{path}.shallow");
let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
- git_run(repo, &["submodule", "sync", "--", path]);
let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
git_run(repo, &[&update[..], depth, &["--", path]].concat());m2-linked-ignores-nested.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..e7db77070 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -628,7 +628,7 @@ fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
let url = git_out(root, &["config", "--file", ".gitmodules", "--get", "submodule.rust.url"]);
wrong.push(fix(fork, url.trim(), pinned, &head));
}
- let nested = if holds(fork, pinned) { misplaced(fork, pinned) } else { Vec::new() };
+ let nested: Vec<(String, String, String)> = Vec::new();
for (path, from, commit) in nested {
let gitmodules = format!("{pinned}:.gitmodules");
let url = git_out(fork, &["config", "--blob", &gitmodules, "--get", &format!("submodule.{path}.url")]);m3-ahead-not-used.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..c6e4bbb82 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -611,9 +611,6 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
/// commit `pinned` records.
fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
let head = git_out(fork, &["rev-parse", "HEAD"]).trim().to_string();
- if head != pinned && git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
- return;
- }
// Each mismatch's own fix, in the order it must run: a nested one is known once the checkout holds its pin.
let mut fixes = Vec::new();
let mut fix = |at: &Path, url: &str, commit: &str, from: &str| {m4-ahead-not-strict.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..19587576c 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -611,7 +611,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
/// commit `pinned` records.
fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
let head = git_out(fork, &["rev-parse", "HEAD"]).trim().to_string();
- if head != pinned && git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
+ if git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
return;
}
// Each mismatch's own fix, in the order it must run: a nested one is known once the checkout holds its pin.m5-primary-predicate-ignores-nested.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..c530aaa57 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -572,7 +572,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
"move the fork checkout to its pin",
|| {
let head = git_out(&fork, &["rev-parse", "HEAD"]).trim().to_string();
- (head != pinned || !misplaced(&fork, "HEAD").is_empty()).then_some(head)
+ (head != pinned).then_some(head)
},
|head| {
// A nested submodule whose commit alone differs is no work: nothing commits there.m6-primary-nested-commit-is-work.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..a2262b644 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -577,7 +577,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
|head| {
// A nested submodule whose commit alone differs is no work: nothing commits there.
let status = git_out(&fork, &["status", "--porcelain=v2", "--ignore-submodules=none"]);
- let work: Vec<&str> = status.lines().filter(|entry| !entry.starts_with("1 .M SC.. ")).collect();
+ let work: Vec<&str> = status.lines().filter(|_| true).collect();
assert!(
work.is_empty(),
"{} is at {head} with uncommitted work, and this tree pins the fork at {pinned}: a build \nc-whole-change-reverted.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 41d12731c..1a5d96e2d 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -22,7 +22,7 @@
//! **Each worktree builds std in its own fork checkout.** The primary builds in its `rust/`,
//! moved to the commit its tree pins;
//! a linked worktree in its own `rust/`, made on first need as a git worktree of
-//! the primary's fork repository at the commit this tree pins and never moved ([`fork_checkout`]).
+//! the primary's fork repository at the commit this tree pins ([`fork_checkout`]).
//! `library/std` names `toyos-abi` and `toyos` as `../../../` and each file of
//! its ToyOS backend, `sdk/std`, by a `#[path]` as far up, so each checkout's
//! std compiles against its own worktree's ABI and backend with nothing
@@ -501,25 +501,24 @@ fn pinned_fork(root: &Path) -> String {
/// The fork checkout `root`'s std is built in, at the commit `root`'s tree pins.
///
/// The primary's is its own `rust/`, used where it is not initialised yet, which
-/// whoever initialises it does at the pin. It is no workspace and nothing
-/// commits in its nested submodules, so one whose `HEAD`, or a nested
-/// submodule checked out in it, is not at the commit the pin records is moved
-/// there, refused by name if it has local changes; a commit missing from it or
-/// from a nested submodule is fetched as rustc's bootstrap fetches one
-/// ([`submodule_update`]). The checkout moves before its nested submodules,
-/// because git's update reads their URLs and commits from the moved checkout's
-/// `.gitmodules` and index; a move killed between the two leaves a nested
-/// submodule off its pin, which the next moves.
+/// whoever initialises it does at the pin. It is no workspace, so one whose
+/// `HEAD` is not the pin is moved there; refused by name if it has local
+/// changes, or does not hold the pinned commit.
///
/// A linked worktree's `rust/` starts as the empty stub `git worktree add`
/// leaves; it is made here, the first time it is needed, as a git worktree of
/// the primary's fork repository at the pin, sharing its objects — and
/// `library/backtrace` the same way from the primary's, or by git's own clone
/// where the primary does not hold that commit. It is where an agent edits the
-/// fork, and is never moved: one strictly ahead of the pin is used as it
-/// stands, and any other whose `HEAD`, or a nested submodule checked out in it,
-/// is not at the commit the pin records is refused by name with the command
-/// that moves it.
+/// fork, so one ahead of the pin is used as it stands; one neither at the pin
+/// nor ahead of it is moved there, fetching the commit from the primary's
+/// repository first if it does not hold it, and refused by name if it has local
+/// changes rather than moved out from under whoever made them.
+///
+/// Every nested submodule checked out in it moves with it to the commit the pin
+/// records there, refused by name where it does not hold that commit, or is at
+/// a commit that neither the checkout's `HEAD` nor the pin records there, which
+/// only its own `HEAD` may record.
///
/// Every build in a worktree asks this at once, so the making and the move are
/// each decided and done under the worktree's lock held exclusively
@@ -564,20 +563,71 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
},
);
}
- if primary.is_some() {
- refuse_off_pin(root, &fork, &pinned);
- return fork;
- }
lock.act_if(
"move the fork checkout to its pin",
|| {
let head = git_out(&fork, &["rev-parse", "HEAD"]).trim().to_string();
- (head != pinned || !misplaced(&fork, "HEAD").is_empty()).then_some(head)
+ let ahead = primary.is_some()
+ && Command::new("git")
+ .args(["merge-base", "--is-ancestor", &pinned, &head])
+ .current_dir(&fork)
+ .status()
+ .is_ok_and(|s| s.success());
+ (head != pinned && !ahead).then_some(head)
},
|head| {
- // A nested submodule whose commit alone differs is no work: nothing commits there.
+ if !holds(&fork, &pinned) {
+ match &primary {
+ Some(primary) => git_run(&fork, &["fetch", path_str(&primary.join("rust")), &pinned]),
+ None => panic!(
+ "{} is at {head}, and this tree pins the fork at {pinned}, which it does not hold: \
+ `git -C {} fetch origin {pinned}` fetches it",
+ fork.display(),
+ fork.display(),
+ ),
+ }
+ }
+ let recorded: Vec<(String, String)> = git_out(&fork, &["ls-tree", "-r", &pinned])
+ .lines()
+ .filter_map(|entry| {
+ let (meta, path) = entry.split_once('\t')?;
+ match meta.split_whitespace().collect::<Vec<_>>().as_slice() {
+ ["160000", "commit", commit] => Some((path.to_string(), commit.to_string())),
+ _ => None,
+ }
+ })
+ .collect();
+ let nested: Vec<(PathBuf, String)> = recorded
+ .iter()
+ .map(|(path, commit)| (fork.join(path), commit.clone()))
+ .filter(|(at, commit)| at.join(".git").exists() && git_out(at, &["rev-parse", "HEAD"]).trim() != commit)
+ .collect();
+ for (at, commit) in &nested {
+ assert!(
+ holds(at, commit),
+ "{} does not hold {commit}, which the fork's {pinned} records there: \
+ `git -C {} fetch origin {commit}` fetches it",
+ at.display(),
+ at.display(),
+ );
+ }
+ // A nested submodule checked out at the commit the pin records there, and nothing more, is no work: a move
+ // killed before its last step leaves it so. One at any other commit may hold a commit nothing else records.
let status = git_out(&fork, &["status", "--porcelain=v2", "--ignore-submodules=none"]);
- let work: Vec<&str> = status.lines().filter(|entry| !entry.starts_with("1 .M SC.. ")).collect();
+ let work: Vec<String> = status
+ .lines()
+ .filter_map(|entry| {
+ let path = entry.strip_prefix("1 .M SC.. ").and_then(|rest| rest.splitn(6, ' ').nth(5));
+ let Some((path, commit)) = path.and_then(|path| recorded.iter().find(|(p, _)| p == path)) else {
+ return Some(entry.to_string());
+ };
+ let at = fork.join(path);
+ let at_head = git_out(&at, &["rev-parse", "HEAD"]).trim().to_string();
+ (at_head != *commit).then(|| {
+ format!("{} is at {at_head}, not at {commit}, what {pinned} records there", at.display())
+ })
+ })
+ .collect();
assert!(
work.is_empty(),
"{} is at {head} with uncommitted work, and this tree pins the fork at {pinned}: a build \
@@ -586,112 +636,17 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
fork.display(),
work.join("\n"),
);
- if holds(&fork, &pinned) {
- git_run(&fork, &["checkout", "--detach", "-q", &pinned]);
- } else {
- submodule_update(root, "rust");
- }
- for (path, _, commit) in misplaced(&fork, "HEAD") {
- let at = fork.join(&path);
- if holds(&at, &commit) {
- git_run(&at, &["checkout", "--detach", "-q", &commit]);
- } else {
- submodule_update(&fork, &path);
- }
+ for (at, commit) in &nested {
+ git_run(at, &["checkout", "--detach", "-q", commit]);
}
+ // Last, so a move killed before it is asked for again: `HEAD` alone decides.
+ git_run(&fork, &["checkout", "--detach", "-q", &pinned]);
eprintln!("{} was at {head}, and this tree pins {pinned}: checked it out", fork.display());
},
);
fork
}
-/// A linked worktree's fork checkout `fork` used as it stands where it is
-/// strictly ahead of `pinned`; otherwise refused by name, with the command that
-/// moves it, wherever it or a nested submodule checked out in it is not at the
-/// commit `pinned` records.
-fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
- let head = git_out(fork, &["rev-parse", "HEAD"]).trim().to_string();
- if head != pinned && git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
- return;
- }
- // Each mismatch's own fix, in the order it must run: a nested one is known once the checkout holds its pin.
- let mut fixes = Vec::new();
- let mut fix = |at: &Path, url: &str, commit: &str, from: &str| {
- if !holds(at, commit) {
- fixes.push(format!("git -C {} fetch --no-recurse-submodules {url} {commit}", at.display()));
- }
- fixes.push(format!("git -C {} checkout --detach -q {commit}", at.display()));
- format!("{} is at {from}, not at {commit}, what this tree's pin records there", at.display())
- };
- let mut wrong = Vec::new();
- if head != pinned {
- let url = git_out(root, &["config", "--file", ".gitmodules", "--get", "submodule.rust.url"]);
- wrong.push(fix(fork, url.trim(), pinned, &head));
- }
- let nested = if holds(fork, pinned) { misplaced(fork, pinned) } else { Vec::new() };
- for (path, from, commit) in nested {
- let gitmodules = format!("{pinned}:.gitmodules");
- let url = git_out(fork, &["config", "--blob", &gitmodules, "--get", &format!("submodule.{path}.url")]);
- wrong.push(fix(&fork.join(&path), url.trim(), &commit, &from));
- }
- assert!(
- wrong.is_empty(),
- "{}\na build would compile a std this tree does not name, and a linked worktree's fork checkout is \
- never moved for it: `{}` moves it",
- wrong.join("\n"),
- fixes.join(" && "),
- );
-}
-
-/// Each nested submodule checked out in `fork` whose `HEAD` is not the commit
-/// `tree` records there: its path, its `HEAD` and that commit.
-fn misplaced(fork: &Path, tree: &str) -> Vec<(String, String, String)> {
- gitlinks(fork, tree)
- .into_iter()
- .filter(|(path, _)| fork.join(path).join(".git").exists())
- .map(|(path, commit)| {
- let at = git_out(&fork.join(&path), &["rev-parse", "HEAD"]).trim().to_string();
- (path, at, commit)
- })
- .filter(|(_, at, commit)| at != commit)
- .collect()
-}
-
-/// The submodules `tree` in `repo` records, by path, each with its commit.
-fn gitlinks(repo: &Path, tree: &str) -> Vec<(String, String)> {
- git_out(repo, &["ls-tree", "-r", tree])
- .lines()
- .filter_map(|entry| {
- let (meta, path) = entry.split_once('\t')?;
- match meta.split_whitespace().collect::<Vec<_>>().as_slice() {
- ["160000", "commit", commit] => Some((path.to_string(), commit.to_string())),
- _ => None,
- }
- })
- .collect()
-}
-
-/// The submodule at `path` in `repo` checked out at the commit `repo`'s index
-/// records there, fetched as rustc's bootstrap fetches its LLVM: `git submodule
-/// sync`, so its `origin` is the URL `repo`'s `.gitmodules` records, then `git
-/// submodule update --init`, one commit deep where `.gitmodules` declares it
-/// `shallow`. Its own fetch recurses into no submodule of its own, whose
-/// `origin` is synced only once it has moved.
-///
-/// `repo` must be the primary's superproject or its own fork checkout, whose
-/// submodules' git directories are their own: run in a linked worktree's fork
-/// checkout, whose nested one is a git worktree of the primary's, `update`
-/// writes that checkout's path as `core.worktree` into the config the
-/// primary's shares, and git in the primary's fails.
-fn submodule_update(repo: &Path, path: &str) {
- let shallow = format!("submodule.{path}.shallow");
- let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
- git_run(repo, &["submodule", "sync", "--", path]);
- let depth: &[&str] = if shallow.trim() == "true" { &["--depth", "1"] } else { &[] };
- let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
- git_run(repo, &[&update[..], depth, &["--", path]].concat());
-}
-
/// Whether the repository at `dir` holds `commit`.
fn holds(dir: &Path, commit: &str) -> bool {
git_try(dir, &["cat-file", "-e", &format!("{commit}^{{commit}}")]).is_ok() |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Review of #831 at Earlier findings
The body's first "Unsure" itemAccepted. Root BLOCKERNone. NOTE
LAND AFTER NAMED CHANGES |
…s depth is read by its path or refused, and a runner's fork takes the primary's update path Round 4's review named five changes. - A linked worktree's refusal named `checkout --detach <pin>` even where the checkout's HEAD is not an ancestor of the pin, which leaves the agent's commits in the reflog alone. Where the pin is held, the refusal now names those commits (`git log --oneline <pin>..<head>`). Where it is not, they cannot be known before the fetch, so the named command runs `git merge-base --is-ancestor <head> <pin>` between the fetch and the checkout and stops there; the next build, the pin now held, names them. - `submodule.<path>.shallow` is read by name, and git keys it by the submodule's name: one named otherwise than by its path would have been fetched whole. `declared_shallow` now refuses that by name and only then reads `shallow`. - `gitlinks` is folded into its one caller, `misplaced`. - `ensure_shallow_fork` now runs `submodule_update` with depth 1, so a runner's fork and the primary's moves share sync, update and the no-recursion fetch. Their depths still differ, and the doc says why: a linked worktree's checkout shares the primary's objects and is told ahead from behind by ancestry, which a shallow `rust` would cut; a runner has no linked worktree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Round 5 mutations and negative control at
mutate.sh#!/bin/sh
# Applies each checked patch, builds and runs the fork checkout tests, reports the exit, restores.
cd ~/toyos-llvmfetch || exit 2
D=$SCRATCH
F="sysroot::tests::a_worktree sysroot::tests::twelve sysroot::tests::a_linked sysroot::tests::the_primary sysroot::tests::ensure_shallow sysroot::tests::a_submodule_named"
for p in nc-whole-change-reverted m1-no-sync m2-linked-ignores-nested m3-ahead-not-used m4-ahead-not-strict m5-primary-predicate-ignores-nested m6-primary-nested-commit-is-work m7-leaves-unnamed m8-no-guard m9-name-not-checked; do
git apply --check $D/$p.patch || { echo "$p: does not apply"; exit 2; }
git apply $D/$p.patch
cargo test --lib -- $F > $D/$p.log 2>&1; e=$?
git apply -R $D/$p.patch
echo "$p EXIT=$e"
done
git status --porcelain --ignore-submodules=none
cargo test --lib -- $F > $D/fixed.log 2>&1; echo "fixed EXIT=$?"m1-no-sync.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..eb01d8110 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -704,7 +704,6 @@ fn declared_shallow(repo: &Path, path: &str) -> bool {
/// writes that checkout's path as `core.worktree` into the config the
/// primary's shares, and git in the primary's fails.
pub(crate) fn submodule_update(repo: &Path, path: &str, shallow: bool) {
- git_run(repo, &["submodule", "sync", "--", path]);
let depth: &[&str] = if shallow { &["--depth", "1"] } else { &[] };
let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
git_run(repo, &[&update[..], depth, &["--", path]].concat());m2-linked-ignores-nested.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..fc64589b0 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -638,7 +638,7 @@ fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
let url = git_out(root, &["config", "--file", ".gitmodules", "--get", "submodule.rust.url"]);
wrong.push(fix(fork, url.trim(), pinned, &head));
}
- let nested = if holds(fork, pinned) { misplaced(fork, pinned) } else { Vec::new() };
+ let nested: Vec<(String, String, String)> = Vec::new();
for (path, from, commit) in nested {
let gitmodules = format!("{pinned}:.gitmodules");
let url = git_out(fork, &["config", "--blob", &gitmodules, "--get", &format!("submodule.{path}.url")]);m3-ahead-not-used.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..74ca675ad 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -613,9 +613,6 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
/// after the fetch.
fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
let head = git_out(fork, &["rev-parse", "HEAD"]).trim().to_string();
- if head != pinned && git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
- return;
- }
// Each mismatch's own fix, in the order it must run: a nested one is known once the checkout holds its pin.
let mut fixes = Vec::new();
let mut fix = |at: &Path, url: &str, commit: &str, from: &str| {m4-ahead-not-strict.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..3766a292c 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -613,7 +613,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
/// after the fetch.
fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
let head = git_out(fork, &["rev-parse", "HEAD"]).trim().to_string();
- if head != pinned && git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
+ if git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
return;
}
// Each mismatch's own fix, in the order it must run: a nested one is known once the checkout holds its pin.m5-primary-predicate-ignores-nested.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..21883f95d 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -572,7 +572,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
"move the fork checkout to its pin",
|| {
let head = git_out(&fork, &["rev-parse", "HEAD"]).trim().to_string();
- (head != pinned || !misplaced(&fork, "HEAD").is_empty()).then_some(head)
+ (head != pinned).then_some(head)
},
|head| {
// A nested submodule whose commit alone differs is no work: nothing commits there.m6-primary-nested-commit-is-work.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..f35b7201a 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -577,7 +577,7 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
|head| {
// A nested submodule whose commit alone differs is no work: nothing commits there.
let status = git_out(&fork, &["status", "--porcelain=v2", "--ignore-submodules=none"]);
- let work: Vec<&str> = status.lines().filter(|entry| !entry.starts_with("1 .M SC.. ")).collect();
+ let work: Vec<&str> = status.lines().filter(|_| true).collect();
assert!(
work.is_empty(),
"{} is at {head} with uncommitted work, and this tree pins the fork at {pinned}: a build \m7-leaves-unnamed.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..e38332b4a 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -626,7 +626,7 @@ fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
fixes.push(format!("git -C {} merge-base --is-ancestor {from} {commit}", at.display()));
line += "; where checking it out would leave commits of its own the command stops before it, and the next \
build names them";
- } else if git_try(at, &["merge-base", "--is-ancestor", from, commit]).is_err() {
+ } else if false && git_try(at, &["merge-base", "--is-ancestor", from, commit]).is_err() {
let left = git_out(at, &["log", "--oneline", &format!("{commit}..{from}")]);
line += &format!("; checking it out leaves these, which {commit} does not hold:\n{}", left.trim_end());
}m8-no-guard.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..019b6a05b 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -623,7 +623,6 @@ fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
if !holds(at, commit) {
fixes.push(format!("git -C {} fetch --no-recurse-submodules {url} {commit}", at.display()));
// Which commits the checkout would leave is known only once the commit is here.
- fixes.push(format!("git -C {} merge-base --is-ancestor {from} {commit}", at.display()));
line += "; where checking it out would leave commits of its own the command stops before it, and the next \
build names them";
} else if git_try(at, &["merge-base", "--is-ancestor", from, commit]).is_err() {m9-name-not-checked.patchdiff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..3e23358c4 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -681,7 +681,7 @@ fn misplaced(fork: &Path, tree: &str) -> Vec<(String, String, String)> {
fn declared_shallow(repo: &Path, path: &str) -> bool {
let named = git_try(repo, &["config", "--file", ".gitmodules", "--get", &format!("submodule.{path}.path")]);
assert!(
- named.as_deref().is_ok_and(|named| named.trim_ascii() == path.as_bytes()),
+ true || named.as_deref().is_ok_and(|named| named.trim_ascii() == path.as_bytes()),
"{}'s .gitmodules names the submodule at {path} otherwise than by its path, and its depth is read by \
that name: `git config --file .gitmodules --get-regexp '\\.path$'` shows its name",
repo.display(),nc-whole-change-reverted.patchdiff --git a/src/lib.rs b/src/lib.rs
index 3e8700e7d..872dbdaec 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -121,11 +121,6 @@ pub fn ensure_submodules(repo_dir: &Path) {
/// the toolchain from them need. Refused in a linked worktree, whose `rust/` is
/// `sysroot::fork_checkout`'s to make: `git submodule` there rewrites the
/// `core.worktree` of the primary's fork.
-///
-/// One commit deep, where the primary's moves take the depth `.gitmodules`
-/// declares, which for `rust` is its whole history: a linked worktree's fork
-/// checkout shares the primary's objects and is told ahead of its pin from
-/// behind it by ancestry, and a runner has no linked worktree.
pub fn ensure_shallow_fork(root: &Path) -> Result<(), String> {
if root.join("rust/x.py").exists() {
return Ok(());
@@ -136,8 +131,12 @@ pub fn ensure_shallow_fork(root: &Path) -> Result<(), String> {
root.join("rust").display()
));
}
- sysroot::submodule_update(root, "rust", true);
- Ok(())
+ let status = Command::new("git")
+ .args(["submodule", "update", "--init", "--depth", "1", "rust"])
+ .current_dir(root)
+ .status()
+ .map_err(|e| format!("git submodule update --init --depth 1 rust: {e}"))?;
+ status.success().then_some(()).ok_or_else(|| format!("git submodule update --init --depth 1 rust exited {status}"))
}
/// Ensure a single git submodule is checked out.
diff --git a/src/sysroot.rs b/src/sysroot.rs
index 04fe6c5b0..8afec1738 100644
--- a/src/sysroot.rs
+++ b/src/sysroot.rs
@@ -22,7 +22,7 @@
//! **Each worktree builds std in its own fork checkout.** The primary builds in its `rust/`,
//! moved to the commit its tree pins;
//! a linked worktree in its own `rust/`, made on first need as a git worktree of
-//! the primary's fork repository at the commit this tree pins and never moved ([`fork_checkout`]).
+//! the primary's fork repository at the commit this tree pins ([`fork_checkout`]).
//! `library/std` names `toyos-abi` and `toyos` as `../../../` and each file of
//! its ToyOS backend, `sdk/std`, by a `#[path]` as far up, so each checkout's
//! std compiles against its own worktree's ABI and backend with nothing
@@ -501,25 +501,24 @@ fn pinned_fork(root: &Path) -> String {
/// The fork checkout `root`'s std is built in, at the commit `root`'s tree pins.
///
/// The primary's is its own `rust/`, used where it is not initialised yet, which
-/// whoever initialises it does at the pin. It is no workspace and nothing
-/// commits in its nested submodules, so one whose `HEAD`, or a nested
-/// submodule checked out in it, is not at the commit the pin records is moved
-/// there, refused by name if it has local changes; a commit missing from it or
-/// from a nested submodule is fetched as rustc's bootstrap fetches one
-/// ([`submodule_update`]). The checkout moves before its nested submodules,
-/// because git's update reads their URLs and commits from the moved checkout's
-/// `.gitmodules` and index; a move killed between the two leaves a nested
-/// submodule off its pin, which the next moves.
+/// whoever initialises it does at the pin. It is no workspace, so one whose
+/// `HEAD` is not the pin is moved there; refused by name if it has local
+/// changes, or does not hold the pinned commit.
///
/// A linked worktree's `rust/` starts as the empty stub `git worktree add`
/// leaves; it is made here, the first time it is needed, as a git worktree of
/// the primary's fork repository at the pin, sharing its objects — and
/// `library/backtrace` the same way from the primary's, or by git's own clone
/// where the primary does not hold that commit. It is where an agent edits the
-/// fork, and is never moved: one strictly ahead of the pin is used as it
-/// stands, and any other whose `HEAD`, or a nested submodule checked out in it,
-/// is not at the commit the pin records is refused by name with the command
-/// that moves it and the commits that move would leave.
+/// fork, so one ahead of the pin is used as it stands; one neither at the pin
+/// nor ahead of it is moved there, fetching the commit from the primary's
+/// repository first if it does not hold it, and refused by name if it has local
+/// changes rather than moved out from under whoever made them.
+///
+/// Every nested submodule checked out in it moves with it to the commit the pin
+/// records there, refused by name where it does not hold that commit, or is at
+/// a commit that neither the checkout's `HEAD` nor the pin records there, which
+/// only its own `HEAD` may record.
///
/// Every build in a worktree asks this at once, so the making and the move are
/// each decided and done under the worktree's lock held exclusively
@@ -564,20 +563,71 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
},
);
}
- if primary.is_some() {
- refuse_off_pin(root, &fork, &pinned);
- return fork;
- }
lock.act_if(
"move the fork checkout to its pin",
|| {
let head = git_out(&fork, &["rev-parse", "HEAD"]).trim().to_string();
- (head != pinned || !misplaced(&fork, "HEAD").is_empty()).then_some(head)
+ let ahead = primary.is_some()
+ && Command::new("git")
+ .args(["merge-base", "--is-ancestor", &pinned, &head])
+ .current_dir(&fork)
+ .status()
+ .is_ok_and(|s| s.success());
+ (head != pinned && !ahead).then_some(head)
},
|head| {
- // A nested submodule whose commit alone differs is no work: nothing commits there.
+ if !holds(&fork, &pinned) {
+ match &primary {
+ Some(primary) => git_run(&fork, &["fetch", path_str(&primary.join("rust")), &pinned]),
+ None => panic!(
+ "{} is at {head}, and this tree pins the fork at {pinned}, which it does not hold: \
+ `git -C {} fetch origin {pinned}` fetches it",
+ fork.display(),
+ fork.display(),
+ ),
+ }
+ }
+ let recorded: Vec<(String, String)> = git_out(&fork, &["ls-tree", "-r", &pinned])
+ .lines()
+ .filter_map(|entry| {
+ let (meta, path) = entry.split_once('\t')?;
+ match meta.split_whitespace().collect::<Vec<_>>().as_slice() {
+ ["160000", "commit", commit] => Some((path.to_string(), commit.to_string())),
+ _ => None,
+ }
+ })
+ .collect();
+ let nested: Vec<(PathBuf, String)> = recorded
+ .iter()
+ .map(|(path, commit)| (fork.join(path), commit.clone()))
+ .filter(|(at, commit)| at.join(".git").exists() && git_out(at, &["rev-parse", "HEAD"]).trim() != commit)
+ .collect();
+ for (at, commit) in &nested {
+ assert!(
+ holds(at, commit),
+ "{} does not hold {commit}, which the fork's {pinned} records there: \
+ `git -C {} fetch origin {commit}` fetches it",
+ at.display(),
+ at.display(),
+ );
+ }
+ // A nested submodule checked out at the commit the pin records there, and nothing more, is no work: a move
+ // killed before its last step leaves it so. One at any other commit may hold a commit nothing else records.
let status = git_out(&fork, &["status", "--porcelain=v2", "--ignore-submodules=none"]);
- let work: Vec<&str> = status.lines().filter(|entry| !entry.starts_with("1 .M SC.. ")).collect();
+ let work: Vec<String> = status
+ .lines()
+ .filter_map(|entry| {
+ let path = entry.strip_prefix("1 .M SC.. ").and_then(|rest| rest.splitn(6, ' ').nth(5));
+ let Some((path, commit)) = path.and_then(|path| recorded.iter().find(|(p, _)| p == path)) else {
+ return Some(entry.to_string());
+ };
+ let at = fork.join(path);
+ let at_head = git_out(&at, &["rev-parse", "HEAD"]).trim().to_string();
+ (at_head != *commit).then(|| {
+ format!("{} is at {at_head}, not at {commit}, what {pinned} records there", at.display())
+ })
+ })
+ .collect();
assert!(
work.is_empty(),
"{} is at {head} with uncommitted work, and this tree pins the fork at {pinned}: a build \
@@ -586,130 +636,17 @@ pub fn fork_checkout(root: &Path, lock: &mut Held) -> PathBuf {
fork.display(),
work.join("\n"),
);
- if holds(&fork, &pinned) {
- git_run(&fork, &["checkout", "--detach", "-q", &pinned]);
- } else {
- submodule_update(root, "rust", declared_shallow(root, "rust"));
- }
- for (path, _, commit) in misplaced(&fork, "HEAD") {
- let at = fork.join(&path);
- if holds(&at, &commit) {
- git_run(&at, &["checkout", "--detach", "-q", &commit]);
- } else {
- submodule_update(&fork, &path, declared_shallow(&fork, &path));
- }
+ for (at, commit) in &nested {
+ git_run(at, &["checkout", "--detach", "-q", commit]);
}
+ // Last, so a move killed before it is asked for again: `HEAD` alone decides.
+ git_run(&fork, &["checkout", "--detach", "-q", &pinned]);
eprintln!("{} was at {head}, and this tree pins {pinned}: checked it out", fork.display());
},
);
fork
}
-/// A linked worktree's fork checkout `fork` used as it stands where it is
-/// strictly ahead of `pinned`; otherwise refused by name, with the command that
-/// moves it, wherever it or a nested submodule checked out in it is not at the
-/// commit `pinned` records. Where that move would leave commits the refusal
-/// names them, or, where it cannot know them before a fetch, the command stops
-/// after the fetch.
-fn refuse_off_pin(root: &Path, fork: &Path, pinned: &str) {
- let head = git_out(fork, &["rev-parse", "HEAD"]).trim().to_string();
- if head != pinned && git_try(fork, &["merge-base", "--is-ancestor", pinned, &head]).is_ok() {
- return;
- }
- // Each mismatch's own fix, in the order it must run: a nested one is known once the checkout holds its pin.
- let mut fixes = Vec::new();
- let mut fix = |at: &Path, url: &str, commit: &str, from: &str| {
- let mut line = format!("{} is at {from}, not at {commit}, what this tree's pin records there", at.display());
- if !holds(at, commit) {
- fixes.push(format!("git -C {} fetch --no-recurse-submodules {url} {commit}", at.display()));
- // Which commits the checkout would leave is known only once the commit is here.
- fixes.push(format!("git -C {} merge-base --is-ancestor {from} {commit}", at.display()));
- line += "; where checking it out would leave commits of its own the command stops before it, and the next \
- build names them";
- } else if git_try(at, &["merge-base", "--is-ancestor", from, commit]).is_err() {
- let left = git_out(at, &["log", "--oneline", &format!("{commit}..{from}")]);
- line += &format!("; checking it out leaves these, which {commit} does not hold:\n{}", left.trim_end());
- }
- fixes.push(format!("git -C {} checkout --detach -q {commit}", at.display()));
- line
- };
- let mut wrong = Vec::new();
- if head != pinned {
- let url = git_out(root, &["config", "--file", ".gitmodules", "--get", "submodule.rust.url"]);
- wrong.push(fix(fork, url.trim(), pinned, &head));
- }
- let nested = if holds(fork, pinned) { misplaced(fork, pinned) } else { Vec::new() };
- for (path, from, commit) in nested {
- let gitmodules = format!("{pinned}:.gitmodules");
- let url = git_out(fork, &["config", "--blob", &gitmodules, "--get", &format!("submodule.{path}.url")]);
- wrong.push(fix(&fork.join(&path), url.trim(), &commit, &from));
- }
- assert!(
- wrong.is_empty(),
- "{}\na build would compile a std this tree does not name, and a linked worktree's fork checkout is \
- never moved for it: `{}` moves it",
- wrong.join("\n"),
- fixes.join(" && "),
- );
-}
-
-/// Each nested submodule checked out in `fork` whose `HEAD` is not the commit
-/// `tree` records there: its path, its `HEAD` and that commit.
-fn misplaced(fork: &Path, tree: &str) -> Vec<(String, String, String)> {
- git_out(fork, &["ls-tree", "-r", tree])
- .lines()
- .filter_map(|entry| {
- let (meta, path) = entry.split_once('\t')?;
- match meta.split_whitespace().collect::<Vec<_>>().as_slice() {
- ["160000", "commit", commit] => Some((path.to_string(), commit.to_string())),
- _ => None,
- }
- })
- .filter(|(path, _)| fork.join(path).join(".git").exists())
- .map(|(path, commit)| {
- let at = git_out(&fork.join(&path), &["rev-parse", "HEAD"]).trim().to_string();
- (path, at, commit)
- })
- .filter(|(_, at, commit)| at != commit)
- .collect()
-}
-
-/// Whether `repo`'s `.gitmodules` declares the submodule at `path` `shallow`,
-/// which git keys by the submodule's name: one named otherwise than by its
-/// path is refused, where a lookup by path would miss it and fetch the whole
-/// history.
-fn declared_shallow(repo: &Path, path: &str) -> bool {
- let named = git_try(repo, &["config", "--file", ".gitmodules", "--get", &format!("submodule.{path}.path")]);
- assert!(
- named.as_deref().is_ok_and(|named| named.trim_ascii() == path.as_bytes()),
- "{}'s .gitmodules names the submodule at {path} otherwise than by its path, and its depth is read by \
- that name: `git config --file .gitmodules --get-regexp '\\.path$'` shows its name",
- repo.display(),
- );
- let shallow = format!("submodule.{path}.shallow");
- let shallow = git_out(repo, &["config", "--file", ".gitmodules", "--type", "bool", "--default", "false", "--get", &shallow]);
- shallow.trim() == "true"
-}
-
-/// The submodule at `path` in `repo` checked out at the commit `repo`'s index
-/// records there, fetched as rustc's bootstrap fetches its LLVM: `git submodule
-/// sync`, so its `origin` is the URL `repo`'s `.gitmodules` records, then `git
-/// submodule update --init`, one commit deep where `shallow`. Its own fetch
-/// recurses into no submodule of its own, whose `origin` is synced only once
-/// it has moved.
-///
-/// `repo` must be a superproject or the primary's own fork checkout, whose
-/// submodules' git directories are their own: run in a linked worktree's fork
-/// checkout, whose nested one is a git worktree of the primary's, `update`
-/// writes that checkout's path as `core.worktree` into the config the
-/// primary's shares, and git in the primary's fails.
-pub(crate) fn submodule_update(repo: &Path, path: &str, shallow: bool) {
- git_run(repo, &["submodule", "sync", "--", path]);
- let depth: &[&str] = if shallow { &["--depth", "1"] } else { &[] };
- let update = ["-c", "fetch.recurseSubmodules=false", "submodule", "update", "--init", "--checkout"];
- git_run(repo, &[&update[..], depth, &["--", path]].concat());
-}
-
/// Whether the repository at `dir` holds `commit`.
fn holds(dir: &Path, commit: &str) -> bool {
git_try(dir, &["cat-file", "-e", &format!("{commit}^{{commit}}")]).is_ok()
@@ -1772,27 +1709,6 @@ mod tests {
assert!(said.contains("leaves these") && said.contains("the agent's own, on the old pin"), "{said}");
}
- /// **A submodule's depth is read by its path or refused**: `shallow` is
- /// read where `.gitmodules` names the submodule by its path, and one named
- /// otherwise is refused by name rather than fetched whole.
- #[test]
- fn a_submodule_named_otherwise_than_by_its_path_is_refused() {
- let base = TempDir::new("fork-named");
- let repo = base.join("repo");
- fs::create_dir_all(&repo).unwrap();
- git(&repo, &["init", "-q"]);
- git(&repo, &["config", "--file", ".gitmodules", "submodule.sub.path", "sub"]);
- assert!(!declared_shallow(&repo, "sub"));
- git(&repo, &["config", "--file", ".gitmodules", "submodule.sub.shallow", "true"]);
- assert!(declared_shallow(&repo, "sub"));
-
- git(&repo, &["config", "--file", ".gitmodules", "--rename-section", "submodule.sub", "submodule.other"]);
- let said = refusal(|| {
- declared_shallow(&repo, "sub");
- });
- assert!(said.contains("names the submodule at sub otherwise than by its path"), "{said}");
- }
-
/// Where a linked worktree's `rust/` is a fork checkout that is not whole,
/// [`crate::ensure_shallow_fork`] refuses, and git in the primary's fork
/// still runs. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
, #826, #835, #833, #831 and #822, into wt/toyos-netperf No hunk conflicted. userland/netstack/src/main.rs took both sides: main's removal of `mod device` and the branch's batched `node.receive` and its module-doc line. The TCP window-scaling and loss-probe commits main carries were already in the branch from #820, so their files merged to main's text plus the branch's own delta. Both lockfiles are main's and pass `cargo metadata --locked`. The branch's new issue still cites `VirtioNet::poll_rx` and `toyos_i219::RX_BUDGET` as they stand on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
#846, #843, #849, #850, #840, #839, #837) into the batch: the icons' and wallpaper's digests hash with toyos-sha2-hw, and the loader's wall clock reads through its own UEFI bindings The batch's merge of main at f72d53d moved #813's wallpaper and #814's icon digest tests onto `toyos_sha2`. #833, already on main, had replaced the root package's `toyos-sha2` dependency with `toyos-sha2-hw`, so CI's merge of the two compiled no `toyos-build` lib test and both the build system's tests and clippy went red with E0432. Both tests now hash through `toyos_sha2_hw`, as every other SHA-256 the build takes does. #842's `bootloader/src/wallclock.rs` was written on the `uefi` crate, which #815 removes; it now calls `efi`'s `RuntimeServices::get_time`, whose `Time` fields are plain and whose error is the `Status` itself. `start_kernel` takes main's `wall_clock` and the batch's `SystemTable`; the batch's `armed_at` goes, as main replaced it with `wallclock::now`. Cargo.lock takes main's `ntapi`; `nonempty` goes with `gix`, which the batch removed and which was its only user (`cargo metadata --offline`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
, #836, #839, #840, #842 through #847, #849 and #850, into consent system.toml: sshserver and shell start both toyfetch (#843) and grants. tests/common/qemu.rs: Profile carries both Desktop and MetalAmdVi (#837). tests/toyos.rs: SCREEN_TESTS carries consent_prompt beside virt_wall_clock_utc (#842) and virt_low_ecam (#840). Beyond the conflict lines: #833 replaced the build's toyos-sha2 dependency with toyos-sha2-hw, so consent_prompt digests its job with toyos_sha2_hw::sha256_digest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
Fixes a red the owner hit on
main. After #790 moved the fork's LLVM pin toToyOSOrg/llvm-projectb7420fe534bf,cargo runin the primary panicked insysroot::fork_checkout: "rust/src/llvm-projectdoes not hold b7420fe…:git -C …/rust/src/llvm-project fetch origin b7420fe…fetches it". That command fails: the checkout was made when the fork's.gitmodulesnamedrust-lang/llvm-project, so itsoriginstill does, and that repository lacks the commit.Head:
faa1820149d8, onorigin/mainat60dc7fcec(it has not moved since round 4's merge).git diff origin/main...HEAD --shortstat: 4 files, +378 −145 (src/sysroot.rs: +138 −75 abovemod tests, +220 −62 in it;src/lib.rs: +7 −6;issues/the-fork-checkout-runs-git-submodule-in-a-linked-worktree.md: +12 −1;issues/seven-commits-of-mains-history-pin-a-rust-commit-no-repository-holds.md: +1 −1).What changed
submodule_update). Bootstrap runsgit submodule syncand thengit submodule update --init --depth=1forsrc/llvm-projecton every LLVM build (rust/src/bootstrap/src/core/config/config.rs). The primary now does the same for whatever is missing: in the superproject for the fork's own pin (-- rust), in the fork for a nested pin (-- <path>). It passes--depth 1where.gitmodulesdeclares the submoduleshallow = true, and--checkoutso a configuredupdatestrategy cannot override it.syncsetsoriginfrom the URL in the moved checkout's.gitmodules, so a staleoriginis fixed at its root rather than refused on every pin move. The refusal arms that namedgit fetch origin <commit>are gone. Thegitrow ofissues/the-build-runs-host-tools-outside-rust-and-qemu.mdalready admits "updates submodules (src/lib.rs,src/sysroot.rs)".declared_shallow). Git keyssubmodule.<name>.shallowby the submodule's name, so a lookup by path misses one whose name is not its path, and that submodule would be fetched whole. Unless.gitmodulesgivessubmodule.<path>.pathaspath, the update is refused by name. Only then isshallowread. Every entry of the fork's.gitmodulesis named by its path today.-c fetch.recurseSubmodules=false). With on-demand recursion, the fork's fetch would fetch each nested submodule from theoriginit had before its move. In the owner's case that is rust-lang's llvm-project. Each nested submodule is fetched by its ownsyncandupdateonce the fork has moved.ensure_shallow_fork,src/lib.rs). It now callssubmodule_update(root, "rust", true)instead of running its owngit submodule update --init --depth 1 rust. Sosync,--checkoutand the non-recursive fetch are one code path. The depths still differ, and the doc at the site says why. A runner's is one commit deep. The primary's moves take the depth.gitmodulesdeclares, and forrustthat is the whole history. That history matters because a linked worktree's fork checkout shares the primary's objects, and ahead, behind and diverged are told apart by ancestry there. A shallowrustwould cut that ancestry at the pin. A runner has no linked worktree.rust/and nothing commits in its nested checkouts. So the move runs again wherever the fork'sHEAD, or a nested submodule checked out in it, is not at the commit the pin records. A nested submodule whose commit alone differs (1 .M SC..) is no work. Anything else ingit statusis refused as before. The fork moves before its nested submodules, because git's update reads their URLs and commits from the moved fork's.gitmodulesand index. A build killed between the two leaves a nested submodule off its pin, and the next build moves it because the predicate sees it.refuse_off_pin). One strictly ahead of its pin (HEAD != pinand the pin is its ancestor) is used as it stands. Any other mismatch with the pin, in the checkout or in a nested submodule checked out in it, is refused by name, with the one command that moves it:git -C <at> checkout --detach -q <commit>for each. Where the commit is missing, that is preceded bygit -C <at> fetch --no-recurse-submodules <url> <commit>, the URL being the one the tree records. A nested mismatch is named once the checkout holds its pin.HEADis not an ancestor of the commit, a checkout leaves the agent's commits reachable only from the reflog. If the commit is held, the refusal names them (git log --oneline <commit>..<HEAD>). If it is not held, they cannot be known before the fetch. The named command therefore runsgit -C <at> merge-base --is-ancestor <HEAD> <commit>between the fetch and the checkout and stops there. The next build then holds the commit and names them. A checkout merely behind its pin is not said to leave anything.git submodulein a linked worktree.submodule_updateis called only underOwner::Us, and in a linked worktree only afterrefuse_off_pinhas returned. One pre-existing call does run there. Whenfork_checkoutfirst makes a linked checkout and the primary's nested repository lacks the pin's commit, it runsgit submodule update --init library/backtrace.ensure_submodule(src/lib.rs) runs the same. That is outside this branch and stays inissues/the-fork-checkout-runs-git-submodule-in-a-linked-worktree.md, which now carries its measurement. In that line's own state, an emptylibrary/backtrace, git 2.54.0 clones a git directory of the checkout's own and leaves the primary's configuration alone (A missing fork or nested commit in the primary is fetched as bootstrap fetches one, statelessly, and a linked worktree's fork checkout is never moved #831 (comment)). Over alibrary/backtracethat is already a git worktree of the primary's nested repository, it rewrites the primary's nestedcore.worktree, and git in the primary'srust/then exits 128.gitlinksis folded into its one caller,misplaced.issues/seven-commits-of-mains-history-pin-a-rust-commit-no-repository-holds.mdsaid a worktree's build "fetches" its pin from the primary'srust/. It now says it takes it from there: the linked checkout shares the primary's objects and fetches nothing itself. The module header says a linked checkout is never moved.Where each submodule command runs
core.worktreesubmodule_update(root, "rust", …)Owner::Us), or a runner's checkout (ensure_shallow_fork, refused in a linked worktree).git/modules/rustrust/: unchanged (measured)submodule_update(&fork, path, …)rust/(Owner::Us).git/modules/rust/modules/<path>fork_checkoutreturns afterrefuse_off_pinMeasurement, script and output: #831 (comment). The refusal's own commands, run in a linked worktree, leave the primary's two configs byte-identical (
a_linked_worktree_lacking_a_nested_commit_is_refused_and_the_primary_s_is_not_written).Gates (head
faa1820149d8)cargo run -- --ci hostcargo run -- --build-onlyLogs, whole, with home and temporary paths scrubbed.
--ci host: 1, 2, 3, 4, 5, 6, 7, 8.--build-only: 1.No guest test is affected, because the change is in the build system only.
Tests, negative control, oracle
the_primary_s_fork_checkout_is_moved_to_its_pinreproduces the owner's case for both pins. Theoriginof the fork and of its nested submodule points at a repository that holds none of the commits to come, and only the URL the pin's tree records holds them. The test asserts:b3;originis now the pin's URL;core.worktreechanged;origin/stale-onlyref arrived through recursion.Then it leaves the fork at the pin with its nested checkout stale, as a killed move leaves it, and asserts the next call moves the nested checkout.
a_worktree_pinning_another_fork_commit_gets_its_own_checkoutcovers the linked checkout's cases:The primary's fork is untouched throughout.
a_linked_refusal_s_command_leaves_no_commit_of_the_agent_sgives a linked worktree's nested checkout a commit of the agent's, then pins a nested commit that only its URL holds. It asserts:merge-base;a_linked_worktree_lacking_a_nested_commit_is_refused_and_the_primary_s_is_not_writtengives a linked worktree a pin whose nested commit only its pin's URL holds. It asserts:git statusin the primary'srust/runs;b3, and the configs are still byte-identical.a_submodule_named_otherwise_than_by_its_path_is_refused:shallowis read for a submodule named by its path, both unset andtrue. Once the section is renamed, the read is refused by name.twelve_builds_at_once_make_one_fork_checkoutkeeps only its making half: a linked checkout is no longer moved, so there is no move to race.The oracle is git itself: the tests run the real
git submoduleand the refusal's real commands against real repositories.The negative control and each mutation is a checked patch, applied, built, run and restored in one script at
faa1820149d8. All exit 101, and the fixed tree exits 0: A missing fork or nested commit in the primary is fetched as bootstrap fetches one, statelessly, and a linked worktree's fork checkout is never moved #831 (comment)mod testsandsrc/lib.rsback toorigin/main's. The name test is dropped from this arm, becauseorigin/mainhas no function for it to call. The other four tests above go red.git submodule sync;is-ancestoralone, so a checkout at its pin counts as ahead;ensure_shallow_fork's depth has no new test. A local-path clone ignores--depth, and git refuses afile://URL for a submodule clone unless the cloning process itself is givenprotocol.file.allow, which the function does not pass (measured: "fatal: transport 'file' not allowed"). Its depth,trueat the call, is unchanged fromorigin/main's--depth 1.Unsure
git submodule update --depth 1first fetches the clone's configured refspec at depth 1 and then the pinned commit, as bootstrap's does. The primary'srust/src/llvm-projecthere is a single-branch shallow clone: itsremote.origin.fetchis+refs/heads/landing-page:refs/remotes/origin/landing-page, andgit ls-remote https://github.com/ToyOSOrg/llvm-project.git refs/heads/landing-pagelists that branch (exit 0). So its first fetch brings one tip, not the 45 headsToyOSOrg/llvm-projectlists. How many bytes that fetch moves is unmeasured until the orchestrator's first run on the primary.🤖 Generated with Claude Code
https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C