fix(storage): recover from full disks with bounded memory fallback

docs/user/compose-and-local-tools.md

@@ -84,6 +84,19 @@ Only the final compose return value crosses the model-context boundary. When its
 
 Kit's working directory is project context, not an operating-system security boundary. A shell command can use absolute paths, `..`, the network, and any credentials or files allowed to the Kit process. Quote untrusted values, inspect destructive commands before running them, and avoid putting secrets into command text or returned output. There is no automatic rollback for shell side effects.
 
+## Read spilled output with `artifact`
+
+Large compose results include an `artifact` path. Read it through `artifact`, which sees both persisted files and output temporarily held in Kit's memory filesystem. Shell commands only see real disk files.
+
+```text
+chunk = artifact({ path: output.artifact, offset: 0, limit: 1024 })
+return chunk
+```
+
+The result contains `content`, `next_offset`, `total_bytes`, and `eof`. Continue from `next_offset` to read another chunk. Reads preserve UTF-8 character boundaries and are limited to 1,024 bytes per call. Only artifacts in the calling session's namespace are accepted; traversal and symlinks are rejected.
+
+An artifact-storage error does not turn an already-completed tool into a failed tool call. Kit returns a bounded preview with `artifact_error` when output cannot be retained. Do not repeat a side-effecting tool merely to obtain its output again.
+
 ## Make exact file changes with `edit`
 
 `edit` operates on one file path with `op: "add"`, `"edit"`, or `"delete"`. Relative paths are resolved from Kit's working directory. Absolute paths, `..`, and paths through symlinks are accepted, so `edit` can change files outside the root when the Kit process has permission. Paths must be non-empty.

docs/user/tui-and-sessions.md

@@ -23,6 +23,18 @@ The catalog requires an existing directory and is workspace-filtered and newest-
 
 A session ID must be 1–128 ASCII letters, digits, `-`, or `_`. `kit prompt` uses the same durable sessions: it prints `session_id: <id>` after its answer, and that ID can be continued by either `kit prompt --resume <session-id>` or `kit tui --resume <session-id>`.
 
+## Recovering from full storage
+
+Kit routes its internal persistence through a shared filesystem service. If a write fails because storage is full or a quota is exceeded, the service retains the pending change in a bounded memory overlay. Internal reads and session listings use the same view, so finishing a turn or closing a session handle does not discard accepted changes. An existing session can be reopened in the same running process while persistence is pending.
+
+Recovery is automatic: internal operations retry pending work, and a process-owned worker retries with bounded backoff while Kit is idle. Once storage recovers, pending changes are persisted in order before newer changes can bypass them. Freeing space through a normal tool call can therefore restore persistence without restarting the session. Kit reports degradation and recovery on stderr. The TUI also shows a persistent memory-only warning outside the optional log pane, including after switching sessions, until storage recovers.
+
+**Pending memory state does not survive process termination.** A different process cannot read it. Keep Kit running until recovery completes if you need those changes to be durable. Before normal exit, Kit makes one final recovery pass; if accepted changes remain unpersisted, it warns and exits unsuccessfully. The TUI waits for its agent process to finish recovery and reports an unsuccessful exit; if graceful shutdown times out, it warns of possible data loss before forcing termination. The default service bounds retained data to 64 MiB and pending operations to 4,096; exhaustion requests cancellation and orderly shutdown rather than silently evicting accepted data. If a fallible fallback allocation itself fails, Kit instead makes a best-effort terminal restoration and exits immediately without allocating an error message.
+
+Explicit `edit` and shell operations still use the real filesystem and report real failures to the model. Native consumers, such as plugin subprocesses, require their inputs to be materialized on disk, and interprocess locks still require real ownership. Unsafe paths, lost ownership, and non-capacity errors are not permission to overwrite another process's data.
+
+Use the `artifact` tool to read spilled output, including memory-only artifacts; a shell cannot see the memory overlay.
+
 ## TUI keys, prompt editing, and navigation
 
 | Key or input | Action |

src/artifacts.rs

@@ -1,27 +1,34 @@
-use std::path::{Path, PathBuf};
+use std::{
+    io::{self, Write as _},
+    path::{Path, PathBuf},
+};
 
-use tokio::fs::File;
+use crate::resilient_fs as fs;
 
-pub(crate) fn directory(root: &Path, session_id: &str, call_id: &str) -> PathBuf {
-    let root = std::env::var_os("HOME")
+pub(crate) fn base(root: &Path) -> PathBuf {
+    std::env::var_os("HOME")
         .filter(|home| !home.is_empty())
         .map(PathBuf::from)
         .map_or_else(
             || root.join(".kit/artifacts"),
             |home| home.join(".kit/artifacts"),
-        );
-    root.join(safe_component(session_id))
-        .join(safe_component(call_id))
+        )
+}
+
+pub(crate) fn session_directory(base: &Path, session_id: &str) -> PathBuf {
+    base.join(safe_component(session_id))
 }
 
-pub(crate) async fn create(path: &Path) -> std::io::Result<(File, PathBuf)> {
+pub(crate) fn directory(root: &Path, session_id: &str, call_id: &str) -> PathBuf {
+    session_directory(&base(root), session_id).join(safe_component(call_id))
+}
+
+/// Retain the complete artifact through the same filesystem as transcripts.
+/// Call from a blocking task; a returned path is readable through ArtifactTool
+/// even when the backing bytes have not yet reached disk.
+pub(crate) fn write(path: &Path, bytes: &[u8]) -> io::Result<PathBuf> {
     let parent = path.parent().unwrap_or_else(|| Path::new("."));
-    tokio::fs::create_dir_all(parent).await?;
-    #[cfg(unix)]
-    {
-        use std::os::unix::fs::PermissionsExt as _;
-        tokio::fs::set_permissions(parent, std::fs::Permissions::from_mode(0o700)).await?;
-    }
+    fs::create_private_dir_all(parent)?;
     let stem = path
         .file_stem()
         .and_then(|value| value.to_str())
@@ -36,19 +43,26 @@ pub(crate) async fn create(path: &Path) -> std::io::Result<(File, PathBuf)> {
         } else {
             path.with_file_name(format!("{stem}-{attempt}.{extension}"))
         };
-        let mut options = tokio::fs::OpenOptions::new();
-        options.write(true).create_new(true);
-        #[cfg(unix)]
-        options.mode(0o600);
-        match options.open(&candidate).await {
-            Ok(file) => return Ok((file, candidate)),
-            Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue,
+        match fs::OpenOptions::new()
+            .write(true)
+            .create_new(true)
+            .private(true)
+            .open(&candidate)
+        {
+            Ok(mut file) => {
+                if let Err(error) = file.write_all(bytes).and_then(|()| file.sync_all()) {
+                    let _ = fs::remove_file(&candidate);
+                    return Err(error);
+                }
+                return Ok(candidate);
+            }
+            Err(error) if error.kind() == io::ErrorKind::AlreadyExists => continue,
             Err(error) => return Err(error),
         }
     }
-    Err(std::io::Error::new(
-        std::io::ErrorKind::AlreadyExists,
-        format!("no available artifact filename under {}", parent.display()),
+    Err(io::Error::new(
+        io::ErrorKind::AlreadyExists,
+        "no available artifact filename",
     ))
 }
 

src/compose_output.rs

@@ -1,9 +1,8 @@
-use std::path::Path;
+use std::{path::Path, sync::Arc};
 
 use agentkit_core::ToolOutput;
 use agentkit_tools_core::ToolError;
 use serde_json::json;
-use tokio::io::AsyncWriteExt as _;
 
 const MAX_MODEL_OUTPUT_BYTES: usize = 8 * 1024;
 
@@ -24,29 +23,37 @@ pub(crate) async fn guard(
         return Ok(output);
     }
 
-    let (mut artifact, path) =
-        crate::artifacts::create(&artifact_directory.join("compose-output.json"))
-            .await
-            .map_err(|error| ToolError::Internal(error.to_string()))?;
-    artifact
-        .write_all(body.as_bytes())
-        .await
-        .map_err(|error| ToolError::Internal(error.to_string()))?;
-    artifact
-        .flush()
-        .await
-        .map_err(|error| ToolError::Internal(error.to_string()))?;
-
-    let marker =
-        format!("\n...[compose output spilled: {original_bytes} bytes; see artifact field]...\n");
-    let artifact = path.display().to_string();
+    let body = Arc::new(body);
+    let artifact_body = Arc::clone(&body);
+    let path = artifact_directory.join("compose-output.json");
+    let stored = tokio::task::spawn_blocking(move || {
+        crate::artifacts::write(&path, artifact_body.as_bytes())
+    })
+    .await
+    .map_err(|error| ToolError::Internal(error.to_string()))?;
+    let (artifact, artifact_error) = match stored {
+        Ok(path) => (Some(path.display().to_string()), None),
+        Err(error) => (None, Some(prefix(&error.to_string(), 256).to_owned())),
+    };
+    // Artifact storage must not turn an already-executed tool into a failed
+    // tool call: retrying that call could duplicate its side effects.
+    let marker = if artifact.is_some() {
+        format!(
+            "\n...[compose output spilled: {original_bytes} bytes; read with artifact(path)]...\n"
+        )
+    } else {
+        format!(
+            "\n...[tool completed; output truncated: {original_bytes} bytes; artifact storage failed]...\n"
+        )
+    };
     let mut preview_budget = MAX_MODEL_OUTPUT_BYTES;
     loop {
         let preview = preview(&body, &marker, preview_budget);
         let replacement = json!({
             "preview": preview,
             "artifact": artifact,
             "original_bytes": original_bytes,
+            "artifact_error": artifact_error,
         });
         let replacement_bytes = serde_json::to_vec(&replacement)
             .map_err(|error| ToolError::Internal(error.to_string()))?

src/config_files.rs

@@ -0,0 +1,77 @@
+//! Symlink-compatible reads for user-selected configuration and context files.
+
+use std::{io, path::Path};
+
+// Resolve components in filesystem order. In particular, `..` applies to the
+// resolved directory, not to the lexical parent of a directory symlink. Each
+// lookup uses the overlay and never requires the final target to exist on disk.
+fn resolve_in(
+    filesystem: &crate::resilient_fs::Fs,
+    path: &Path,
+    links: &mut usize,
+) -> io::Result<std::path::PathBuf> {
+    use std::path::{Component, PathBuf};
+
+    let absolute = if path.is_absolute() {
+        path.to_path_buf()
+    } else {
+        std::env::current_dir()?.join(path)
+    };
+    let mut resolved = PathBuf::new();
+    for component in absolute.components() {
+        match component {
+            Component::Prefix(_) | Component::RootDir => resolved.push(component.as_os_str()),
+            Component::CurDir => {}
+            Component::ParentDir => {
+                if !filesystem.metadata(&resolved)?.is_dir() {
+                    return Err(io::ErrorKind::NotADirectory.into());
+                }
+                // Popping a root has no effect, as with native path traversal.
+                resolved.pop();
+            }
+            Component::Normal(name) => {
+                if !filesystem.metadata(&resolved)?.is_dir() {
+                    return Err(io::ErrorKind::NotADirectory.into());
+                }
+                resolved.push(name);
+                if filesystem
+                    .symlink_metadata(&resolved)?
+                    .file_type()
+                    .is_symlink()
+                {
+                    if *links == 40 {
+                        return Err(io::Error::new(
+                            io::ErrorKind::InvalidInput,
+                            "too many symbolic links in configuration path",
+                        ));
+                    }
+                    *links += 1;
+                    let target = filesystem.read_link(&resolved)?;
+                    let target = if target.is_absolute() {
+                        target
+                    } else {
+                        resolved
+                            .parent()
+                            .unwrap_or_else(|| Path::new("."))
+                            .join(target)
+                    };
+                    resolved = resolve_in(filesystem, &target, links)?;
+                }
+            }
+        }
+    }
+    Ok(resolved)
+}
+
+pub fn read_in(filesystem: &crate::resilient_fs::Fs, path: &Path) -> io::Result<Vec<u8>> {
+    filesystem.read(resolve_in(filesystem, path, &mut 0)?)
+}
+
+pub fn read(path: &Path) -> io::Result<Vec<u8>> {
+    read_in(crate::resilient_fs::global(), path)
+}
+
+pub fn read_to_string(path: &Path) -> io::Result<String> {
+    String::from_utf8(read(path)?)
+        .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))
+}