Skip to content

architecture.md

Latest commit

 

History

History
258 lines (193 loc) · 12.4 KB

architecture.md

File metadata and controls

258 lines (193 loc) · 12.4 KB
Raw

Architecture

This document describes LocalScript's components, the session state machine, the generation and validation pipeline, and the optional RAG flow.

Table of Contents

System Overview

LocalScript is a small microservice system with three first-party components and two optional third-party dependencies:

┌──────────┐    ┌─────────────────┐    ┌───────────────┐    ┌─────────────────┐
│  curl /  │───▶│   llm-service   │───▶│     Ollama    │───▶│  sandbox-service│
│   TUI    │    │  FastAPI  :8080 │    │ qwen2.5-coder │    │  Axum  :6778    │
│          │    │  state machine  │    │     :11434    │    │  AST + sandbox  │
└──────────┘    └────────┬────────┘    └───────────────┘    └────────┬────────┘
                         │                                           │
                         └──────── Plan / Code ──── User ────────────┘
                                                          │
                                                   (optional)
                                                          │
                                                   ┌──────▼───────┐
                                                   │   Qdrant     │
                                                   │  :6333 (RAG) │
                                                   └──────────────┘

All components communicate over plain HTTP/JSON on a private Docker bridge network (backend).

Components

llm-service — Python / FastAPI (port 8080)

The orchestrator and the only public API surface. Responsibilities:

  • Accepts generation requests (POST /generate) and health checks (GET /health).
  • Hosts the session state machine and an in-memory session store.
  • Drives the LLM through three roles: architect (plans), coder (code), critic (code review).
  • Calls sandbox-service to validate every generated or revised code snippet.
  • Optionally builds RAG context from Qdrant before code generation.

Source structure:

llm-service/app/
├── main.py            # FastAPI app, routes /generate and /health
├── config.py          # All environment-based configuration
├── api/
│   ├── schemas.py     # Pydantic request/response models
│   └── state_machine.py  # Session states, transitions, and orchestration
├── core/
│   ├── pipeline.py    # GenerationPipeline: plan / code / critique calls
│   └── prompts.py     # System prompts for architect, coder, critic
├── clients/
│   ├── ollama.py      # Ollama chat API client
│   ├── sandbox.py     # sandbox-service client + feedback extraction
│   └── rag.py         # Qdrant search + embedding helpers
├── utils/
│   └── json_parser.py # Extracts workflow context JSON from the task text
└── scripts/           # run_model.py, qdrant_delete.py

sandbox-service — Rust / Axum (port 6778)

The code validator. It is not meant to be exposed publicly (see Operational Notes). Responsibilities:

  • Parses Lua with tree-sitter and validates the AST (syntax).
  • Runs static safety checks (dangerous text patterns, forbidden calls).
  • Optionally executes the code in a hardened mlua sandbox with memory and time limits.
  • Returns structured errors with line numbers and source snippets.

Source structure:

sandbox-service/src/
├── main.rs            # Axum server, routes /pipeline and /health
├── models.rs          # PipelineRequest / PipelineResponse models
├── routes/
│   └── pipeline.rs    # Validation pipeline orchestration
├── ast/
│   ├── parser.rs      # tree-sitter Lua parsing + error walking
│   ├── extractor.rs   # Function-call extraction from the AST
│   └── safety.rs      # Dangerous patterns + forbidden call rules
└── executor/
    └── sandbox.rs     # mlua runtime sandbox, context injection, error mapping

llm-tui — Rust / Ratatui

A terminal client. It talks only to llm-service and renders the session as a chat interface. See tui.md.

ollama — Local LLM runtime (port 11434)

Hosts the generation model (qwen2.5-coder:7b by default). A dedicated ollama-init container pulls the model on first startup.

qdrant — Vector database (port 6333, optional)

Stores Lua pattern/reference documents for RAG. See RAG Flow.

Request Lifecycle

User ── task ──▶ llm-service
                  │
                  ├── create session (GENERATING_PLAN)
                  ├── generate plan  (architect prompt) ──▶ Ollama
                  ▼
              AWAITING_PLAN_CONFIRMATION  ◀── plan returned to user
                  │
        user approves / requests edits
                  │
                  ▼
              build RAG context (optional) ──▶ Qdrant
                  │
                  ├── generate code (coder prompt) ──▶ Ollama
                  ├── validate in sandbox-service (retry loop, up to 20)
                  ├── LLM critique (optional) ──▶ Ollama
                  ▼
              AWAITING_CODE_APPROVAL  ◀── code returned to user
                  │
        user approves / requests edits
                  │
                  ▼
                  DONE
  1. A client sends a task (optionally with inline workflow context JSON) to POST /generate. A new session is created.
  2. llm-service asks Ollama (architect role) to produce a step-by-step plan and returns it.
  3. The user either approves the plan or sends revision feedback.
  4. On approval, llm-service optionally builds RAG context from the plan, then asks Ollama (coder role) to generate Lua code.
  5. The code is sent to sandbox-service. If validation fails, the code is regenerated with the sandbox's feedback (loop).
  6. Optionally, the code is reviewed by Ollama (critic role), which replies CODE_OK or a list of issues.
  7. The user approves the final code (or requests more edits). On approval the session transitions to done.

Session State Machine

Sessions are tracked server-side in memory (sessions: dict[str, SessionData]), keyed by session_id.

States

State Description
generating_plan Initial state; plan generation is in progress.
awaiting_plan_confirmation Plan is ready; waiting for approval or edits.
generating_code Code generation and validation in progress.
awaiting_code_approval Code is ready and validated; waiting for approval or edits.
done Code approved; session is complete.

Transitions

From Condition To / action
generating_plan plan generated awaiting_plan_confirmation
awaiting_plan_confirmation user_response is an approval word generating_code
awaiting_plan_confirmation any other user_response plan revision → awaiting_plan_confirmation
generating_code sandbox (+ critic) pass awaiting_code_approval
generating_code validation fails awaiting_code_approval (with feedback)
awaiting_code_approval user_response == "подтвердить" done
awaiting_code_approval any other user_response code revision → awaiting_code_approval
done any request done (returns the approved code)

Approval words

  • Plan approval accepts a list of words (case-insensitive): подтвердить, да, согласен, утверждаю, approve, confirm, yes, ok, хорошо, принять, ок, plus some demo aliases (78, 67, docker, борзячка).
  • Code approval is stricter: only the exact word подтвердить (case-insensitive) completes the session. Any other response is treated as revision feedback.

Note: server-generated human messages are currently in Russian (e.g., "План сгенерирован. Подтвердите или укажите исправления.").

Generation Pipeline

GenerationPipeline (llm-service/app/core/pipeline.py) wraps an OllamaClient and three prompt builders (llm-service/app/core/prompts.py):

Method LLM role Purpose
_generate_plan(task, context) architect Produces a plan, with steps separated by $:.
_generate_code(plan, task, rag_data, previous_code, critic_feedback, context) coder Generates or revises Lua code.
_critique_code(code, rag_data, context) critic Reviews the code; returns CODE_OK or issues.

Prompts encode "LowCode environment" rules the model must follow, for example:

  • Access workflow data via the global wf table (wf.vars, wf.initVariables), not JsonPath.
  • Iterate arrays with for _, item in ipairs(...) do ... end.
  • Store newly created variables under wf.vars.
  • Use table.remove instead of niling array holes; avoid the # operator on arrays with holes.

The model and Ollama endpoint are configured via GENERATION_MODEL and OLLAMA_URL (see configuration.md).

Validation Pipeline

After code is generated (or revised), llm-service calls sandbox-service (POST /pipeline). The sandbox pipeline has four stages:

  1. Parse — tree-sitter parses the code; syntax errors are reported with line/column and a source snippet.
  2. Static safety — text-level dangerous patterns and AST-level forbidden calls are checked.
  3. Execute (optional, controlled by execute) — the code runs in a hardened mlua sandbox.
  4. Report — a structured response with status, logs, AST analysis, and execution stats.

The sandbox retry loop in llm-service (validate_code) regenerates code with feedback for up to CODE_RETRIES_SANDBOX (default 20) attempts. When execute is off, the response still returns AST analysis.

Detailed security rules: sandbox-security.md.

LLM critique

With llm_validation: true (the default), llm-service asks Ollama (critic role) to review the code. The response is compared against CONFIRM_WORD (default CODE_OK), case-insensitively. If it matches, the code proceeds to the user; otherwise, the critic's comments are returned as sandbox_feedback for the user to act on.

The llm_validation flag can be set to false per request to skip this step (see api-reference.md).

RAG Flow

RAG is optional and degrades gracefully — if Qdrant or the embeddings endpoint is unreachable, generation continues without reference context.

  1. When the plan is approved, llm-service splits the plan into chunks on $: markers.
  2. Each chunk is embedded via an OpenAI-compatible embeddings endpoint (EMBEDDINGS_URL, model bge-m3).
  3. The embedding searches the Qdrant collection (QDRANT_COLLECTION, default lua_patterns, top-1 result).
  4. Matched documents (description + validation_checklist payload fields) are concatenated into rag_data.
  5. rag_data is injected into the coder prompt as Reference data and into the critic prompt.

Implementation: llm-service/app/clients/rag.py. Required configuration: configuration.md.

Project Layout

.
├── docker-compose.yml         # Service orchestration (ollama, qdrant, sandbox, api, tui)
├── README.md                  # Project overview and quick start
├── LICENSE                    # Apache 2.0
├── llm-service/               # FastAPI orchestrator (Python 3.12)
├── sandbox-service/           # Lua sandbox validator (Rust)
├── llm-tui/                   # Terminal client (Rust / Ratatui)
├── qdrant_data/               # Qdrant storage (bind mount, persists)
└── docs/                      # This documentation