Skip to content

Rewrite README for Spark/agent use case #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AdrianBinDC merged 1 commit into from
Jul 18, 2026
Merged

Rewrite README for Spark/agent use case #2

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 61 additions & 52 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,100 +1,109 @@
# wirecache

**Cache a curated newswire locally and query it** — instead of letting an agent wander the open web for “today’s news.”
**Stop burning tokens on news you already could have cached.**

wirecache pulls RSS/Atom feeds you choose into PostgreSQL, then gives you (or [Hermes](https://nousresearch.com/)) a small CLI to filter by category, keyword, source, and time. Deterministic plumbing; the LLM owns judgment and TTS.
Built for people running agents on local iron — [DGX Spark](https://www.nvidia.com/en-us/products/workstations/dgx-spark/), homelab GPUs, always-on boxes — who want headlines and research bites **without** the model wandering the open web for twenty tool calls.

## Prerequisites
wirecache pulls a curated RSS/Atom wire into PostgreSQL and gives you a tiny CLI. Your agent asks once, gets JSON back, and spends tokens on *judgment* — not rediscovery.

- [Docker](https://docs.docker.com/get-docker/) (PostgreSQL runs in Compose)
- [uv](https://docs.astral.sh/uv/)
- Python 3.11+
Deterministic plumbing. The LLM owns the briefing (and TTS, if you want it spoken).

## Why this exists

Generic “search the news” is slow, flaky, and expensive in context. A Spark (or any local agent host) is perfect for a **personal newswire cache**: you pick the sources, wirecache keeps them warm, Hermes (or whatever agent you use) queries them.

```mermaid
flowchart LR
Feeds[Curated RSS feeds] --> Fetch[wirecache fetch]
Fetch --> DB[(PostgreSQL)]
Agent[Your agent] -->|query JSON| CLI[wirecache CLI]
CLI --> DB
Agent -->|brief / speak| You[You]
```

Physics for poets: **feeds in → cache → ask → answers out.** No crawling safari.

## Quick start

```bash
git clone <your-repo-url> wirecache
git clone https://github.com/AdrianBinDC/wirecache.git
cd wirecache
uv sync --extra dev

# First run copies feeds.example.yaml → feeds.yaml and .env.example → .env
uv run wirecache fetch
uv run wirecache query --category ai --format text --limit 10
uv run wirecache query --category ai --limit 10
```

PostgreSQL starts automatically on the first DB command. Defaults are local-dev only (`news`/`news` on port `5432`).
Needs [Docker](https://docs.docker.com/get-docker/) (Postgres via Compose), [uv](https://docs.astral.sh/uv/), Python 3.11+. Postgres starts on the first DB command. Defaults are local-dev only (`news`/`news` on port `5432`).

## Use cases
## Agent-shaped commands

| Goal | Command |
|------|---------|
| Multi-beat briefing (spoken) | `uv run wirecache query --category ai --category tech --format voice --limit 8` |
| Human-readable scan | `uv run wirecache query --category tech --format text` |
| Topic watch | `uv run wirecache query --keyword NVIDIA --since 2026-07-18T12:00:00Z` |
| Agent / JSON | `uv run wirecache query --keyword CUDA --limit 20` |
| Fresh then query | `uv run wirecache query --category ai --fetch-first --format text` |
| You want | Run |
|----------|-----|
| Structured data for the model | `uv run wirecache query --category ai --limit 15` |
| Spoken briefing (Hermes TTS) | `uv run wirecache query --category ai --format voice --limit 6` |
| Human scan with links | `uv run wirecache query --category tech --format text` |
| Topic watch | `uv run wirecache query --keyword CUDA --days 3` |
| Fresh then ask | `uv run wirecache query --category ai --fetch-first --limit 10` |
| Health check | `uv run wirecache status` |
| Import subscriptions | `uv run wirecache import-opml ~/subscriptions.opml` |
| Add a feed | `uv run wirecache add-feed --url URL --name NAME --categories tech,ai` |
| Retention | `uv run wirecache purge --days 14` |

Background refresh (cron / Hermes): `scripts/fetch.sh`.

## Output formats
Background refresh: `scripts/fetch.sh` (cron / Hermes).

- **`json`** (default) — `{ query, count, stories }` for agents and scripts
- **`text`** — scannable terminal output with URLs
- **`voice`** — speakable prose for Hermes TTS (no URLs, no markdown). wirecache does not synthesize audio.
**Formats:** `json` (default, agents), `text` (terminal), `voice` (speakable prose — no URLs/markdown; audio is Hermes’s job).

## Your feeds stay local
## Your feeds, your wire

| File | Role |
|------|------|
| `feeds.example.yaml` | Committed starter (includes a strong **AI** section: labs, research, digests) |
| `feeds.yaml` | **Your** registry (gitignored). Created on first run; change via CLI |
| `feeds.example.yaml` | Starter set (strong **AI** section: labs, research, digests) |
| `feeds.yaml` | **Yours** — gitignored; created on first run |

```bash
uv run wirecache list-feeds --category ai
uv run wirecache add-feed --url URL --name NAME --categories ai,tech
uv run wirecache import-opml ~/subscriptions.opml --category ai
```

Tag by primary beat: AI-primary feeds get `ai`; general tech stays `tech` only. Full agent procedures: [SKILL.md](SKILL.md).
Tag by primary beat: AI-primary → `ai`; general tech stays `tech`. Agent procedures: [SKILL.md](SKILL.md).

## Architecture
## Under the hood

```
CLI → feeds.registry / feeds.opml
→ fetch.rss → store.stories → PostgreSQL
→ output (json|text|voice)
→ docker_lifecycle
```mermaid
flowchart TB
CLI[CLI] --> Registry[Feed registry / OPML]
CLI --> Fetch[Parallel RSS fetch]
CLI --> Store[Story store]
CLI --> Out[json / text / voice]
Fetch --> Store
Store --> PG[(Postgres + FTS)]
CLI --> Docker[Compose lifecycle]
Docker --> PG
```

| Want to change… | Look in |
|-----------------|---------|
| Feed list / categories | `feeds.yaml` or `src/wirecache/feeds/` |
| How RSS is pulled | `src/wirecache/fetch/rss.py` |
| SQL / FTS / purge | `src/wirecache/store/stories.py`, `schema.sql` |
| Text / voice phrasing | `src/wirecache/output/` |
| Compose / DB | `docker-compose.yml`, `.env` |
| Change this… | Look here |
|--------------|-----------|
| Feeds / categories | `feeds.yaml`, `src/wirecache/feeds/` |
| Fetch behavior | `src/wirecache/fetch/rss.py` |
| Query / purge / FTS | `src/wirecache/store/stories.py`, `schema.sql` |
| Voice phrasing | `src/wirecache/output/voice_fmt.py` |

## Hermes skill
## Hermes

See [SKILL.md](SKILL.md) for agent procedures. Install by placing this directory where Hermes loads skills, then call `uv run wirecache …`.
Drop this repo where Hermes loads skills (or symlink it). Prefer **wirecache** over open-web news search and over legacy `news` / `news-fetcher` skills. Details in [SKILL.md](SKILL.md).

## Logging

Stdout is reserved for command results (JSON / text / voice). Logs go to **stderr** and, by default, `data/wirecache.log`.
Stdout = command results. Logs → stderr and `data/wirecache.log`.

```bash
uv run wirecache -v fetch # DEBUG
uv run wirecache -q status # warnings/errors only
uv run wirecache --log-file '' query --category ai # no log file
uv run wirecache -v fetch
uv run wirecache -q status
```

Env: `WIRECACHE_LOG_LEVEL`, `WIRECACHE_LOG_FILE` (see `.env.example`).
See `.env.example` for `WIRECACHE_LOG_LEVEL` / `WIRECACHE_LOG_FILE`.

## Development
## Dev

```bash
uv sync --extra dev
Expand All @@ -103,4 +112,4 @@ uv run pytest -q

## License

MIT — see [LICENSE](LICENSE).
MIT — [LICENSE](LICENSE).
Loading