Add opt-in i18n foundation for apps (Language setting + injected helpers)

APPS_README.md

@@ -204,6 +204,100 @@ Use these keys to synchronize related fields:
 }
 ```
 
+## Optional: Localization & Shared Helpers
+
+`fetch()` can opt into runtime-injected helpers by **declaring extra parameters**.
+The runtime inspects your signature and only passes a helper if you name it, so the
+classic four-argument `fetch(settings, format_lines, get_rows, get_cols)` keeps
+working unchanged — every helper below is purely opt-in and backward compatible.
+
+### Localization (`i18n`)
+
+If your app shows words (day/month names, status labels), add an `i18n=None`
+parameter. The runtime binds it to the global **Language** setting; on a host
+without it (or with `babel` not installed) `i18n` is `None` and you fall back to
+English.
+
+```python
+def fetch(settings, format_lines, get_rows, get_cols, i18n=None):
+    from datetime import datetime
+    now = datetime.now()
+    weekday = i18n.weekday(now) if i18n else now.strftime("%A").upper()
+    label   = i18n.t("SUNRISE") if i18n else "SUNRISE"
+    return [format_lines(weekday, label)]
+```
+
+- `i18n.weekday(dt, short=False)` / `i18n.month(dt, short=False)` — CLDR-correct,
+  UPPERCASE day/month names for every language.
+- `i18n.date(dt, short=False, year=False)` — day + month (and optional year) in the
+  locale's **own order and wording**: `JULY 9` in English but `9 JUILLET` (fr),
+  `9. JULI` (de). Don't hand-assemble `month + " " + day` — order is language-specific.
+- `i18n.time(dt, seconds=False)` — wall-clock time: `3:48 PM` in English, `15:48`
+  elsewhere. `i18n.is_24h` exposes the same decision if you branch yourself.
+- `i18n.unit("D")` — localized compact duration suffix for `D`/`H`/`M`/`S`, so `175D`
+  becomes `175J` in French (jour), `175T` in German (Tag).
+- `i18n.number(value, decimals=2, grouping=True)` — a number with the locale's own
+  separators: `1,234.50` (en) vs `1.234,50` (de) vs `1 234,50` (fr). Use it for any
+  price/rate/percent — never hardcode `f"{v:,.2f}"`.
+- `i18n.base_currency()` / `i18n.country()` — the currency / country a language-region
+  implies (a sensible default; prefer `get_location` below for geography).
+- `i18n.t("ENGLISH LABEL")` — a translated UI word; with no translation it returns the
+  English key, so nothing ever breaks. All data lives in `server/i18n_data.json` (the
+  language list, translations, and per-language default currency/country). Add keys or
+  languages there rather than in your app, keep them short (modules are narrow), and
+  give each key a generic `context` note so translators know what the word means. A
+  regional variant like `pt-BR` automatically inherits every `pt` translation.
+
+Once your app adapts to the language, add `"i18n": true` to its `manifest.json`. When
+internationalization is enabled the Apps grid shows a 🌐 badge on those cards, and the
+app automatically gets a per-app **Language** override in its settings (blank = follow
+the global Language). Only Western-European (Windows-1252) languages are offered — the
+modules can't display Greek, Cyrillic, CJK, etc.
+
+Internationalization is **off by default**: unless the user turns on *Enable
+internationalization* in Settings, `i18n` is not injected and your app runs its
+English `i18n=None` fallback — so always keep that fallback correct. The toggle exists
+because accented glyphs only render on modules whose firmware/character map supports
+them.
+
+### Shared current weather (`get_weather`)
+
+If your app shows the weather, don't hardcode a provider — add a `get_weather=None`
+parameter and call it:
+
+```python
+def fetch(settings, format_lines, get_rows, get_cols, get_weather=None):
+    if get_weather is None:            # host without the helper
+        return [format_lines("NO WEATHER")]
+    w = get_weather()                  # uses the *global* provider + key + location
+    if not w["ok"]:
+        return [format_lines("WEATHER", "UNAVAILABLE")]
+    return [format_lines(w["city"], f'{w["temp_f"]}F {w["desc"]}')]
+```
+
+`get_weather()` returns a dict: `ok`, `city`, `temp_f`, `feels_like_f`, `hi_f`,
+`lo_f`, `desc`, `humidity`, `wind_mph`, `cloud_cover`, `provider`, `lat`, `lon`
+(temps in °F; `ok` is `False` with an `error` key on failure). The default provider
+is keyless Open-Meteo, so weather works with **no API key**.
+
+### Location → country / currency (`get_location`)
+
+Anything tied to *geography* — which currency, which country's holidays — should key
+off the configured **Location**, not the language. Declare a `get_location`
+parameter for the shared resolver:
+
+```python
+def fetch(settings, format_lines, get_rows, get_cols, get_location=None):
+    loc = get_location() if get_location else {}
+    country     = loc.get("country")      # ISO 3166-1 alpha-2, e.g. "CA"
+    subdivision = loc.get("subdivision")  # ISO 3166-2, e.g. "CA-QC"; may be None
+    currency    = loc.get("currency")     # ISO 4217, e.g. "CAD" (None if unknown)
+```
+
+It reverse-geocodes the global Location once (cached) and is keyless. Declaring
+`get_location` also gives the app an automatic per-app **Location** override in its
+settings. Helpers compose: `def fetch(..., get_weather=None, get_location=None, i18n=None)`.
+
 ## Final Checklist
 
 - Confirm your app directory contains `manifest.json` and `app.py`.

server/app.py

@@ -11,11 +11,15 @@
 import unicodedata
 import yfinance as yf
 import importlib.util
+import inspect
 import urllib.request
 import shutil
 from datetime import datetime
 from flask import Flask, render_template, request, jsonify
 from tuning import build_tuning_adjust_commands
+import i18n
+import location
+import weather
 from hardware.universal_firmware import (
     UniversalFirmwareError,
     UniversalFirmwareManager,
@@ -207,6 +211,9 @@ def load_settings():
         "location_lon":  "",
         "location_name": "",
         "timezone":      sys_tz,
+        "i18n_enabled":  False,
+        "language":      "en-US",
+        "weather_provider": "openmeteo",
         "weather_api_key": "",
         "mbta_stop":     "",
         "mbta_route":    "",
@@ -1369,6 +1376,69 @@ def _load_functional_module(app_id, app_dir):
         logging.error(f"Plugin {app_id}: error importing app.py: {e}")
 
 
+def _fetch_accepts(fn, name):
+    """True if an app's fetch() declares a parameter called ``name`` (how an app
+    opts into an injected helper like ``i18n`` / ``get_weather`` / ``get_location``),
+    or accepts arbitrary keywords. Classic 4-arg apps accept neither and are called
+    unchanged, so this whole mechanism is fully backward compatible."""
+    try:
+        params = inspect.signature(fn).parameters
+    except (TypeError, ValueError):
+        return False
+    return name in params or any(p.kind == p.VAR_KEYWORD for p in params.values())
+
+
+def _i18n_enabled():
+    """Master switch for internationalization. Off by default: the display stays
+    English-only and looks exactly as it did before i18n existed. Should only be
+    turned on for modules whose firmware/character map can render accented glyphs."""
+    return bool(settings.get("i18n_enabled", False))
+
+
+def _resolve_app_language(app_id):
+    """The Language an app should render in: a per-app override
+    (``plugin_<id>_language``) when set, otherwise the global Language. A blank or
+    unset override means "follow the global Language"."""
+    override = settings.get(f"plugin_{app_id}_language")
+    return override or settings.get("language", "en-US")
+
+
+def _apply_location_override(plugin_settings, value):
+    """Fold a per-app Location override (a ``/location_search`` chip value, format
+    ``"lat,lon|name"``) into the ``location_*`` keys the location/weather helpers
+    read, so a per-app override is honored without touching the global location.
+    A blank/unset value leaves the global location in place."""
+    if not value:
+        return
+    try:
+        coords, _, name = str(value).partition("|")
+        lat, _, lon = coords.partition(",")
+        lat, lon = lat.strip(), lon.strip()
+        if lat and lon:
+            plugin_settings["location_lat"] = lat
+            plugin_settings["location_lon"] = lon
+            plugin_settings["location_name"] = name.strip() or plugin_settings.get("location_name", "")
+    except Exception:
+        pass
+
+
+def _plugin_fetch_kwargs(app_id, fetch_fn, plugin_settings):
+    """Optional keyword args a plugin's fetch() opted into by declaring them. Only
+    the helpers the signature names are built, so nothing is injected into apps that
+    don't ask for it."""
+    kwargs = {}
+    if _fetch_accepts(fetch_fn, "get_weather"):
+        kwargs["get_weather"] = lambda s=None: weather.fetch_current(
+            s if s is not None else plugin_settings)
+    if _fetch_accepts(fetch_fn, "get_location"):
+        kwargs["get_location"] = lambda: location.resolve(plugin_settings)
+    # Only bind a Localizer when i18n is enabled; otherwise the app's i18n=None
+    # fallback runs and it renders exactly as it did before i18n existed (English).
+    if _i18n_enabled() and _fetch_accepts(fetch_fn, "i18n"):
+        kwargs["i18n"] = i18n.Localizer(_resolve_app_language(app_id))
+    return kwargs
+
+
 def get_plugin_pages(app_id):
     manifest = _plugin_registry.get(app_id)
     if not manifest:
@@ -1404,7 +1474,13 @@ def get_plugin_pages(app_id):
                 else:
                     key = f"plugin_{app_id}_{s['key']}"
                     plugin_settings[s["key"]] = settings.get(key, s.get("default", ""))
-            pages = mod.fetch(plugin_settings, format_lines, get_rows, get_cols)
+            # A per-app Location override wins over the global location for this
+            # app's injected get_location / get_weather helpers (blank = follow global).
+            _apply_location_override(plugin_settings, settings.get(f"plugin_{app_id}_location"))
+            # Optional helpers (i18n / get_weather / get_location) are injected only
+            # when fetch() declares the matching parameter; 4-arg apps are unchanged.
+            kwargs = _plugin_fetch_kwargs(app_id, mod.fetch, plugin_settings)
+            pages = mod.fetch(plugin_settings, format_lines, get_rows, get_cols, **kwargs)
             if not isinstance(pages, list):
                 pages = [str(pages)]
             _plugin_caches[app_id] = {"pages": pages, "fetched_at": now}
@@ -1433,6 +1509,8 @@ def get_plugin_app_list():
             "desc": manifest.get("description", "")[:30],
             "plugin": True,
             "plugin_id": app_id,
+            # 🌐 badge: the app adapts to Language AND i18n is globally enabled.
+            "i18n": bool(manifest.get("i18n")) and _i18n_enabled(),
         }
         if "min_rows" in manifest:
             entry["min_rows"] = manifest["min_rows"]
@@ -1443,6 +1521,13 @@ def get_plugin_app_list():
     return entries
 
 
+# The languages an app can be localized into (the global Language select and every
+# per-app Language override read from here). Defined once in server/i18n_data.json —
+# only Windows-1252 (Western/Latin-1) languages, since the modules can't display
+# Greek, Cyrillic, CJK, etc.
+LANGUAGE_OPTIONS = i18n.LANGUAGE_OPTIONS
+
+
 _SETTING_PASSTHROUGH_KEYS = (
     "size",
     "ph",
@@ -1560,6 +1645,42 @@ def map_related_key(raw_related_key):
     return field
 
 
+def _app_uses_location(app_id):
+    """True if the app's fetch() opts into the injected ``get_location`` helper."""
+    mod = _plugin_modules.get(app_id)
+    return bool(mod) and _fetch_accepts(mod.fetch, "get_location")
+
+
+def _auto_setting_fields(app_id, manifest, manifest_settings):
+    """Per-app override fields the runtime adds automatically (not in the manifest):
+    a Language override for any i18n app, and a Location override for any app that
+    reads the injected location. Each is stored under its own ``plugin_<id>_*`` key,
+    and a blank value means "follow the global setting"."""
+    declared = {s["key"] for s in manifest_settings}
+    fields = []
+    if manifest.get("i18n") and _i18n_enabled() and "language" not in declared:
+        fields.append({
+            "key": f"plugin_{app_id}_language",
+            "label": "Language",
+            "type": "select",
+            "opts": [{"value": "", "label": "Follow global"}] + LANGUAGE_OPTIONS,
+            "ph": "",
+            "note": "Override the global Language for this app only.",
+        })
+    if _app_uses_location(app_id) and "location" not in declared:
+        fields.append({
+            "key": f"plugin_{app_id}_location",
+            "label": "Location",
+            "type": "search_chips",
+            "searchUrl": "/location_search",
+            "resultKey": "results",
+            "maxItems": 1,
+            "ph": "",
+            "note": "Override the global Location for this app only (place search).",
+        })
+    return fields
+
+
 def get_plugin_settings_config():
     configs = {}
     for app_id, manifest in _plugin_registry.items():
@@ -1569,6 +1690,7 @@ def get_plugin_settings_config():
             _build_plugin_setting_field(app_id, setting, resolved_keys)
             for setting in manifest_settings
         ]
+        fields += _auto_setting_fields(app_id, manifest, manifest_settings)
 
         configs[f"plugin_{app_id}"] = {
             "title": f"{manifest.get('icon', '🧩')} {manifest.get('name', app_id)}",
@@ -2080,7 +2202,7 @@ def _trigger_loop():
 @app.route('/')
 def index():
     version = _read_version()
-    return render_template('index.html', version=version)
+    return render_template('index.html', version=version, language_options=LANGUAGE_OPTIONS)
 
 @app.route('/current_state')
 def current_state():

server/gateway_transport.py

@@ -151,7 +151,12 @@ def _on_message(self, client, userdata, msg):
             return
         # Re-append the newline the gateway stripped so downstream parsers that
         # look for '\n' terminators behave exactly as with raw serial.
-        data = frame.encode("utf-8", errors="ignore") + b"\n"
+        # Encode with latin-1 (not utf-8): the bus is single-byte cp1252, so a
+        # frame may carry raw high bytes (e.g. a module's A-command char map with
+        # 0xE9/0x80). latin-1 maps every code point 0x00-0xFF back to that exact
+        # byte, so downstream .decode('cp1252') sees the original bytes. utf-8
+        # here would drop lone high bytes (errors="ignore") and corrupt the map.
+        data = frame.encode("latin-1", errors="ignore") + b"\n"
         with self._rx_lock:
             self._rx_buf.extend(data)
 
@@ -162,8 +167,13 @@ def _extract_frame(payload):
         Accepts the gateway's JSON form ``{"command":"..."}`` and also a bare
         plain-text frame, for robustness against future gateway changes.
         """
+        # Decode with latin-1, a transparent byte<->code-point codec: the bus is
+        # single-byte cp1252 and a frame (e.g. an A-command char-map response) can
+        # contain raw high bytes that are invalid standalone utf-8. latin-1 never
+        # raises and preserves every byte, so json.loads still parses the ASCII
+        # structure and the raw bytes survive to the caller (re-encoded latin-1).
         try:
-            text = payload.decode("utf-8", errors="ignore")
+            text = payload.decode("latin-1")
         except Exception:
             return None
         text = text.strip()