Feat/dual os - #14
Draft
Spiritreader wants to merge 61 commits into
Draft
Conversation
- Add encoder/priority_windows.go (PriorityClass via x/sys/windows) - Add encoder/priority_linux.go (nice levels via syscall.Setpriority) - Remove windows import from encoder.go, replace with setProcessPriority() call - Fix unmasked bug: err -> vErrify in verification log (was shadowed by windows err var)
- Add Makefile + build scripts for windows/linux targets - Extend CI workflow to windows+linux matrix - Add Dockerfile + compose.yaml for Komodo deployment - Add .dockerignore
- docker/config.example.json: converted Windows UNC paths to /media/... paths for compose mount (/mnt/user/media -> /media) - docker/README.md: setup instructions, hardware acceleration notes, software encoder fallback reference - compose.yaml: add commented GPU devices passthrough section
…pport - Use linuxserver/ffmpeg:latest as runtime base (ships QSV-enabled ffmpeg) - Enable /dev/dri GPU passthrough in compose.yaml via devices section - Update README: confirm QSV works out of box on Unraid with ARC GPU
- compose.yaml: drop image: avior-go:latest so docker compose up always builds when extra_args --build is passed (avoids stale cached images) - docker/README.md: document Komodo stack config with branch and extra_args = "--build"
…ations GetClientForMachine() looked up clients by strings.ToUpper(hostname) but inserted with the raw hostname. On Linux the hostname is lowercase (container ID), so the lookup never matched and every restart created a new duplicate client entry. Now hostname is uppercased once before both lookup and insert, consistent with the historical UPPERCASE registry (VDR-U, PHOENIX, ...).
… in Mongo instead of container ID
Add PathMappings (config.json map of UNC prefix -> local container path) and translatePath() in the worker, applied once at job-processing time. The Docker instance can now process DB jobs whose Path is a Windows UNC path (e.g. \\192.168.178.75\recording_pool\... -> /recording_pool/...). No mappings configured = identity, so Windows instances are unaffected.
Spiritreader
marked this pull request as draft
August 2, 2026 14:07
Spiritreader
commented
Aug 2, 2026
| state := globalstate.Instance() | ||
| state.HostName = hostname | ||
| var thisMachine *structs.Client | ||
| err := ds.Db().Collection("clients").FindOne(ctx, bson.M{"Name": strings.ToUpper(hostname)}).Decode(&thisMachine) |
Owner
Author
There was a problem hiding this comment.
Does changing this cause issues with machine detection?
If machines names aren't normalized, we would would henceforth produce duplicate clients in the DB when signing in, or not find a client anymore that was previously uppercased.
This is generally fine, but either the DB needs to be adjusted one-time to be compliant with the new name format, or we perform normalization during name read and writes throughout the codebase (all to lowercase when checking etc).
/data/avior-go may be the currently running binary (previous start exec'd from the same volume). cp onto a running executable fails with 'Text file busy', so the entrypoint aborted and the container crash-looped on every restart after the first deploy. rm -f first, then copy.
Run the app as the configured UID/GID (LinuxServer convention) so every file it creates - logs in /data/log, .INFO.log next to media, config.json - is owned by that user on the host, matching the mm:users ownership the Windows instances produced. umask 002 keeps group/other rw. Default without PUID/PGID: root (previous behavior).
su-exec is an Alpine package and not available in the Ubuntu-based linuxserver/ffmpeg image; apt install failed with exit 100. setpriv is preinstalled in util-linux. Keep su-exec as fallback if present.
Same module settings as the software example (Threshold, Mode, MaxSize, MinResolution, Accuracy, Difference/SampleCount/Fraction) and desired enablement/priority.
…ip threshold 8, Length) in QSV example
Port from test/DockerwoARC: new config field ClientName overrides the container hostname for DB client registration, preventing phantom clients (container ID) on every restart. Empty = previous os.Hostname() behavior.
glg.FileWriter passes its perm argument to BOTH os.MkdirAll (directory) and OpenFile (file), so one value cannot be correct for both: a directory needs the execute bit (0755), a file should be 0644. The earlier 0644 change left the dir as rw-r--r-- (no x) - still unenterable. MkdirAll with 0755 before FileWriter gives dir 0755 and file 0644.
A legacy log/ directory created with mode 000 (old os.ModeAppend bug) stays 000 forever because os.MkdirAll is a no-op on existing paths. Explicit os.Chmod(logDir, 0755) repairs pre-existing directories too.
The host exposes two GPUs: iGPU (renderD128, UHD 630, no AV1 encode) and ARC (renderD129). ffmpeg defaults to the first device, so av1_qsv failed with 'runtime doesn't support AV1 encoding'. Add -qsv_device /dev/dri/renderD129 to all encoder profiles.
Same fix as the ARC test config: the host has two GPUs (iGPU renderD128 without AV1, ARC renderD129), ffmpeg defaults to the first, so av1_qsv failed. Pin all encoder profiles to the ARC device.
The library cache stays valid for the Redis TTL (24h). If MediaPaths change during that window (config reload), files in newly added paths are never seen as duplicates until a restart. Fingerprint the MediaPaths list and invalidate the cache on change; TTL behavior unchanged.
Under Docker the MediaPaths walk is a direct local FS access (<1s), so the 24h Redis-TTL cache adds staleness bugs (newly added files not seen as duplicates) without perf benefit. New config flag CacheLibScan (default true, Windows behavior unchanged); Docker example configs set false. Redis job broadcast for multi-instance stays untouched.
User's production config for the UNRAID instance with Intel ARC: - ClientName UNRAID, full MediaPaths (re_encode/transcoded/tv) - PathMappings for UMS + IP hostnames (both recording shares) - QSV pinned to /dev/dri/renderD129 (ARC) in all encoder profiles - CacheLibScan false (fresh scan per job, <1s on local FS)
…PORT, DATA_DIR, GPU_DEVICE) Remove hard-coded container_name and per-instance values. Each Komodo stack sets environment vars; container name derives from the Compose project name (set per stack via project_name) to avoid collisions. Client registration is separated via ClientName in each instance's config.json.
…aming ClientName stays identical across instances (all 'UNRAID'); the Instance field in config.json separates them (UNRAID, UNRAID-1, ...) and sets the listening port 10000+Instance. Compose now exposes CONTAINER_PORT so the host mapping follows the instance's actual container port.
The app logs with time.Local which reads the TZ env var; unset defaults to UTC in the container. Add TZ with Europe/Berlin default, overridable per stack via environment.
…ave() json omitempty on a bool omits false when marshaling. config.Save() runs on every start (api.go), so CacheLibScan:false vanished from config.json after each restart. Remove omitempty so false is written explicitly.
…multi-instance setup
Without container_name Compose derives <project>-<service>-<replica> (avior-go-avior-go-1), which is confusing. Set container_name from INSTANCE_SUFFIX so instances are simply avior-go, avior-go-1, ...
…_SUFFIX usage and client naming conventions
When the exact output name already exists in the library but the release
year differs (e.g. 'Die Löwin' 2011 vs 2024), the new film is renamed to
'Die Löwin (2024)' before the duplicate modules decide — treating
same-title/different-year films as separate instead of duplicates.
- media/year.go: ExtractYear/ExtractYearFromFile (subtitle -> .txt -> .log
fallback, ported from movie_nfo_lib), NormalizeName, HasYearSuffix
- config: YearAwareDupes flag (default true)
- worker: findDuplicateYear + suffix append before checkForDuplicates
- RE2 port notes: no lookahead (decade check manual), \w is ASCII
(\p{L} for umlauts)
ExtractYearFromFile scanned ALL .txt lines and took the first year — the Created=/Date= lines carry the RECORDING date (e.g. 03.08.2026), which is not the release year, so 'Die Löwin' (2024 film, 2026 recording) got suffixed as 'Die Löwin (2026)'. Restrict .txt source to Info= and Title= lines. Regression test added.
… custom regexes
Replace the simplified custom patterns with faithful Go ports of the
library's proven logic (extract_txt_metadata / _extract_log_metadata):
COUNTRY_HINT_PATTERN, TXT_META_PATTERN, LOG_META_LINE(_STRICT)_PATTERN,
TIMER_NAME_META_PATTERN, TYPE2_* patterns, _slice_log_lines_for_metadata.
This fixes the multi-country cases (Deutschland/Estland/Lettland) that the
simplified version missed, and reuses the battle-tested extraction order
(subtitle -> .txt Info=/Description= -> .log Timer Name -> meta line).
RE2 port notes: no lookahead (decade check manual), \w is ASCII (\p{L}),
and a capturing country-list alternation wrapped in a repeat breaks RE2 —
country lists are non-capturing, year is group 1.
…ry context) The year is only recognized in Genre+Country+Year context — the full COUNTRY_HINT_PATTERN backbone distinguishes 4-digit years from other numbers. Port the complete _extract_from_candidate cascade: preprocessing (FSK/parens/subtitle-truncation/Min-strip/dupe-collapse), start_match truncation, TXT_META -> TYPE2 -> loose -> short_genre -> permissive stages, all gated by NARRATIVE_MARKER (excludes 'Jahr 2022', 'im 2024'). Add scored Description=-segment selection, episode detection (IsProbablyEpisodeFilename), and real-case regression tests (new film .log TimerName 2024 vs TimerStart 2026 -> 2024; old film 'Melodram Südafrika/2011' -> 2011).
The year-aware duplicate flow was silent on all negative branches, so a film with an unreadable .log (e.g. 'Die Löwin .log' with a stray space) was silently treated as an exact duplicate and replaced. Now every step is visible in the log: - year extraction: which source (subtitle/txt Info=/Description=/log) yielded the year, or why none was found - collision decision: no year / duplicate year unknown / same year / different year -> suffix appended Regression test for the real production case (subtitle wins over recording year in .log) added.
glg.LOG has no file writer (AddLevelWriter(glg.LOG, log) is commented out in app.go), so glg.Logf only reaches the console, not main.log. The new year extraction / collision logs must use Infof to be visible in the log file. Pre-existing Logf calls (input file, media struct, walker positions) left untouched.
CacheLibScan=false disables the library cache (always fresh FS scan), which makes the Redis job broadcast redundant. AutoManage now treats Redis as disabled unless BOTH Redis.Enabled and CacheLibScan are true — no Redis connection/session is held, and the worker's Running() checks skip broadcast/cache paths. Re-enabling CacheLibScan (or Redis.Enabled) restores Redis automatically; the coupling is logical, not a hard config default, so Redis stays available if caching is ever needed again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.