audiobook-merge/README.md

Audiobook TUI

Audiobook TUI is a keyboard-driven queue for converting MP3 audiobooks into validated M4B files on a headless Linux server. It is designed to be copied to Enterprise-D, launched through SSH, and run one book at a time with explicit operator approval.

The application discovers a source in pending, previews its tracks and metadata, creates chapters, runs m4b-tool, validates the resulting M4B with ffprobe, publishes it to ready, and only then moves the source to done. An operator can instead move a source to review after a separate typed confirmation.

Safety model

• Conversion requires typing CONVERT exactly. Review moves require REVIEW.
• Existing final, partial, work, done, and review destinations are never overwritten.
• Inputs remain unchanged during conversion. A source moves only after the exact validated output has been published.
• State moves use an atomic no-replace rename when the filesystem supports it. Across mounts, or when that operation is unavailable, the application makes exclusive destination claims and verifies source and destination identities and content before removing the source.
• All configured paths must be absolute and are canonicalized before use. The four media roots cannot overlap one another, and state/work cannot overlap a media root.
• A selected source and every track are checked against the canonical pending root, and source/track symlinks are rejected before any ffprobe process is started.
• Interrupted and failed conversions leave source material in pending and preserve partial/work artifacts for explicit recovery.
• The container runs as the numeric PUID:PGID supplied by Compose, drops all Linux capabilities, enables no-new-privileges, is not privileged, and does not mount the Docker socket.
• UID 0 and GID 0 are refused. The application must never run as root, even if root IDs are supplied accidentally in .env.

This safety model is not a backup. Keep independent backups of source audio and the ready library.

Enterprise-D prerequisites

Enterprise-D must provide:

• Linux with Docker Engine and the Docker Compose plugin (docker compose)
• SSH access with terminal allocation (ssh -t)
• a user that can read and write all four audiobook directories
• enough free space for the source, temporary merge, partial M4B, and final M4B
• LAN access to Discovery at http://192.168.86.168:3333 when optional Audnexus lookup is enabled
• either linux/amd64 or linux/arm64 for the default sandreas/m4b-tool:0.5.2 image

The Docker CLI and daemon were not available on the development desktop used to create this package. The Python tests were run locally; every Docker command below must be run and verified on Enterprise-D before the service is considered accepted there. In particular, do not treat the presence of compose.yaml as proof that Enterprise-D can pull, build, or run the image.

Copy and install

The dist/audiobook-tui-0.1.0.tar.gz release bundle is the preferred copy artifact. It contains one top-level audiobook-tui/ directory and omits Git data, development caches, local environment files, runtime state/work data, and audiobook media. Verify the SHA-256 reported with the release, then copy and extract it:

shasum -a 256 audiobook-tui-0.1.0.tar.gz
scp audiobook-tui-0.1.0.tar.gz Enterprise-D:/opt/
ssh Enterprise-D
cd /opt
tar -xzf audiobook-tui-0.1.0.tar.gz
cd audiobook-tui

If copying the source tree instead, exclude local artifacts:

From the machine containing this project, copy it to Enterprise-D without local caches, state, or audio:

rsync -av \
  --exclude .git --exclude .pytest_cache --exclude __pycache__ \
  --exclude .env --exclude state --exclude '*.mp3' --exclude '*.m4b' \
  ./audiobook-tui/ Enterprise-D:/opt/audiobook-tui/

Then connect and enter the project:

ssh Enterprise-D
cd /opt/audiobook-tui

Create the four sibling workflow directories and persistent log directory. Run this as the account that will operate the application:

install -d -m 0775 \
  /mnt/riker/alexandria/process/audiobook \
  /mnt/riker/alexandria/process/audiobook-ready \
  /mnt/riker/alexandria/process/audiobook-done \
  /mnt/riker/alexandria/process/audiobook-review \
  ./state

Create the deployment environment file:

cp .env.example .env
id -u
id -g

Edit .env: set PUID to the id -u result and PGID to the id -g result. Both results must be nonzero; the entrypoint refuses root UID or GID. Confirm all four AUDIOBOOK_*_HOST values and AUDIOBOOK_STATE_HOST. All are required host-side deployment settings. Do not commit .env.

Build and verify on Enterprise-D

Run each command on Enterprise-D and require a zero exit status:

docker compose --env-file .env config --quiet
docker compose --env-file .env build --pull
docker compose --env-file .env run --rm audiobook-tui doctor
docker compose --env-file .env run --rm \
  --entrypoint /app/scripts/container-smoke.sh audiobook-tui

The last command must finish with:

container smoke test passed

The smoke test uses only a temporary directory inside the container. It prints the installed m4b-tool, ffmpeg, and ffprobe versions and checks the pinned m4b-tool 0.5.2 merge help before conversion. It refuses an upgrade missing any option the planner can emit: --jobs, --output-file, --name, --artist, --writer, --series, --series-part, --year, --description, or --cover. It creates two synthetic one-second MP3 tracks and a generated cover, exercises every optional metadata field, runs the real conversion services, and checks the final duration and two chapter titles. A second deterministic tone/silence/tone fixture runs through the real bounded silence detector and must report its known midpoint. Temporary media is removed on every normal, failed, or interrupted exit. The smoke test does not touch host audiobook paths. It also requires active and inactive records for all four progress phases and exact terminal-callback/log mirroring.

Launch over SSH

From an interactive Enterprise-D shell:

cd /opt/audiobook-tui
docker compose --env-file .env run --rm audiobook-tui

For a one-command remote launch, allocate a terminal explicitly:

ssh -t Enterprise-D \
  'cd /opt/audiobook-tui && docker compose --env-file .env run --rm audiobook-tui'

Diagnostics can be repeated at any time:

docker compose --env-file .env run --rm audiobook-tui doctor

Queue workflow

1. Put one loose MP3, or one directory containing MP3 tracks, directly beneath the pending host directory.
2. Launch the TUI and select the source.
3. Choose Convert or Review. Review requires typing REVIEW and moves the source without converting it.
4. For conversion, inspect the naturally sorted tracks, choose local/manual or optional Audnexus metadata, select a chapter mode, and inspect every path, chapter, and escaped merge/chapter-application argument vector in the final preview. These command renderings are display text only and are never passed to a shell.
5. Type CONVERT exactly. On success, the validated .m4b appears in ready and the original source appears in done.

The directories are the queue database: pending means waiting, ready contains validated output, done contains completed sources, and review contains sources intentionally set aside.

Chapter modes

• Embedded: each sorted MP3 boundary becomes a chapter, using that track's embedded title. Every track must have a title.
• Filenames: each sorted MP3 boundary becomes a chapter; a conservative leading track-number prefix is removed from the filename.
• Manual: a UTF-8 chapters.txt provides explicit timestamps and titles. For a directory book it must be the direct chapters.txt child; for a loose MP3 it must be the chapters.txt sibling. The application derives this path from the selected book and never accepts arbitrary chapter-file path text.
• Audnexus exact: publisher titles and offsets are selectable only when local and remote runtimes differ by no more than the smaller of 1% or 120 seconds, and every imported offset fits the local audio.
• Audnexus aligned: when runtimes differ by no more than 5%, publisher boundaries are projected onto the local timeline and snapped to nearby detected silence. At least 90% of nonzero boundaries must match. Preview shows alignment confidence plus predicted, snapped, and delta offsets.
• Silence detection: estimated boundaries come only from qualifying local silences with minimum chapter spacing. Titles are generic (Chapter 1, Chapter 2, and so on), never publisher titles.

The exact manual format is one nonblank chapter per line:

00:00:00 Opening Credits
00:01:42.500 Chapter 1
01:14:09 Chapter 2

Timestamps are HH:MM:SS or HH:MM:SS.mmm. The first must be 00:00:00; later values must be strictly increasing and precede total audio duration. A title is required after one or more spaces. Invalid UTF-8, duplicate timestamps, malformed lines, and out-of-range timestamps block conversion.

Metadata and covers

Local inference uses a field only when its embedded book tag is present and consistent across every selected track. Consistent embedded title, author, narrator, series, series position, year, and description values take precedence over folder/filename inference. Manual edits still take precedence over the selected local or Audnexus candidate.

Without an explicit manual cover choice, local cover selection checks the selected book location in this order: cover.jpg, cover.jpeg, then cover.png. Cover symlinks are not followed. A conventional local cover takes precedence over an accepted Audnexus cover.

Audnexus

Audnexus is optional enrichment served by Discovery on private LAN HTTP:

[audnexus]
enabled = true
base_url = "http://192.168.86.168:3333"
region = "us"
connect_timeout = 3.0
read_timeout = 8.0

Book chapter requests use /books/{ASIN}/chapters?region=us. Metadata choice and chapter choice are independent: operator may keep local metadata while using chapters returned for entered ASIN. Network failure, malformed responses, rate limits, wrong returned ASINs, and missing results leave Audnexus modes unavailable while local chapter modes remain usable. No API key is accepted by this client.

Private HTTP is deliberately narrow. URL host must be literal RFC1918 IPv4 or literal loopback IPv4/IPv6. HTTP hostnames, public addresses, credentials, non-root paths, queries, and fragments are rejected; redirects are not followed. HTTPS remains valid for a normal syntactic host. This rule validates operator configuration, not Discovery's availability.

API responses and cover downloads are capped at 10 MiB. Returned ASINs and external text are validated. Cover handling remains HTTPS-only even when API base URL uses private HTTP. Remote cover is downloaded only after its metadata candidate is accepted, into persistent work staging; validation or download failure continues without that cover. To disable enrichment:

[audnexus]
enabled = false

The rest of the workflow remains available without network access.

Chapter-analysis configuration

[chapters]
exact_runtime_percent = 1.0
exact_runtime_max_seconds = 120
alignment_runtime_percent = 5.0
alignment_required_percent = 90.0

[chapters.silence]
threshold_db = -40
minimum_silence_seconds = 1.5
minimum_chapter_minutes = 5
snap_window_seconds = 45
analysis_timeout_seconds = 1800
max_output_bytes = 10485760

Analysis reads sorted tracks once and does not modify source media. Each chapter candidate includes an availability reason. Selection opens detailed preview and does not approve conversion; operator must still type CONVERT exactly after reviewing paths, commands, timestamps, titles, and alignment diagnostics.

Known limitations

• Runtime agreement cannot prove audio is same edition. Confirm title, runtime, chapter count, and several boundaries before conversion.
• Silence detection estimates structural pauses. Long dramatic pauses may become chapters; music or continuous narration may hide real boundaries.
• Audnexus-aligned titles are publisher data only after 90% configured match; silence-only titles remain generic.
• Conversion exposes observable work, but no percentage or ETA can be derived reliably from m4b-tool. No probe runs against growing output.
• Remote enrichment and silence failures fall back; neither failure moves source data or creates ready output.

Enterprise-D LAN acceptance

Before trusting remote chapters, replace placeholder with real ASIN from your Audible library and test Discovery directly from Enterprise-D:

ASIN='REPLACE_WITH_REAL_ASIN'
curl --fail --show-error --silent \
  "http://192.168.86.168:3333/books/${ASIN}/chapters?region=us"

Then launch TUI, enter same real ASIN, verify exact or aligned preview against local audio, inspect silence-only preview, and cancel once to prove analysis does not approve or move source. For disposable backed-up book, approve one conversion and verify output chapter count, titles, playback, and pending to ready-plus-done transition.

Conversion progress

After exact approval, terminal tracker reports observable phase, elapsed time, output size, active/inactive state, exit status, and sanitized error. It updates every five seconds by default while work remains active. Phases cover converting audio, applying chapters, validating output, and moving completed files. Output size appears only when a regular conversion artifact can be observed safely.

Tracker does not report percentage or ETA because m4b-tool does not expose reliable values for either. It reads growing output size from filesystem metadata and never runs ffprobe against a growing output. Validation invokes ffprobe only after chapter application finishes.

For every terminal event, that same progress record is appended exactly once to current UTC-timestamped log beneath ./state/logs. Pressing Ctrl-C retains existing cancellation behavior: conversion child process group is stopped, final inactive event records interruption, source remains pending, and recoverable partial/work artifacts remain available for explicit cleanup. Moving starts only after successful validation; moving failure receives its own inactive error record and keeps existing workflow recovery guarantees.

Collisions and recovery

If the selected output name already exists in ready, choose another name or cancel. The application will not replace it. The same rule applies when a source name already exists in done or review.

Pressing Ctrl-C during conversion terminates the child conversion process. The source stays in pending; a <name>.partial.m4b and/or <name>.work directory may remain. An interrupted cross-device source move may also leave a direct .moving-* artifact beneath done or review. On filesystems that cannot publish that temporary with an atomic no-replace rename, directories are claimed with an exclusive mkdir, while loose files are claimed exclusively and copied through the held file descriptor. Directory entries are likewise created exclusively relative to a held root descriptor, so a replaced claim or an injected child is not followed or overwritten. A claimed directory can therefore be visible while its verified copy is in progress, but the source remains in place until the destination and final source manifests match.

Caught copy, verification, and publication failures remove only temporary or final roots and child entries whose identities were recorded by the application's exclusive claims. Untracked or replaced children are never removed automatically. If an entry was replaced, changed identity, or could not be safely cleaned, the application preserves the source and conflicting destination and reports that manual reconciliation is required. A failure while removing a fully verified source keeps the final destination; the source may remain whole or partially removed and must be reconciled from the final copy and an independent backup before retrying.

On the next launch, the TUI lists supported exact stale paths. Preserve them for investigation or select the displayed artifacts and confirm cleanup. Only direct .moving-* entries are stale state-move artifacts: a canonical entry in done or review is never selected or removed automatically. Symlinks, nested selections, and unsupported artifact shapes are refused. The application does not resume ambiguous work or delete outside the configured ready, work, done, and review recovery boundaries.

Persistent conversion logs use UTC-timestamped names beneath the directory mounted at /app/state, normally ./state/logs on Enterprise-D. Back up logs if they are part of your audit trail. Conversion work and generated chapter metadata live beneath persistent /app/state/work, normally ./state/work on Enterprise-D. The entrypoint creates that work directory only after confirming the mounted state directory is writable by PUID:PGID. Interrupted work therefore survives container removal and remains available to the TUI's explicit stale-recovery flow. It is operational recovery data, not a backup.

Permissions troubleshooting

If the entrypoint reports an identity mismatch, compare .env with the Enterprise-D operator account:

id -u
id -g

Both IDs must be nonzero. Do not work around the root-ID refusal by adding privileges; use the ordinary account that owns the host directories.

If doctor reports an unreadable or unwritable root, inspect every host path and the state directory with ls -ld. Correct ownership or group permissions on the host; do not add privileged mode, capabilities, or a Docker socket. Files created in ready, done, review, and state should be owned by the configured numeric identity. ACLs, NFS root-squash, or mismatched NAS IDs may also prevent writes even when Unix mode bits look plausible.

Pinned image upgrades

The normal build path is pinned to sandreas/m4b-tool:0.5.2; it never defaults to latest. To upgrade intentionally:

1. Read the upstream release notes and confirm the candidate supports the Enterprise-D architecture.
2. Change M4B_TOOL_IMAGE in .env (and .env.example when adopting the pin for everyone) to a stable version tag or immutable digest.
3. Run Compose configuration, build --pull, doctor, and the synthetic smoke command from the Enterprise-D verification section. The smoke gate confirms every planner-emitted merge option before exercising complete metadata and a generated cover.
4. Run the manual acceptance checklist with disposable or backed-up test data.
5. Keep the old pin recorded so rollback is a one-line .env change and rebuild.

Never promote a new pin solely because the image builds.

Manual Enterprise-D acceptance checklist

• docker compose --env-file .env config --quiet exits zero.
• The image pulls/builds for Enterprise-D's CPU architecture.
• doctor reports all mounts, the TTY, and three media tools healthy.
• The container smoke test prints container smoke test passed.
• Launch through ssh -t is interactive and remains usable at 80 columns.
• Pending content is visible and all destination paths are writable.
• A disposable conversion produces a playable M4B with expected chapters.
• A nontrivial conversion visibly reports all four progress phases without percentage or ETA, and its persistent log contains same records once.
• Output, moved source, and log ownership match PUID:PGID.
• An existing output collision is refused without changing either file.
• Existing done and review collisions are refused without changing the pending source or the existing destination.
• Ctrl-C leaves the source pending and stale cleanup lists only expected paths.
• Ctrl-C adds final inactive interruption record to persistent conversion log.
• Removing the interrupted container leaves stale work under ./state/work.
• Successful conversion yields the expected pending → ready + done transition.
• REVIEW yields the expected pending → review transition.
• If a move reports manual reconciliation, both surviving paths are inspected and backed up before either is removed or the operation is retried.
• Independent source and ready-library backups have been confirmed.

Development verification status

Local Python tests are the only verified checks from the authoring desktop. Docker config/build/doctor/smoke are NOT RUN locally because Docker was absent. Run the Enterprise-D commands above and retain their output before declaring deployment acceptance.

License

MIT. See LICENSE.