docs: ship private Audnexus chapter analysis
This commit is contained in:
@@ -2,6 +2,31 @@
|
||||
|
||||
All notable changes to this project are documented in this file.
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
- Private-LAN Audnexus metadata and chapter retrieval from Discovery at
|
||||
`192.168.86.168:3333`, with independent metadata/chapter approval.
|
||||
- Exact-runtime publisher chapters, conservative silence-guided alignment, and
|
||||
deterministic silence-only chapter candidates with detailed preview reasons.
|
||||
- Bounded FFmpeg silence analysis across sorted multi-track input and a
|
||||
deterministic container smoke fixture covering real detector integration.
|
||||
|
||||
### Security
|
||||
|
||||
- Plain HTTP Audnexus URLs accept only literal RFC1918 IPv4 or loopback
|
||||
addresses. Public addresses, hostnames, credentials, paths, queries,
|
||||
fragments, and redirects remain rejected.
|
||||
- Remote cover downloads remain HTTPS-only and bounded despite private HTTP API
|
||||
access.
|
||||
|
||||
### Operations
|
||||
|
||||
- Documented Discovery configuration, chapter thresholds, known limitations,
|
||||
real-ASIN Enterprise-D acceptance, explicit approval, and absence of invented
|
||||
percentage or ETA.
|
||||
|
||||
## 0.1.0 - 2026-08-26
|
||||
|
||||
### Added
|
||||
|
||||
@@ -47,7 +47,8 @@ Enterprise-D must provide:
|
||||
- SSH access with terminal allocation (`ssh -t`)
|
||||
- a user that can read and write all four audiobook directories
|
||||
- enough free space for the source, temporary merge, partial M4B, and final M4B
|
||||
- outbound HTTPS only if optional Audnexus lookup is enabled
|
||||
- LAN access to Discovery at `http://192.168.86.168:3333` when optional
|
||||
Audnexus lookup is enabled
|
||||
- either `linux/amd64` or `linux/arm64` for the default
|
||||
`sandreas/m4b-tool:0.5.2` image
|
||||
|
||||
@@ -144,8 +145,11 @@ any option the planner can emit: `--jobs`, `--output-file`, `--name`,
|
||||
`--artist`, `--writer`, `--series`, `--series-part`, `--year`,
|
||||
`--description`, or `--cover`. It creates two synthetic one-second MP3 tracks
|
||||
and a generated cover, exercises every optional metadata field, runs the real
|
||||
conversion services, and checks the final duration and two chapter titles. It
|
||||
does not touch host audiobook paths.
|
||||
conversion services, and checks the final duration and two chapter titles. A
|
||||
second deterministic tone/silence/tone fixture runs through the real bounded
|
||||
silence detector and must report its known midpoint. Temporary media is removed
|
||||
on every normal, failed, or interrupted exit. The smoke test does not touch host
|
||||
audiobook paths.
|
||||
|
||||
## Launch over SSH
|
||||
|
||||
@@ -198,6 +202,16 @@ The directories are the queue database: `pending` means waiting,
|
||||
For a directory book it must be the direct `chapters.txt` child; for a loose
|
||||
MP3 it must be the `chapters.txt` sibling. The application derives this path
|
||||
from the selected book and never accepts arbitrary chapter-file path text.
|
||||
- **Audnexus exact:** publisher titles and offsets are selectable only when
|
||||
local and remote runtimes differ by no more than the smaller of 1% or 120
|
||||
seconds, and every imported offset fits the local audio.
|
||||
- **Audnexus aligned:** when runtimes differ by no more than 5%, publisher
|
||||
boundaries are projected onto the local timeline and snapped to nearby
|
||||
detected silence. At least 90% of nonzero boundaries must match. Preview
|
||||
shows alignment confidence plus predicted, snapped, and delta offsets.
|
||||
- **Silence detection:** estimated boundaries come only from qualifying local
|
||||
silences with minimum chapter spacing. Titles are generic (`Chapter 1`,
|
||||
`Chapter 2`, and so on), never publisher titles.
|
||||
|
||||
The exact manual format is one nonblank chapter per line:
|
||||
|
||||
@@ -227,14 +241,35 @@ precedence over an accepted Audnexus cover.
|
||||
|
||||
## Audnexus
|
||||
|
||||
Audnexus is optional, unauthenticated enrichment. Network failure, malformed
|
||||
responses, rate limits, and missing results fall back to local/manual metadata.
|
||||
No API key is accepted. API responses and cover downloads are capped at 10 MiB,
|
||||
returned ASINs and external text are validated, and only HTTPS cover URLs are
|
||||
accepted. A remote cover is downloaded only after its Audnexus candidate is
|
||||
accepted, into persistent work staging; validation or download failure simply
|
||||
continues without that cover. If the public service becomes paid, credentialed,
|
||||
or otherwise unsuitable, disable it in `config.toml`:
|
||||
Audnexus is optional enrichment served by Discovery on private LAN HTTP:
|
||||
|
||||
```toml
|
||||
[audnexus]
|
||||
enabled = true
|
||||
base_url = "http://192.168.86.168:3333"
|
||||
region = "us"
|
||||
connect_timeout = 3.0
|
||||
read_timeout = 8.0
|
||||
```
|
||||
|
||||
Book chapter requests use
|
||||
`/books/{ASIN}/chapters?region=us`. Metadata choice and chapter choice are
|
||||
independent: operator may keep local metadata while using chapters returned for
|
||||
entered ASIN. Network failure, malformed responses, rate limits, wrong returned
|
||||
ASINs, and missing results leave Audnexus modes unavailable while local chapter
|
||||
modes remain usable. No API key is accepted by this client.
|
||||
|
||||
Private HTTP is deliberately narrow. URL host must be literal RFC1918 IPv4 or
|
||||
literal loopback IPv4/IPv6. HTTP hostnames, public addresses, credentials,
|
||||
non-root paths, queries, and fragments are rejected; redirects are not followed.
|
||||
HTTPS remains valid for a normal syntactic host. This rule validates operator
|
||||
configuration, not Discovery's availability.
|
||||
|
||||
API responses and cover downloads are capped at 10 MiB. Returned ASINs and
|
||||
external text are validated. Cover handling remains HTTPS-only even when API
|
||||
base URL uses private HTTP. Remote cover is downloaded only after its metadata
|
||||
candidate is accepted, into persistent work staging; validation or download
|
||||
failure continues without that cover. To disable enrichment:
|
||||
|
||||
```toml
|
||||
[audnexus]
|
||||
@@ -243,6 +278,59 @@ enabled = false
|
||||
|
||||
The rest of the workflow remains available without network access.
|
||||
|
||||
### Chapter-analysis configuration
|
||||
|
||||
```toml
|
||||
[chapters]
|
||||
exact_runtime_percent = 1.0
|
||||
exact_runtime_max_seconds = 120
|
||||
alignment_runtime_percent = 5.0
|
||||
alignment_required_percent = 90.0
|
||||
|
||||
[chapters.silence]
|
||||
threshold_db = -40
|
||||
minimum_silence_seconds = 1.5
|
||||
minimum_chapter_minutes = 5
|
||||
snap_window_seconds = 45
|
||||
analysis_timeout_seconds = 1800
|
||||
max_output_bytes = 10485760
|
||||
```
|
||||
|
||||
Analysis reads sorted tracks once and does not modify source media. Each chapter
|
||||
candidate includes an availability reason. Selection opens detailed preview and
|
||||
does not approve conversion; operator must still type `CONVERT` exactly after
|
||||
reviewing paths, commands, timestamps, titles, and alignment diagnostics.
|
||||
|
||||
### Known limitations
|
||||
|
||||
- Runtime agreement cannot prove audio is same edition. Confirm title, runtime,
|
||||
chapter count, and several boundaries before conversion.
|
||||
- Silence detection estimates structural pauses. Long dramatic pauses may
|
||||
become chapters; music or continuous narration may hide real boundaries.
|
||||
- Audnexus-aligned titles are publisher data only after 90% configured match;
|
||||
silence-only titles remain generic.
|
||||
- Conversion exposes observable work, but no percentage or ETA can be derived
|
||||
reliably from `m4b-tool`. No probe runs against growing output.
|
||||
- Remote enrichment and silence failures fall back; neither failure moves source
|
||||
data or creates ready output.
|
||||
|
||||
### Enterprise-D LAN acceptance
|
||||
|
||||
Before trusting remote chapters, replace placeholder with real ASIN from your
|
||||
Audible library and test Discovery directly from Enterprise-D:
|
||||
|
||||
```sh
|
||||
ASIN='REPLACE_WITH_REAL_ASIN'
|
||||
curl --fail --show-error --silent \
|
||||
"http://192.168.86.168:3333/books/${ASIN}/chapters?region=us"
|
||||
```
|
||||
|
||||
Then launch TUI, enter same real ASIN, verify exact or aligned preview against
|
||||
local audio, inspect silence-only preview, and cancel once to prove analysis
|
||||
does not approve or move source. For disposable backed-up book, approve one
|
||||
conversion and verify output chapter count, titles, playback, and pending to
|
||||
ready-plus-done transition.
|
||||
|
||||
## Collisions and recovery
|
||||
|
||||
If the selected output name already exists in `ready`, choose another name or
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
# Conversion Progress Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make long m4b-tool conversions visibly active through honest phase, elapsed-time, output-size, exit, and error records in both terminal and persistent log.
|
||||
|
||||
**Architecture:** Add one focused progress module that owns formatting and cadence. Converter emits immutable progress events through one callback and serializes all log writes on the conversion thread; workflow emits finalization phase without probing growing media or inventing percent/ETA.
|
||||
|
||||
**Tech Stack:** Python 3.12 standard library (`threading`, `time`, `pathlib`), pytest 8.4.2, existing `m4b-tool`/FFmpeg converter.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-08-29-audnexus-chapters-progress-design.md`
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Default update interval is five seconds and must be finite and positive.
|
||||
- Never run ffprobe against a growing output.
|
||||
- Never display percentage or ETA that m4b-tool does not provide.
|
||||
- Report only observable phase, elapsed duration, output size, active state, exit status, and error.
|
||||
- Every progress line goes to terminal callback and persistent log.
|
||||
- Do not block child stdout consumption, race log writes, leak threads, or weaken process-group cancellation.
|
||||
- Preserve all source, partial-output, work-directory, validation, and recovery invariants.
|
||||
- Use strict TDD and do not push, merge, publish, or mutate servers.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Progress Events and Deterministic Formatting
|
||||
|
||||
**Files:**
|
||||
- Create: `app/progress.py`
|
||||
- Create: `tests/test_progress.py`
|
||||
- Modify: `app/models.py`
|
||||
- Modify: `app/config.py`
|
||||
- Modify: `tests/test_config.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces `ProgressPhase(StrEnum)` values `CONVERTING`, `APPLYING_CHAPTERS`, `VALIDATING`, and `MOVING`.
|
||||
- Produces immutable `ProgressEvent(phase: ProgressPhase, elapsed_seconds: float, active: bool = True, output_bytes: int | None = None, exit_status: int | None = None, error: str = "")`.
|
||||
- Produces `format_progress(event: ProgressEvent) -> str`.
|
||||
- Produces `Settings.progress_update_seconds: float = 5.0`.
|
||||
|
||||
- [ ] **Step 1: Add failing formatting tests**
|
||||
|
||||
Assert exact lines for zero, hour-plus elapsed time, IEC output sizes, phases without output, inactive success with `exit=0`, and escaped/single-line error text. Reject non-finite/negative elapsed and output values at event construction or formatting boundary.
|
||||
|
||||
- [ ] **Step 2: Verify missing module failure**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_progress.py
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement immutable event and pure formatter**
|
||||
|
||||
Format elapsed as `HH:MM:SS`, sizes as `B`, `KiB`, `MiB`, or `GiB`, and normalize control/newline characters in errors so log framing cannot be forged.
|
||||
|
||||
- [ ] **Step 4: Add/implement configuration tests**
|
||||
|
||||
Parse `[progress].update_seconds`, default `5`, and reject boolean, zero, negative, NaN, and infinity with `progress.update_seconds must be positive`.
|
||||
|
||||
- [ ] **Step 5: Run focused/full tests and commit**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_progress.py tests/test_config.py
|
||||
pytest -q
|
||||
git add app/progress.py app/models.py app/config.py tests/test_progress.py tests/test_config.py
|
||||
git commit -m "feat: model honest conversion progress"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Converter Heartbeats and Phase Transitions
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/converter.py`
|
||||
- Modify: `tests/test_converter.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes Task 1 types.
|
||||
- Extends `Converter.__init__` with `on_progress: Callable[[ProgressEvent], None] = lambda event: None`, `clock: Callable[[], float] = time.monotonic`, and injected wait/tick primitive suitable for deterministic tests.
|
||||
- Keeps `Converter.run(plan, log_path) -> ValidationResult` unchanged.
|
||||
|
||||
- [ ] **Step 1: Add failing phase-order test**
|
||||
|
||||
Using fake clock/process, assert converting starts, periodic converting heartbeat includes `plan.merge_output.stat().st_size` when regular/non-symlink, applying begins after successful merge, validating begins after chapter application, and terminal events appear in order.
|
||||
|
||||
- [ ] **Step 2: Verify expected failure**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_converter.py -k progress
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement nonblocking heartbeat design**
|
||||
|
||||
Use one daemon heartbeat thread only while a child runs. Thread computes events and enqueues them; conversion thread drains queue and alone calls terminal callback/writes log, preventing concurrent `TextIO` writes. Stop/join heartbeat in `finally` with a bounded join. Filesystem observation uses `lstat`; symlink/missing/error yields no size rather than failure.
|
||||
|
||||
- [ ] **Step 4: Add failing log-mirroring/cadence tests**
|
||||
|
||||
Assert each formatted progress event appears once in callback and once in log, interval honors injected clock, rapid child completion has start/final events without heartbeat, and no event occurs after run returns.
|
||||
|
||||
- [ ] **Step 5: Add failing exit/error/cancellation tests**
|
||||
|
||||
Assert nonzero child, output-handler exception, KeyboardInterrupt, SIGTERM success, and SIGKILL fallback produce final inactive event with status/error while retaining existing `ValidationResult` messages. Assert heartbeat thread joins in every path.
|
||||
|
||||
- [ ] **Step 6: Implement failure-finalization events minimally**
|
||||
|
||||
Do not change existing subprocess vectors, validation semantics, or cleanup paths. Sanitize error before formatting; retain full trusted internal error in `ValidationResult`.
|
||||
|
||||
- [ ] **Step 7: Run focused/full tests and commit**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_converter.py tests/test_progress.py
|
||||
pytest -q
|
||||
git add app/converter.py tests/test_converter.py
|
||||
git commit -m "feat: track long conversion phases"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: TUI Wiring, Finalization Phase, Docs, and Runtime Gate
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/tui.py`
|
||||
- Modify: `app/main.py`
|
||||
- Modify: `tests/test_tui.py`
|
||||
- Modify: `tests/test_main.py`
|
||||
- Modify: `README.md`
|
||||
- Modify: `CHANGELOG.md`
|
||||
- Modify: `scripts/container-smoke.sh`
|
||||
- Modify: `tests/test_packaging.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes `ProgressEvent`/`format_progress` and updated Converter.
|
||||
- Extends `UIAdapter` with `show_progress(event: ProgressEvent) -> None` or routes formatted lines through existing `stream` while tests assert one stable public behavior.
|
||||
- Emits `MOVING` immediately before `finalize_success` and a final inactive completion/error event afterward.
|
||||
|
||||
- [ ] **Step 1: Add failing application wiring test**
|
||||
|
||||
Assert normal composition sends converter progress to TerminalUI, conversion log, and that moving phase occurs only after validation success and before finalizer. Assert finalization failure records moving-phase error and source behavior remains unchanged.
|
||||
|
||||
- [ ] **Step 2: Implement UI/workflow wiring**
|
||||
|
||||
Use plain non-markup terminal output. Do not duplicate converter-owned log lines at application layer. If moving events need same log, inject one scoped progress sink owning current log path and close it before result display.
|
||||
|
||||
- [ ] **Step 3: Add failing smoke/docs tests**
|
||||
|
||||
Assert README explains observable tracker fields, five-second default, lack of percent/ETA, no growing-file ffprobe, cancellation, and log location. Smoke harness captures at least converting/applying/validating/moving phase records with a short configurable interval without making runtime flaky.
|
||||
|
||||
- [ ] **Step 4: Implement smoke/docs/package updates**
|
||||
|
||||
Update changelog and audited release inputs consistently with chapter-analysis plan. Keep synthetic media short.
|
||||
|
||||
- [ ] **Step 5: Run all gates**
|
||||
|
||||
```sh
|
||||
pytest -q
|
||||
python -m compileall -q app tests
|
||||
git diff --check
|
||||
docker compose --env-file .env config --quiet
|
||||
```
|
||||
|
||||
When Docker exists, rebuild and run container smoke. Otherwise record Enterprise-D runtime gate as pending.
|
||||
|
||||
- [ ] **Step 6: Commit and independent review**
|
||||
|
||||
```sh
|
||||
git add app/tui.py app/main.py tests/test_tui.py tests/test_main.py README.md CHANGELOG.md scripts/container-smoke.sh tests/test_packaging.py config.toml
|
||||
git commit -m "feat: show persistent conversion progress"
|
||||
```
|
||||
|
||||
Dispatch whole-feature review against spec, fix findings with red tests, rerun every gate, and do not push or merge.
|
||||
@@ -0,0 +1,368 @@
|
||||
# Private Audnexus and Chapter Analysis Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Let audiobook-tui reach Discovery over narrowly permitted private HTTP and offer exact Audnexus, Audnexus-aligned, or silence-only chapters without weakening existing manual approval.
|
||||
|
||||
**Architecture:** Keep transport normalization in `app/audnexus.py`, audio-only silence extraction in a new `app/silence.py`, and deterministic candidate policy in a new `app/chapter_analysis.py`. TUI receives already validated candidates and passes the selected existing `Chapter` tuple into the unchanged planner/converter boundary.
|
||||
|
||||
**Tech Stack:** Python 3.12, `httpx==0.28.1`, FFmpeg `silencedetect`, pytest 8.4.2, Docker Compose.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-08-29-audnexus-chapters-progress-design.md`
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- HTTPS remains required except HTTP to literal IPv4 RFC1918 or IPv4/IPv6 loopback addresses.
|
||||
- Reject HTTP hostnames, public/link-local/multicast/unspecified addresses, credentials, query strings, fragments, and non-root paths.
|
||||
- Redirects remain disabled and covers remain HTTPS-only.
|
||||
- Audnexus responses remain bounded to 10 MiB and all network failure remains optional fallback.
|
||||
- Exact runtime tolerance is `min(local * 1%, 120 seconds)`.
|
||||
- Guided alignment eligibility is at most 5% runtime mismatch and selectability requires at least 90% nonzero-boundary silence matches.
|
||||
- Silence defaults are `-40 dB`, `1.5 seconds`, five-minute chapter spacing, and a 45-second snap window.
|
||||
- Analysis never mutates source media or moves workflow state.
|
||||
- Final conversion approval remains mandatory.
|
||||
- Use strict TDD: add one failing behavior test, observe expected failure, implement minimally, then run focused and full tests.
|
||||
- Do not push, merge, publish, or mutate Enterprise-D/Discovery during implementation.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Configuration and Private-HTTP Boundary
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/models.py`
|
||||
- Modify: `app/config.py`
|
||||
- Modify: `config.toml`
|
||||
- Modify: `tests/test_config.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `valid_audnexus_base_url(value: str) -> bool` in `app/models.py`.
|
||||
- Produces Settings fields: `audnexus_region: str`, `chapter_exact_runtime_percent: float`, `chapter_exact_runtime_max_seconds: float`, `chapter_alignment_runtime_percent: float`, `chapter_alignment_required_percent: float`, `silence_threshold_db: float`, `silence_minimum_seconds: float`, `silence_minimum_chapter_minutes: float`, `silence_snap_window_seconds: float`, `silence_analysis_timeout_seconds: float`, and `silence_max_output_bytes: int`.
|
||||
- Consumed by Tasks 2-5.
|
||||
|
||||
- [ ] **Step 1: Add failing URL and configuration tests**
|
||||
|
||||
Add parameterized cases proving HTTPS works, `http://192.168.86.168:3333`, `http://10.0.0.1`, `http://172.16.0.1`, `http://127.0.0.1`, and `http://[::1]` work, while `http://example.com`, `http://8.8.8.8`, link-local, multicast, unspecified, user-info, path, query, and fragment fail. Add a load-settings test for every exact default and for `region="us"`. Add aggregated validation tests for NaN/infinite/nonpositive timeouts and thresholds, percentages outside `(0, 100]`, and nonpositive output ceiling.
|
||||
|
||||
- [ ] **Step 2: Verify tests fail for missing fields/private HTTP support**
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_config.py
|
||||
```
|
||||
|
||||
Expected: failures state private HTTP is rejected and new Settings attributes/config tables are absent.
|
||||
|
||||
- [ ] **Step 3: Implement exact URL predicate and settings**
|
||||
|
||||
Use `urllib.parse.urlparse` plus `ipaddress.ip_address`; do not resolve DNS. For HTTP, explicitly accept `address.is_loopback` or membership in `10/8`, `172.16/12`, or `192.168/16`, not broad `is_private`. Require empty username/password/query/fragment and path in `{"", "/"}`. Parse `[chapters]` and `[chapters.silence]` as separate tables. The conversion-progress plan owns `[progress]`.
|
||||
|
||||
- [ ] **Step 4: Update deployed configuration**
|
||||
|
||||
Set:
|
||||
|
||||
```toml
|
||||
[audnexus]
|
||||
enabled = true
|
||||
base_url = "http://192.168.86.168:3333"
|
||||
region = "us"
|
||||
connect_timeout = 3.0
|
||||
read_timeout = 8.0
|
||||
```
|
||||
|
||||
Add all spec defaults without changing path or conversion settings.
|
||||
|
||||
- [ ] **Step 5: Run focused and full tests**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_config.py tests/test_main.py
|
||||
pytest -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```sh
|
||||
git add app/models.py app/config.py config.toml tests/test_config.py
|
||||
git commit -m "feat: allow private Audnexus configuration"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Audnexus Chapter Transport and Normalization
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/audnexus.py`
|
||||
- Modify: `tests/test_audnexus.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Settings.audnexus_region` from Task 1 at composition call sites later.
|
||||
- Produces immutable `AudnexusChapter(start_ms: int, title: str)`.
|
||||
- Produces immutable `AudnexusChapters(asin: str, runtime_ms: int, is_accurate: bool, chapters: tuple[AudnexusChapter, ...])`.
|
||||
- Produces `AudnexusClient.chapters(asin: str, *, region: str) -> AudnexusChapters | None`.
|
||||
|
||||
- [ ] **Step 1: Add failing documented-response test**
|
||||
|
||||
Use `httpx.MockTransport` to assert `GET /books/B08G9PRS1K/chapters?region=us`, no authorization header, and normalization of `runtimeLengthMs`, `isAccurate`, `startOffsetMs`, and titles.
|
||||
|
||||
- [ ] **Step 2: Verify missing method failure**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_audnexus.py -k chapters
|
||||
```
|
||||
|
||||
Expected: fail because `AudnexusClient.chapters` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement dataclasses and happy-path normalization**
|
||||
|
||||
Reuse `_bounded_body`, `_ASIN`, `_external_string`, and client timeouts. Require matching ASIN, positive integer runtime, boolean accuracy, nonempty list, integer nonnegative offsets, zero first offset, strict increase, offsets below runtime, and validated titles. Return `None` for invalid input or network/HTTP/JSON failure.
|
||||
|
||||
- [ ] **Step 4: Add adversarial failing tests one behavior at a time**
|
||||
|
||||
Cover mismatched/invalid ASIN, oversized body, missing/invalid runtime, empty/non-list chapters, bool-as-int offsets, negative/non-increasing/out-of-range offsets, empty/oversized/control-character titles, timeout, 404, and region propagation. Assert every failure returns `None` and no partial document.
|
||||
|
||||
- [ ] **Step 5: Implement minimal validation for each red case**
|
||||
|
||||
Keep book metadata normalization unchanged. Do not add authentication—the self-hosted server owns Audible credentials.
|
||||
|
||||
- [ ] **Step 6: Run focused and full tests**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_audnexus.py
|
||||
pytest -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```sh
|
||||
git add app/audnexus.py tests/test_audnexus.py
|
||||
git commit -m "feat: retrieve bounded Audnexus chapters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Bounded FFmpeg Silence Extraction
|
||||
|
||||
**Files:**
|
||||
- Create: `app/silence.py`
|
||||
- Create: `tests/test_silence.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Book`, silence Settings values from Task 1.
|
||||
- Produces immutable `Silence(start_ms: int, end_ms: int)` with `midpoint_ms` and `duration_ms` properties.
|
||||
- Produces `SilenceError`.
|
||||
- Produces `detect_silences(book: Book, settings: Settings, *, popen_factory: Callable[..., Any] = subprocess.Popen) -> tuple[Silence, ...]`.
|
||||
|
||||
- [ ] **Step 1: Add failing single-track command/parser test**
|
||||
|
||||
Fake `Popen` and assert explicit vector contains `ffmpeg`, `-nostdin`, input path, `silencedetect=noise=-40dB:d=1.5`, and null output. Feed paired `silence_start`/`silence_end` lines and assert millisecond rounding and midpoint.
|
||||
|
||||
- [ ] **Step 2: Verify missing module failure**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_silence.py
|
||||
```
|
||||
|
||||
Expected: import failure for `app.silence`.
|
||||
|
||||
- [ ] **Step 3: Implement one-track bounded parser**
|
||||
|
||||
Read merged stdout/stderr incrementally, count encoded bytes before retaining/parsing, enforce ceiling, use monotonic deadline, launch with `start_new_session=True`, and reuse converter-equivalent SIGTERM/10-second/SIGKILL cleanup semantics. Accept only finite paired events within probed track duration.
|
||||
|
||||
- [ ] **Step 4: Add failing multi-track timeline test**
|
||||
|
||||
Use two tracks and prove second-track silence offsets include cumulative first-track duration. Prove returned values sort and exact duplicate midpoint retains longest silence.
|
||||
|
||||
- [ ] **Step 5: Implement deterministic multi-track translation**
|
||||
|
||||
Analyze tracks separately in existing sorted order; add cumulative duration to every event; deduplicate only identical midpoints; exclude midpoint `<= 0` or `>= total duration`.
|
||||
|
||||
- [ ] **Step 6: Add and satisfy failure/cancellation tests**
|
||||
|
||||
Add one red-green cycle each for malformed numeric values, reversed/unmatched events, output overflow, timeout, nonzero exit, `KeyboardInterrupt`, SIGTERM success, and SIGKILL fallback. Assert no source writes and no lingering fake child.
|
||||
|
||||
- [ ] **Step 7: Run focused and full tests**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_silence.py
|
||||
pytest -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```sh
|
||||
git add app/silence.py tests/test_silence.py
|
||||
git commit -m "feat: detect bounded audiobook silences"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Exact, Aligned, and Silence-Only Candidate Policy
|
||||
|
||||
**Files:**
|
||||
- Create: `app/chapter_analysis.py`
|
||||
- Create: `tests/test_chapter_analysis.py`
|
||||
- Modify: `app/models.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `AudnexusChapters`, `Silence`, `Book`, and Settings fields from Tasks 1-3.
|
||||
- Produces immutable `Alignment(boundary_index: int, predicted_ms: int, selected_ms: int, snapped: bool, delta_ms: int)`.
|
||||
- Produces immutable `ChapterCandidate(mode: ChapterMode, available: bool, chapters: tuple[Chapter, ...] = (), reason: str = "", local_runtime_ms: int = 0, remote_runtime_ms: int | None = None, confidence_percent: float | None = None, alignments: tuple[Alignment, ...] = ())`.
|
||||
- Produces `analyze_remote_chapters(book: Book, remote: AudnexusChapters | None, silences: tuple[Silence, ...], settings: Settings) -> tuple[ChapterCandidate, ChapterCandidate]` returning exact then aligned.
|
||||
- Produces `silence_chapters(book: Book, silences: tuple[Silence, ...], settings: Settings) -> ChapterCandidate`.
|
||||
|
||||
- [ ] **Step 1: Add failing exact-match boundary tests**
|
||||
|
||||
Prove acceptance at `min(local*1%, 120s)`, rejection one millisecond beyond, preservation of titles/offsets, and rejection when a remote offset reaches local duration.
|
||||
|
||||
- [ ] **Step 2: Verify missing analysis failure, then implement exact candidate**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_chapter_analysis.py -k exact
|
||||
```
|
||||
|
||||
Expected before implementation: import/name failure. After minimal implementation: pass.
|
||||
|
||||
- [ ] **Step 3: Add failing guided-alignment tests**
|
||||
|
||||
Prove proportional projection, nearest midpoint inside 45 seconds, earlier tie, unused silence rule, unsnapped projected offsets, 5% eligibility edge, 90% confidence edge, zero boundary excluded from denominator, five-minute spacing rejection, strict order, and local-duration bound. Assert low confidence produces no selectable chapters or remote titles.
|
||||
|
||||
- [ ] **Step 4: Implement deterministic guided alignment**
|
||||
|
||||
Use integer millisecond arithmetic with one documented rounding rule (`round`). Match each nonzero projected boundary independently in order against unused silences; validate complete selected sequence after matching. Emit exact reasons containing mismatch/confidence values for TUI display.
|
||||
|
||||
- [ ] **Step 5: Add failing silence-only tests**
|
||||
|
||||
Prove chapter zero, earliest eligible midpoint selection, five-minute spacing, duplicate-midpoint longest preference, generic sequential titles, no-candidate unavailability, strict order, and end exclusion.
|
||||
|
||||
- [ ] **Step 6: Implement silence-only candidate**
|
||||
|
||||
Do not reuse Audnexus titles. Require at least one detected nonzero boundary so silence mode never advertises a guessed one-chapter result.
|
||||
|
||||
- [ ] **Step 7: Run focused and full tests**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_chapter_analysis.py
|
||||
pytest -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```sh
|
||||
git add app/models.py app/chapter_analysis.py tests/test_chapter_analysis.py
|
||||
git commit -m "feat: analyze exact aligned and silence chapters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: TUI Integration and Chapter Preview
|
||||
|
||||
**Files:**
|
||||
- Modify: `app/tui.py`
|
||||
- Modify: `app/main.py`
|
||||
- Modify: `tests/test_tui.py`
|
||||
- Modify: `tests/test_main.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes all Task 1-4 interfaces.
|
||||
- Extends `AudnexusService` with `chapters(asin: str, *, region: str) -> AudnexusChapters | None`.
|
||||
- Extends `UIAdapter` with `choose_chapter_candidate(book: Book, candidates: tuple[ChapterCandidate, ...]) -> ChapterCandidate | ChapterMode | None` and candidate diagnostics in preview.
|
||||
- Produces end-to-end selected `tuple[Chapter, ...]` for unchanged planner interface.
|
||||
|
||||
- [ ] **Step 1: Add failing metadata-independent remote chapter flow test**
|
||||
|
||||
Script UI to enter ASIN, retain local metadata, select exact Audnexus chapters, preview, approve, and assert planner receives remote chapters. Assert one metadata request and one chapter request.
|
||||
|
||||
- [ ] **Step 2: Verify test fails at missing protocol/mode integration**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_tui.py -k audnexus_chapter
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement chapter-data caching and candidate composition**
|
||||
|
||||
Return entered normalized ASIN alongside metadata lookup result. Fetch chapters once. Run silence detection only when exact is unavailable or silence candidate is requested for display; cache result for one conversion preparation. Preserve existing exception-to-fallback behavior.
|
||||
|
||||
- [ ] **Step 4: Add failing menu/reason/preview tests**
|
||||
|
||||
Assert exact/aligned/silence availability labels, mismatch and confidence reasons, every chapter timestamp/title, alignment predicted/selected/delta fields, and estimated warning. Assert back/cancel performs no planner/converter/finalizer action.
|
||||
|
||||
- [ ] **Step 5: Implement terminal UI and builder routing**
|
||||
|
||||
Keep embedded, filename, and manual behavior. Candidate selection returns validated chapters directly; local modes continue through `_build_chapters`. Never auto-select best candidate.
|
||||
|
||||
- [ ] **Step 6: Add failure fallback tests**
|
||||
|
||||
Cover Audnexus 404/timeout/malformed chapters, silence timeout/error, >5% mismatch, <90% confidence, local metadata selection, and no ASIN. Existing local modes must remain selectable and no analysis failure may move source state.
|
||||
|
||||
- [ ] **Step 7: Run focused and full tests**
|
||||
|
||||
```sh
|
||||
pytest -q tests/test_tui.py tests/test_main.py
|
||||
pytest -q
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```sh
|
||||
git add app/tui.py app/main.py tests/test_tui.py tests/test_main.py
|
||||
git commit -m "feat: select Audnexus and detected chapters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Documentation, Synthetic Smoke Analysis, and Release Gate
|
||||
|
||||
**Files:**
|
||||
- Modify: `README.md`
|
||||
- Modify: `CHANGELOG.md`
|
||||
- Modify: `scripts/container-smoke.sh`
|
||||
- Modify: `tests/test_packaging.py`
|
||||
- Modify: any audited packaging manifest/version file discovered by existing package tests
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes completed Tasks 1-5.
|
||||
- Produces operator documentation and release-ready container checks.
|
||||
|
||||
- [ ] **Step 1: Add failing documentation/package assertions**
|
||||
|
||||
Assert README includes Discovery URL, exact/aligned/silence semantics, configuration keys, private-HTTP restriction, cover HTTPS rule, known limitations, and Enterprise-D acceptance commands. Assert packaged config contains Discovery URL and new tables.
|
||||
|
||||
- [ ] **Step 2: Add failing synthetic-silence smoke fixture**
|
||||
|
||||
Generate deterministic tone/silence/tone media with FFmpeg, call real silence detector, and assert expected midpoint within a narrow tolerance before existing conversion validation. Keep smoke runtime short and source under `mktemp`.
|
||||
|
||||
- [ ] **Step 3: Implement docs and smoke updates**
|
||||
|
||||
Document no fake ETA, no automatic approval, alignment confidence, generic silence titles, and real-ASIN LAN acceptance. Update changelog and release/package audit inputs consistently.
|
||||
|
||||
- [ ] **Step 4: Run all local gates**
|
||||
|
||||
```sh
|
||||
pytest -q
|
||||
python -m compileall -q app tests
|
||||
git diff --check
|
||||
docker compose --env-file .env config --quiet
|
||||
```
|
||||
|
||||
Run existing packaging/static commands documented in README. If Docker is available, rebuild without cache and run `scripts/container-smoke.sh`; otherwise record Docker runtime gate as pending for Enterprise-D.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```sh
|
||||
git add README.md CHANGELOG.md scripts/container-smoke.sh tests/test_packaging.py config.toml
|
||||
git commit -m "docs: ship private Audnexus chapter analysis"
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Final independent review**
|
||||
|
||||
Dispatch whole-branch spec and security review. Fix every load-bearing finding with a failing regression first, rerun all gates, and re-review. Do not push or merge.
|
||||
@@ -0,0 +1,319 @@
|
||||
# Private Audnexus, Chapter Analysis, and Conversion Progress Design
|
||||
|
||||
## Purpose
|
||||
|
||||
Extend audiobook-tui so Enterprise-D can use the self-hosted Audnexus service
|
||||
at `http://192.168.86.168:3333`, import publisher chapter data when it matches
|
||||
the local audio, align remote chapter data to local silence when runtimes differ
|
||||
slightly, generate conservative silence-only chapters as a fallback, and show
|
||||
honest progress during long conversions.
|
||||
|
||||
The workflow remains keyboard-only over SSH. Analysis never modifies source
|
||||
media, and conversion still requires explicit final approval.
|
||||
|
||||
## Scope
|
||||
|
||||
This change adds:
|
||||
|
||||
- narrowly scoped private-network HTTP validation for the Audnexus base URL;
|
||||
- an Audnexus chapter endpoint client and strict response normalization;
|
||||
- exact-runtime Audnexus chapters;
|
||||
- conservative Audnexus-guided silence alignment;
|
||||
- silence-only chapter detection with generic titles;
|
||||
- chapter-source availability reasons and detailed preview data;
|
||||
- phase, elapsed-time, process-activity, and output-size tracking during
|
||||
conversion;
|
||||
- configuration, documentation, unit tests, integration tests, and a synthetic
|
||||
container smoke case for the new behavior.
|
||||
|
||||
This change does not add a desktop GUI, modify source audio during analysis,
|
||||
infer publisher chapter titles from silence alone, follow HTTP redirects, or
|
||||
automatically approve a conversion.
|
||||
|
||||
## Security Boundary for Audnexus URLs
|
||||
|
||||
An Audnexus base URL is valid when all these rules hold:
|
||||
|
||||
- `https` may address a syntactically valid host.
|
||||
- `http` may address only a literal IPv4 or IPv6 loopback address or a literal
|
||||
IPv4 RFC1918 address in `10.0.0.0/8`, `172.16.0.0/12`, or
|
||||
`192.168.0.0/16`.
|
||||
- HTTP hostnames are rejected, even when they currently resolve to a private
|
||||
address. This avoids DNS rebinding and resolution-time ambiguity.
|
||||
- Public, link-local, multicast, and unspecified HTTP addresses are rejected.
|
||||
- User information, query strings, and fragments are rejected.
|
||||
- A non-root path is rejected so endpoint construction cannot escape or double
|
||||
an operator-supplied path prefix.
|
||||
- Redirect following remains disabled.
|
||||
|
||||
The deployed configuration uses:
|
||||
|
||||
```toml
|
||||
[audnexus]
|
||||
enabled = true
|
||||
base_url = "http://192.168.86.168:3333"
|
||||
region = "us"
|
||||
```
|
||||
|
||||
Cover URLs remain HTTPS-only and retain existing bounded streaming, media-type,
|
||||
signature, staging, and no-overwrite checks.
|
||||
|
||||
## Domain Model
|
||||
|
||||
`ChapterMode` gains three values:
|
||||
|
||||
- `AUDNEXUS_EXACT`
|
||||
- `AUDNEXUS_ALIGNED`
|
||||
- `SILENCE`
|
||||
|
||||
Audnexus normalization produces a chapter document containing requested ASIN,
|
||||
remote runtime, accuracy flag, and ordered remote chapters. Analysis produces a
|
||||
chapter candidate containing mode, chapters, availability, reason, local and
|
||||
remote runtimes when applicable, confidence percentage when applicable, and
|
||||
per-boundary alignment diagnostics when applicable.
|
||||
|
||||
The selected candidate is converted to the existing immutable `Chapter` tuple
|
||||
before planning. Planner and converter continue to consume one validated
|
||||
chapter sequence without knowing how it was produced.
|
||||
|
||||
## Audnexus Chapter Retrieval
|
||||
|
||||
For a valid ten-character ASIN, the client requests:
|
||||
|
||||
```text
|
||||
GET /books/{ASIN}/chapters?region={configured-region}
|
||||
```
|
||||
|
||||
The chapter response uses the existing connection/read timeouts and 10 MiB
|
||||
response ceiling. Failure returns an unavailable candidate rather than blocking
|
||||
local chapter modes.
|
||||
|
||||
Normalization requires:
|
||||
|
||||
- a response object;
|
||||
- returned ASIN equal to the requested ASIN;
|
||||
- finite, positive remote runtime;
|
||||
- a nonempty chapter array;
|
||||
- finite, nonnegative integer millisecond offsets;
|
||||
- first offset equal to zero;
|
||||
- strictly increasing offsets;
|
||||
- every offset less than remote runtime;
|
||||
- nonempty titles no longer than 10,000 Unicode characters;
|
||||
- no C0, DEL, or C1 control characters in titles.
|
||||
|
||||
Extra response fields are ignored. Malformed or oversized data is rejected as
|
||||
one remote failure. Metadata acceptance and chapter acceptance remain
|
||||
independent: an entered ASIN may supply chapters while the operator retains
|
||||
local metadata.
|
||||
|
||||
## Exact Runtime Matching
|
||||
|
||||
Let `L` be scanned local duration in milliseconds and `R` be reported remote
|
||||
duration. Exact chapters are available when:
|
||||
|
||||
```text
|
||||
abs(L - R) <= min(L * 0.01, 120 seconds)
|
||||
```
|
||||
|
||||
Remote offsets and titles are used unchanged after verifying every offset is
|
||||
less than `L`. This test intentionally uses the smaller tolerance. It prevents
|
||||
the absolute allowance from becoming too loose for short books and the
|
||||
percentage allowance from becoming too loose for long books.
|
||||
|
||||
## Silence Analysis
|
||||
|
||||
Silence analysis runs FFmpeg against sorted local tracks without modifying
|
||||
them. Multi-file input is analyzed in track order with cumulative offsets so
|
||||
results share the same local timeline used by conversion. The subprocess uses
|
||||
an explicit argument vector, bounded captured output, a configured timeout,
|
||||
and the existing process-group termination pattern for interruption.
|
||||
|
||||
Default configuration:
|
||||
|
||||
```toml
|
||||
[chapters]
|
||||
exact_runtime_percent = 1.0
|
||||
exact_runtime_max_seconds = 120
|
||||
alignment_runtime_percent = 5.0
|
||||
alignment_required_percent = 90.0
|
||||
|
||||
[chapters.silence]
|
||||
threshold_db = -40
|
||||
minimum_silence_seconds = 1.5
|
||||
minimum_chapter_minutes = 5
|
||||
snap_window_seconds = 45
|
||||
analysis_timeout_seconds = 1800
|
||||
max_output_bytes = 10485760
|
||||
```
|
||||
|
||||
FFmpeg silence events must form valid start/end pairs. A candidate boundary is
|
||||
the midpoint of a qualifying silence. Events before zero, beyond the track,
|
||||
non-finite values, reversed pairs, malformed lines, unmatched pairs, output
|
||||
beyond the configured ceiling, timeout, cancellation, and nonzero process exit
|
||||
make silence analysis unavailable with a concise reason.
|
||||
|
||||
Candidates are translated by cumulative track duration, sorted, deduplicated,
|
||||
and filtered so boundaries are not placed at zero or at/after total local
|
||||
duration.
|
||||
|
||||
## Audnexus-Guided Alignment
|
||||
|
||||
Guided alignment is considered only when exact matching is unavailable and:
|
||||
|
||||
```text
|
||||
abs(L - R) / R <= 0.05
|
||||
```
|
||||
|
||||
Each remote boundary after the first is projected onto the local timeline:
|
||||
|
||||
```text
|
||||
predicted_local_offset = remote_offset * L / R
|
||||
```
|
||||
|
||||
The projected boundary snaps to the nearest unused silence midpoint within 45
|
||||
seconds. Ties resolve to the earlier midpoint. A boundary with no qualifying
|
||||
silence remains at its projected offset and is marked unsnapped. Snapped and
|
||||
unsnapped boundaries must remain strictly increasing, respect the five-minute
|
||||
minimum chapter spacing, and precede local duration. A candidate that violates
|
||||
those rules is unavailable even when its raw match percentage would pass. The
|
||||
first chapter remains at zero and does not require a silence match.
|
||||
|
||||
Alignment confidence is:
|
||||
|
||||
```text
|
||||
matched nonzero remote boundaries / total nonzero remote boundaries * 100
|
||||
```
|
||||
|
||||
Aligned chapters are available only at confidence of at least 90%. When that
|
||||
threshold passes, remote titles are retained for both snapped and projected
|
||||
boundaries, and preview identifies every unsnapped boundary. Otherwise the
|
||||
candidate is unavailable; the application must not attach remote titles to a
|
||||
partial or low-confidence alignment.
|
||||
|
||||
## Silence-Only Chapters
|
||||
|
||||
Silence-only mode remains independent of Audnexus availability and runtime.
|
||||
It sorts qualifying midpoint candidates by time, then greedily selects the
|
||||
earliest candidate at least five minutes after the previously selected
|
||||
boundary. Exact duplicate midpoints retain the longest silence. This makes the
|
||||
result deterministic without allowing silence duration to pull a boundary far
|
||||
away from the earliest eligible transition.
|
||||
|
||||
The first chapter always starts at zero. Subsequent titles are generated in
|
||||
sequence:
|
||||
|
||||
```text
|
||||
Chapter 1
|
||||
Chapter 2
|
||||
...
|
||||
```
|
||||
|
||||
The UI labels these as estimated chapters and never presents them as publisher
|
||||
chapter data. An empty qualifying set leaves silence mode unavailable rather
|
||||
than creating a misleading one-chapter result.
|
||||
|
||||
## TUI Flow
|
||||
|
||||
After track scan and optional ASIN lookup, the application analyzes applicable
|
||||
chapter sources. Existing embedded, filename, and manual modes remain
|
||||
available under their current rules.
|
||||
|
||||
The chapter menu reports usable candidates and concise unavailable reasons,
|
||||
including runtime difference and alignment confidence. Example:
|
||||
|
||||
```text
|
||||
Audnexus exact unavailable: runtime differs by 3.4%
|
||||
Audnexus aligned unavailable: 76% of boundaries matched silence
|
||||
Silence detection: 31 estimated chapters
|
||||
```
|
||||
|
||||
Selection preview includes:
|
||||
|
||||
- chapter method;
|
||||
- local and remote runtimes when applicable;
|
||||
- match confidence when applicable;
|
||||
- every chapter timestamp and title;
|
||||
- predicted, snapped, and delta offsets for aligned boundaries;
|
||||
- existing paths and escaped command vectors.
|
||||
|
||||
The operator may go back and choose another source. Final conversion approval
|
||||
remains exact and mandatory.
|
||||
|
||||
## Conversion Progress
|
||||
|
||||
`m4b-tool` suppresses useful FFmpeg percentage output, so the application must
|
||||
not invent a percentage or ETA. A tracker reports observable state every five
|
||||
seconds by default:
|
||||
|
||||
```text
|
||||
[Converting audio] elapsed 00:37:12 | output 94.3 MiB | active
|
||||
[Applying chapters] elapsed 00:00:08
|
||||
[Validating output] elapsed 00:00:03
|
||||
[Moving completed files]
|
||||
```
|
||||
|
||||
Phases are converting audio, applying chapters, validating output, and moving
|
||||
completed files. During conversion, output size is read with filesystem
|
||||
metadata only. The application never runs `ffprobe` against a growing output.
|
||||
Each update goes to both terminal callback and persistent conversion log.
|
||||
|
||||
Tracker implementation must not block subprocess output consumption, leak a
|
||||
thread after completion, race log writes, or weaken Ctrl+C behavior. Final
|
||||
phase record includes elapsed time, child exit status, and error when present.
|
||||
Existing source, partial-output, work-directory, and recovery invariants remain
|
||||
unchanged.
|
||||
|
||||
Configuration:
|
||||
|
||||
```toml
|
||||
[progress]
|
||||
update_seconds = 5
|
||||
```
|
||||
|
||||
The interval must be finite and positive.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- Audnexus metadata or chapter failures fall back to local choices.
|
||||
- Silence-analysis failures fall back to non-silence modes.
|
||||
- Exact mismatch is informational, not fatal.
|
||||
- Guided mismatch beyond 5% is unavailable without running alignment.
|
||||
- Low-confidence alignment never produces a selectable candidate.
|
||||
- Invalid configuration blocks startup with deterministic aggregated errors.
|
||||
- Conversion failure leaves source pending and preserves existing controlled
|
||||
recovery behavior.
|
||||
- No analysis failure moves source data or creates a ready output.
|
||||
|
||||
## Testing
|
||||
|
||||
Tests follow red-green-refactor and cover:
|
||||
|
||||
- HTTPS and private HTTP acceptance plus public, hostname, credential,
|
||||
path/query/fragment, link-local, multicast, and unspecified rejection;
|
||||
- IPv4 and IPv6 loopback cases;
|
||||
- Audnexus chapter request path, region, timeout, body bound, ASIN matching,
|
||||
schema, title, and offset validation;
|
||||
- exact-match tolerance boundaries;
|
||||
- FFmpeg silence command construction, parsing, multi-track offset translation,
|
||||
malformed/oversized output, timeout, cancellation, and exit failure;
|
||||
- deterministic silence filtering and generic titles;
|
||||
- guided projection, nearest snapping, tie behavior, spacing, confidence, 5%
|
||||
eligibility, and wrong-edition rejection;
|
||||
- TUI candidate display, unavailable reasons, selection, preview, backtracking,
|
||||
and final approval;
|
||||
- progress cadence, observable fields, terminal/log mirroring, child exit,
|
||||
cancellation, and cleanup;
|
||||
- synthetic container smoke audio containing known silences;
|
||||
- regression coverage for embedded, filenames, manual chapters, metadata,
|
||||
covers, planning, conversion validation, workflow transitions, and packaging.
|
||||
|
||||
Runtime acceptance on Enterprise-D must verify a real ASIN against Discovery,
|
||||
an exact or aligned chapter import, a silence-only preview, visible progress
|
||||
during a nontrivial conversion, final chapter count/titles, and playable output.
|
||||
|
||||
## Delivery
|
||||
|
||||
Implementation stays on `feat/audiobook-tui`, is committed in reviewable tasks,
|
||||
and is not pushed or merged without explicit approval. README and example
|
||||
configuration document Discovery at `192.168.86.168:3333`, safe-match rules,
|
||||
alignment confidence, silence limitations, progress semantics, and recovery.
|
||||
@@ -53,6 +53,13 @@ ffmpeg -nostdin -hide_banner -loglevel error \
|
||||
-f lavfi -i 'color=c=navy:s=64x64:d=0.1' \
|
||||
-frames:v 1 -q:v 2 \
|
||||
"$smoke_root/pending/Synthetic Book/cover.jpg"
|
||||
ffmpeg -nostdin -hide_banner -loglevel error \
|
||||
-f lavfi -i 'sine=frequency=440:duration=0.4' \
|
||||
-f lavfi -i 'anullsrc=r=44100:cl=mono:d=0.8' \
|
||||
-f lavfi -i 'sine=frequency=880:duration=0.4' \
|
||||
-filter_complex '[0:a][1:a][2:a]concat=n=3:v=0:a=1[out]' \
|
||||
-map '[out]' -q:a 7 \
|
||||
"$smoke_root/silence-fixture.mp3"
|
||||
|
||||
python3 - <<'PY'
|
||||
import os
|
||||
@@ -108,6 +115,42 @@ if not (roots.done / source.name).is_dir():
|
||||
raise SystemExit("source was not moved to done")
|
||||
PY
|
||||
|
||||
python3 - <<'PY'
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from app.models import Book, Roots, Settings, Track
|
||||
from app.silence import detect_silences
|
||||
|
||||
root = Path(os.environ["AUDIOBOOK_SMOKE_ROOT"])
|
||||
roots = Roots(
|
||||
pending=root / "pending",
|
||||
ready=root / "ready",
|
||||
done=root / "done",
|
||||
review=root / "review",
|
||||
)
|
||||
fixture = root / "silence-fixture.mp3"
|
||||
book = Book(source=fixture, tracks=(Track(path=fixture, duration_ms=1600),))
|
||||
settings = Settings(
|
||||
roots=roots,
|
||||
state_dir=root / "state",
|
||||
work_dir=root / "state" / "work",
|
||||
jobs=1,
|
||||
audnexus_enabled=False,
|
||||
silence_threshold_db=-35,
|
||||
silence_minimum_seconds=0.4,
|
||||
silence_analysis_timeout_seconds=30,
|
||||
silence_max_output_bytes=1_048_576,
|
||||
)
|
||||
silences = detect_silences(book, settings)
|
||||
if len(silences) != 1:
|
||||
raise SystemExit(f"silence smoke expected one interval; found {silences!r}")
|
||||
midpoint_ms = silences[0].midpoint_ms
|
||||
if abs(midpoint_ms - 800) > 150:
|
||||
raise SystemExit(f"silence smoke midpoint outside tolerance: {midpoint_ms}ms")
|
||||
print("silence detector smoke passed")
|
||||
PY
|
||||
|
||||
ffprobe -v error \
|
||||
-show_entries \
|
||||
'format=duration:chapter_tags=title:stream=codec_name,codec_tag_string,codec_type:stream_disposition=attached_pic' \
|
||||
|
||||
@@ -111,6 +111,13 @@ def _run_container_smoke_with_probe_payload(
|
||||
" printf '%s\\n' 'ffmpeg fake'\n"
|
||||
" exit 0\n"
|
||||
"fi\n"
|
||||
"case \"$*\" in\n"
|
||||
" *silencedetect=*)\n"
|
||||
" printf '%s\\n' '[silencedetect] silence_start: 0.400'\n"
|
||||
" printf '%s\\n' '[silencedetect] silence_end: 1.200'\n"
|
||||
" exit 0\n"
|
||||
" ;;\n"
|
||||
"esac\n"
|
||||
"for output do :; done\n"
|
||||
": > \"$output\"\n",
|
||||
encoding="utf-8",
|
||||
@@ -191,6 +198,54 @@ def test_config_defaults_match_container_roots() -> None:
|
||||
assert root in text
|
||||
|
||||
|
||||
def test_packaged_config_enables_discovery_chapter_analysis() -> None:
|
||||
config = tomllib.loads(Path("config.toml").read_text(encoding="utf-8"))
|
||||
|
||||
assert config["audnexus"] == {
|
||||
"enabled": True,
|
||||
"base_url": "http://192.168.86.168:3333",
|
||||
"region": "us",
|
||||
"connect_timeout": 3.0,
|
||||
"read_timeout": 8.0,
|
||||
}
|
||||
assert config["chapters"] == {
|
||||
"exact_runtime_percent": 1.0,
|
||||
"exact_runtime_max_seconds": 120,
|
||||
"alignment_runtime_percent": 5.0,
|
||||
"alignment_required_percent": 90.0,
|
||||
"silence": {
|
||||
"threshold_db": -40,
|
||||
"minimum_silence_seconds": 1.5,
|
||||
"minimum_chapter_minutes": 5,
|
||||
"snap_window_seconds": 45,
|
||||
"analysis_timeout_seconds": 1800,
|
||||
"max_output_bytes": 10_485_760,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_operator_docs_cover_private_audnexus_chapter_analysis() -> None:
|
||||
readme = Path("README.md").read_text(encoding="utf-8")
|
||||
required_operator_guidance = (
|
||||
"http://192.168.86.168:3333",
|
||||
"Audnexus exact",
|
||||
"Audnexus aligned",
|
||||
"Silence detection",
|
||||
"exact_runtime_percent",
|
||||
"alignment_required_percent",
|
||||
"minimum_silence_seconds",
|
||||
"literal RFC1918",
|
||||
"HTTPS-only",
|
||||
"no percentage or ETA",
|
||||
"does not approve",
|
||||
"real ASIN",
|
||||
"/books/{ASIN}/chapters?region=us",
|
||||
)
|
||||
|
||||
for guidance in required_operator_guidance:
|
||||
assert guidance in readme
|
||||
|
||||
|
||||
def test_work_directory_is_persistent_in_the_state_mount() -> None:
|
||||
config = tomllib.loads(Path("config.toml").read_text(encoding="utf-8"))
|
||||
service = _service()
|
||||
@@ -430,6 +485,7 @@ def test_container_smoke_accepts_expected_final_stream_layout(
|
||||
)
|
||||
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "silence detector smoke passed" in result.stdout
|
||||
assert "container smoke test passed" in result.stdout
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user