From 0e22839ae4197a20f6a4b98cf438e0132f26deaf Mon Sep 17 00:00:00 2001 From: justinzeus Date: Sat, 3 Oct 2026 00:02:22 +0200 Subject: [PATCH 1/3] Define the integrated Scholarr product sprint --- AGENTS.md | 24 ++- README.md | 14 +- SPRINT.md | 314 ++++++++++++++++++++++++++++++++++++ TASKS.md | 95 ----------- docs/specs/README.md | 20 ++- docs/specs/data-model.md | 24 +-- runbooks/rewrite/charter.md | 15 +- 7 files changed, 361 insertions(+), 145 deletions(-) create mode 100644 SPRINT.md delete mode 100644 TASKS.md diff --git a/AGENTS.md b/AGENTS.md index e1eadcc..e1a4367 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # Scholarr agent brief -This file is the canonical brief for every coding agent working in this repository. Read it and -`TASKS.md` before making changes. +For sprint work, read `SPRINT.md` before changing the repository. It is the +single product, decision, provider-safety, and execution reference. ## Product @@ -41,14 +41,11 @@ Do not silently change a locked decision. Record a proposed change as open and a ## Decision record -On tank, the authoritative project runbook is: +`SPRINT.md` governs an explicitly owner-approved sprint. The historical charter +and identity/calibration/testing records live in `runbooks/rewrite/`. -`/opt/stacks/server-docs/runbooks/scholarr-rewrite/README.md` - -It owns doctrine, locked decisions, empirical calibration results, and project state. Its -companion files own the identity model and testing strategy. This brief is self-contained for -agents without tank access, but any state or design decision reached on tank must also be written -to that runbook in the same session. +Track acceptance and next actions in Forgejo. Record sprint implementation +decisions in `SPRINT.md`; do not add another plan or a separate freeze queue. ## Frozen UI contract @@ -67,10 +64,11 @@ are expected. ## Work process -- Work through `TASKS.md` in order. Each task has a goal and acceptance criteria. -- Freeze one spec at a time. Decided items are labeled decided; open items remain open for owner - review. -- Do not start implementation for an area whose spec is not frozen. +- Work the integration issue's acceptance items in order, against `SPRINT.md`. +- Preparation does not authorize execution. Start only after the owner approves + the concrete nightly plan and explicitly says to launch. +- An approved sprint replaces separate spec-freeze ceremonies with the bundled + defaults in `SPRINT.md`. Preserve locked requirements; report conflicts. - Keep commits focused and use plain imperative commit messages without trailers. - Stage only files you changed. - Do not use em dashes in prose, code comments, commit messages, or user-facing copy. diff --git a/README.md b/README.md index b83dea5..03fec63 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,8 @@ Scholarr is a self-hosted author watchlist and publication tracker. Follow researchers, learn when they publish, and find legal open-access copies of their work. -The rewrite is pre-alpha. Its product direction and interface are settled, and the architecture -specs are being frozen before implementation begins. +The rewrite is pre-alpha. [SPRINT.md](SPRINT.md) is the single reference for its +proposed integrated product sprint; preparation does not authorize a launch. ## Non-negotiable rule @@ -21,13 +21,13 @@ from documented APIs such as OpenAlex, Crossref, arXiv, Unpaywall, and ORCID. - Internal auth, generic OIDC, and trusted-header auth in v1 - Config-driven operation and simple first-run onboarding -See [AGENTS.md](AGENTS.md) for the full project constraints and [TASKS.md](TASKS.md) for the -ordered work queue. +Read [SPRINT.md](SPRINT.md) for scope, decisions, provider safety, and acceptance. +[AGENTS.md](AGENTS.md) supplies the repository constraints. ## Repositories - Primary: -- Forgejo copy: +- Forgejo tracker and sprint delivery: - Preserved legacy implementation: ## Frozen interface contract @@ -38,5 +38,5 @@ are the source for the Vue port, not an application implementation. ## License -The license has not been selected yet. Until a LICENSE file is added, normal copyright rules -apply. License selection is an explicit project decision in [TASKS.md](TASKS.md). +The sprint proposes MIT, subject to owner approval of [SPRINT.md](SPRINT.md). +Until a LICENSE file is added, normal copyright rules apply. diff --git a/SPRINT.md b/SPRINT.md new file mode 100644 index 0000000..d862204 --- /dev/null +++ b/SPRINT.md @@ -0,0 +1,314 @@ +# Scholarr product sprint + +## Approval and authority + +Justin authorized preparation, issue triage, and a new baseline commit. He has +not authorized launch. Review this document as the single product and execution +reference for the proposed 24-hour run. Approval tracking: justinzeus/scholarr#9. +Runner preparation: justinzeus/nightly#92; reset supervisor: justinzeus/nightly#93. + +An explicit owner go for the concrete plan approves the proposed defaults below +and replaces the old, separate owner-freeze gates for this sprint. The agents +may then choose implementation details and record their rationale here without +stopping for another spec ceremony. They must preserve the locked requirements. +A requirement conflict, missing credential needed for a real feature, or unsafe +provider operation remains a blocker; agents must report it rather than invent +a fallback or claim completion. Preparation alone does not activate this authority. + +The reset preserves Git history, the frozen design, and prior decision evidence. +It supplies one new starting commit on an isolated branch. It does not erase the +legacy repository, dirty local files, old decision issues, or production data. + +## Intended result + +Build a self-hosted author watchlist and publication tracker that Justin's dad +can use daily and other households can install. A reader follows researchers, +sees their publications, tracks read state, resolves ambiguous identities, and +finds legal open-access copies. An administrator can understand provider health, +recover access, audit changes, and back up or restore the installation. + +The target after the run is one integrated, runnable product branch and one +reviewed PR, with executable acceptance evidence and a usable installation path. +A pile of independent PRs, a static UI demonstration, mock-only provider clients, +or a service with omitted authentication modes does not meet the target. +Twenty-four hours is a delivery target, not a promise that unfinished work is done. + +The run does not deploy, merge into main, publish a release, import the real +household database, or replace the existing workflow. Justin still performs +manual UI approval and the real-data shadow period before a production cutover. + +## Locked requirements + +- Go backend, Vue 3 frontend, SQLite only. Use a serialized writer, foreign keys, + WAL, numbered migrations, and restart-safe persistence. +- A source-agnostic `FollowedAuthor` is the author root. Names are display/search + labels, never identity keys. Preserve external source identities and provenance. +- Share deduplicated authors and publications globally. Keep follows, read state, + visibility, and user preferences private to their user. +- Support internal username/password, generic OIDC, and trusted-header auth. + Internal auth remains the zero-dependency default and recovery path. +- Use one validated declarative configuration, environment-variable references + for secrets, safe first-admin setup, and a straightforward first-run flow. +- Use the frozen `design/DESIGN.md`, `design/tokens.css`, and bundled assets. + Implement dashboard, publication list/detail, author list/detail, review, + author search/add, bulk import, sources, activity, settings, login, onboarding, + and not-found behavior. Keep loading, empty, populated, and degraded states + honest, with responsive navigation, keyboard access, and light/dark themes. +- Serve system fonts and bundled assets locally. The design proof's + `support.js` and synthetic example records do not ship as product behavior. +- Show source freshness and confidence words rather than invented certainty or + raw match percentages. Keep new-publication counts scoped to the current user. +- Preview bulk imports and destructive admin actions before applying them. + Audit consequential writes and support conflict-checked undo. +- Implement documented clients for OpenAlex, Crossref, arXiv, Unpaywall, and + ORCID. Missing keys disable that client's network lane with an actionable + health state; they do not authorize anonymous scraping or a substitute API. +- Zero application-initiated Google contact. Scholar URLs/IDs are inert import + metadata or human-clickable links. Parse them locally; never dereference them. + No scraping, CAPTCHA solving, anti-bot evasion, or Semantic Scholar v1 expansion. + +The data-model draft supplies detailed table contracts and its 22 falsifiable +acceptance cases. The frozen visual design supplies the appearance contract. +This document controls sprint scope and defaults where the old process differs. + +## Proposed decisions approved together at launch + +| Decision | Sprint default | +|---|---| +| D1: public identity | Internal integer joins; immutable UUIDv7 public IDs only on externally referenced or merge-aliased rows. | +| D2a: unfollow | Hide records without an active follow path; retain read/favorite state and restore it on refollow. | +| D2b: legacy favorites | Retain migrated favorite state for lossless import/export; do not add a favorite UI outside the frozen design. | +| D2c: legacy reading | Treat a publication as read if any legacy link for that user says read. | +| D2d: NEW | Derive at read time from first incremental discovery within 14 days; baseline, manual, and legacy imports are not NEW. | +| D3: attachment | Reuse exact identities, explicit provider crosswalks, and completed high-confidence calibration evidence. Review weaker or conflicting evidence; names alone never attach or merge. | +| D4: undo | No arbitrary expiry while recorded row-version preconditions hold. A conflict prevents the entire undo. | +| D5: orphans | Retain automatically. Admin garbage collection requires a backup, dry-run preview, reference checks, and audit. | +| D6: shared identity | Only administrators merge shared identity or write global negative-identity evidence. Reader decisions remain per-user. | +| D7: purge | Hard purge with no resurrection tombstone. A later sanctioned strong-ID ingest can create a new canonical row; legacy-only data cannot recreate it automatically. | +| Distribution | A self-contained Go binary with embedded built Vue assets, plus a non-root container image. Compose is an installation example, not a live deployment. | +| License | Propose MIT with Justin's personal copyright identity. Add the license only after the owner approves this packet. | +| Acquisition | OpenAlex is primary. Crossref/arXiv add work metadata; ORCID anchors identity; Unpaywall resolves legal OA locations. Persist cursors, deduplicate, and retain provenance. | +| Sync | One active run per user, shared provider budgets, idempotent resume, additive baseline import, scheduled and manual sync through the same queue. | +| Matching | Exact strong identifiers first; weak/ambiguous candidates go to review. Keep recorded calibration expectations, using public-safe fixtures only. | +| OA | Store location, license, and provenance. Expose a legal human-clickable location; do not fetch PDFs or arbitrary publisher URLs during the sprint. | +| Config changes | Validate before applying. Document restart requirements; do not add hot reload for settings whose safe transition is unclear. | + +Security details follow maintained Go libraries: memory-hard password hashing, +secure expiring sessions, CSRF defenses, bounded login attempts, OIDC issuer and +signature validation with state/nonce and PKCE, and trusted headers accepted only +from explicitly trusted proxies. Reject direct-header spoofing. Isolate user +queries and authorize admin operations on the server, not only in the UI. + +## Provider safety and the sprint's network boundary + +Spend Codex quota on coding, testing, and reviewing, not on harvesting papers. +Automated tests and the repeatable integration smoke use deterministic local +HTTP fixtures. Production clients must be real. Justin additionally authorized +a bounded live-provider acceptance check, not bulk acquisition. + +After fixture tests prove throttling and circuit breaking, an operator-owned +host check may issue at most **two requests each** to OpenAlex and Crossref, +**one each** to arXiv and Unpaywall, and **zero** to ORCID until its credentials +and non-commercial eligibility are confirmed. Six requests total, including +retries; a failure does not replenish the allowance. Use public example works, +the operator's existing contact identity, and the escrowed OpenAlex key. +Keep credentials, private calibration caches, and the household dump out of +writer sandboxes. Do not perform these checks during preparation. + +Record actual endpoint, status, timestamp, and remaining allowance without keys +or full private payloads. Skip a provider whose prerequisite is missing and +report it as unverified, not passed. A 429/auth/ban response stops that provider +immediately. Honor the conservative spacing below even for this small check. +The application defaults acquisition to disabled until the operator explicitly +enables configured providers. Installation never starts a historical crawl; +confirmation queues limited work and never bypasses the scheduler. + +The shipped provider scheduler must enforce: + +- One outstanding request per provider and no burst allowance. Share the limit + across users, search, sync, OA lookups, retries, and manual refreshes. +- Persist the next-request time, cooldown, daily allowance consumption, and + in-progress work before issuing requests. Restart never clears a cooldown or + resets a daily budget. A daily allowance uses the provider's documented reset; + if none exists, enforce a rolling 24-hour window. +- Every attempted request, including retries, consumes the budget. Cache by + strong identifiers/query parameters; coalesce duplicate queued work. Expose + queued, cooling, failed, and disabled states without retry loops in the UI. +- A 429 immediately opens the shared provider circuit. Honor a valid + `Retry-After` date/duration, then permit at most one rate-limited probe. + Missing or invalid `Retry-After` requires an administrator-set cooldown; + do not guess a short retry interval. A second 429 in a rolling 24-hour period + requires manual re-arm after the administrator reviews the cause. +- On 401/403, a ban/challenge response, or an unexpected redirect, stop the lane + and require operator attention. Do not rotate keys, IPs, accounts, or user agents. +- Treat 503 as a provider cooldown, since some APIs use it for burst limiting. + Other transient failures get at most two retries with exponential backoff and + jitter under the same budgets. Malformed responses stop the work item; they + do not trigger an endpoint retry loop. Retain resumable work and surface errors. +- Use honest identification and configured contact details where the provider + requires them. An absent required identity or key fails closed. +- Restrict production acquisition to the provider's approved HTTPS host and path. + Reject arbitrary base URLs, Google hosts, private/link-local targets, and + unapproved redirects. Local fixture endpoints exist only in test configuration. +- Persist budgets per provider credential/IP scope; an application limiter cannot + account for other services using tank's IP. Before live enablement, the owner + must allocate that shared-IP allowance. Use the smaller of published limits, + this product's conservative default, and the allocated operator allowance. + +No rate limit can guarantee that a provider will never block an IP. The zero-live +sprint budget removes this run's data-API traffic; conservative client limits and +circuit breakers constrain a later enabled installation. + +### Conservative live-enable defaults + +These are local engineering limits for a later owner-approved connection check, +not promised provider entitlements or the sprint's budget. The sprint budget +remains zero. One outstanding request and no burst apply to each row. +Official documentation was checked on 2026-10-02. + +| Provider | Local check allowance | Access and policy constraints | +|---|---|---| +| OpenAlex | 1 request per 10 seconds; 20 attempts/day; $0.05/day spend cap | Current docs conflict about keyless access. Require an owner-managed API key and policy clarification before live enablement. Price requests by operation; request count alone is not the daily budget. Exclude content/PDF endpoints. [Authentication](https://help.openalex.org/api/authentication/), [costs](https://help.openalex.org/access/example-costs/), [deprecations](https://help.openalex.org/api/deprecations/). | +| Crossref | 1 request per 10 seconds; 80 attempts/day | Use polite `mailto` and an identifying User-Agent. Honor rate-interval and concurrency response headers; do not interpret pool counts as a fixed RPS entitlement. No published daily cap was established. [Access and authentication](https://www.crossref.org/documentation/retrieve-metadata/rest-api/access-and-authentication/). | +| arXiv | 1 request per 6 seconds; 80 attempts/day | Published terms require at least 3 seconds between calls and one connection across all machines under your control. Use the sanctioned query route, not paper scraping. Validate the approved TLS endpoint before live enablement. [Terms](https://info.arxiv.org/help/api/tou.html), [manual](https://info.arxiv.org/help/api/user-manual.html). | +| Unpaywall | 1 request per 10 seconds; 80 attempts/day | DOI lookup requires a configured email. Docs ask for at most 100,000 calls/day but state no RPS or quota-accounting key. The search route retired on 2026-09-18; do not implement it. [API](https://unpaywall.org/products/api). | +| ORCID | Disabled; conditional check: 1 request per 10 seconds, 80 reads/day | Require owner confirmation of non-commercial eligibility and an individually registered public client. Implement client-credentials OAuth and fixtures; do not automate account registration. Anonymous/public limits and 503 burst responses are not an invitation to use their burst allowance. [Limits](https://info.orcid.org/ufaqs/what-are-the-api-limits/), [registration](https://info.orcid.org/documentation/integration-guide/registering-a-public-api-client/), [public-client terms](https://info.orcid.org/public-client-terms-of-service/). | + +OpenAlex's published 100-RPS ceiling and daily dollar budgets, and ORCID's +published anonymous/public read quotas, do not supersede these smaller local +allowances. ORCID Public API terms prohibit use connected with a +revenue-generating product/service. Public release must document this constraint +and leave that lane disabled unless the operator meets the terms or has an +approved Member API path. No paid provider use is authorized by this sprint. + + +## Build order and the acceptance ledger + +Use **one integration issue**, with one verifiable behavior per acceptance box. +Nightly works its boxes sequentially on the same branch; each later item inherits +the previous verified commit. Both review roles run on each candidate. Keep the +specification in this document and evidence/ticks in Forgejo, rather than copying +another plan into each role prompt. + +Work in this dependency order: + +1. Runnable Go process, validated config, SQLite lifecycle, health/readiness, and + a default gate that runs offline. The first real feature supplies the Go module; + do not spend a separate item on an empty scaffold. +2. Numbered schema migrations, database invariants, public identity, and the + data-model's independent acceptance cases. Exercise migrations up/down and + serialized concurrent writes before acquisition code uses them. +3. Safe first-admin creation, internal authentication/session recovery, tenant + isolation, generic OIDC, and trusted-proxy authentication. +4. The shared persisted provider limiter and block/cooldown behavior, then one + documented provider adapter at a time against local fixtures. +5. Exact identity matching, conservative candidate review, author follow/unfollow, + publication dedup/provenance, per-user reading, and NEW projection. +6. Restart-safe queued baseline/incremental sync, partial failure, cancellation, + dry-run import, audited merge/undo, legal OA resolution, and export. +7. Vue's real API integration and the frozen screens, each a separate checked + behavior. No production fixture data and no missing buttons hidden as polish. +8. Safe installation, license, embedded assets/container packaging, offline demo, + backup/restore, operational documentation, and the final full-flow smoke. + +Every implementation item states its check. Keep tests focused on observable +behavior, invariant violations, failure transitions, or a plausible regression. +Use a fake clock for cooldowns, not real sleeps. Use the race detector and parser +fuzz smoke in the final gate. Synthetic fixtures carry no real household identity. +Do not invent a claim that the private calibration corpus passed if the run did +not exercise a sanitized copy; list that validation as owner-side acceptance. + +## Execution envelope + +| Setting | Proposed value | +|---|---| +| Project | Scholarr only; base is the preserved sprint-baseline branch. | +| Work | One integrated product issue, ordered by the build ledger. | +| Provider/model | `openai-codex/gpt-6.1-sol` in scope, writer, escalation, spec review, and standards review. | +| Thinking | `xhigh` in all five slots. Never change this sprint to `max`. | +| Codex budget/floor | 100 window points / 0% remaining reserve; the available subscription balance is the practical bound. | +| Night/issue wall | 1,440 active minutes each; subscription pauses do not consume the issue wall, but do consume the night wall. | +| Issue token guard | 100 million non-cached tokens; finite secondary guard, not a target. | +| Role session | Existing 120-minute cap and idle detection. | +| Concurrency | One project issue; the two independent review roles may run together. | +| Repairs/refusals/resources | Existing caps, sandbox isolation, admission reserves, and broker protections. | +| Data-provider requests | Local fixtures in writer/review sandboxes; separate operator acceptance allowance of six real requests, never a crawl. | +| End | All acceptance complete, quota exhaustion, wall cutoff, or a reported safety/stuck condition. No filler work to burn remaining quota. | + +Nightly checks the night wall between role sessions. A role or verification +already running may finish after the 24-hour scheduler cutoff; this plan does not +claim an exact process-kill deadline. Only reviewed, verified commits count as +output. Exhausting quota may interrupt an item and discard its uncommitted work; +completed items remain on the integration branch. + +Operator-owned ceiling configuration permits the larger wall/token/budget values; +ordinary presets keep their existing defaults. The Scholarr preset requests the +larger values explicitly. Freeze the runner settings in the saved draft and keep +no model fallback. Approval builds/checks the Go and Node image without calling a +model. Launch spends subscription quota and requires Justin's explicit go on the +resolved plan, not merely permission to prepare it. + +### Expiring Codex reset credit + +Read-only account inspection on 2026-10-02 found 98% of the weekly window +remaining, an ordinary weekly reset at **2026-10-09T21:10:30Z**, and three saved +reset credits. The earliest credit expires at +**2026-10-04T05:32:19.368242Z**. This saved credit, not the weekly rollover, is +the urgent deadline. Re-read live usage and credit details before approval. + +Preparation does not redeem a credit. A separate explicit owner go may arm a +host-only, single-use supervisor for the soonest-expiring available credit. +Use the official installed Codex app-server account/reset-credit protocol, not +cache invalidation or guessed private HTTP calls. Pin the expected account and +credit; persist one idempotency UUID before redemption. Never expose OAuth +credentials to runner images, prompts, logs, or repository files. + +The proposed trigger is a quiescent run with at least 99% weekly usage, or the +expiry guard at **2026-10-04T05:17:19.368242Z** after safely stopping work. +The expiry guard explicitly trades remaining old-window quota for preserving +the expiring reset. Confirm backend eligibility first; a rejected or ambiguous +redemption must not select another credit or create a new UUID. Record and +reconcile the result using the same request ID. Redeem at most one credit; +never automatically launch a second night or spend the newly reset window. + +This policy maximizes useful work, not artificial token burn. Subscription +throttling, an early finished product, or an unmet backend reset prerequisite +can prevent complete exhaustion; do not promise either exhaustion or redemption. + +## Final product gate + +Before marking the integration issue complete, prove: + +- The built binary serves the built Vue assets and survives a SQLite reopen. +- Two users have isolated follows/read state over shared deduplicated works. +- First admin, login/logout, recovery, OIDC, and trusted-header modes behave under + both valid and hostile inputs using local identity-provider fixtures. +- An offline seeded smoke drives onboarding, search/import preview, confirmation, + baseline sync, restart/resume, incremental sync, review, undo, reading, and export + through the real HTTP interface. Tests do not make public-provider calls. +- Injected 429, banned/auth responses, daily exhaustion, redirect rejection, and + restart during cooldown prove the provider safeguards with a controlled clock. +- Migrations, race tests, parser fuzz smoke, frontend type/build gates, and a + backup/restore into a fresh data directory pass. +- README states exact setup, disabled-provider defaults, key/contact requirements, + real-provider enablement, backup/restore, and limits. The shipped app does not + contain the design tool runtime, private records, tokens, or placeholder features. + +Report unmet boxes as unmet. The bounded live-provider check is part of the +sprint acceptance allowance above. Justin's visual review, private-corpus/import +validation, and the existing 2-3 week household shadow run remain acceptance +before production use; the shadow run cannot finish within this sprint. + +## Owner go + +The launch packet must name the pushed baseline commit, runner commit/release, +actual saved plan ID, full resolved role table and caps, image checks, and the +integration issue. It must show any failed preflight rather than hiding it. + +A launch go approves this product scope and bundled defaults, the six-request +public-provider allowance, use of the available Codex subscription at `xhigh`, +and launch of that exact plan. Reset-credit redemption needs explicit approval +of the separate single-use trigger policy. Neither go authorizes a main merge, +publication, production deployment, real-data import, bulk provider crawl, +or a second night. Start launch/monitoring in a fresh session from the approval +issue after the matching runner/plugin and reset supervisor preflights pass. diff --git a/TASKS.md b/TASKS.md deleted file mode 100644 index a95bb6e..0000000 --- a/TASKS.md +++ /dev/null @@ -1,95 +0,0 @@ -# Scholarr task cards - -Work in order. A card is complete only when its acceptance criteria are met. Specs are reviewed -and frozen one at a time before implementation begins. - -## Card 0: publish the clean repository **COMPLETE 2026-07-22** - -**Goal:** publish this clean-history bootstrap as the public project. - -The legacy GitHub repository was preserved as `JustinZeus/scholarr-legacy`. The clean bootstrap is -public at `JustinZeus/scholarr` on GitHub, with the same history copied to Forgejo. GitHub is the -primary remote. - -**Acceptance:** public `justinzeus/scholarr` repositories exist on GitHub (primary) and Forgejo -(copy), both point at this clean history, no legacy history is present, and no secret or work -identity appears anywhere. - -## Card 1: freeze the data model spec **REVIEW-READY 2026-07-23, NOT FROZEN** - -**Goal:** specify global author/publication identity, per-user follows/read state, review records, -and merge/undo behavior. - -Inputs: the project identity decision record, the legacy schema concepts worth salvaging, and the -global `FollowedAuthor` plus `AuthorSourceIdentity` direction. - -**Acceptance:** `docs/specs/data-model.md` is owner-reviewed and marked frozen; constraints, -migrations, duplicate prevention, audit/undo behavior, and every remaining open question are -explicit. - -**Draft result:** [`docs/specs/data-model.md`](docs/specs/data-model.md) is complete for owner and -independent review. It records fixed decisions, proposes a concrete SQLite model, and isolates five -owner decisions. Acceptance remains open until the owner reviews those decisions and explicitly -marks the spec frozen. Do not begin implementation. - -## Card 2: freeze provider contracts and health - -**Goal:** specify sanctioned API contracts, etiquette, persisted rate clocks/cooldowns, and -user-visible source health for OpenAlex, Crossref, arXiv, Unpaywall, and ORCID. - -**Acceptance:** `docs/specs/providers.md` is owner-reviewed and marked frozen; keyed and anonymous -lanes, contact identity, retry semantics, restart behavior, fixture boundaries, and open questions -are explicit. - -## Card 3: freeze the sync engine and run lifecycle - -**Goal:** define safe, resumable acquisition within SQLite's concurrency model. - -**Acceptance:** `docs/specs/sync-engine.md` is owner-reviewed and marked frozen; it defines one -active run per user, idempotent reruns, additive-only baseline import, resumability, writer -serialization, dry runs, and failure recovery. - -## Card 4: freeze PDF resolution - -**Goal:** define legal open-access discovery and provenance. - -**Acceptance:** `docs/specs/pdf-resolution.md` is owner-reviewed and marked frozen; source order, -license/provenance handling, retry behavior, and the absence of all Google access are explicit. - -## Card 5: freeze auth - -**Goal:** define internal, generic OIDC, and trusted-header auth without provider-specific coupling. - -**Acceptance:** `docs/specs/auth.md` is owner-reviewed and marked frozen; identity linking, roles, -session/security behavior, first-admin recovery, and mode-specific trust boundaries are explicit. - -## Card 6: freeze the config schema - -**Goal:** define one declarative configuration surface for the service. - -**Acceptance:** `docs/specs/config.md` is owner-reviewed and marked frozen; defaults, validation, -secret references, provider settings, reload/restart behavior, and upgrade compatibility are -explicit. - -## Card 7: freeze onboarding UX - -**Goal:** make first run straightforward: search or paste Scholar URLs, review candidates, confirm, -and begin tracking. - -**Acceptance:** `docs/specs/onboarding.md` is owner-reviewed and marked frozen; empty, ambiguous, -cooldown, partial-success, and unresolved-shell paths match the frozen UI and honest calibration -expectations. - -## Card 8: choose the public license - -**Goal:** make the repository genuinely open source under a consciously selected OSI license. - -**Acceptance:** the owner selects the license, `LICENSE` is added with correct copyright identity, -and README/project metadata are updated. - -## Card 9: plan build phases - -**Goal:** split implementation into reviewable vertical slices against the frozen specs and UI. - -**Acceptance:** build cards cover schema/migrations, providers, sync, auth, API, Vue port, -packaging, backup/restore, and release gates. Every card names its tests and demo proof. diff --git a/docs/specs/README.md b/docs/specs/README.md index 29add5e..03df895 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -1,16 +1,14 @@ # Architecture specs -Specs are frozen in the order listed in `TASKS.md`. Each spec must: +For the integrated product sprint, read [SPRINT.md](../../SPRINT.md) first. +The owner approves the proposed decisions together with the concrete nightly +plan; preparation alone does not authorize implementation. -- distinguish locked decisions from open questions; -- define data and trust boundaries; -- state failure, recovery, migration, and observability behavior; -- identify deterministic fixtures and acceptance tests; -- receive explicit owner review before being marked frozen. +Detailed references still define data/trust boundaries, failure recovery, +migrations, and deterministic acceptance. They do not create a second freeze +queue for an approved sprint. -No implementation for a subsystem starts before its spec is frozen. +## Detailed references -## Current review - -- [Data model](data-model.md): review-ready draft dated 2026-07-23, not frozen. Owner decisions - D1 through D5 remain open. +- [Data model](data-model.md): the reviewed table contracts, proposed D1-D7 + defaults, and 22 falsifiable acceptance cases adopted by an approved sprint. diff --git a/docs/specs/data-model.md b/docs/specs/data-model.md index da77a68..26375ce 100644 --- a/docs/specs/data-model.md +++ b/docs/specs/data-model.md @@ -1,19 +1,19 @@ # Scholarr data model -**Status:** DRAFT FOR OWNER REVIEW, 2026-07-23. This spec is review-ready and is not frozen. -No implementation is authorized by this document. This revision (2026-07-23) incorporates the -Card 1 architecture review round; owner decisions are now D1 through D7. +**Status:** historical reviewed draft; this document is not independently frozen. +For an explicitly owner-approved sprint, [SPRINT.md](../../SPRINT.md) selects +the proposed defaults and supplies execution authority. Preparation alone does +not authorize implementation. -This is Card 1 in [`TASKS.md`](../../TASKS.md). It defines the persistent domain model for -global author and publication identity, per-user follows and reading state, review records, -duplicate prevention, merge and undo, and migration from the legacy Scholarr database. +This detailed reference defines persistent author/publication identity, per-user +state, duplicate prevention, audit/undo, and legacy migration. ## Status language - **DECIDED** records a project decision that this spec must not reopen. -- **PROPOSED** is the concrete default in this draft. It becomes decided only when the owner - approves and freezes the spec. -- **OPEN Dn** identifies an owner decision listed in [Owner decisions](#owner-decisions). +- **PROPOSED** records the reviewed default selected by an owner-approved + `SPRINT.md`; without that approval it remains a proposal. +- **OPEN Dn** identifies the historical decision entry below. ## Scope and boundaries @@ -813,6 +813,6 @@ The draft recommends one answer for each unresolved choice: | D6 | Global identity authority | Restrict merges and negative-identity tombstones to an administrator role; non-admin review actions affect only that user's own follow and library rows. | | D7 | Deletion vs re-ingestion | Hard purge with no resurrection tombstone; sanctioned re-ingest of the same strong identifier recreates a new canonical row, all-`legacy_import` provenance does not. | -Freezing this spec means the owner has answered D1 through D7, approved any resulting edits, and -explicitly changed the status at the top to `FROZEN` with the approval date. Until then, no schema -or implementation work begins. +The historical process required separate answers and an explicit freeze. +An owner-approved [SPRINT.md](../../SPRINT.md) supersedes that process for the +integrated sprint and adopts the defaults together. Preparation grants no execution authority. diff --git a/runbooks/rewrite/charter.md b/runbooks/rewrite/charter.md index 0ba29f5..bbd55db 100644 --- a/runbooks/rewrite/charter.md +++ b/runbooks/rewrite/charter.md @@ -141,14 +141,15 @@ adversarial dimensions over the `e36e486` draft) returned **NO-FREEZE with seven blockers**, all spec-text gaps rather than design flaws, and the revision applying every finding is committed at `a8c5939`. -Freezing means the owner has answered **D1 through D7** (D2 split into D2a-D2d), -approved any resulting edits, and marked the spec frozen. **No implementation and -no Card C.2 work begins before that freeze.** That rule is not enforced by -convention any more: the implementation issue carries real Forgejo dependencies -on all seven decision issues, so Forgejo itself refuses to close it while any -decision is open. +The historical process required separate answers to **D1 through D7** and a +spec freeze before any later work. Those issues preserve the decision evidence. -The **UI spec is already frozen** and needs no freeze card. +For an explicitly owner-approved integrated sprint, `SPRINT.md` replaces the +separate freeze gates and adopts its proposed defaults together. The owner must +approve the concrete nightly plan and explicitly authorize launch first. +Preparation and a baseline commit alone do not authorize implementation. + +The **UI spec is already frozen**. Its appearance contract still binds the sprint. ## Related -- 2.49.1 From 44e7518071206918646487f63001d27a518321a9 Mon Sep 17 00:00:00 2001 From: justinzeus Date: Sat, 3 Oct 2026 00:39:07 +0200 Subject: [PATCH 2/3] Record the three-hour reset wind-down margin --- SPRINT.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/SPRINT.md b/SPRINT.md index d862204..b7229f3 100644 --- a/SPRINT.md +++ b/SPRINT.md @@ -264,12 +264,18 @@ credit; persist one idempotency UUID before redemption. Never expose OAuth credentials to runner images, prompts, logs, or repository files. The proposed trigger is a quiescent run with at least 99% weekly usage, or the -expiry guard at **2026-10-04T05:17:19.368242Z** after safely stopping work. +final expiry guard at **2026-10-04T05:17:19.368242Z**. Justin selected a separate +three-hour safe wind-down margin: request a safe stop of that exact run by +**2026-10-04T02:32:19.368242Z** if it is still active. Do not redeem at wind-down +alone when usage is below 99%; wait for the final guard and backend eligibility. +Require a terminal run receipt and positive absence of its systemd/Docker work. +The margin reduces expiry risk but cannot guarantee stuck verification settles. + The expiry guard explicitly trades remaining old-window quota for preserving -the expiring reset. Confirm backend eligibility first; a rejected or ambiguous -redemption must not select another credit or create a new UUID. Record and -reconcile the result using the same request ID. Redeem at most one credit; -never automatically launch a second night or spend the newly reset window. +the expiring reset. A rejected, ineligible, or ambiguous redemption must not +select another credit, create a new UUID, or be reported as successful. Record +and reconcile using the same request ID. Redeem at most one credit; never +automatically launch a second night or spend the newly reset window. This policy maximizes useful work, not artificial token burn. Subscription throttling, an early finished product, or an unmet backend reset prerequisite -- 2.49.1 From 19c33057b25c6bf7faa47f7befb585105673113d Mon Sep 17 00:00:00 2001 From: justinzeus Date: Sat, 3 Oct 2026 12:16:15 +0200 Subject: [PATCH 3/3] Prioritize the expiring reset window without reducing product scope --- SPRINT.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/SPRINT.md b/SPRINT.md index b7229f3..21e2181 100644 --- a/SPRINT.md +++ b/SPRINT.md @@ -4,7 +4,7 @@ Justin authorized preparation, issue triage, and a new baseline commit. He has not authorized launch. Review this document as the single product and execution -reference for the proposed 24-hour run. Approval tracking: justinzeus/scholarr#9. +reference for the proposed expiry-priority run. Approval tracking: justinzeus/scholarr#9. Runner preparation: justinzeus/nightly#92; reset supervisor: justinzeus/nightly#93. An explicit owner go for the concrete plan approves the proposed defaults below @@ -31,7 +31,9 @@ The target after the run is one integrated, runnable product branch and one reviewed PR, with executable acceptance evidence and a usable installation path. A pile of independent PRs, a static UI demonstration, mock-only provider clients, or a service with omitted authentication modes does not meet the target. -Twenty-four hours is a delivery target, not a promise that unfinished work is done. +The original target was twenty-four hours. Justin selected expiry priority once +the remaining credit window became shorter: reduce the time budget, not these +product requirements. An unfinished gate is not completed delivery. The run does not deploy, merge into main, publish a release, import the real household database, or replace the existing workflow. Justin still performs @@ -227,7 +229,7 @@ not exercise a sanitized copy; list that validation as owner-side acceptance. | Provider/model | `openai-codex/gpt-6.1-sol` in scope, writer, escalation, spec review, and standards review. | | Thinking | `xhigh` in all five slots. Never change this sprint to `max`. | | Codex budget/floor | 100 window points / 0% remaining reserve; the available subscription balance is the practical bound. | -| Night/issue wall | 1,440 active minutes each; subscription pauses do not consume the issue wall, but do consume the night wall. | +| Night/issue wall | At most 1,440 active minutes each, shortened for the actual go window to leave the 120-minute session allowance before the three-hour wind-down. Subscription pauses do not consume issue wall but do consume night wall. | | Issue token guard | 100 million non-cached tokens; finite secondary guard, not a target. | | Role session | Existing 120-minute cap and idle detection. | | Concurrency | One project issue; the two independent review roles may run together. | @@ -235,15 +237,20 @@ not exercise a sanitized copy; list that validation as owner-side acceptance. | Data-provider requests | Local fixtures in writer/review sandboxes; separate operator acceptance allowance of six real requests, never a crawl. | | End | All acceptance complete, quota exhaustion, wall cutoff, or a reported safety/stuck condition. No filler work to burn remaining quota. | -Nightly checks the night wall between role sessions. A role or verification -already running may finish after the 24-hour scheduler cutoff; this plan does not -claim an exact process-kill deadline. Only reviewed, verified commits count as -output. Exhausting quota may interrupt an item and discard its uncommitted work; -completed items remain on the integration branch. +The final approval packet records the requested wall and latest safe go time, +derived from the pinned wind-down deadline minus that wall and the 120-minute +session allowance. If owner go arrives later, do not claim the original packet +still fits: re-evaluate timing before launch or arming. Nightly's walls are soft; +a running role or verification may finish after cutoff, and a draining stop is +honored between work items. This is not a process-kill deadline or a guarantee +of redemption. Only reviewed, verified commits count as output. Quota exhaustion +may interrupt an item and discard uncommitted work; completed items remain on +the integration branch. Operator-owned ceiling configuration permits the larger wall/token/budget values; -ordinary presets keep their existing defaults. The Scholarr preset requests the -larger values explicitly. Freeze the runner settings in the saved draft and keep +ordinary presets keep their existing defaults. The Scholarr preset explicitly +requests the selected expiry-priority walls and larger token/budget guards. +Freeze the runner settings in the saved draft and keep no model fallback. Approval builds/checks the Go and Node image without calling a model. Launch spends subscription quota and requires Justin's explicit go on the resolved plan, not merely permission to prepare it. -- 2.49.1