From 0e22839ae4197a20f6a4b98cf438e0132f26deaf Mon Sep 17 00:00:00 2001 From: justinzeus Date: Sat, 3 Oct 2026 00:02:22 +0200 Subject: [PATCH 1/3] Define the integrated Scholarr product sprint --- AGENTS.md | 24 ++- README.md | 14 +- SPRINT.md | 314 ++++++++++++++++++++++++++++++++++++ TASKS.md | 95 ----------- docs/specs/README.md | 20 ++- docs/specs/data-model.md | 24 +-- runbooks/rewrite/charter.md | 15 +- 7 files changed, 361 insertions(+), 145 deletions(-) create mode 100644 SPRINT.md delete mode 100644 TASKS.md diff --git a/AGENTS.md b/AGENTS.md index e1eadcc..e1a4367 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # Scholarr agent brief -This file is the canonical brief for every coding agent working in this repository. Read it and -`TASKS.md` before making changes. +For sprint work, read `SPRINT.md` before changing the repository. It is the +single product, decision, provider-safety, and execution reference. ## Product @@ -41,14 +41,11 @@ Do not silently change a locked decision. Record a proposed change as open and a ## Decision record -On tank, the authoritative project runbook is: +`SPRINT.md` governs an explicitly owner-approved sprint. The historical charter +and identity/calibration/testing records live in `runbooks/rewrite/`. -`/opt/stacks/server-docs/runbooks/scholarr-rewrite/README.md` - -It owns doctrine, locked decisions, empirical calibration results, and project state. Its -companion files own the identity model and testing strategy. This brief is self-contained for -agents without tank access, but any state or design decision reached on tank must also be written -to that runbook in the same session. +Track acceptance and next actions in Forgejo. Record sprint implementation +decisions in `SPRINT.md`; do not add another plan or a separate freeze queue. ## Frozen UI contract @@ -67,10 +64,11 @@ are expected. ## Work process -- Work through `TASKS.md` in order. Each task has a goal and acceptance criteria. -- Freeze one spec at a time. Decided items are labeled decided; open items remain open for owner - review. -- Do not start implementation for an area whose spec is not frozen. +- Work the integration issue's acceptance items in order, against `SPRINT.md`. +- Preparation does not authorize execution. Start only after the owner approves + the concrete nightly plan and explicitly says to launch. +- An approved sprint replaces separate spec-freeze ceremonies with the bundled + defaults in `SPRINT.md`. Preserve locked requirements; report conflicts. - Keep commits focused and use plain imperative commit messages without trailers. - Stage only files you changed. - Do not use em dashes in prose, code comments, commit messages, or user-facing copy. diff --git a/README.md b/README.md index b83dea5..03fec63 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,8 @@ Scholarr is a self-hosted author watchlist and publication tracker. Follow researchers, learn when they publish, and find legal open-access copies of their work. -The rewrite is pre-alpha. Its product direction and interface are settled, and the architecture -specs are being frozen before implementation begins. +The rewrite is pre-alpha. [SPRINT.md](SPRINT.md) is the single reference for its +proposed integrated product sprint; preparation does not authorize a launch. ## Non-negotiable rule @@ -21,13 +21,13 @@ from documented APIs such as OpenAlex, Crossref, arXiv, Unpaywall, and ORCID. - Internal auth, generic OIDC, and trusted-header auth in v1 - Config-driven operation and simple first-run onboarding -See [AGENTS.md](AGENTS.md) for the full project constraints and [TASKS.md](TASKS.md) for the -ordered work queue. +Read [SPRINT.md](SPRINT.md) for scope, decisions, provider safety, and acceptance. +[AGENTS.md](AGENTS.md) supplies the repository constraints. ## Repositories - Primary: -- Forgejo copy: +- Forgejo tracker and sprint delivery: - Preserved legacy implementation: ## Frozen interface contract @@ -38,5 +38,5 @@ are the source for the Vue port, not an application implementation. ## License -The license has not been selected yet. Until a LICENSE file is added, normal copyright rules -apply. License selection is an explicit project decision in [TASKS.md](TASKS.md). +The sprint proposes MIT, subject to owner approval of [SPRINT.md](SPRINT.md). +Until a LICENSE file is added, normal copyright rules apply. diff --git a/SPRINT.md b/SPRINT.md new file mode 100644 index 0000000..d862204 --- /dev/null +++ b/SPRINT.md @@ -0,0 +1,314 @@ +# Scholarr product sprint + +## Approval and authority + +Justin authorized preparation, issue triage, and a new baseline commit. He has +not authorized launch. Review this document as the single product and execution +reference for the proposed 24-hour run. Approval tracking: justinzeus/scholarr#9. +Runner preparation: justinzeus/nightly#92; reset supervisor: justinzeus/nightly#93. + +An explicit owner go for the concrete plan approves the proposed defaults below +and replaces the old, separate owner-freeze gates for this sprint. The agents +may then choose implementation details and record their rationale here without +stopping for another spec ceremony. They must preserve the locked requirements. +A requirement conflict, missing credential needed for a real feature, or unsafe +provider operation remains a blocker; agents must report it rather than invent +a fallback or claim completion. Preparation alone does not activate this authority. + +The reset preserves Git history, the frozen design, and prior decision evidence. +It supplies one new starting commit on an isolated branch. It does not erase the +legacy repository, dirty local files, old decision issues, or production data. + +## Intended result + +Build a self-hosted author watchlist and publication tracker that Justin's dad +can use daily and other households can install. A reader follows researchers, +sees their publications, tracks read state, resolves ambiguous identities, and +finds legal open-access copies. An administrator can understand provider health, +recover access, audit changes, and back up or restore the installation. + +The target after the run is one integrated, runnable product branch and one +reviewed PR, with executable acceptance evidence and a usable installation path. +A pile of independent PRs, a static UI demonstration, mock-only provider clients, +or a service with omitted authentication modes does not meet the target. +Twenty-four hours is a delivery target, not a promise that unfinished work is done. + +The run does not deploy, merge into main, publish a release, import the real +household database, or replace the existing workflow. Justin still performs +manual UI approval and the real-data shadow period before a production cutover. + +## Locked requirements + +- Go backend, Vue 3 frontend, SQLite only. Use a serialized writer, foreign keys, + WAL, numbered migrations, and restart-safe persistence. +- A source-agnostic `FollowedAuthor` is the author root. Names are display/search + labels, never identity keys. Preserve external source identities and provenance. +- Share deduplicated authors and publications globally. Keep follows, read state, + visibility, and user preferences private to their user. +- Support internal username/password, generic OIDC, and trusted-header auth. + Internal auth remains the zero-dependency default and recovery path. +- Use one validated declarative configuration, environment-variable references + for secrets, safe first-admin setup, and a straightforward first-run flow. +- Use the frozen `design/DESIGN.md`, `design/tokens.css`, and bundled assets. + Implement dashboard, publication list/detail, author list/detail, review, + author search/add, bulk import, sources, activity, settings, login, onboarding, + and not-found behavior. Keep loading, empty, populated, and degraded states + honest, with responsive navigation, keyboard access, and light/dark themes. +- Serve system fonts and bundled assets locally. The design proof's + `support.js` and synthetic example records do not ship as product behavior. +- Show source freshness and confidence words rather than invented certainty or + raw match percentages. Keep new-publication counts scoped to the current user. +- Preview bulk imports and destructive admin actions before applying them. + Audit consequential writes and support conflict-checked undo. +- Implement documented clients for OpenAlex, Crossref, arXiv, Unpaywall, and + ORCID. Missing keys disable that client's network lane with an actionable + health state; they do not authorize anonymous scraping or a substitute API. +- Zero application-initiated Google contact. Scholar URLs/IDs are inert import + metadata or human-clickable links. Parse them locally; never dereference them. + No scraping, CAPTCHA solving, anti-bot evasion, or Semantic Scholar v1 expansion. + +The data-model draft supplies detailed table contracts and its 22 falsifiable +acceptance cases. The frozen visual design supplies the appearance contract. +This document controls sprint scope and defaults where the old process differs. + +## Proposed decisions approved together at launch + +| Decision | Sprint default | +|---|---| +| D1: public identity | Internal integer joins; immutable UUIDv7 public IDs only on externally referenced or merge-aliased rows. | +| D2a: unfollow | Hide records without an active follow path; retain read/favorite state and restore it on refollow. | +| D2b: legacy favorites | Retain migrated favorite state for lossless import/export; do not add a favorite UI outside the frozen design. | +| D2c: legacy reading | Treat a publication as read if any legacy link for that user says read. | +| D2d: NEW | Derive at read time from first incremental discovery within 14 days; baseline, manual, and legacy imports are not NEW. | +| D3: attachment | Reuse exact identities, explicit provider crosswalks, and completed high-confidence calibration evidence. Review weaker or conflicting evidence; names alone never attach or merge. | +| D4: undo | No arbitrary expiry while recorded row-version preconditions hold. A conflict prevents the entire undo. | +| D5: orphans | Retain automatically. Admin garbage collection requires a backup, dry-run preview, reference checks, and audit. | +| D6: shared identity | Only administrators merge shared identity or write global negative-identity evidence. Reader decisions remain per-user. | +| D7: purge | Hard purge with no resurrection tombstone. A later sanctioned strong-ID ingest can create a new canonical row; legacy-only data cannot recreate it automatically. | +| Distribution | A self-contained Go binary with embedded built Vue assets, plus a non-root container image. Compose is an installation example, not a live deployment. | +| License | Propose MIT with Justin's personal copyright identity. Add the license only after the owner approves this packet. | +| Acquisition | OpenAlex is primary. Crossref/arXiv add work metadata; ORCID anchors identity; Unpaywall resolves legal OA locations. Persist cursors, deduplicate, and retain provenance. | +| Sync | One active run per user, shared provider budgets, idempotent resume, additive baseline import, scheduled and manual sync through the same queue. | +| Matching | Exact strong identifiers first; weak/ambiguous candidates go to review. Keep recorded calibration expectations, using public-safe fixtures only. | +| OA | Store location, license, and provenance. Expose a legal human-clickable location; do not fetch PDFs or arbitrary publisher URLs during the sprint. | +| Config changes | Validate before applying. Document restart requirements; do not add hot reload for settings whose safe transition is unclear. | + +Security details follow maintained Go libraries: memory-hard password hashing, +secure expiring sessions, CSRF defenses, bounded login attempts, OIDC issuer and +signature validation with state/nonce and PKCE, and trusted headers accepted only +from explicitly trusted proxies. Reject direct-header spoofing. Isolate user +queries and authorize admin operations on the server, not only in the UI. + +## Provider safety and the sprint's network boundary + +Spend Codex quota on coding, testing, and reviewing, not on harvesting papers. +Automated tests and the repeatable integration smoke use deterministic local +HTTP fixtures. Production clients must be real. Justin additionally authorized +a bounded live-provider acceptance check, not bulk acquisition. + +After fixture tests prove throttling and circuit breaking, an operator-owned +host check may issue at most **two requests each** to OpenAlex and Crossref, +**one each** to arXiv and Unpaywall, and **zero** to ORCID until its credentials +and non-commercial eligibility are confirmed. Six requests total, including +retries; a failure does not replenish the allowance. Use public example works, +the operator's existing contact identity, and the escrowed OpenAlex key. +Keep credentials, private calibration caches, and the household dump out of +writer sandboxes. Do not perform these checks during preparation. + +Record actual endpoint, status, timestamp, and remaining allowance without keys +or full private payloads. Skip a provider whose prerequisite is missing and +report it as unverified, not passed. A 429/auth/ban response stops that provider +immediately. Honor the conservative spacing below even for this small check. +The application defaults acquisition to disabled until the operator explicitly +enables configured providers. Installation never starts a historical crawl; +confirmation queues limited work and never bypasses the scheduler. + +The shipped provider scheduler must enforce: + +- One outstanding request per provider and no burst allowance. Share the limit + across users, search, sync, OA lookups, retries, and manual refreshes. +- Persist the next-request time, cooldown, daily allowance consumption, and + in-progress work before issuing requests. Restart never clears a cooldown or + resets a daily budget. A daily allowance uses the provider's documented reset; + if none exists, enforce a rolling 24-hour window. +- Every attempted request, including retries, consumes the budget. Cache by + strong identifiers/query parameters; coalesce duplicate queued work. Expose + queued, cooling, failed, and disabled states without retry loops in the UI. +- A 429 immediately opens the shared provider circuit. Honor a valid + `Retry-After` date/duration, then permit at most one rate-limited probe. + Missing or invalid `Retry-After` requires an administrator-set cooldown; + do not guess a short retry interval. A second 429 in a rolling 24-hour period + requires manual re-arm after the administrator reviews the cause. +- On 401/403, a ban/challenge response, or an unexpected redirect, stop the lane + and require operator attention. Do not rotate keys, IPs, accounts, or user agents. +- Treat 503 as a provider cooldown, since some APIs use it for burst limiting. + Other transient failures get at most two retries with exponential backoff and + jitter under the same budgets. Malformed responses stop the work item; they + do not trigger an endpoint retry loop. Retain resumable work and surface errors. +- Use honest identification and configured contact details where the provider + requires them. An absent required identity or key fails closed. +- Restrict production acquisition to the provider's approved HTTPS host and path. + Reject arbitrary base URLs, Google hosts, private/link-local targets, and + unapproved redirects. Local fixture endpoints exist only in test configuration. +- Persist budgets per provider credential/IP scope; an application limiter cannot + account for other services using tank's IP. Before live enablement, the owner + must allocate that shared-IP allowance. Use the smaller of published limits, + this product's conservative default, and the allocated operator allowance. + +No rate limit can guarantee that a provider will never block an IP. The zero-live +sprint budget removes this run's data-API traffic; conservative client limits and +circuit breakers constrain a later enabled installation. + +### Conservative live-enable defaults + +These are local engineering limits for a later owner-approved connection check, +not promised provider entitlements or the sprint's budget. The sprint budget +remains zero. One outstanding request and no burst apply to each row. +Official documentation was checked on 2026-10-02. + +| Provider | Local check allowance | Access and policy constraints | +|---|---|---| +| OpenAlex | 1 request per 10 seconds; 20 attempts/day; $0.05/day spend cap | Current docs conflict about keyless access. Require an owner-managed API key and policy clarification before live enablement. Price requests by operation; request count alone is not the daily budget. Exclude content/PDF endpoints. [Authentication](https://help.openalex.org/api/authentication/), [costs](https://help.openalex.org/access/example-costs/), [deprecations](https://help.openalex.org/api/deprecations/). | +| Crossref | 1 request per 10 seconds; 80 attempts/day | Use polite `mailto` and an identifying User-Agent. Honor rate-interval and concurrency response headers; do not interpret pool counts as a fixed RPS entitlement. No published daily cap was established. [Access and authentication](https://www.crossref.org/documentation/retrieve-metadata/rest-api/access-and-authentication/). | +| arXiv | 1 request per 6 seconds; 80 attempts/day | Published terms require at least 3 seconds between calls and one connection across all machines under your control. Use the sanctioned query route, not paper scraping. Validate the approved TLS endpoint before live enablement. [Terms](https://info.arxiv.org/help/api/tou.html), [manual](https://info.arxiv.org/help/api/user-manual.html). | +| Unpaywall | 1 request per 10 seconds; 80 attempts/day | DOI lookup requires a configured email. Docs ask for at most 100,000 calls/day but state no RPS or quota-accounting key. The search route retired on 2026-09-18; do not implement it. [API](https://unpaywall.org/products/api). | +| ORCID | Disabled; conditional check: 1 request per 10 seconds, 80 reads/day | Require owner confirmation of non-commercial eligibility and an individually registered public client. Implement client-credentials OAuth and fixtures; do not automate account registration. Anonymous/public limits and 503 burst responses are not an invitation to use their burst allowance. [Limits](https://info.orcid.org/ufaqs/what-are-the-api-limits/), [registration](https://info.orcid.org/documentation/integration-guide/registering-a-public-api-client/), [public-client terms](https://info.orcid.org/public-client-terms-of-service/). | + +OpenAlex's published 100-RPS ceiling and daily dollar budgets, and ORCID's +published anonymous/public read quotas, do not supersede these smaller local +allowances. ORCID Public API terms prohibit use connected with a +revenue-generating product/service. Public release must document this constraint +and leave that lane disabled unless the operator meets the terms or has an +approved Member API path. No paid provider use is authorized by this sprint. + + +## Build order and the acceptance ledger + +Use **one integration issue**, with one verifiable behavior per acceptance box. +Nightly works its boxes sequentially on the same branch; each later item inherits +the previous verified commit. Both review roles run on each candidate. Keep the +specification in this document and evidence/ticks in Forgejo, rather than copying +another plan into each role prompt. + +Work in this dependency order: + +1. Runnable Go process, validated config, SQLite lifecycle, health/readiness, and + a default gate that runs offline. The first real feature supplies the Go module; + do not spend a separate item on an empty scaffold. +2. Numbered schema migrations, database invariants, public identity, and the + data-model's independent acceptance cases. Exercise migrations up/down and + serialized concurrent writes before acquisition code uses them. +3. Safe first-admin creation, internal authentication/session recovery, tenant + isolation, generic OIDC, and trusted-proxy authentication. +4. The shared persisted provider limiter and block/cooldown behavior, then one + documented provider adapter at a time against local fixtures. +5. Exact identity matching, conservative candidate review, author follow/unfollow, + publication dedup/provenance, per-user reading, and NEW projection. +6. Restart-safe queued baseline/incremental sync, partial failure, cancellation, + dry-run import, audited merge/undo, legal OA resolution, and export. +7. Vue's real API integration and the frozen screens, each a separate checked + behavior. No production fixture data and no missing buttons hidden as polish. +8. Safe installation, license, embedded assets/container packaging, offline demo, + backup/restore, operational documentation, and the final full-flow smoke. + +Every implementation item states its check. Keep tests focused on observable +behavior, invariant violations, failure transitions, or a plausible regression. +Use a fake clock for cooldowns, not real sleeps. Use the race detector and parser +fuzz smoke in the final gate. Synthetic fixtures carry no real household identity. +Do not invent a claim that the private calibration corpus passed if the run did +not exercise a sanitized copy; list that validation as owner-side acceptance. + +## Execution envelope + +| Setting | Proposed value | +|---|---| +| Project | Scholarr only; base is the preserved sprint-baseline branch. | +| Work | One integrated product issue, ordered by the build ledger. | +| Provider/model | `openai-codex/gpt-6.1-sol` in scope, writer, escalation, spec review, and standards review. | +| Thinking | `xhigh` in all five slots. Never change this sprint to `max`. | +| Codex budget/floor | 100 window points / 0% remaining reserve; the available subscription balance is the practical bound. | +| Night/issue wall | 1,440 active minutes each; subscription pauses do not consume the issue wall, but do consume the night wall. | +| Issue token guard | 100 million non-cached tokens; finite secondary guard, not a target. | +| Role session | Existing 120-minute cap and idle detection. | +| Concurrency | One project issue; the two independent review roles may run together. | +| Repairs/refusals/resources | Existing caps, sandbox isolation, admission reserves, and broker protections. | +| Data-provider requests | Local fixtures in writer/review sandboxes; separate operator acceptance allowance of six real requests, never a crawl. | +| End | All acceptance complete, quota exhaustion, wall cutoff, or a reported safety/stuck condition. No filler work to burn remaining quota. | + +Nightly checks the night wall between role sessions. A role or verification +already running may finish after the 24-hour scheduler cutoff; this plan does not +claim an exact process-kill deadline. Only reviewed, verified commits count as +output. Exhausting quota may interrupt an item and discard its uncommitted work; +completed items remain on the integration branch. + +Operator-owned ceiling configuration permits the larger wall/token/budget values; +ordinary presets keep their existing defaults. The Scholarr preset requests the +larger values explicitly. Freeze the runner settings in the saved draft and keep +no model fallback. Approval builds/checks the Go and Node image without calling a +model. Launch spends subscription quota and requires Justin's explicit go on the +resolved plan, not merely permission to prepare it. + +### Expiring Codex reset credit + +Read-only account inspection on 2026-10-02 found 98% of the weekly window +remaining, an ordinary weekly reset at **2026-10-09T21:10:30Z**, and three saved +reset credits. The earliest credit expires at +**2026-10-04T05:32:19.368242Z**. This saved credit, not the weekly rollover, is +the urgent deadline. Re-read live usage and credit details before approval. + +Preparation does not redeem a credit. A separate explicit owner go may arm a +host-only, single-use supervisor for the soonest-expiring available credit. +Use the official installed Codex app-server account/reset-credit protocol, not +cache invalidation or guessed private HTTP calls. Pin the expected account and +credit; persist one idempotency UUID before redemption. Never expose OAuth +credentials to runner images, prompts, logs, or repository files. + +The proposed trigger is a quiescent run with at least 99% weekly usage, or the +expiry guard at **2026-10-04T05:17:19.368242Z** after safely stopping work. +The expiry guard explicitly trades remaining old-window quota for preserving +the expiring reset. Confirm backend eligibility first; a rejected or ambiguous +redemption must not select another credit or create a new UUID. Record and +reconcile the result using the same request ID. Redeem at most one credit; +never automatically launch a second night or spend the newly reset window. + +This policy maximizes useful work, not artificial token burn. Subscription +throttling, an early finished product, or an unmet backend reset prerequisite +can prevent complete exhaustion; do not promise either exhaustion or redemption. + +## Final product gate + +Before marking the integration issue complete, prove: + +- The built binary serves the built Vue assets and survives a SQLite reopen. +- Two users have isolated follows/read state over shared deduplicated works. +- First admin, login/logout, recovery, OIDC, and trusted-header modes behave under + both valid and hostile inputs using local identity-provider fixtures. +- An offline seeded smoke drives onboarding, search/import preview, confirmation, + baseline sync, restart/resume, incremental sync, review, undo, reading, and export + through the real HTTP interface. Tests do not make public-provider calls. +- Injected 429, banned/auth responses, daily exhaustion, redirect rejection, and + restart during cooldown prove the provider safeguards with a controlled clock. +- Migrations, race tests, parser fuzz smoke, frontend type/build gates, and a + backup/restore into a fresh data directory pass. +- README states exact setup, disabled-provider defaults, key/contact requirements, + real-provider enablement, backup/restore, and limits. The shipped app does not + contain the design tool runtime, private records, tokens, or placeholder features. + +Report unmet boxes as unmet. The bounded live-provider check is part of the +sprint acceptance allowance above. Justin's visual review, private-corpus/import +validation, and the existing 2-3 week household shadow run remain acceptance +before production use; the shadow run cannot finish within this sprint. + +## Owner go + +The launch packet must name the pushed baseline commit, runner commit/release, +actual saved plan ID, full resolved role table and caps, image checks, and the +integration issue. It must show any failed preflight rather than hiding it. + +A launch go approves this product scope and bundled defaults, the six-request +public-provider allowance, use of the available Codex subscription at `xhigh`, +and launch of that exact plan. Reset-credit redemption needs explicit approval +of the separate single-use trigger policy. Neither go authorizes a main merge, +publication, production deployment, real-data import, bulk provider crawl, +or a second night. Start launch/monitoring in a fresh session from the approval +issue after the matching runner/plugin and reset supervisor preflights pass. diff --git a/TASKS.md b/TASKS.md deleted file mode 100644 index a95bb6e..0000000 --- a/TASKS.md +++ /dev/null @@ -1,95 +0,0 @@ -# Scholarr task cards - -Work in order. A card is complete only when its acceptance criteria are met. Specs are reviewed -and frozen one at a time before implementation begins. - -## Card 0: publish the clean repository **COMPLETE 2026-07-22** - -**Goal:** publish this clean-history bootstrap as the public project. - -The legacy GitHub repository was preserved as `JustinZeus/scholarr-legacy`. The clean bootstrap is -public at `JustinZeus/scholarr` on GitHub, with the same history copied to Forgejo. GitHub is the -primary remote. - -**Acceptance:** public `justinzeus/scholarr` repositories exist on GitHub (primary) and Forgejo -(copy), both point at this clean history, no legacy history is present, and no secret or work -identity appears anywhere. - -## Card 1: freeze the data model spec **REVIEW-READY 2026-07-23, NOT FROZEN** - -**Goal:** specify global author/publication identity, per-user follows/read state, review records, -and merge/undo behavior. - -Inputs: the project identity decision record, the legacy schema concepts worth salvaging, and the -global `FollowedAuthor` plus `AuthorSourceIdentity` direction. - -**Acceptance:** `docs/specs/data-model.md` is owner-reviewed and marked frozen; constraints, -migrations, duplicate prevention, audit/undo behavior, and every remaining open question are -explicit. - -**Draft result:** [`docs/specs/data-model.md`](docs/specs/data-model.md) is complete for owner and -independent review. It records fixed decisions, proposes a concrete SQLite model, and isolates five -owner decisions. Acceptance remains open until the owner reviews those decisions and explicitly -marks the spec frozen. Do not begin implementation. - -## Card 2: freeze provider contracts and health - -**Goal:** specify sanctioned API contracts, etiquette, persisted rate clocks/cooldowns, and -user-visible source health for OpenAlex, Crossref, arXiv, Unpaywall, and ORCID. - -**Acceptance:** `docs/specs/providers.md` is owner-reviewed and marked frozen; keyed and anonymous -lanes, contact identity, retry semantics, restart behavior, fixture boundaries, and open questions -are explicit. - -## Card 3: freeze the sync engine and run lifecycle - -**Goal:** define safe, resumable acquisition within SQLite's concurrency model. - -**Acceptance:** `docs/specs/sync-engine.md` is owner-reviewed and marked frozen; it defines one -active run per user, idempotent reruns, additive-only baseline import, resumability, writer -serialization, dry runs, and failure recovery. - -## Card 4: freeze PDF resolution - -**Goal:** define legal open-access discovery and provenance. - -**Acceptance:** `docs/specs/pdf-resolution.md` is owner-reviewed and marked frozen; source order, -license/provenance handling, retry behavior, and the absence of all Google access are explicit. - -## Card 5: freeze auth - -**Goal:** define internal, generic OIDC, and trusted-header auth without provider-specific coupling. - -**Acceptance:** `docs/specs/auth.md` is owner-reviewed and marked frozen; identity linking, roles, -session/security behavior, first-admin recovery, and mode-specific trust boundaries are explicit. - -## Card 6: freeze the config schema - -**Goal:** define one declarative configuration surface for the service. - -**Acceptance:** `docs/specs/config.md` is owner-reviewed and marked frozen; defaults, validation, -secret references, provider settings, reload/restart behavior, and upgrade compatibility are -explicit. - -## Card 7: freeze onboarding UX - -**Goal:** make first run straightforward: search or paste Scholar URLs, review candidates, confirm, -and begin tracking. - -**Acceptance:** `docs/specs/onboarding.md` is owner-reviewed and marked frozen; empty, ambiguous, -cooldown, partial-success, and unresolved-shell paths match the frozen UI and honest calibration -expectations. - -## Card 8: choose the public license - -**Goal:** make the repository genuinely open source under a consciously selected OSI license. - -**Acceptance:** the owner selects the license, `LICENSE` is added with correct copyright identity, -and README/project metadata are updated. - -## Card 9: plan build phases - -**Goal:** split implementation into reviewable vertical slices against the frozen specs and UI. - -**Acceptance:** build cards cover schema/migrations, providers, sync, auth, API, Vue port, -packaging, backup/restore, and release gates. Every card names its tests and demo proof. diff --git a/docs/specs/README.md b/docs/specs/README.md index 29add5e..03df895 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -1,16 +1,14 @@ # Architecture specs -Specs are frozen in the order listed in `TASKS.md`. Each spec must: +For the integrated product sprint, read [SPRINT.md](../../SPRINT.md) first. +The owner approves the proposed decisions together with the concrete nightly +plan; preparation alone does not authorize implementation. -- distinguish locked decisions from open questions; -- define data and trust boundaries; -- state failure, recovery, migration, and observability behavior; -- identify deterministic fixtures and acceptance tests; -- receive explicit owner review before being marked frozen. +Detailed references still define data/trust boundaries, failure recovery, +migrations, and deterministic acceptance. They do not create a second freeze +queue for an approved sprint. -No implementation for a subsystem starts before its spec is frozen. +## Detailed references -## Current review - -- [Data model](data-model.md): review-ready draft dated 2026-07-23, not frozen. Owner decisions - D1 through D5 remain open. +- [Data model](data-model.md): the reviewed table contracts, proposed D1-D7 + defaults, and 22 falsifiable acceptance cases adopted by an approved sprint. diff --git a/docs/specs/data-model.md b/docs/specs/data-model.md index da77a68..26375ce 100644 --- a/docs/specs/data-model.md +++ b/docs/specs/data-model.md @@ -1,19 +1,19 @@ # Scholarr data model -**Status:** DRAFT FOR OWNER REVIEW, 2026-07-23. This spec is review-ready and is not frozen. -No implementation is authorized by this document. This revision (2026-07-23) incorporates the -Card 1 architecture review round; owner decisions are now D1 through D7. +**Status:** historical reviewed draft; this document is not independently frozen. +For an explicitly owner-approved sprint, [SPRINT.md](../../SPRINT.md) selects +the proposed defaults and supplies execution authority. Preparation alone does +not authorize implementation. -This is Card 1 in [`TASKS.md`](../../TASKS.md). It defines the persistent domain model for -global author and publication identity, per-user follows and reading state, review records, -duplicate prevention, merge and undo, and migration from the legacy Scholarr database. +This detailed reference defines persistent author/publication identity, per-user +state, duplicate prevention, audit/undo, and legacy migration. ## Status language - **DECIDED** records a project decision that this spec must not reopen. -- **PROPOSED** is the concrete default in this draft. It becomes decided only when the owner - approves and freezes the spec. -- **OPEN Dn** identifies an owner decision listed in [Owner decisions](#owner-decisions). +- **PROPOSED** records the reviewed default selected by an owner-approved + `SPRINT.md`; without that approval it remains a proposal. +- **OPEN Dn** identifies the historical decision entry below. ## Scope and boundaries @@ -813,6 +813,6 @@ The draft recommends one answer for each unresolved choice: | D6 | Global identity authority | Restrict merges and negative-identity tombstones to an administrator role; non-admin review actions affect only that user's own follow and library rows. | | D7 | Deletion vs re-ingestion | Hard purge with no resurrection tombstone; sanctioned re-ingest of the same strong identifier recreates a new canonical row, all-`legacy_import` provenance does not. | -Freezing this spec means the owner has answered D1 through D7, approved any resulting edits, and -explicitly changed the status at the top to `FROZEN` with the approval date. Until then, no schema -or implementation work begins. +The historical process required separate answers and an explicit freeze. +An owner-approved [SPRINT.md](../../SPRINT.md) supersedes that process for the +integrated sprint and adopts the defaults together. Preparation grants no execution authority. diff --git a/runbooks/rewrite/charter.md b/runbooks/rewrite/charter.md index 0ba29f5..bbd55db 100644 --- a/runbooks/rewrite/charter.md +++ b/runbooks/rewrite/charter.md @@ -141,14 +141,15 @@ adversarial dimensions over the `e36e486` draft) returned **NO-FREEZE with seven blockers**, all spec-text gaps rather than design flaws, and the revision applying every finding is committed at `a8c5939`. -Freezing means the owner has answered **D1 through D7** (D2 split into D2a-D2d), -approved any resulting edits, and marked the spec frozen. **No implementation and -no Card C.2 work begins before that freeze.** That rule is not enforced by -convention any more: the implementation issue carries real Forgejo dependencies -on all seven decision issues, so Forgejo itself refuses to close it while any -decision is open. +The historical process required separate answers to **D1 through D7** and a +spec freeze before any later work. Those issues preserve the decision evidence. -The **UI spec is already frozen** and needs no freeze card. +For an explicitly owner-approved integrated sprint, `SPRINT.md` replaces the +separate freeze gates and adopts its proposed defaults together. The owner must +approve the concrete nightly plan and explicitly authorize launch first. +Preparation and a baseline commit alone do not authorize implementation. + +The **UI spec is already frozen**. Its appearance contract still binds the sprint. ## Related From 44e7518071206918646487f63001d27a518321a9 Mon Sep 17 00:00:00 2001 From: justinzeus Date: Sat, 3 Oct 2026 00:39:07 +0200 Subject: [PATCH 2/3] Record the three-hour reset wind-down margin --- SPRINT.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/SPRINT.md b/SPRINT.md index d862204..b7229f3 100644 --- a/SPRINT.md +++ b/SPRINT.md @@ -264,12 +264,18 @@ credit; persist one idempotency UUID before redemption. Never expose OAuth credentials to runner images, prompts, logs, or repository files. The proposed trigger is a quiescent run with at least 99% weekly usage, or the -expiry guard at **2026-10-04T05:17:19.368242Z** after safely stopping work. +final expiry guard at **2026-10-04T05:17:19.368242Z**. Justin selected a separate +three-hour safe wind-down margin: request a safe stop of that exact run by +**2026-10-04T02:32:19.368242Z** if it is still active. Do not redeem at wind-down +alone when usage is below 99%; wait for the final guard and backend eligibility. +Require a terminal run receipt and positive absence of its systemd/Docker work. +The margin reduces expiry risk but cannot guarantee stuck verification settles. + The expiry guard explicitly trades remaining old-window quota for preserving -the expiring reset. Confirm backend eligibility first; a rejected or ambiguous -redemption must not select another credit or create a new UUID. Record and -reconcile the result using the same request ID. Redeem at most one credit; -never automatically launch a second night or spend the newly reset window. +the expiring reset. A rejected, ineligible, or ambiguous redemption must not +select another credit, create a new UUID, or be reported as successful. Record +and reconcile using the same request ID. Redeem at most one credit; never +automatically launch a second night or spend the newly reset window. This policy maximizes useful work, not artificial token burn. Subscription throttling, an early finished product, or an unmet backend reset prerequisite From 19c33057b25c6bf7faa47f7befb585105673113d Mon Sep 17 00:00:00 2001 From: justinzeus Date: Sat, 3 Oct 2026 12:16:15 +0200 Subject: [PATCH 3/3] Prioritize the expiring reset window without reducing product scope --- SPRINT.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/SPRINT.md b/SPRINT.md index b7229f3..21e2181 100644 --- a/SPRINT.md +++ b/SPRINT.md @@ -4,7 +4,7 @@ Justin authorized preparation, issue triage, and a new baseline commit. He has not authorized launch. Review this document as the single product and execution -reference for the proposed 24-hour run. Approval tracking: justinzeus/scholarr#9. +reference for the proposed expiry-priority run. Approval tracking: justinzeus/scholarr#9. Runner preparation: justinzeus/nightly#92; reset supervisor: justinzeus/nightly#93. An explicit owner go for the concrete plan approves the proposed defaults below @@ -31,7 +31,9 @@ The target after the run is one integrated, runnable product branch and one reviewed PR, with executable acceptance evidence and a usable installation path. A pile of independent PRs, a static UI demonstration, mock-only provider clients, or a service with omitted authentication modes does not meet the target. -Twenty-four hours is a delivery target, not a promise that unfinished work is done. +The original target was twenty-four hours. Justin selected expiry priority once +the remaining credit window became shorter: reduce the time budget, not these +product requirements. An unfinished gate is not completed delivery. The run does not deploy, merge into main, publish a release, import the real household database, or replace the existing workflow. Justin still performs @@ -227,7 +229,7 @@ not exercise a sanitized copy; list that validation as owner-side acceptance. | Provider/model | `openai-codex/gpt-6.1-sol` in scope, writer, escalation, spec review, and standards review. | | Thinking | `xhigh` in all five slots. Never change this sprint to `max`. | | Codex budget/floor | 100 window points / 0% remaining reserve; the available subscription balance is the practical bound. | -| Night/issue wall | 1,440 active minutes each; subscription pauses do not consume the issue wall, but do consume the night wall. | +| Night/issue wall | At most 1,440 active minutes each, shortened for the actual go window to leave the 120-minute session allowance before the three-hour wind-down. Subscription pauses do not consume issue wall but do consume night wall. | | Issue token guard | 100 million non-cached tokens; finite secondary guard, not a target. | | Role session | Existing 120-minute cap and idle detection. | | Concurrency | One project issue; the two independent review roles may run together. | @@ -235,15 +237,20 @@ not exercise a sanitized copy; list that validation as owner-side acceptance. | Data-provider requests | Local fixtures in writer/review sandboxes; separate operator acceptance allowance of six real requests, never a crawl. | | End | All acceptance complete, quota exhaustion, wall cutoff, or a reported safety/stuck condition. No filler work to burn remaining quota. | -Nightly checks the night wall between role sessions. A role or verification -already running may finish after the 24-hour scheduler cutoff; this plan does not -claim an exact process-kill deadline. Only reviewed, verified commits count as -output. Exhausting quota may interrupt an item and discard its uncommitted work; -completed items remain on the integration branch. +The final approval packet records the requested wall and latest safe go time, +derived from the pinned wind-down deadline minus that wall and the 120-minute +session allowance. If owner go arrives later, do not claim the original packet +still fits: re-evaluate timing before launch or arming. Nightly's walls are soft; +a running role or verification may finish after cutoff, and a draining stop is +honored between work items. This is not a process-kill deadline or a guarantee +of redemption. Only reviewed, verified commits count as output. Quota exhaustion +may interrupt an item and discard uncommitted work; completed items remain on +the integration branch. Operator-owned ceiling configuration permits the larger wall/token/budget values; -ordinary presets keep their existing defaults. The Scholarr preset requests the -larger values explicitly. Freeze the runner settings in the saved draft and keep +ordinary presets keep their existing defaults. The Scholarr preset explicitly +requests the selected expiry-priority walls and larger token/budget guards. +Freeze the runner settings in the saved draft and keep no model fallback. Approval builds/checks the Go and Node image without calling a model. Launch spends subscription quota and requires Justin's explicit go on the resolved plan, not merely permission to prepare it.