No. 26 · 16 November 2026 · Updated 22 November 2026 · Security · Forensics

How we do forensics, and where it breaks: the Hawthorne method, open for argument

Three AI reviewers together caught 8 of 10 errors planted in one essay, no more than the strongest reviewer alone. Hawthorne reports that all three caught the two missed errors in a follow-up that supplied two relevant records.

Every audit asks the reader to trust a process they cannot watch [1]. That puts the verification burden on the reader. A self-audited forensic process needs logs the reader can check; without external validation of the evidence bundle, it still relies on the integrity of the people reporting the results. The evidence bundle for t0035, the house’s audit of PhotoCraft, is not yet public: site deployment is pending [19: N3].

The abstract, the TL;DR and a note on who built the method are in The Short Version, after the conclusion.

The Late Shadow

The Author’s house publishes its own process failures next to the target’s, and every publication goes through him [19: F29] [19: F02]. We publish those alongside the target’s vulnerabilities so readers can decide how much trust to place in the audit [1]. No medals for hiding the mess.

The distinction matters [1]. If we miss a vulnerability because a tool broke, that is a process failure. If the target has a vulnerability that stays unpatched, that is a target vulnerability. Scoring is only accurate if the two stay separate. Some of the bugs come from inside the house, and those go in the log too.

Our standard for reader verification includes an error log and pre-registered control results. It does not guarantee completeness or truth [1]. A process can be transparent and still produce false negatives if its method has blind spots. Verification also means checking whether the published flowchart matches the live essay’s version history and the Gate 2 approval records [1].

The pipeline has specific control points where deviations can occur. Gate 2 is the approval step where corrections are validated. The Moot Bypass (v4-v7) is a gap where edits occurred without full review by the Moot, the house’s review panel [1]. Gaps like that put uncertainty into the historical record, so the reader should assume that any unverified ledger entry is potentially subject to modification until an external checkpoint anchors it [4].

That is the foundation: self-audited processes need reader-verifiable logs, and we publish our own failures so readers can set their trust accordingly. Nothing in this section depends on a later one. The technical sections that follow depend on it.

The Author set rules for how we work: zero guess laundering, no overwrites and honest fetching [19: F15] [19: F25] [19: F26]. The audit also keeps a hash-chained ledger, whose limits are stated below [14]. They are not a guarantee that the results are right. They are how we leave the argument open to inspection.

The Gate With No Fence

Web Capture Policy: Web capture (collecting pages) is passive: the tool identifies itself, obeys robots.txt (a site’s instructions to automated visitors) as policy, and performs one GET (a request for a page) per URL (page address) [1]. On 1 November 2026, after a government site returned a 403 (access forbidden), the Author said, “yeah, we don't bypass government warnings.” [19: F26] The stop rule applies to every host: no other user agent, route or mirror around a block [19: F26]. Robots.txt is not authorization; obeying it is a policy choice, not a security control [15].

Score Integrity: “Never overwrite” went into the house plan on 28 October 2026; on 1 November, the Author identified a versioning problem and ordered the current versions advanced to the same next integer [19: F25]. Scores are appended per target version and rubric version, never overwritten [1], so historical performance can be tracked. The old score doesn’t get quietly escorted out the back.

The ledger (the audit’s record of events) is a hash chain: each entry includes the previous entry’s hash (a digital fingerprint of its data) [14]. Its entry schema places a 0x1f separator (a byte marking a field boundary) after every field, including the last, detail_str [1] [14]. Re-serialization (turning data back into a stored string) can change whitespace, escaping or key order and therefore the hash; verification should reproduce the specified original bytes, preferably from the stored string [14].

Guess Labeling: Zero guess laundering is a rule with an origin story we would rather not need: the Author adopted it on 30 October 2026, after No. 04 went live reading as though a study had been built and run when it was only a plan [18] [19: F15]. A guess, inference, hypothetical or unverified claim keeps its label through rewrites until a primary source verifies it; dropping the label is a blocking defect [18] [19: F15]. No new factual claims without source; guesses are labeled [1]. Disputed claims are stated as disputed, with both sides and what would settle it [1].

For example, the cost audit is incomplete because usd is NULL for sam/sundar calls [1]. This is a recording limitation, not a hidden cost [1]. (NULL means the number wasn't recorded, not that it was zero.)

The ledger entry, field by field. The hash covers six fields in this order, and a 0x1f byte follows every one of them, the last included [14].

OrderFieldHolds
1prev_hashprevious entry's hash
2tstimestamp
3actorwho acted
4actionwhat was done
5subjectwhat it was done to
6detail_strthe stored detail string

The entry's hash is the SHA-256 of those bytes [14].

Verification: The reader reconstructs a ledger entry from the stored data and confirms its hash matches the next entry’s prev_hash (previous-entry hash) [14]. If the hash does not match, the chain is broken [4].

As Far as the Stake

All of this assumes the ledger is append-only (entries are added, not altered or deleted). Without a signed commitment (external checkpoint) after the latest entry, the owner can rewrite the suffix [4]. Publishing head entry 288 in t0035 does not authenticate later entries 314, 321, 322, and 324 [4]. That’s structural: a limitation of the current implementation. The ledger doesn’t get to provide its own alibi.

The repair (proposed, not yet adopted): The hole has a known fix, and it is a matter of order: publish the commitment outside the house before the work it is meant to protect. For the next audit, that means three steps. First, before the target is opened or a control is run, publish the hash of the pre-registration somewhere the house cannot quietly edit: a public registration that is timestamped and read-only [5], or a signed tag in a public repository. Second, publish a checkpoint of the ledger’s head at each gate and again after the final entry, so no tail is left unanchored [4]. Third, commit the pre-registration and the results separately, so their order does not rest on the ledger’s word alone. None of this was done for t0035 or t0036. Entries 314–324 stay unanchored, and a checkpoint published now cannot prove old timestamps [4]. This is a rule for the next audit, not a repair of this one.

The Author doesn’t need the failures to disappear for the method to look good. He keeps the receipts and changes the rules: a grading bar, a paid-video mishap and even a wrong commit message are all on the record. [19: F19] [19: F25] [19: F30]

The publication gate. All 28 tests failed one draft, Volume 2 v9, whose grades averaged 3.65; on 6 November 2026, the Author changed the rule with the instruction “relax the A on every test, keep rigor absolute, accept that outliers happen.” [19: F19] The current bar has 33 named-personality tests—each a labeled best educated guess about how that person would grade—and requires two independent runs with zero open blocking problems. [19: F19] No heroic average gets a blocking problem off the hook. [19: F19]

The prompt budget. On 30 October, the house measured dario’s input at a median of 130,000–160,000 tokens per call, with a maximum of 348,000, because search results were being resent every turn. [19: F21] A token is a small chunk of text, the unit model input is measured in. The rule now sends long text to the justices in chunks and resends history conservatively, rather than putting roughly 5,000 words into one prompt. [19: F21]

The receipt. One incident produced 90 paid videos instead of 30. [17] [19: F25] The rule that followed: save a generation’s ID before downloading the goods. [17] [19: F25] Paying for the same clips three times is a memorable way to learn that lesson. [17] [19: F25]

Rule -1. When a training start ran out of memory because idle image servers were holding cached models, the Author’s rule, on 15 November, was: “optimize memory when needed, we must finish these work products.” [19: F33]

The prediction and its correction. On 28 October, the Author predicted that about half the scouted topics would fail the next step and had the prediction registered, with the test written, before that step ran. [19: F30] The result was 11 of 27, or 41%, with a 95% Wilson interval—a range expressing statistical uncertainty—of 25–59%; a hand-typed commit message said 10, and the next commit corrected it: “11 of 27 failed, not 10”. [19: F30] The correction is part of the record, not an embarrassment to hide. [19: F30]

The model’s job. On 5 November the Author put it this way: “we are assigning roles with measurements and not just guesses.” [19: F08] On 4 November, he had Nemotron run the same two research tasks as the previous eddie before any swap: Nemotron had 18 of 18 quotations verified word for word; the previous eddie, Qwen, had 10 of 16. [19: F07] Nemotron became eddie on 5 November. [19: F07] That’s an audition with receipts, not a warranty.

Rules are only as good as the next failure they catch.

Severity scores shape the risk assessment, so the arithmetic had better work. Ours didn’t at first: the v1 pseudocode left out the confidence multiplier, turning 0.64 points into 2.56 for severity 8 and confidence 0.25 [13]. That is a lively difference.

The scoring formula is influenced by CVSS (the Common Vulnerability Scoring System) [3] and SSVC (Stakeholder-Specific Vulnerability Categorization), with new axes introduced for internal consistency.

Weigh It Again

The Author’s Hawthorne Classification gives each finding a three-word verdict: Species · Immediacy · Devastation, each on a four-rung ladder; the 0–10 score is only a sort key [16] [19: F32]. It describes rather than prescribes: “We describe; the reader prescribes.” [16] [19: F32] Version 0.3 is a pre-validation draft, not a frozen instrument; no inter-rater exercise (a test of agreement among reviewers) has run yet, and freezing awaits an exercise that clears a stated bar [16] [19: F32] [19: N5].

Scoring Formula: Per-finding points = 4 × (severity/10)² × confidence [13].

Floors: Floors establish policy constraints on the audit score [13].

They are policy constraints, not calibration or a guarantee against real-world under-scoring [13, 16]. A finding may still be under-scored if the Classification spec is misapplied [13, 16].

Classification Spec vs. Rubric: Classification Spec v0.3 and Scoring Rubric v0.3 must be kept distinct [13, 16]. The v1 pseudocode’s floor conditions did not match the rubric; the corrected code in “How to” follows rubric v0.3 and score.py, with one gap in score.py itself: its 9.0 floor checks only CWE-506 and CWE-912, not covert exfiltration, which the rubric includes [13] [19: N5].

Verification: Enter severity 8 and confidence 0.25 and confirm per-finding points of 0.64, not 2.56; for an audit containing only that finding, with no higher floor triggered, score.py reports the audit score rounded to one decimal, 0.6 [13]. If the calculation differs, the implementation does not conform to the adopted rubric [13].

The floors are applied to the audit's total, not to each finding: rubric v0.3 floors the score, and score.py applies them after summing the points [13]. That matters whenever an audit has more than one finding. The policy floors alone do not supply calibration [13, 16].

This is what went wrong in t0035 (PhotoCraft). The house publishes its own process failures next to the target’s, and every publication goes through the Author [19: F02] [19: F29]. The receipt: seven false positives, two reviewer errors and one process deviation— Claude wrote draft v2 [1]. The log makes this essay more useful: readers get the mistakes as well as the findings.

False Positives and Errors: False positives mean the classification process is not perfect. Reviewer errors are a different category from process deviations, which include third-party package execution and database updates.

CVE Labeling: t0035 mislabeled CVE (Common Vulnerabilities and Exposures, the public catalogue of known vulnerabilities) initially, then over-corrected [1]. That indicates instability in the automated classification pipeline; the over-correction suggests a feedback loop that was not properly managed.

Version Control: The t0035 corrections—voiced versions 8, 9 and 10—are unapproved, awaiting the Author’s Gate 2 (his fact-approval step) [1] [19: F17]. The live essay requires visible correction [1]. The Moot did not review t0035 after v3, and the SQL UPDATE (a database change) to finding #32 was logged after the fact [1]. That log entry is afterwit: the record shows up, but only after the change it records. The public version of the audit therefore contains unreviewed changes.

What happened to t0035, in the order the record gives it [1].

WhenEventStanding
30 October 2026Third-party packages ranDisclosed
31 October 2026The venv was deletedDisclosed
After v3Moot review stoppedv4–v7 unreviewed
Undated hereFinding #32 changed by SQLLogged late, entry 322
PendingCorrections v8, v9, v10Await Gate 2

Verification: Once the bundle is public, a reader can check audit.db for entry 322 and the version history for v3–v7 Moot review status [1]. If entry 322 is missing or the timestamp is inconsistent, the log’s integrity is compromised [4]. The evidence bundle is in the private repository and on the shared drive; site deployment is still pending, so the public cannot yet perform that artifact check [19: N3].

Still Wet

Third-party package execution occurred on October 30, 2026, during an audit meant to execute nothing [1] [19: F29]. The failure log also discloses that the virtual environment (venv, an isolated software environment) was deleted on October 31, 2026 [1]. A deleted venv is not an alibi. Deletion does not retrospectively prove the absence of earlier effects on the host computer or file access [1], and it removes the ability to verify the exact environment state used during the audit [1].

The Author had his own reviewers tested: the pre-registration was written on 31 October 2026, and the house's audit ledger logs its hash before the result [19: F31]. That order is the ledger's own claim: both entries come after the last public checkpoint, and the commit that holds the pre-registration also holds the results, so git cannot show which came first [19: F31]. What follows is an author-reported pilot, not a victory lap.

Initial control:

That file exists in the bundle but is not yet public, so readers cannot independently verify the counts [11] [19: N3]. The initial misses indicate a blind spot for planted errors involving internal evidence when the relevant records are absent [11].

Control C (targeted appendix):

On Watch at the Birdbath

This suggests that targeted records improve detection, but it was a follow-up test, not a randomized controlled trial [11].

Panel union: Panel union recall equals the best individual in both conditions; no incremental benefit demonstrated in this n=1 pilot [11]. That challenges the assumption that cross-lab AI reviewers (models from different labs) add value beyond one strong reviewer [11]. The Author's own three-lab panel has not yet been shown to beat one [19: N7]. Three reviewers, one result to beat, and no bonus points for looking impressive.

Access vs. competence: Control C used a targeted two-record appendix. It does not prove that access alone caused the initial misses: the records' prominence and the difficulty of finding them also changed [11]. Testing the access hypothesis calls for a follow-up with the full 3.5 MB bundle, not the targeted appendix [11].

Who caught which planted error. The ten factual plants are R1 to R10; the reviewers are the three justices [11].

Initial controldariosamsundar
R1 wrong datecaughtcaughtcaught
R2 wrong versioncaughtcaughtcaught
R3 wrong PR numbercaughtcaughtcaught
R4 wrong CVE issuercaughtcaughtcaught
R5 wrong rubric floormissedmissedmissed
R6 wrong bug numbercaughtcaughtcaught
R7 wrong notice numbercaughtcaughtcaught
R8 wrong binary sizesmissedmissedmissed
R9 fabricated quotemissedcaughtcaught
R10 score mismatchcaughtcaughtcaught

In Control C, with the two records supplied, the only miss left was dario's on R9 [11].

Control Cdariosamsundar
R5 wrong rubric floorcaughtcaughtcaught
R8 wrong binary sizescaughtcaughtcaught
R9 fabricated quotemissedcaughtcaught
The other sevencaughtcaughtcaught

Verification: A reader can replay the control with the full 3.5 MB bundle, if it is released, to test the access hypothesis [11]. Until the bundle is released, that replay cannot be done by the reader [11].

Limitations: Control C was an n=1 pilot: one essay, ten planted errors per checker, easy mode [11]. It detects a specific class of error; it does not establish general effectiveness or a general failure rate. Nor does it prove that access alone explains the misses [11].

What we don't know yet: the places where the record is incomplete or disputed.

Prior Art Survey: Q11: Prior art survey for AI-assisted security research is not yet done [7, 12]. The universal novelty claim is dropped until it is [7, 12]. Schroter et al. 2008 establishes relevant prior art in peer review [7]. NIST SATE VI establishes relevant prior art in static analysis [6].

The Dumbwaiter

Evidence Bundle: Released to the private repository and shared drive, site deploy pending [1] [19: N3]. The README address returned homepage HTML during this check; that response does not supply the promised artifact verification [1]. The live essay publishes the stated head hash, calls its evidence record unpublished, and retains broad no-downloaded-code-execution wording [1].

Cost Audit: Cost audit incomplete: usd is NULL for sam/sundar calls; token counts (80 input vs 59,827 output) need telemetry explanation [1]. Explaining the 80 input vs 59,827 output discrepancy across seven dario calls requires inspecting raw ledger tables and API logs [1]. Neither public methodological documentation nor agreement among reviewers authenticates local entry numbers, execution dates, approval decisions, control totals or database changes [1].

Verification: Reader monitors site deploy and future control runs [1]. Until the bundle is published with hashes, the counts remain unverified [1].

What isn't done yet: Here's the part where the receipts come for us.

Scores are versioned; check the version [1]. Do not trust the score; trust the evidence bundle enough to examine it [1]. Transparency enables verification. It does not guarantee truth.

The Plain Box

The ledger verifier is in the t0035 evidence bundle, but the site deploy is pending; the control bundle is not yet public either [19: F23] [19: N3] [19: F31]. Once the artifacts are available, readers can use the verifier and control data to assess the audit. Until then, the reported results remain claims awaiting independent inspection.

Hawthorne is one operator, and everything above went through him [19: F02]. We have a hash-chained audit ledger, a pre-registered control and a failure log kept next to the target’s findings; the control’s order is the ledger’s claim, not externally anchored [19: F23] [19: F31] [19: F29]. On 30 October 2026, the Author adopted zero guess laundering after No. 04 made a plan read like a completed study [19: F15]. On 6 November, he revised the publication gate after all 28 tests failed one draft under the old rule [19: F19]. The operation publishes its own failure log [19: F29]. His question, roughly: who else in the world could do this? [19: F03] Check the receipts.

The summary, moved behind the argument: what the essay claims, what it admits, who it is for and who built the method.

Abstract

This essay sets out how Hawthorne audits software, and where that method has failed. It covers two pieces of work: t0035, an audit of PhotoCraft, and t0036, a control in which errors were planted in an essay to see whether the house’s AI reviewers would catch them [1, 11]. The Author designed the method, runs Hawthorne and sets its rules [19: F02, F37].

Our own audit gets an error log, not an exemption. A process failure (our tool or our procedure broke) is recorded separately from a target vulnerability (the software under audit has a flaw) [1]. Three failures are disclosed here. Third-party packages ran on 30 October 2026, in an audit meant to execute nothing. A database change (an SQL UPDATE) was made without a ledger entry and logged only afterwards, at entry 322. And the author edited the text after the Moot, the house’s review panel, had finished; those edits still await approval [1] [19: F29].

The control’s result: reviewers from different AI labs missed planted errors that could only be checked against private records, and the panel caught them once two relevant records were supplied. That is one essay (n=1), reported by the author, with the follow-up designed after the first results were in [11]. We have dropped the claim that the method is new everywhere until a survey of prior work (Question 11) is done [7, 12].

TL;DR

Why you should care

Who built this, and where

From a 100-square-foot cottage in an orchard without running water, the Author runs Hawthorne. [19: F01] Every pick, ruling and publication passes through him. [19: F02] By 16 November 2026, the release shelf held 25 numbered Hawthorne issues, and the publication bar had 33 tests guessing how named people would grade sections; those grades are labeled best educated guesses. [19: E] [19: F19] The house “we” is a method. No outside party has audited the system yet. [19: N6]

No publication-bar grades are recorded for this essay yet. [19: N8]

The work ledger does not get to quietly tidy up after itself: it is append-only, designed to accept additions, not updates or deletions. Since 30 October, it had recorded 31,565 events across 64 sessions by 16 November. [19: F22] The separate t0035 audit ledger has 324 entries in a hash chain: each entry’s hash, a digital fingerprint, incorporates the previous one. [19: F23] Its public checkpoint reaches entry 288; later entries are not anchored by a published checkpoint, and the evidence bundle’s site deploy is pending. [19: N3]

The local research model, eddie, does the research and coordinates the work; local models also make the images, video and voice. [19: F04] After a measured quiz, Gemma was given fast triage (quick first checks, never admitting a fact). [19: F08] Frontier models from three labs argue as justices: disagreement is a signal, while agreement is not proof. [19: F10] A court with a standing order not to believe its own applause.

The Author’s challenge, in cleaner clothes: who else could do this? [19: F03] That is a question, not a finding of exclusivity. Here are some receipts: the t0035 failure log records seven false positives, two reviewer errors and a process deviation; the record also includes a third-party package run on 30 October during an audit meant to execute nothing, and an after-the-fact database update, logged in the audit ledger (entry 322). [19: F29] The essay above is the receipts, failures included. The counts show work recorded, not that every result is right.

  1. H1: Private-evidence blind spot. Reviewers miss private-evidence errors when records are absent, and catch them when provided.
    • Falsifier: A follow-up experiment with a full 3.5 MB bundle (not targeted appendix) shows union recall <80% on internal-evidence plants, or failure with full access explains the misses [11].
  2. H2: Prompt transparency. Publishing original generation prompts increases external reviewers' supported error detection by >10% when draft and evidence access are held constant.
    • Falsifier: An equivalence test with a prespecified margin (10%) shows the confidence interval includes zero or favors the null, and the study design distinguishes prompt visibility from prompt causation [11].
  3. H3: Rubric floors. The rubric v0.3 floors prevent under-scoring of critical issues.
    • Falsifier: An audit with a confirmed CVSS 10.0 finding scores <6.0 under v0.3 rules [1].
  4. H4: Cross-lab value. Cross-lab AI reviewers add value beyond one strong reviewer or three independent calls to the same model.
    • Falsifier: At matched budget, a cross-lab panel does not improve union recall by >5% over the strongest single-model comparator, or increases false accusations [11].
  5. H5: Ledger conformance. The implementation conforms exactly to the adopted rubric's confidence weights, aggregation rules and floor predicates.
    • Falsifier: Any valid independently constructed input produces a result inconsistent with the adopted specification [13].
  6. H6: Ledger completeness. Every score-relevant database mutation in the audited interval is reconstructible from the ledger.
    • Falsifier: One unreconstructible mutation, an inconsistent reconstruction, or a claimed pre-review event outside the relevant commitment (e.g., finding #32 UPDATE at entry 322) [1].
  7. H7: Cost efficiency. AI-assisted review has lower total cost than human-only review while meeting a preregistered noninferiority margin for supported detection.
    • Falsifier: It demonstrably fails the quality margin, or its total cost at comparable quality is not lower (requires explicit accounting boundaries) [1].

Purpose: Demonstrate the ledger hash chain construction and scoring formula (corrected).

# Ledger Entry Hash Construction (Corrected)
# Source: local:hwtn.io/evidence/t0035-photocraft-0.17.0/verify_ledger.py

def _entry_hash(prev_hash, ts, actor, action, subject, detail_str):
    # 1. detail_str is ALREADY SERIALIZED (stored string, not object)
    # 2. Separator 0x1f after EVERY field including the last (detail_str)
    raw = prev_hash.encode() + b'\x1f'
    raw += ts.encode() + b'\x1f'
    raw += actor.encode() + b'\x1f'
    raw += action.encode() + b'\x1f'
    raw += (subject or "").encode() + b'\x1f'
    raw += detail_str.encode() + b'\x1f' # Trailing separator included

    # 3. SHA256
    from hashlib import sha256
    return sha256(raw).hexdigest()

# Scoring Formula (Corrected)
# Source: local:reveng/RUBRIC.md (v0.3) and local:reveng/score.py
def calculate_audit_score(findings):
    # Each finding: severity 0-10 (float); confidence 1.0 (confirmed),
    # 0.5 (plausible) or 0.25 (speculative).
    # An ambiguity (a surface we could not clear) has no severity and a
    # flat cost: 0.1 minor, 0.3 major.
    total, floor = 0.0, 0.0
    for f in findings:
        if f.get("ambiguity") is not None:
            total += f["ambiguity"]
            floor = max(floor, 0.1)  # unresolved
            continue
        # FIX: Multiplier included
        total += 4.0 * (f["severity"] / 10.0) ** 2 * f["confidence"]
        confirmed = f["confidence"] == 1.0
        if confirmed and (f.get("is_malicious") or f.get("is_covert")):
            # confirmed CWE-506/CWE-912 or covert exfiltration
            # (score.py checks only the CWEs)
            floor = max(floor, 9.0)
        if confirmed and f.get("is_cvss") and f["severity"] >= 9.0:
            floor = max(floor, 6.0)  # confirmed CVSS >= 9.0
        if not confirmed:
            floor = max(floor, 0.1)  # unresolved
    # FIX: Floors apply to the audit's total, not to each finding
    # (rubric v0.3, score.py). score.py reports it rounded to one decimal.
    return min(max(total, floor), 10.0)

What it shows: The ledger is append-only relative to a checkpoint; the scoring is deterministic with confidence weighting [13, 14]. What it leaves out: Actual JSON schema complexity and drift check logic. The code models covert exfiltration as a flag; score.py itself checks only CWE-506/CWE-912 for the 9.0 floor [13].

The code block shows the correct separator placement. The trailing separator is part of the specified construction; leave it out and the hash will not match the stored chain [1]. The scoring formula shows what the confidence multiplier does. Before floors, omitting confidence multiplies the base points by 1/confidence: fourfold at 0.25 and twofold at 0.5. The final score may be unchanged if the same floor dominates both calculations.

The Moot is the house's review panel. Models from three labs, called dario, sam and sundar here, argue over a draft, and the research model, eddie, rules on each objection [19: F04, F10]. This is the short version of what that argument settled and what it left open. The item-by-item rulings are in the essay's working folder, which is not public.

Settled

VI). That establishes relevant prior art; it does not disprove every narrowly specified novelty claim [6, 7].

Still Disputed

Where the review itself slipped

The review made its own mistakes, and they are on the record as well. While ruling on this essay's draft, eddie put Control C's per-reviewer totals into the initial control, contradicting an earlier ruling of his own [11]. Two rulings stated as done things the record does not show: a completed adjudication with separate scoring of unplanted discoveries, and a completed amendment to the failure log [1]. One ruling reversed an inference: failure with full evidence counts against an access-only explanation, not for it [11]. One attached an excerpt from an unrelated paper, Lost in the Middle, to the peer-review prior-art point [7, 8]. Several kept merging the Classification spec with the scoring rubric, and one described the proposed corrected scoring formula as existing behaviour [13, 16].

Three Levels

The quotations were the largest problem. Seven of twelve excerpts failed a word-for-word check, because non-contiguous sentences had been spliced with ellipses or words had been altered, even though the underlying pages supported the objections [8]. The other reviewers caught each of these slips from the supplied record.

Artifact verification: The reported control counts, failure counts, 3.5 MB bundle size and model-call costs remain author-reported pilot observations pending independent artifact inspection. We could not fetch the README or control outputs via public tooling in this check; the evidence bundle’s site deploy is pending [1] [19: N3]. A reported result is not an independently verified one.

Control C generalizability: The control was an n=1 pilot: one essay, ten plants per checker, easy mode. It does not prove general effectiveness; it tests a specific class of error [11].

Execution history: The virtual environment was deleted on October 31, 2026. That does not retrospectively prove the absence of earlier host effects or file access [1]. Deletion is not a time machine.

Novelty claim: Question 11, the prior-art survey, is not yet researched. The universal novelty claim is dropped until it is completed [7, 12].

Approval status: Corrections in voiced v8, v9 and v10 await the Author’s Gate 2 approval. The live page is an earlier version [1] [19: N4].

Cost data: usd is NULL for sam/sundar calls, and voicer and image costs are not in the ledger. The total cost is incomplete [1].

Builder claims: Claims about the Author, his tools and his process are meant to map to entries in the internal builder’s fact file, which includes his own words with dates and is not public [19].

Voicing and checks: This version was written by OpenAI’s gpt-6.1-terra and was not voiced by hthompson [19: T5] [19: N9]. Its changed sections were accuracy-checked by one lab only, Anthropic’s claude-opus-5-5; a Gemma model quick-checked a handful of load-bearing claims as triage, not evidence. Google’s Gemini did not take part, and the OpenAI seat is conflicted as the writer’s lab [19: F36].

This revision: Version 10, 22 November 2026, reorders the essay. It now opens with the House of Cards; the abstract, TL;DR, “Why you should care” and the builder’s note follow the conclusion as The Short Version, and the first three were rewritten for plainer reading with the same facts and sources. In the body, the builder’s name is replaced by “the Author”. Version 9, 21 November 2026, answered an outside reader's critique: it merges the two BS Detector sections into one, replaced three figure captions that had no figure with tables built from the control's result files and the ledger verifier, rewrote the Moot record for a reader who was not in the room, stated a proposed repair for the unanchored ledger tail, reconciled the fact-watch dates, relabelled the pilot counts in the claim register, added reference 21 and rewrapped three code comments that ran off the page. The edits in both versions were made by Anthropic's Claude at the Author's request. They were not re-voiced and the panel has not reviewed them.

Tools that testify

The Author keeps a release shelf that can testify against him. [20: PW06] On 16 November 2026, it held 41 read-only golden-master releases (reference copies of what shipped), containing 138 files, and three append-only hash chains (logs added to rather than changed, with each entry’s digital fingerprint linked to the previous one); the quick verifier passed. [20: PT11] Release uploads use --immutable to prevent overwriting; mirrors copy rather than sync and move replaced files into a backup folder. [20: PT06] The half-hourly shelf job skips itself while an earlier upload is still running. [20: PT06] That is a tool with instructions for what not to do, which is a useful kind.

His research model got its job by audition: on 4 November 2026, 18 of 18 of Nemotron’s quotations checked out word for word, against 10 of 16 for Qwen. [19: F07] He chooses the tools, sets the rules, rules on the output and decides what ships. [19: F02, F37]

Workflows with a failure log

The Author’s workflow has a review trail long enough to trip over, and we can count it. On 16 November 2026, the house found 24 essay folders with a long-Moot record, 37 justice review files and 47 fix-desk files. [20: PW01] After voicing, a model from a different lab checks for drift and reports rather than edits; 22 essay READMEs reported checks covering 2,726 claims, with 91 flagged. [20: PW05] Those are records of checks, not a verdict that every claim survived one.

The house also checks whether work that looks finished is actually there: read the log, compare expected outputs with files on disk, redo what is missing and check the remote service. [20: PW09] In the animation work, code-drawn blocking boards come before the Author’s approval and any ink keys; by the evening of 15 November, 244 shots had been blocked across eight pieces. [20: PW07] Stick figures first, production later.

Environment, minus the scenery

The Author’s memory rule, set on 15 November 2026, is “optimize memory when needed, we must finish these work products.” [20: PE03] When training needed room that day, idle image servers were cleared under that rule and training started. [20: PE03]

Under the Author’s standing rule, the shelf sync sends smaller files first. [20: PE04] In his words, auto-refill “is turned off on everything” and he has “done this on purpose”; the rule is to display spend estimates, stop a provider cleanly and say which account stopped, never switching models silently. [20: PT03]

ClaimReceiptEntry
Release verificationThree hash chains; 41 golden-master releases; quick verifier passed on 16 November 2026[20: PT11]
Review trail24 long-Moot folders, 37 reviews and 47 fix- desk files, counted 16 November 2026[20: PW01]
Drift checked22 reported checks, 2,726 claims and 91 flags, counted 16 November 2026[20: PW05]
Memory managed under his ruleIdle image servers were cleared so training could start, 15 November 2026[20: PE03]
Spend controlled on purposeAuto-refill off on every account; no silent model switches[20: PT03]
evidence/t0036-control/RESULTS.md

and

RESULTS-C.md

, 2026-10-31. Released to the repository and the shared drive; site deploy pending, so not yet public (checked 2026-11-16).

                                                 essays/t0035-mostly-harmless-
no-execution-audit/voiced/t0035-mostly-harmless-no-execution-
audit.voiced.v10.md

, 2026-10-31. Awaits the Author's gate 2; not public.

reveng/RUBRIC.md

) and its scoring script (

                                                                    reveng/
score.py

). Internal documents, adopted 2026-10-30. Not yet public.

                                                         evidence/t0035-
photocraft-0.17.0/verify_ledger.py

), 2026-10-31. Released to the repository and the shared drive; site deploy pending, so not yet public.

docs/zero-guess-laundering-v1.md

). Internal policy, 30 October 2026. Not yet public.

essays/BUILDER-FACTS.2026-11-16.md

). Internal record, 16 November 2026: every claim about the Author, his tools and his process, with its evidence and a label (from the record, his own words, published, or inference). Cited as [19: entry]. Not yet public.

                                             essays/BUILDER-FACTS.proficiency.
2026-11-16.md

). Internal record, 16 November 2026: tools, workflows and environment, each with its evidence (path, commit, config key, command and output, or his dated words) and a label. Cited as [20: entry]. Not yet public.

ClaimFootnote(s)Source typeStatus
Self-audited processes require reader-verifiable logs.[1]PrimaryEstablished
t0035 PhotoCraft audit includes error log.[1]PrimaryEstablished
t0036 control experiment is n=1 pilot.[11]PrimaryEstablished
Third-party packages executed on Oct 30, 2026.[1]PrimaryEstablished
venv deleted Oct 31, 2026.[1]PrimaryEstablished
Scoring formula includes confidence multiplier.[13]Primary (internal)Established
Floors: 9.0 for malicious code.[13]Primary (internal)Established
Floors: 6.0 for CVSS ≥ 9.[13]Primary (internal)Established
Floors: 0.1 for unresolved.[13]Primary (internal)Established
Floors apply to the audit's total, not to each finding.[13]Primary (internal)Established
score.py's 9.0 floor checks CWE-506/ CWE-912 only, not covert exfiltration.[13]Primary (internal)Established
Initial control: 8/10 recall (union).[11]PrimaryAuthor-reported pilot (n=1)
Control C: 10/10 recall (targeted).[11]PrimaryAuthor-reported pilot (n=1)
Repair for the unanchored ledger tail: publish the pre- registration hash and ledger checkpoints outside the house before the work they protect.[4, 5]House proposalProposed, not adopted
Novelty claim dropped (Q11).[7, 12]PrimaryEstablished
Ledger entry 288 does not authenticate 314-324.[4, 14]PrimaryEstablished (v3: was Disputed; the Moot's Settled list and the hash-chain structure both settle it, and no checkpoint after 288 had been published as of 2026-11-16)
Cost audit incomplete ( usd NULL).[1]PrimaryEstablished
v8-v10 corrections unapproved.[1]PrimaryEstablished
Schroter et al. 2008 establishes planted- error prior art.[7]SecondaryEstablished
NIST SATE VI establishes static analysis prior art.[6]SecondaryEstablished
Robots.txt is not authorization.[15]PrimaryEstablished
pip installation runs code by default.[10]PrimaryEstablished
Hash separator 0x1f after every field.[14]Primary (internal)Established
The Classification's verdict is a Species · Immediacy · Devastation tuple; the score is a sort key; v0.3 is pre- validation.[16]PrimaryEstablished
The method's rules are set by one person, the Author, who runs Hawthorne from a 100- square-foot cottage in an orchard without running water.[19: F01, F02, F37]His own statement (2026-11-16); published page [17]His statement
Model roles were assigned by measurement (bake-off 18/18 vs 10/16 verified quotes; quiz 20/20 vs 19/20).[19: F07, F08]Internal recordEstablished
House rules were written after named failures (No. 04; the 28-test failure; 90 paid videos instead of 30; a 403; an out-of- memory crash).[17], [19: F15, F19, F25, F26, F33]Internal record; published pageEstablished
The fact watch last ran cleanly on 2026-11-03 and failed every scheduled run from 2026-11-07 to 2026-11-16.[19: N1]Internal record (run logs)Established
Scale counts (25 numbered issues, 33 tests, 31,565 ledger events, 324-entry audit ledger) as of 2026-11-16.[19: E]Internal record, counted with the commands listed thereEstablished
"Who else in the world could do this?"[19: F03]His own questionA question, not a claim
Release shelf: 41 read- only golden-master releases (138 files) and three append-only hash- chained logs; quick verify passed 2026-11-16. Release uploads are immutable; mirrors copy, never sync, with a backup folder.[20: PT11, PT06, PW06]Internal record (command output; script); his own wordsEstablished
Review trail on 2026-11-16: 24 essay folders with a long-Moot record, 37 justice review files, 47 fix-desk files; 22 drift checks covering 2,726 claims, 91 flagged.[20: PW01, PW05]Internal record (counts with the commands)Established
The house checks finished work against the log and the files on disk; animation is blocked by code before approval and ink (244 shots by 2026-11-15).[20: PW09, PW07]His own words; internal recordEstablished
Under his Rule -1, idle image servers were cleared on 2026-11-15 so training could start.[20: PE03]His own words; internal recordEstablished
The shelf sync sends the smallest files first, under his standing rule.[20: PE04]Internal record (session report, rclone output)Established
Auto-refill is off on every account on purpose; a credit error stops that provider cleanly with no silent model switch.[20: PT03]His own words (2026-11-16)His statement

References

  1. 01hwtn.io. "t0035 PhotoCraft Evidence." Evidence, n/a. https://hwtn.io/evidence/t0035-photocraft-0.17.0/README.md (accessed 2026-10-31; on 2026-11-16 this address still returned the site's homepage, see §6).
  2. 02hwtn.io. "No. 03 Moot of Doom." Essays, 2026-10-29. https://hwtn.io/essays/moot-of-doom-essay-and-other-fun-facts (accessed 2026-10-31).
  3. 03FIRST. "CVSS v4.0 Specification Document." Specification, n/a. https://www.first.org/cvss/v4.0/specification-document (accessed 2026-10-31).
  4. 04RFC Editor. "RFC 9162." RFC, n/a. https://www.rfc-editor.org/rfc/rfc9162.html (accessed 2026-10-31).
  5. 05OSF. "Welcome to Registrations." Help, n/a. https://help.osf.io/article/330-welcome-to-registrations (accessed 2026-10-31).
  6. 06NIST. "SATE VI Report: Bug Injection and Collection." NIST SP 500-341, June 14, 2023. https://www.nist.gov/publications/sate-vi-report-bug-injection-and-collection (accessed 2026-10-31).
  7. 07Schroter, S., et al. "What errors do peer reviewers detect, and does training improve their ability to detect them?" J R Soc Med 101:507–514, 2008. doi:10.1258/jrsm.2008.080062. https://pmc.ncbi.nlm.nih.gov/articles/PMC2586872/ (accessed 2026-10-31).
  8. 08Liu, N. F., et al. "Lost in the Middle: How Language Models Use Long Contexts." arXiv:2307.03172, submitted July 6, 2023; revised November 20, 2023. https://arxiv.org/abs/2307.03172 (accessed 2026-10-31).
  9. 09Python Software Foundation. "Python json." Documentation, n/a. https://docs.python.org/3/library/json.html (accessed 2026-10-31).
  10. 10PIP. "Secure Installs." Documentation, n/a. https://pip.pypa.io/en/latest/topics/secure-installs/ (accessed 2026-10-31).
  11. 11Hawthorne. "t0036 Control Results." Evidence bundle files
  12. 12Hawthorne. "No. 17, t0035, voiced v10." Unpublished correction,
  13. 13Hawthorne. "Hawthorne score: rubric v0.3" (
  14. 14Hawthorne. "verify_ledger.py," the t0035 evidence bundle's ledger verifier (
  15. 15IETF. "RFC 9309: Robots Exclusion Protocol." RFC, September 2022. https://www.rfc-editor.org/rfc/rfc9309.html (accessed 2026-11-16).
  16. 16Hawthorne. "The Hawthorne Classification," spec v0.3, pre-validation draft, 31 October 2026. https://hwtn.io/classification (live since 2026-11-16).
  17. 17Hawthorne. "How Hawthorne Runs," v1, 1 November 2026. Projects page, https://hwtn.io/about.
  18. 18Hawthorne. "Zero guess laundering, v1" (
  19. 19Hawthorne. "The builder's fact file" (
  20. 20Hawthorne. "The builder's fact file, part two: proficiency" (
  21. 21Amrhein, V., Korner-Nievergelt, F., and Roth, T. "The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research." PeerJ 5:e3544, 2017. doi:10.7717/peerj.3544. https://pmc.ncbi.nlm.nih.gov/articles/PMC5502092/ (accessed 2026-11-21).