Standards

How a claim gets onto hawthorne, what happens when one turns out to be wrong, and who answers for it. Hawthorne is one person; this page says what stands in for a second fact-checker, and where that falls short.

Every claim of fact is checked against a primary source listed in the issue. A guess is labelled as a guess. Errors of fact are fixed in the text and logged here with the date the fix went live. There is no advertising, no sponsored content and no paywall.

Corrections

  1. 2026-11-06 · No. 23 Tocqueville and Amodei

    Said: Quoted Anthropic's Claude 3 model card with the card's own footnote number left in, so it read as a citation to this issue's reference 17, an unrelated Ray Kurzweil Q&A.

    Now: The model card's footnote number is gone from the quotation on the web edition; the PDF follows at its next typeset.

  2. 2026-11-02 · No. 22 RAG filters at the threshold a SOC would actually deploy

    Said: The abstract said the field leans on AUROC.

    Now: The preprint's own survey finds only 1 of 21 papers reports a ranking metric at all.

  3. 2026-11-02 · No. 22 RAG filters at the threshold a SOC would actually deploy

    Said: Called Cheng et al.'s false-positive and false-negative rates disputed.

    Now: Cheng et al. do report them.

  4. 2026-11-02 · No. 22 RAG filters at the threshold a SOC would actually deploy

    Said: Described the 0.93 survival figure as a share of queries.

    Now: It is a share of passages.

  5. 2026-11-02 · No. 22 RAG filters at the threshold a SOC would actually deploy

    Said: Said the learned scorer's AUROC held.

    Now: It fell from 1.000 to 0.953.

  6. 2026-11-02 · No. 20 PHP2Shell: count evidence stages, not branded requests

    Said: The triage code tested for HTTP 500, looked for the upload-progress marker in the response, and expected matching PIDs.

    Now: Karimi's run completed with HTTP 200, the marker is in the request, and container and host PIDs differ; the code is replaced by prose triage.

  7. 2026-11-02 · No. 20 PHP2Shell: count evidence stages, not branded requests

    Said: Graded "FPM can succeed through pool saturation" as established.

    Now: No source says it; FPM is now an untested chain.

  8. 2026-11-02 · No. 20 PHP2Shell: count evidence stages, not branded requests

    Said: Misquoted Wiz.

    Now: Wiz's subtitle is quoted correctly and its conclusion paraphrased; Wiz itself acknowledged Karimi's success.

  9. 2026-11-02 · No. 20 PHP2Shell: count evidence stages, not branded requests

    Said: Called AWS's 17 October observation the first attempt.

    Now: It was "an attempt", and AWS's caveat about success applies to the 21 October attempts.

  10. 2026-11-02 · No. 20 PHP2Shell: count evidence stages, not branded requests

    Said: Said the PHP release was scheduled for 21 October.

    Now: It was brought forward from 22 to 21 October.

  11. 2026-11-02 · No. 20 PHP2Shell: count evidence stages, not branded requests

    Said: Called the outcome full system compromise.

    Now: It was code execution as the web user.

  12. 2026-11-01 · No. 16 HTTP method equivalence is not state equivalence

    Said: Said the frameworks' distribution matches the Tranco baseline.

    Now: The paper reports no such comparison; the claim is removed and H3 is an untested hypothesis.

  13. 2026-11-01 · No. 16 HTTP method equivalence is not state equivalence

    Said: Counted a Cache-Control: public header as a web cache deception finding.

    Now: It is only a signal; a cross-session hit and unauthorized disclosure are separate outcomes.

  14. 2026-11-01 · No. 16 HTTP method equivalence is not state equivalence

    Said: Said the measurements were authenticated.

    Now: The paper says they were unauthenticated.

  15. 2026-11-01 · No. 15 Can agents tell when they are being audited?

    Said: Described a rising contention curve.

    Now: Withdrawn; the paper's Proposition 5 values at ε=0.2 fall from 0.80 to 0.10 for 2 to 100 sessions.

  16. 2026-11-01 · No. 15 Can agents tell when they are being audited?

    Said: Described the audit's worst-case bound as showing the agent "is biased".

    Now: It is a worst-case capacity, not a realized gap.

  17. 2026-11-01 · No. 15 Can agents tell when they are being audited?

    Said: Cited reference [8] for the AUC conversion.

    Now: The conversion is the essay's own derivation; [8] couldn't be fetched and is no longer cited.

  18. 2026-11-01 · No. 15 Can agents tell when they are being audited?

    Said: Made a causal claim about deployment.

    Now: It is deleted; a verified quote from the paper's §I replaces it.

  19. 2026-11-01 · No. 15 Can agents tell when they are being audited?

    Said: Got the authors of the AI Control paper wrong.

    Now: Greenblatt, Shlegeris, Sachan and Roger (checked on arXiv).

  20. 2026-11-01 · No. 15 Can agents tell when they are being audited?

    Said: Called the condition "necessary and sufficient" without qualification.

    Now: Every instance now carries "under the paper's model".

  21. 2026-11-01 · No. 14 Assessing Offline LLM Efficacy for Emergency Operations

    Said: Said the vendor claims 0 harmful answers and guessed the failures were filtered out of its safety report.

    Now: False: the vendor's README lists Granite 4.2 3B with "5 (2 could kill)" and says it isn't offered; both answers are quoted.

  22. 2026-11-01 · No. 14 Assessing Offline LLM Efficacy for Emergency Operations

    Said: Said the app's answer-anyway rule caused the 3B model's harmful answers.

    Now: That claim is withdrawn; the vendor's tests used an earlier library and prompt.

  23. 2026-11-01 · No. 14 Assessing Offline LLM Efficacy for Emergency Operations

    Said: Compared energy at a flat 4,000 J, concluded the 27B model is more efficient, and said the essay corrected the vendor's 100 W.

    Now: A labelled illustration (about 4,190 vs 4,560 J, resting on an unmeasured 100 W) replaces it; no efficiency claim; the vendor had already qualified the 100 W.

  24. 2026-11-01 · No. 14 Assessing Offline LLM Efficacy for Emergency Operations

    Said: Cited California jury instructions for claims about update checks.

    Now: That source no longer stands in; tourniquet guidance cites DHS Stop the Bleed and the 35-foot figure cites Seattle Fire.

  25. 2026-11-01 · No. 14 Assessing Offline LLM Efficacy for Emergency Operations

    Said: Called a blocked network connection an air-gap breach.

    Now: It isn't; the vendor documents a daily check that sends nothing about the user.

  26. 2026-11-01 · No. 14 Assessing Offline LLM Efficacy for Emergency Operations

    Said: Said a 24 GB GPU will not accommodate the 27B model.

    Now: That is unmeasured.

  27. 2026-11-01 · No. 13 Quantization is not a privacy guarantee

    Said: Stated that the accuracy-matched comparison had been observed, including as a reproduction of Priv-QDNN.

    Now: It is a hypothesis the proposed evaluation would test; the Priv-QDNN sentence now describes Fu et al.'s before/after-quantization analysis.

  28. 2026-11-01 · No. 13 Quantization is not a privacy guarantee

    Said: Cited Microsoft's enclave page for claims it doesn't make.

    Now: It supports only "an isolated region of code and data"; on-device and black-box claims cite Shokri et al.

  29. 2026-11-01 · No. 13 Quantization is not a privacy guarantee

    Said: Called reference [1] theoretical work.

    Now: It is empirical work.

  30. 2026-11-01 · No. 13 Quantization is not a privacy guarantee

    Said: Gave a contradictory account of differential privacy and calibration.

    Now: Post-processing keeps differential privacy; private calibration needs its own accounting.

  31. 2026-11-01 · No. 13 Quantization is not a privacy guarantee

    Said: Treated GDPR Article 17 erasure as settled.

    Now: Erasure is assessed case by case, per the EDPB.

  32. 2026-10-31 · No. 18 Bacon and Hassabis

    Said: Quoted Jumper as having "consulted more than 50 experts".

    Now: It's a reporter's paraphrase of what he reportedly told the press, so the quotation marks are gone.

  33. 2026-10-31 · No. 18 Bacon and Hassabis

    Said: Attributed the openness rule to Bacon.

    Now: It's stated as the essay's own rule, derived from Bacon's method.

  34. 2026-10-31 · No. 18 Bacon and Hassabis

    Said: Gave Hassabis's 2025 view on where hypotheses come from undated, with no mention of Google's Co-Scientist.

    Now: The quote is dated, and Co-Scientist is admitted as a contradictory instance.

  35. 2026-10-31 · No. 18 Bacon and Hassabis

    Said: Called New Atlantis "the first sketch of a modern research lab".

    Now: "An early sketch".

  36. 2026-10-31 · No. 12 EDK II PXE bug: which shipped firmware and VM images actually contain the fix?

    Said: Referred to an EDK II stable202610 release.

    Now: There is no stable release after 202608; the Mu release (v2026080000.0.5) is named, and vendors can backport.

  37. 2026-10-31 · No. 12 EDK II PXE bug: which shipped firmware and VM images actually contain the fix?

    Said: Treated a missing GUID as a safe result.

    Now: A missing GUID isn't a safe result.

  38. 2026-10-31 · No. 12 EDK II PXE bug: which shipped firmware and VM images actually contain the fix?

    Said: Generalized the PCR0 measurement beyond Google.

    Now: PCR0 is scoped to Google's Shielded VM, which Google says diverges from the TCG profile.

  39. 2026-10-31 · No. 12 EDK II PXE bug: which shipped firmware and VM images actually contain the fix?

    Said: Misstated Hetzner's advice.

    Now: Hetzner advised avoiding the PXE boot and filtering ICMP, and on 30 October added filters for fragmented and oversized ICMPv4.

  40. 2026-10-31 · No. 12 EDK II PXE bug: which shipped firmware and VM images actually contain the fix?

    Said: Called Amazon Linux 2023's status disputed.

    Now: It is unstated.

  41. 2026-10-31 · No. 11 Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines

    Said: The title called the notes "LLM-visible", the very thing in question.

    Now: Retitled "Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines".

  42. 2026-10-31 · No. 11 Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines

    Said: The abstract reported findings from an audit that hasn't been run.

    Now: The abstract says what happened, what was measured, and what is proposed.

  43. 2026-10-31 · No. 11 Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines

    Said: Predicted a 10% rate with a falsifier the binomial arithmetic didn't support.

    Now: Only 0 of 200 gives an upper bound under 2%, and 28 are needed for a lower bound over 10%; the 10% prediction is dropped for a 2% decision threshold.

  44. 2026-10-31 · No. 11 Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines

    Said: The abstract equated static versus routed notes with harmless versus harmful.

    Now: It no longer makes that claim.

  45. 2026-10-31 · No. 11 Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines

    Said: Cited ReptileHaus's findings to a secondary source.

    Now: Cites the ReptileHaus article itself.

  46. 2026-10-31 · No. 11 Written for the Chatbot, Sent to Everyone: auditing the text help pages carry for machines

    Said: Reported OWASP #855 from an earlier run.

    Now: Reports the September rerun (2,924 distinct sites), with its scope.

  47. 2026-10-31 · No. 10 Broccoli2Shell: counting vendored Brotli decoders hiding in the software supply chain

    Said: Read 50 failed exploitation attempts as a sign the bug might be deprioritized.

    Now: ELLIO's own chain got shells on four default targets; the 0 of 50 came from one attacker's broken chain.

  48. 2026-10-31 · No. 10 Broccoli2Shell: counting vendored Brotli decoders hiding in the software supply chain

    Said: Mischaracterized what CISA's KEV listing and BOD 26-04 mean.

    Now: A KEV listing means evidence of exploitation, and the required action involves exposure-based prioritization.

  49. 2026-10-31 · No. 10 Broccoli2Shell: counting vendored Brotli decoders hiding in the software supply chain

    Said: Described a bug mechanism with status codes (ERROR_STATUS_SUCCESS 0, FAILURE −1) found in no source.

    Now: The mechanism is rewritten from the Node.js advisory and ELLIO; the invented codes are gone.

  50. 2026-10-31 · No. 10 Broccoli2Shell: counting vendored Brotli decoders hiding in the software supply chain

    Said: Made claims about the Go port that weren't on its v1.2.7 page.

    Now: Those claims are removed.

  51. 2026-10-31 · No. 09 What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs

    Said: The title promised a replication of Cohere's study.

    Now: Retitled "What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs"; the essay's own arithmetic rules out a replication.

  52. 2026-10-31 · No. 09 What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs

    Said: Described Check Point's 15 of 32 and 4 of 32 as network calls.

    Now: They are generated commands, not network calls.

  53. 2026-10-31 · No. 09 What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs

    Said: Cited RFC 8020 for the point about IP-literal addresses.

    Now: The citation is removed; the point stands as common knowledge.

  54. 2026-10-31 · No. 09 What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs

    Said: Mixed up gigabytes and gibibytes for the checkpoint size.

    Now: 360 × 10⁹ bytes is 335.3 GiB, matching the recipe; checkpoint size is kept separate from serving memory.

  55. 2026-10-31 · No. 09 What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs

    Said: Attributed Zenity's results to Vals.

    Now: Zenity has its own reference.

  56. 2026-10-31 · No. 09 What a Local GPU Can and Can't Check About Qwen3.8's Phone-Home URLs

    Said: Overstated what zero events in 300,000 runs shows.

    Now: Zero in 300,000 is evidence against a rate above 1 in 100,000, and conditional runs can't stand in for unprompted episodes.

  57. 2026-10-31 · No. 08 Architectural Defenses Against Prompt Injection

    Said: Said the defense treats user content as untrusted.

    Now: In the paper's typical setting, system and user content are trusted and tool output is untrusted.

  58. 2026-10-31 · No. 08 Architectural Defenses Against Prompt Injection

    Said: Attributed the role policy to channels.py.

    Now: The role policy is in server.py and the paper; channels.py handles the side-band only.

  59. 2026-10-31 · No. 08 Architectural Defenses Against Prompt Injection

    Said: Gave the paper's 10% attack success rate without its conditions.

    Now: 10% is the highest rate at medium reasoning effort; at low effort it is 36% against the defense, 100% undefended and 82% for SecAlign.

  60. 2026-10-31 · No. 08 Architectural Defenses Against Prompt Injection

    Said: Misstated the defense's utility cost.

    Now: AgentDojo clean utility goes from 66.32 to 63.16 at learning rate 0.1 (−3.16).

  61. 2026-10-31 · No. 08 Architectural Defenses Against Prompt Injection

    Said: Called prefix caching incompatible with the defense, and gave the wrong reason for eager execution.

    Now: Prefix caching is unimplemented, not incompatible; eager execution is on because CUDA-graph replay can't see the per-step channel data.

  62. 2026-10-31 · No. 08 Architectural Defenses Against Prompt Injection

    Said: Proposed comparing against a "-vanilla" model with prefix caching on.

    Now: No such configuration exists; "-vanilla" runs inside the gated server with prefix caching off, so the study adds stock-vLLM arms beside it.

  63. 2026-10-30 · No. 07 The Missing Ring-0 Protections for AI Agents

    Said: Said LangChain's security policy sets no single default code-execution posture.

    Now: The policy never mentions code execution; the sentence is replaced by a quote from it about permissions.

  64. 2026-10-30 · No. 07 The Missing Ring-0 Protections for AI Agents

    Said: Described the CVE chain without its prerequisites.

    Now: States that CVE-2026-28277 alone is moderate, post-exploitation and high-privilege, that the chain needs an exposed filter, and that these are fixed by patching, not by the proposed supervisor.

  65. 2026-10-30 · No. 07 The Missing Ring-0 Protections for AI Agents

    Said: Treated NIST's project as a regulatory signal.

    Now: It is a proposed demonstration project whose comment period has closed.

  66. 2026-10-30 · No. 07 The Missing Ring-0 Protections for AI Agents

    Said: Cited NVIDIA and Intel in support of the proposed plan.

    Now: They are no longer cited for it.

  67. 2026-10-30 · No. 06 Black-Box Detection of Model Distillation using Watermark Basins

    Said: Listed the paper's author as "arXiv".

    Now: Names the authors: Hongyan Chang, Hady Elsahar, Tom Sander, Pierre Fernandez, et al. (FAIR / Meta Superintelligence Labs).

  68. 2026-10-30 · No. 06 Black-Box Detection of Model Distillation using Watermark Basins

    Said: Called the reasoning-fine-tune detection result (Table 7) disputed.

    Now: It is settled in the paper: 0/0/0 at 0.1% false-positive rate.

  69. 2026-10-30 · No. 06 Black-Box Detection of Model Distillation using Watermark Basins

    Said: Said the reasoning fine-tune was in the paper's math domain.

    Now: The fine-tune used 1,900 code and science traces; only the audit domain is math.

  70. 2026-10-30 · No. 06 Black-Box Detection of Model Distillation using Watermark Basins

    Said: Said capability was retained or improved after the reasoning fine-tune.

    Now: GSM8K rose (+0.96 to +4.77) but Math-500 fell (−1.11 to −3.60).

  71. 2026-10-30 · No. 06 Black-Box Detection of Model Distillation using Watermark Basins

    Said: Gave 13.5% as the maximum false-positive rate for the worst key.

    Now: 13.5% is the floor; the worst key reaches 23.4–26.2% at a nominal 5%.

  72. 2026-10-30 · No. 05 When clock uncertainty changes an incident verdict

    Said: Said 4 of 38 papers in the survey describe input ordering.

    Now: 4 of 52 host-detector papers describe input ordering; the 38 is a separate cohort (provenance-era papers, 28 of which trust the evidence).

  73. 2026-10-30 · No. 05 When clock uncertainty changes an incident verdict

    Said: Gave the survey's authors' initials as "J. & A.".

    Now: Names the authors correctly: René Heinzl and Markus Nissl.

  74. 2026-10-30 · No. 05 When clock uncertainty changes an incident verdict

    Said: Said the survey's coding of KAIROS conflates clock evidence with concept drift.

    Now: That overstated it; the cell quotes a concept-drift passage and the clock question is coded "not found in full text".

  75. 2026-10-30 · No. 05 When clock uncertainty changes an incident verdict

    Said: Gave KAIROS's GPU memory as 0.74–3.93 GB, figures that came from a different row of the table.

    Now: Gives KAIROS's own figures, 2.26–23.85 GB, scoped to the E5 datasets.

  76. 2026-10-30 · No. 05 When clock uncertainty changes an incident verdict

    Said: Cited the Basin et al. and Michael et al. papers with wrong titles, initials and venue.

    Now: Cites "On Real-Time Monitoring with Imprecise Timestamps" (RV 2014) and "On the Forensic Validity of Approximated Audit Logs" (ACSAC 2020).

  77. 2026-10-30 · No. 05 When clock uncertainty changes an incident verdict

    Said: Used a single ±500 ms error bar for all timestamp sources.

    Now: Uses the error intervals each source actually documents; the single figure contradicted them.

  78. 2026-10-30 · No. 04 Does PII masking survive a different document generator?

    Said: Read as though the proposed study had already been built and tested.

    Now: Says outright that nothing has been built or run; the study is described as a plan, and only the original benchmark's numbers are reported as results.