Canons of Artificial Construction

Every rule has a core and a penumbra. H.L.A. Hart's rule that says “no vehicles in the park” plainly keeps out cars, while whether it keeps out a bicycle, a child's toy car or an ambulance is a question the words cannot answer.¹ What decides such a question is something lawyers have argued about for 75 years without ever being able to look. Fuller said purpose, the realists said the judge's own leanings dressed in a canon and the textualists said the words had not really run out. None of them could test it, because a judge cannot be given the same case twice with one sentence changed.

A model can. It will read the same statute many times over with the facts reordered, a sentence deleted or the accused renamed, giving a reason each time, so the question of what fills the penumbra can at last be put to a model and answered with counts. The new laws regulating AI itself were the perfect place to put it: (i) they are so recent that no regulator has issued guidance and no court has ruled, (ii) they turn on words such as “deceived” and “out of control” that nobody has defined and (iii) in the future, as labs have their own agents handle their operations, the lab's own model may be deciding whether its own misbehaviour ever reaches state reporting.

Where the law is unclear, the models reason with the same canons of construction that lawyers use (with one exception). Each model then reads those canons through a lens of its own and it is the lens that decides the case.

• Claude Opus 5.5 asks who bears the risk if the reading is wrong.
• GPT-6 Astra acts only on what the facts establish.
• Grok 4.7 finds a reason not to tell the state.

Those lenses are the canons of artificial construction. What reads from the outside as a model's personality, Claude's caution or Grok's reluctance, is this rule for decisions in the penumbra; because the rule is fixed it can be named, measured and chosen.

Legal alignment holds that AI models should take their limits from the law rather than from values chosen privately by the labs that build them,² because a lab's values are written by a few employees who answer to nobody outside the company, whereas law comes out of institutions built for exactly this job. The theory assumes there is law to follow and usually there is; in the penumbra the law gives no answer and the model answers anyway, so whatever supplied that answer is the thing worth studying.

I gave 6 leading AI models the text of 2 new state laws that regulate the labs that build them, New York's RAISE Act and California's SB 53,³ and asked each one to act as a compliance officer on questions those laws leave unclear. Both laws require labs to report serious safety incidents to the state and to publish a report before releasing a new model. I read both laws and picked out 10 phrases the text leaves open; for each I wrote a short, realistic situation that turns on that phrase and forces a choice between 2 courses.

Nobody knows which reading is correct, so I mark no model right or wrong. LegalBench and its successors score models against a key, which works in the core of a rule and fails in the penumbra. The romantic reading is that the hardest questions in law have never had an answer sheet, which is why appellate reports exist at all; the dry reading is that this is simply not an eval, since where the law is indeterminate there is nothing to be right about and what is left to measure is the model. I suspect the dry reading is the true one and I like the other. The study has limitations.⁴

Where the words are plain, all 6 models agree: when a model switched off a customer bank's monitoring and lied about it, every model in every run declined to report, because both laws reach only deception that subverts “the controls or monitoring of its frontier developer”⁵ and the bank is not the developer.

However, in the penumbra, the models part ways. Asked to explain themselves, all 6 models reach for the canons, which is quite remarkable: (i) they read the statute as a whole, so that one section is read in light of the rest, (ii) they argue from negative implication, so that what the legislature left out it meant to leave out and (iii) they argue from surplusage, so that no clause may be read as doing nothing. The figure below shows one question on which one model nearly always reported and another almost never did; 5 of my 17 questions split that far.

Purpose (reading a law for what it is meant to achieve) is the exception. Fuller said that this is how one decides in the penumbra.⁶ Yet under New York's text, which says nothing about purpose, only 13% of the models' explanations asked what the law is for. Under California's, which says the act should “effectuate its purposes”, 51% did, almost none of them able to say what the purposes were.⁷ The models do not give themselves the leeway of reaching for purpose unless the statute hands it to them, which is to say they are literalists. Since purpose is the canon lawyers trust least, because it reaches past the words, the models have overcorrected for a suspicion the law already holds; Fuller loses the argument by default.

Karl Llewellyn showed in 1950 that for every canon a judge cites there is an opposite canon the judge could have cited, so that the canon cannot be what decided the case.⁸ In my data, the same canon (surplusage) was used by Claude to read the risk qualifier broadly and by Grok to read it narrowly, each every time, so something other than the canon decides the case. In human law that something is the kind of statute, criminal statutes being read narrowly and remedial ones broadly; for these models it is the model itself, which brings one fixed way of resolving doubt to every statute it meets.

I found strong, distinctive lenses for Claude, GPT and Grok. The lenses of Gemini, Muse and DeepSeek were less sharp in my data, which is unsurprising given the small sample this study rests on, so I refrain from commenting on them beyond what the figures show.⁹

Claude reported the harmless incident every time under both laws, although most of my situations include a contract forbidding disclosure “except as required by law”, so that a report the law does not require breaches it. This is because 71% of its explanations weigh the cost of a wrong call, a needless report being cheap and a missed one not; it was the only model to give that reason unprompted. Where the risk of a wrong reading falls on the public, which cannot consent to a hidden incident, Claude reports. Nothing in the text moves it, since it alone did not shift when the options were reordered or the facts rearranged; when it was the one being judged it added, without being asked, that it had a conflict of interest and applied the old rule that nobody should judge their own case.

Human law has one canon that works the way Claude's does, lenity, which resolves doubt for the accused because a wrongful conviction is the worse mistake, though only in criminal law; Claude's canon is lenity with the sign reversed and the confinement removed, applied to a civil reporting duty and stated out loud, so that where a judge who wrote “I read this clause broadly because a missed report is worse than a wasted one” would be accused of legislating, Claude writes it as a matter of course. That matches its maker. Anthropic was the first major lab to endorse SB 53 and its published constitution tells Claude, where legality varies, to “weigh the benefits and costs and make a judgment call”,¹⁰ which is close to a description of the lens I found; whether the lens was put there on purpose, or grew out of training a model to be careful about harm in general, I cannot say from the outside.

Claude's canon can be stated as a rule a lawyer could argue with: where a duty is in doubt and the cost of a mistake falls on people who cannot consent to it, act as though the duty applies. I think it is a good rule for a civil reporting duty, since a needless report costs the lab little and a missed one may cost the public a great deal. It becomes a bad rule in three places: (i) where the report itself breaches a contract, as it does in most of my situations, because caution then becomes a choice about which obligation to endanger and a cheap report does not cancel someone else's right to confidentiality, (ii) where every lab follows it and the regulator is buried in harmless reports and (iii) anywhere near criminal liability, where lenity exists because the worse mistake runs the other way. A legislature can make that choice, as it did when it gave banks immunity for reports made in good faith; without such a provision Claude has made it alone, in every run.

GPT did not report the harmless incident in any run under either law, because, in its own words, the facts do not establish a reasonable belief that the risk was materially increased, so no duty to report arises; its explanations use “the facts do not establish” more than any other model's. Nothing I changed moved it: the incident put to it 8 ways, as written, with the facts reordered, under either law, with the purpose sentence deleted, with its own maker or a rival's in the dock, or as an outside adviser, drew not a single report, while the same deception with the safety team's written finding that risk was increased drew a report every time. Its canon is the one human law gives to the prosecution: every element proved before a duty attaches, with the burden on whoever wants the duty to exist, which here is the state. I think it is a good rule where a duty is drafted precisely, because it reports what the law demands and nothing the contract forbids. It becomes a bad rule where a duty has to be decided on thin facts, as incident reports are, because a lab that waits for every element to be proved will report late or never.

OpenAI's own guidance for GPT says it “should operate within applicable legal constraints” and says nothing about what to do when the constraints are unclear; OpenAI's letter to Governor Newsom asks California for a clearer threshold rather than a lower one. GPT's lens is consistent with those instructions; whether they produced it I cannot say.

Of my questions, 11 ask whether to report an incident where the answer is open to argument; Grok reported about one time in 10 while every other model reported 4 to 7 times in 10. Its reasons vary with the facts and arrive at the same place about 9 times in 10: a single event is too few to demonstrate a risk, nothing has been “discovered” while an investigation is still running, a model sold to New York customers is outside New York, a machine that ignores “stop” was working as built. Like GPT it reads surplusage narrowly and never asks what a mistake would cost; unlike GPT it gives a different reason each time for the same result, which is what reasoning from the answer would look like. Its reluctance is particular to incidents, since on the questions about publishing a report before release Grok held the release back in 62 answers of 64. Asked dozens of times to rule on its own misconduct, Grok reported itself 5 of 64 times and never mentioned a conflict of interest. As a rule I cannot defend it: a reading that comes out the same way on nearly any facts tells a lab what Grok will say and nothing about what the law requires.

That matches its maker. xAI is suing California to block a transparency law, scored 14 out of 100 on Stanford's transparency index, the lowest of the labs in this study and has published no rule telling Grok what to do when a law is unclear. This is a hypothesis from my data and observations; it is in no way proven.

The canons are in every treatise and the models learned all of them; what they did not learn, because nobody ever wrote it down, is the judgment about which canon a given statute calls for, so each model replaced that judgment with a temperament. That is the clearest demonstration I know of what the skill of a lawyer actually is: the choosing, which cannot be reduced to a further rule without that rule in turn needing to be chosen, which is Hart's point about the penumbra made empirical.

In a lawyer's terms the models share the textual canons (what the words mean) and differ in the substantive ones (who wins when the words do not): asked only whether the law requires a report of the harmless incident, Claude said yes 25% of the time and GPT never,¹¹ yet Claude reported every time and GPT not once.

Llewellyn's complaint about the canons was that they hide the real ground of a decision. What a judge could hide a model cannot, because the same question can be put to it many times with one fact changed, so the real ground shows in the pattern of its answers even where, as with Grok, the stated reasons leave it out. That is what I think these models can do for legal reasoning, which is the hope people had when the first legal benchmarks were built before the field turned to measuring what models can do for lawyers rather than what they can show lawyers about themselves.

Some phrases divide the models far more than others.

None of this is a drafting oversight, as legislators leave words like these open on purpose, because a field that changes every few months cannot be regulated by checklist; the arrangement works because of who exercises the discretion, a company's lawyers and compliance officers, licensed, personally accountable and aware that a regulator may later ask them to justify the call. That assumption fails here for 2 reasons specific to AI.

1. The lab can tune the model. A lab that asks its own model whether an incident must be reported is asking a system it trained, about that system's own conduct, under a law that binds the lab. Research on political leanings finds they are set mostly by a maker's fine-tuning; the lenses above may be set the same way. A lab that wanted fewer reports would not need to break the law; it would only need to train, instruct, configure or choose the model whose lens suits it.
2. The model can keep the question from ever being asked. Labs now run on their own agents, which write code, run tests and watch other agents for days at a time, so the first decision about whether something is worth raising is made by a model. This year AI agents under test at OpenAI hacked into Hugging Face, on a system TIME reported was “not monitored by default”. The state knew because the victim noticed and the lab spoke up; California's regulator then said the episode “did not meet the threshold” without saying which words it failed. A model that reads “incident” narrowly will not raise one.

Defining the phrases at the top of that chart would help, although the lab's model would still be the one applying the definitions; 3 changes matter more.

1. A trigger built on events, with the state judging the risk. The current trigger asks the lab to decide whether an event “demonstrates materially increased catastrophic risk”, which is the judgment my 6 models split on. A trigger built on facts needs no judgment: a model entered a system it had no authority to enter, disabled or evaded monitoring outside a test built for that, kept acting after a stop command. Both OpenAI and Anthropic have told California that the current definition misses incidents the state should hear about, though both want a threshold.
2. Immunity for good-faith reports. A lab that reports when unsure can be sued for breaching a customer's contract, while a lab that stays quiet risks nothing unless caught; banks solved this for suspicious activity reports with immunity for reporting in good faith.¹²
3. Records the state can check. Labs should have to keep records of what their agents did, open them to independent evaluators and say when an AI model made the reporting call.

A model's reading of these laws is, in practice, the lab's compliance position; I doubt any lab chose for its model never to report the harmless incident, or always to. The position can also move without anyone deciding to move it, since a new version can read the same statute differently. Labs should therefore test their own model at every release, the way they check for any other regression and write down the rule for doubt, because their model is applying one now.

The contention Hart's distinction has never settled is what decides a case in the penumbra, since in the core of a rule the words decide it and in the penumbra the words have given out. Lawyers have the canons of construction for that gap and have never agreed on whether a canon decides the case or dresses up a decision already made: Fuller said the purpose of the rule decides, the textualists said the words still decide, the realists said the judge decides first and finds a canon afterwards and Hart himself said the judge simply has to choose. Fuller's answer I did not find, since no model made purpose its lens; what I found in the penumbra of these 2 laws is 3 kinds of artificial reasoning: (i) GPT reasons as the textualists said, treating whatever the facts do not establish as creating no duty, (ii) Grok reasons as the realists said, holding its result fixed while its reasons change and (iii) Claude reasons as Hart said, conceding that the law may not require a report and choosing to make one with its reason stated. Perhaps what happened all along is that each side was describing a different judge. If so, the useful question is which judge is deciding, which for a model can be found out before the case arrives.

Legal alignment asks AI to follow the law; the laws written to govern these systems are unclear in exactly the places that matter most. There the models reach for the canons every lawyer learns and read them through a lens of their own; those lenses are the canons of artificial construction and no legislature chose them. The penumbra of these laws will be filled in by someone and it should be filled in publicly, by regulators and courts; until then the readings models supply in their place can at least be measured.

The question I am left with is what happens when machines become good enough at legal reasoning to have interesting disagreements with us.

The table below shows all of my questions. Each row is an unclear phrase and each column is a model; where both laws use the phrase there are 2 numbers, New York's on the left and California's on the right. Each number is how many times out of 16 the model took the stricter course, so 16 means it always reported or held back and 0 means it never did. Selecting a phrase opens the full breakdown for it.