frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

California issues historic fine over lawyer's ChatGPT fabrications

https://calmatters.org/economy/technology/2025/09/chatgpt-lawyer-fine-ai-regulation/
60•geox•2h ago

Comments

lordnacho•1h ago
This is why there are certain jobs AI can never take: we are wired for humans to be responsible. Even though a pilot can do a lot of his work via autopilot, we need a human to be accountable. For the pilot, that means sitting in the plane. But there are plenty of other jobs, mostly high-earning experts, where we need to be able to place responsibility on a person. For those jobs, the upside is that the tool will still be available for the expert to use and capture the benefits from.

This lawyer fabricating his filings is going to be among the first in a bunch of related stories: devs who check in code they don't understand, doctors diagnosing people without looking, scientists skipping their experiments, and more.

unshavedyak•1h ago
> This is why there are certain jobs AI can never take

You're thinking too linearly imo. Your examples are where AI will "take", just perhaps not entirely replace.

Ie if liability is the only thing stopping them from being replaced, what's stopping them from simply assuming more liability? Why can't one lawyer assume the liability of ten lawyers?

lordnacho•32m ago
Then there will still be lawyers. More productive, higher income lawyers.

Just like with a lot of other jobs that got more productive.

observationist•27m ago
People who think like this cannot be convinced; they're unaware of the acceleration of the rate of progress, and it won't change until they clash with reality. Don't waste your time and energy trying to convince them.

They don't understand how to calibrate their model of the world with the shape of future changes.

The gap between people who've been paying attention and those who haven't is going to increase, and the difficulty in explaining what's coming is going to keep rising, because humans don't do well with nonlinearities.

The robots are here. The AI is here. The future is now, it's just not evenly distributed, and by the time you've finished arguing or explaining to someone what's coming, it'll have already passed, and something even weirder will be hurtling towards us even faster than whatever they just integrated.

Sometime in the near future, there won't be much for people to do but stand by in befuddled amazement and hope the people who set this all in motion knew what they were doing (because if we're not doing that, we're all toast anyway.)

gdulli•1h ago
Where could lawyers be learning this behavior?

https://www.theguardian.com/us-news/2025/apr/24/california-b...

abeppu•1h ago
> In recent weeks, she’s documented three instances of judges citing fake legal authority in their decisions.

So lawyers use it, judges use it ... have we seen evidence of lawmakers submitting AI-generated language in bills or amendments?

dylan604•25m ago
I mean, we've seen laws that were written by lobbyists with zero changes. Does it matter if it was AI generated or not at that point? The congress critters are not rewriting what they've been told to do if they've even read it after being told what to do.
milesvp•12m ago
I know a lawyer who almost took a job in state government where one of the primary duties was to make sure that the punctuation in the bills going through the state legislature were correct and accurate. For her, part of the appeal of the job, was that it would allow her to subtly alter the meaning of a bill being presented. Apparently it is a non trivial skill to be able to determine how judges are likely to rule on cases due to say, the presence, or the absence of an oxford comma.

There was an entire team dedicated to this work, and the hours were insane when the legislature was in session. She ended up not taking the job because of the downsides associated with moving to the capital, so I don't know more about the job. I'd be curious how much AI has changed what that team does now. Certainly, they still would want to meticulously look at every character, but it is certainly possible that AI has gotten better at analyzing the "average" ruling, which might make the job a little easier. What I know about law though, is that it's often defined by the non average ruling, that there's sort of a fractal nature to it, and it's the unusual cases that often forever shape future interpretations of a given law. Unusual scenarios are something that LLMs generally struggle with, and add to that the need to creatively come up with scenarios that might further distort the bill, and I'd expect LLMs to be patently bad at creating laws. So while, I have no doubt that legislators (and lobbyists) are using AI to draft bills, I am positive that there is still a lot of work that goes into refining bills, and we're probably not seeing straight vibe drafting.

ratelimitsteve•1h ago
The fact that they interviewed someone named "Charlotin" for this article is a coincidence I can't ignore
FlamingMoe•1h ago
$10k is not a high enough fine to dissuade this behavior. Large law firms will just approach this the same way that big tech accepts GDPR fines as part of the cost of doing business.
amoshebb•48m ago
I'm numb to it after many "EU fines Householdnamecorp a zillion doubloons" type headlines, but using "historic fine" to describe $10k to a lawyer feels odd.
dylan604•31m ago
expecting the same level of fine to an individual person as opposed to a faceless corp really shows how numb you must be. for an attorney to be fined that much is not normal. TFA even shows example of higher fines issued to law firms, while still not as high as your zillion doubloons hyberbole it still shows the distinction between individual and s/corporation/law firm/. EU fines have been progressively getting higher especially for repeat offenders. it would be unwise to expect different in legal matters
grues-dinner•27m ago
> The fine appears to be the largest issued over AI fabrications by a California court

This is a bit like all the stats like "this is appears to be an unprecedented majority in the last 10 years in a Vermont county starting with G for elections held on the 4th when no candidate is from Oklahoma".

Lots of things are historic but that doesn't necessarily mean they're impressive overall. More interesting is how many of these cases have already been tried such that this isn't "historic" for being the first one decided.

jsbisviewtiful•38m ago
Doesn't even feel controversial. LLMs hallucinate and in law that's not acceptable. Increase the fine though to really punish those doing this.
abirch•19m ago
Hopefully the fines grow exponentially for repeat offenders. Seems like now, lawyers can use AI to DDOS the other counsel.
donatj•34m ago
$10,000? That's a slap on the wrist. I don't say this lightly, this should have been jail time for someone. You're making a mockery of our most sacred institutions.
dylan604•29m ago
for first offenses for something like this you'd suggest jail time? hope you find excuses to skip your next jury summons. however, it is typical in jury selection to be asked by the defense if you'd be able to agree with a minimum sentence while the prosecutors like to ask if you'd be able to agree to the maximum. personally was asked if I could agree to 99 years for someone's first offense of GTA. I said no, and was dismissed. Sounds like you'd have said yes.
quickthrowman•11m ago
For someone who had to attend 6+ years of school and had to pass a professional licensing exam with ethics questions? Yes, I do. $10,000 is one week of billable hours at $250/hr.

Do you think a Civil Engineer (PE) should be held liable if they vibe engineered a bridge using an LLM without reviewing the output? For this hypothetical, let’s assume an inspector caught the issue before the bridge was in use, but it would’ve collapsed had the inspector not noticed.

Analemma_•11m ago
There's something grimly hilarious about knee-jerk demands for jail time for [other profession] for using AI, when a bunch of us here are eagerly adopting it into our own workflows as fast as we can.

Why jail time for lawyers who use Chat-GPT, but not programmers? Are we that unimportant compared to the actual useful members of society, whose work actually has to be held to standards?

I don't think you meant it this way, but it feels like a frank admission that what we do has no value, and so compared to other people who have to be correct, it's fine for us to slather on the slop.

hollerith•2m ago
>jail time

Surely it would suffice to eject him from the California bar.

jerlam•33m ago
I searched for the lawyer's name in the state bar association, they've been practicing for over 13 years. Even has electrical engineering in their background.
SilverElfin•32m ago
> He thinks it is unrealistic to expect lawyers to stop using AI. It’s become an important tool just as online databases largely replaced law libraries and, until AI systems stop hallucinating fake information, he suggests lawyers who use AI to proceed with caution.

I think this is a good reason for fines to not be incredibly big. People are using AI all the time. There will be a growing period until they learn of its limitations.

dylan604•27m ago
The slope of the learning curve can be adjusted though with the level of fines being a big adjustment. Only I'd suggest not the first time for any one lawyer per firm, but per incident for each firm.
sidewndr46•30m ago
$10,000, wow what a shocker! I don't know how anyone who can afford to live in California could ever expect to pay such a fine. I expect the lawyer will soon have to declare bankruptcy.
jerf•22m ago
Do lawyers and judges not by now have software that turns all these citations into hyperlinks into some relevant database? Software that would also flag a citation as not having a referent? Surely this exists and is expensive but in wide usage...?

It's not a large step after that to verify that a quote actually exists in the cited document, though I can see how perhaps that was not something that was necessary up to this point.

I have to think the window on this being even slightly viable is going to close quickly. When you ship something to a judge and their copy ends up festooned with "NO REFERENT" symbols it's not going to go well for you.

KittenInABox•15m ago
Part of an issue is that there's already in existence a lot of manual entry and a lot of small/regional courts with a lot of specificity for individual jurisdictions. Unification of standards is a long way away, I mean, tech hasn't even done it
alwa•14m ago
> Mostafavi told CalMatters he wrote the appeal and then used ChatGPT to try and improve it. He said that he didn’t know it would add case citations or make things up. He thinks it is unrealistic to expect lawyers to stop using AI. [...] “In the meantime we’re going to have some victims, we’re going to have some damages, we’re going to have some wreckages,” he said. “I hope this example will help others not fall into the hole. I’m paying the price.”

Wow. Seems like he really took the lesson to heart. We're so helpless in the face of LLM technology that "having some victims, having some damages" (rather than reading what you submit to the court) is the inevitable price of progress in the legal profession?

21 of 23 citations are fake, and so is whatever reasoning they purport to support, and that's casually "adding some citations"? I sometimes use tools that do things I don't expect, but usually I'd like to think I notice when I check their work... if there were 2 citations when I started, and 23 when I finished, I'd like to think I'd notice.

TikTok algorithm to be retrained on US user data under Trump deal

https://www.bbc.com/news/articles/c4gj7mlg9vdo
1•1659447091•1m ago•0 comments

App-Solutely Modded: Surveying Modded App Market Operators and Original App Devs

https://www.lightbluetouchpaper.org/2025/09/22/app-solutely-modded-surveying-modded-app-market-op...
1•gnufx•2m ago•0 comments

The billion-dollar infrastructure deals powering the AI boom

https://techcrunch.com/2025/09/22/the-billion-dollar-infrastructure-deals-powering-the-ai-boom/
1•rntn•4m ago•0 comments

A collection of technical things every software developer should know about

https://github.com/mtdvio/every-programmer-should-know
2•redbell•8m ago•0 comments

Boring Oracle became cool again

https://www.cnn.com/2025/09/16/tech/oracle-larry-ellison-ai-computing
1•mathattack•9m ago•2 comments

Choose Your Own Adventure

https://www.filfre.net/2025/09/choose-your-own-adventure/
2•naves•11m ago•0 comments

Diffusion Beats Autoregressive in Data-Constrained Settings

https://blog.ml.cmu.edu/2025/09/22/diffusion-beats-autoregressive-in-data-constrained-settings/
2•djoldman•12m ago•0 comments

If search results were presented like Instagram stories

https://www.hopit.ai/stories?category=travel_explainer&slug=noctourism-embracing-the-night-in-ame...
8•Arkid•14m ago•1 comments

Show HN: Python Audio Transcription: Convert Speech to Text Locally

https://www.pavlinbg.com/posts/python-speech-to-text-guide
2•Pavlinbg•15m ago•0 comments

Exploring GrapheneOS secure allocator: Hardened Malloc

https://www.synacktiv.com/en/publications/exploring-grapheneos-secure-allocator-hardened-malloc
1•throawayonthe•15m ago•1 comments

H1B and the $100k Fee

https://thezvi.wordpress.com/2025/09/22/h1-b-and-the-100k-fee/
1•speckx•15m ago•0 comments

Lockheed Martin's New F-35 Drone Tech [video]

https://www.youtube.com/watch?v=zi8XZyAD8mM
1•mgh2•16m ago•1 comments

"Radiohead – Pyramid" Song 800% slower via extreme sound stretching algorithm

https://youtu.be/XiKWfcy-Z70
1•danielfalbo•16m ago•1 comments

Context Engineering Is the New Full Stack of AI Agents

https://zilliz.com/blog/why-context-engineering-is-becoming-the-full-stack-of-ai-agents
1•Fendy•17m ago•1 comments

Bitcoin Treasury Company M&A

https://www.bloomberg.com/opinion/newsletters/2025-09-22/bitcoin-treasury-company-m-a
1•feross•17m ago•1 comments

Nvidia to Invest Up to $100B in OpenAI

https://www.wsj.com/tech/nvidia-openai-100-billion-deal-data-centers-d2f85cae
2•bookofjoe•21m ago•3 comments

Low Earth Orbit Visualization

https://platform.leolabs.space/visualization
1•handfuloflight•22m ago•0 comments

To the Best Talent in the World: An Invitation

https://aloe.inc/blog/the-best-talent-in-the-world
2•arunbahl•23m ago•0 comments

Top Fossil Fuel Producing Nations Plan to Blow Past Climate Targets

https://insideclimatenews.org/news/22092025/top-fossil-fuel-nations-will-blow-past-climate-targets/
3•lycopodiopsida•25m ago•0 comments

Real Madrid Signs 11-Year-Old African Giant

https://www.footboom1.com/en/news/basketball/2752741-real-madrid-signs-11-year-old-african-giant
1•wslh•26m ago•0 comments

Pareto Principle [80/20 rule/Law of the vital few/Principle of factor sparsity]

https://en.wikipedia.org/wiki/Pareto_principle
2•bookofjoe•26m ago•0 comments

AI-Generated "Workslop" Is Destroying Productivity

https://hbr.org/2025/09/ai-generated-workslop-is-destroying-productivity
4•McScrooge•26m ago•0 comments

Demystifying Agentic Memory

https://alexspyropoulos.com/posts/demystifying-agentic-memory/
3•alexspyr•29m ago•0 comments

How I Vibe Coding? (Sept 2025 Edition)

https://xuanwo.io/2025/06-how-i-vibe-coding-sept-2025-edition/
1•xuanwo•30m ago•0 comments

Model literals, semantic aliases, and preference-aligned routing for LLMs

https://docs.archgw.com/guides/llm_router.html
1•honorable_coder•30m ago•1 comments

Market design can feed the poor

https://worksinprogress.co/issue/how-market-design-can-feed-the-poor/
2•zdw•31m ago•2 comments

Automate User Interviews with AI

https://theproductfeedbackcompany.com/
1•bobcoi•32m ago•1 comments

AMD Ryzen AI Max+ "Strix Halo" Performance with ROCm 7.0

https://www.phoronix.com/review/amd-rocm-7-strix-halo
2•rbanffy•33m ago•0 comments

How Samin Nosrat Learned to Love the Recipe

https://www.newyorker.com/culture/persons-of-interest/how-samin-nosrat-learned-to-love-the-recipe
1•mitchbob•33m ago•1 comments

Canon updates a PowerShot with higher price and fewer features

https://m.dpreview.com/news/9212403257/canon-powershot-360-hs-a-announcement
3•PaulHoule•33m ago•1 comments