diz, diz@awful.systems

And the other "nuanced" take, common on my linkedin feed, is that people who learn how to use (useless) AI are gonna replace everyone with their much increased productive output.

Even if AI becomes not so useless, the only people whose productivity will actually improve are the people who aren't using it now (because they correctly notice that its a waste of time).

That philosophy always ends in stepping into dogshit to try to boost stock prices.

When they tested on bugs not in SWE-Bench, the success rate dropped to 57‑71% on random items, and 50‑68% on fresh issues created after the benchmark snapshot. I’m surprised they did that well.

After the benchmark snapshot. Could still be before LLM training data cut off, or available via RAG.

edit: For a fair test you have to use git issues that had not been resolved yet by a human.

This is how these fuckers talk, all of the time. Also see Sam Altman's not-quite-denials of training on Scarlett Johansson's voice: they just asserted that they had hired a voice actor, but didn't deny training on actual Scarlett Johansson's voice. edit: because anyone with half a brain knows that not only did they train on her actual voice, they probably gave it and their other pirated movie soundtracks massively higher weighting, just as they did for books and NYT articles.

Anyhow, I fully expect that by now they just use everything they can to cheat benchmarks, up to and including RAG from solutions past the training dataset cut off date. With two of the paper authors being from Microsoft itself, expect that their "fresh issues" are gamed too.

Yeah I'm thinking that people who think their brains work like LLM may be somewhat correct. Still wrong in some ways as even their brains learn from several orders of magnitude less data than LLMs do, but close enough.

You can film with an actual camera then use video to video to make it look very AI. If you're just grifting, that would be the way to go I think.

They're also very gleeful about finally having one upped the experts with one weird trick.

Up until AI they were the people who were inept and late at adopting new technology, and now they get to feel that they're ahead (because this time the new half-assed technology was pushed onto them and they didn't figure out they needed to opt out).

I was writing some math code, and not being an idiot I'm using an open source math library for doing something called "QR decomposition", and its efficient, and it supports sparse matrices (matrices where many numbers are 0), etc.

Just out of curiosity I checked where some idiot vibecoder would end up. AI simply plagiarizes from some shit sample snippets which exist purely to teach people what QR decomposition is. It's actually unusable, due to being numerically unstable.

Who in the fuck even needs this shit to be plagiarized, anyway?

It can't plagiarize a production quality implementation, because you can count those on the fingers of one hand, they're complex as fuck and you can't just blend a few together to try to pretend you didn't plagiarize.

The answer is, people who are peddling the AI. They are the ones who ordered plagiarism with extra plagiarism on top. These are not coding tools, these are demos to convince the investors to buy the actual product, which is company's stock. There's a little bit of tool functionality (you can ask them to refactor the code), but it's just you misusing a demo to try to get some value out of it.

And to that end, the demos take every opportunity to plagiarize something, and to talk about how the "AI" wrote the code from scratch based on its supposed understanding of fairly advanced math.

And in coding, it is counter productive to plagiarize. Many of the open source libraries can be used in commercial projects. You get upstream fixes for free. You don't end up with some bugs or worse yet security exploits that may have been fixed since the training cut-off date.

No fucking one in the right mind would willingly want their product to contain copy pasted snippets from stale open source libraries, passed through some sort of variable-renaming copyright laundering machine.

Except of course the business idiots who are in charge of software at major companies, who don't understand software. Who just failed upwards.

They look at plagiarized lines and count them as improved productivity.

Indistinguishable from a business idiot.

Its also interesting that this is the most conservative, pro “its not just memorizing” estimation possible : they multiplied the probabilities of consequent tokens. Basically it means if it starts shitting out a quote it will not be able to stop quoting until their anti copy the whole book finetuning kicks in after 50 words or so.

It can probably output far more under a realistic test (always picking the top token, temperature =0)

If it was a basement dweller with a chatbot that could be mistaken for a criminal co-conspirator, he would've gotten arrested and his computer seized as evidence, and then it would be a crapshoot if he would even be able to convince a jury that it was an accident. Especially if he was getting paid for his chatbot. Now, I'm not saying that this is right, just stating how it is for normal human beings.

It may not be explicitly illegal for a computer to do something, but you are liable for what your shit does. You can't just make a robot lawnmower and run over a neighbor's kid. If you are using random numbers to steer your lawnmower... yeah.

But because it's OpenAI with 300 billion dollar "valuation", absolutely nothing can happen whatsoever.

In theory, at least, criminal justice's purpose is prevention of crimes. And if it would serve that purpose to arrest a person, it would serve that same purpose to court-order a shutdown of a chatbot.

There's no 1st amendment right to enter into criminal conspiracies to kill people. Not even if "people" is Sam Altman.

It's curious how if ChatGPT was a person - saying exactly the same words - he would've gotten charged with a criminal conspiracy, or even shot, as its human co-conspirator in Florida did.

And had it been a foreign human in the middle east, radicalizing random people, he would've gotten a drone strike.

"AI" - and the companies building them - enjoy the kind of universal legal immunity that is never granted to humans. That needs to end.

If I pirated a book and wrote a review of it, would that make the review copyright infringement?

That would leave pirating the book be copyright infringement. But if you used AI (trained on the pirated book) to write a review, then pirating the book wouldn't be copyright infringement, at least according to this judge.

Re Anthropic, their ruling was that downloading itself was copyright infringement, regardless of whether Anthropic distributed works while torrenting them.

In Meta's case the plaintiffs will likely find it impossible to demonstrate that the data which Meta uploaded contained specifically their books. Maybe all the other torrent users who were downloading that torrent were downloading other parts of it.

I appreciate the sentiment but I also hate the whole "AI is a power loom for coding".

The power loom for coding is called "git clone".

What "AI" (LLM) tools provide is just English as a programming language with plagiarized sum total of all open source as the standard library. English is a shit programming language. LLMs are shit at compiling it. Open source is awesome. Plagiarized open source is "meh" - you can not apply upstream patches.

It's not about ceding victory, it's about whether we accept shit talking plaintiff's lawyers as an adequate substitute for a slap on the wrist, or not. Clearly the judge wants to appear impartial.

Plaintiff made a perfectly good argument that meta downloaded the books illegally, and that this downloading wasn't necessary to enable a (fair or not) use. A human critic does not get a blanket license to pirate any work he might want to criticize, even though critique is fair use.

Thought about it some more, the most charitable I can get is that Meta's judge thinks someone else could win the case if they have a specific book that was torrented and then they point at the general situation with AI slop in bookstores and argue that AI harms book sales.

I can not imagine that working. At all. So the AI is producing slop of infinitesimally higher quality because it was trained on a pirated copy of your book in particular. Clearly the extra harm to specifically your business due to piracy of specifically your books would be rather small, as this very judge would immediately point out. In fact the AI slop is so shit that people only buy it by mistake, so its quality doesn't really matter.

Maybe news companies could sometimes win lawsuits like this, but book authors, no way.

I think it is just pure copium to see this ruling in any kind of positive light. Alsup (misanthropic's judge) at least was willing to ding an AI company for pirating books (although he was probably only willing to ding them for that because it wouldn't be fatal to them the way it would be to Meta). This guy wouldn't even do that bare minimum.

And the whole approach is insane. You can't make a movie without getting a movie rights contract with the author. A movie adaptation of a book is far more transformative than anything AI does. Especially the "training" which is just fucking gradient descent, you nudge a bunch of numbers towards replicating the works, over and over again, in a purely mechanical process.

Nobody ever had to successfully argue that movie sales harm book sales just to treat movie adaptations as derivative work.

But he’s saying that plaintiffs need to demonstrate said disruption, to even get to the jury.

He said:

In cases involving uses like Meta’s, it seems like the plaintiffs will often win, at least where those cases have better-developed records on the market effects of the defendant’s use

And what do those records are supposed to look like? Harm has to be specific, always had been. How do you ever demonstrate that a specific AI harmed the market for a specific book?

I honestly think both judges had to try to appear neutral and Meta’s had to work harder to appear neutral because Meta’s conduct was worse, hence a more bizarre argument. Misanthropic can be slapped with a small fine.

So, the judge says:

In cases involving uses like Meta’s, it seems like the plaintiffs will often win, at least where those cases have better-developed records on the market effects of the defendant’s use.

And what is that supposed to ever look like? Do authors need a better developed record of effects of movies on book sales, to get paid for movie adaptations, too?

As somewhat of an author I fucking can't understand how.

To win, they'd need to demonstrate specific harms (from specific infringer, to specific book), the "amazon is full of slop" won't do.

It's like someone makes a movie without licensing from the book author, and then the judge says that authors must argue that movies harm book sales.

edit: except much much worse because good luck pointing at specific instances of slop and connecting them to specific ai and its training on a specific work. At least with a movie you can point at a movie and at the book its made from.

edit: frankly the whole thing just sounds like both of the judges had to sound neutral, and because Meta's conduct was more egregious the judge had to write weirder stuff to sound neutral.

Anthropic's judge can simply slap them with (likely insignificant) fines in the light of theirs displaying "good faith" by buying a bunch of books legally. Meta's judge had to invent whole new theory of unfair use that plaintiff lawyers can't possibly support with evidence.

It's called sarcasm.