Reminds me of "Rainbow's End" by Vinge. There's a machine that's like a giant worm, which slithers down the stacks in a library, vacuuming up all the books. They pass through shredders, and then the shredded remains fly down the body of the worm, which is studded with cameras. The cameras photograph the pieces and then software reconstructs the content of the books based on unique shapes of the shreds, like solving a million simultaneous jigsaw puzzles. The paper is excreted and recycled or burned.
Interestingly there was a real attempt to build an E-Puzzler for shredded documents, to reconstruct the torn Stasi files after the German reunification. But while the system worked for defined stuff, but failed for mass reconstruction of documents with different formats:
Getting flashbacks to Vernor Vinge's book Rainbows End, where there's a project to rapidly digitize the collection of the UCSD's Geisel Library by shredding all the books and photographing the fragments of pages and reassembling them via computer programs.
> In the process, the company cut millions of print books from their bindings, scanned them into digital files, and threw away the originals solely for the purpose of training AI
Oh boy. The more I learn about how genAI companies work, the more detestable they appear to be.
I didn't get suckered by anything. I'm aware of the practice. I find it objectionable. That they did this is just another thing on the growing list of objectionable things that genAI companies seem to enjoy doing.
To be honest, I probably wouldn't have even commented on it if it were the only bad thing these companies do.
> Ultimately, Judge William Alsup ruled that this destructive scanning operation qualified as fair use—but only because Anthropic had legally purchased the books first, destroyed each print copy after scanning, and kept the digital files internally rather than distributing them. The judge compared the process to "conserv[ing] space" through format conversion and found it transformative.
Very laws that the publishing industry has lobbied so heavily to make so strict are the reasons for this behavior.
If you believe that destroying books is bad, your issue is with copyright law, not the AI companies. The AI companies are just following copyright law -- they are allowed to move data from one format to another (thereby destroying the original), but not copy it.
Not everything objectionable or unethical should or could necessarily be outlawed. "It's not illegal" is not really an argument or justification for anything.
I don't think CaptainFever's point is that it's acceptable because it's legal, but rather that copyright law is what prevents them from, say, donating the originals instead of throwing them away.
I mean, they could have gotten e-book versions of the books, or even preprint PDFs.
In an era where people are starting to calculate the environmental impact of the jobs they run on the cloud and start to optimize it, adding that much load on recycling system is not a wise choice, but only a selfish one.
I strongly suspect that dealing with ebooks on this scale might actually be even more onerous than the physical volumes.
The physical stuff is straightforward. Buy books from bulk sellers, rip off everything and put them into off-the-self rigs for digitization. It's straightforward, directly scalable, can use any book, and your main issue is format shifting, which anthropic successfully argued here. No DRM, you buy exactly the books you need, and every book is processed exactly the same way.
If you try to buy ebooks, you get wrapped up in onerous licensing terms about copying, and how you're able to use them, how long you're able to access them, and so on. Many books won't even be available (or can only be licensed alongside a bunch of others) and you have to deal with DRM you can't strip without creating additional copyright issues.
We've somehow created a world where physical objects are more free than bits.
No, they probably couldn't have. eBooks are notoriously DRMed and the DMCA makes it illegal to circumvent an effective copy protection mechanism even if you otherwise have legal access to work. Furthermore, first sale doctrine doesn't apply to any digital files and they can't be obtained legally in bulk.
I'm sure they would have loved to save the hassle and expense of disassembling physical books. Presumably something legal related or cost related prevented them from going that route.
Why does a good LLM require millions of books? No person requires/reads nearly that many books to function!
Most well-educated persons have read far fewer books yet are presumably more intelligent than any LLM. This is a hint toward solving the problem of AI: the industry would do well to pay attention to that hint.
In other words, the industry should concentrate on creating an AI that has a far more limited corpus. Of course, maybe they are doing so but not telling anyone about it. I certainly wouldn't reveal my best ideas. At the same time one must remember that, only a few centuries ago, people learned to read English by studying the King James version of the Old Testament, a single book [not to say that a lot of work didn't go into that particular text: it really is a work of art in the English language].
Perhaps the AI industry must maintain a illusion for the unwashed masses (and especially investors) that each generation of LLM AI must always be bigger to reach AGI, whereas the truth is that some guy in his study will find the key that ties LLMs to human intelligence and will, any day, render the entire industry obsolete.
I don't like Anthropic, I think their "marketing through fear" approach to be shitty and frankly I'm over the AI "boom" anyway.
BUT... here's the only line in that whole article that really matters, because this is a headline meant to create an impression that isn't corrected for quite a while.
> The court documents don't indicate that any rare books were destroyed in this process—Anthropic purchased its books in bulk from major retailers
Books are routinely pulped and recycled, they aren't holy, and if they aren't rare then frankly who cares what techniques they use to scan them? The issue is whether or not "AI" learning represents fair use, which the courts so far have ruled that it does.
Does it matter? It's waste at the end of the day. Instead they could have bought e-books. Just because we can recycle paper, it doesn't mean we have the luxury to create waste as we see fit, esp. when climate change became this severe.
> which the courts so far have ruled that it does.
Any concrete cases you can cite?
From [0], for example, while the course said that the authors failed to argue their case, the second observation is complete opposite of what you said. Citing the article directly:
Opinion suggests AI models do generally violate law.
In the same spirit, I think I can safely assume that they violated copyright law, since they earn money by circumventing it, and fair use doesn't like for-profit copying.
As someone who finds the act objectionable, I actually do think this is an important point. Destroying commodity books in this way is objectionable. Destroying precious books in this way would be abominable.
They'd have to agree to special terms that go beyond the normal first sale doctrine. If those terms don't hold up their own terms against training on their model data for foundation models might not hold up, so you can see their perverse incentive to burn books.
More likely they are taking a waste stream of books and reusing and possibly even recycling.
Few people want old books, and many people that have books are throwing them out or donating them. I don't think I know anybody under 30 with a bookshelf of books they obviously intend to keep for life. Bookshelves used to be an elite status symbol, now I often see them as image rather than reference (e.g. part off backdrop behind influencer vid).
It is likely they didn't destroy much of value, since they will have minimized their purchasing costs. Modern DRM is not helping.
Depending on their approach this might not be as bad as it seems.
Some large percentage of physical books really aren't a precious commodity. They may be precious to you as an individual but chances are no-one else is going to be that bothered. For example, a second hand book wholesaler I just found with a quick google will sell you pallets of books for 25c/book which would only go down as your purchase size went up. If they don't sell then it seems pretty likely those books are going to get pulped anyway.
If they were buying the books new and then essentially pulping them after scanning that does feel wasteful, but if they were buying in bulk from a second hand wholesaler then Im kinda glad the content of those books is going into something useful before they get pulped.
When they run out of training data, they have to rely upon better reasoning algos which take time to develop and that’s when the party ends. The purpose of IT investment is to increase competitiveness through better efficiency and thus capture more of the market. If everyone is using the same model trained on the same max dataset, it stops being much of a competitive advantage. I’ve already heard the stock-option Billionaires bloviating with their intuition shaped by narrow corporate experiences that the AI objective is to increase GDP or similar, but obviously the ol’ man in the corner office ain’t payin’ for that and so at some point, the question of how to keep the Three Mile Island running to power this sucker becomes very real. The answer is sort of scary, if you think about it…
