He asks g2.5 flash to assign a topic. I am also interested in the best way to develop a general schema- there is a good deal of literature on this but nothing stands out, I think the standard approach is open ended classification generation using a single model then binning. Actually the novelty in his approach is first asking if a chunk is useful (ie adding a filter for non-semantic information) which I would normally do at the dataset creation stage.

I don't think the Venn diagram of those people and everyone else is as separate as you imagine.

I'm a Literature major and avid reader, but projects like this are still incredibly exciting to me. I salivate at the thought of new kinds of literary analysis that AI is going to open up.

Sure but those people don't have to be mutually exclusive. At the very least, a tool like this can help me decide what to read next.

I can't wait to have the LLM autopilot my neuralink whilst i'm in VR mario kart.

I still read a lot of books and I use LLMs all the time. I have even got a bunch of book recommendations from LLMs. Imagine that. You actually have agency over these tools. I know it is hard to believe for some.

I agree that we should be reading books with our eyes and that feeding a book into an LLM doesn't constitute reading it and confers few of the same benefits.

But this thing isn't (so far as I can tell) even slightly proposing that we feed books into an LLM instead of reading them. It looks to me more like a discovery mechanism: you run this thing, it shows you some possible links between books, and maybe you think "hmm, that little snippet seems well written" or "well, I enjoyed book X, let's give book Y a try" or whatever.

I don't think it would work particularly well for me; I'd want longer excerpts to get a sense of whether a book is interesting, and "contains a fragment that has some semantic connection with a fragment of a book I liked" doesn't feel like enough recommendation. Maybe it is indeed a huge waste of time. But if it is, it isn't because it's encouraging people to substitute LLM use for reading.

I zgrep my epubs, is that a problem too?

I need a name for people who dismiss an entirely new and revolutionary class of technology without even trying it, so much so that they'll not even read about any new ideas that involve it.

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."

https://news.ycombinator.com/newsguidelines.html

One has to be the special kind of stupid that is blinded by efficiency promises from the LLM Church to think the article is any worth.

It's the usual jargon soup. Publish a vetted paper with repeatable steps instead of a hyped-up, garbage, supposed 100x productivity bomb.

And his best result is mechanical findings from where the LLM got the highest correlations between its vectors: Bravo; there's always going to be a top item in any ordered list, but it doesn't make it automatically interesting. Reading literature is about witnessing the journey the characters take. Reading technical material is about memorizing enough of it. In both cases the material has to go through a brain. I find it idiotic to assign any value to outputs like "Oh King Lear's X is highly correlated to Antigone's Y"

The incumbants Goodreads and their owner Amazon have indeed done such a poor job at this. Seven years ago I tried creating a basic graph using collaborative-filtering (effectively using our actual reading patterns as the embeddings space instead of semantics [human X likes book Y so likers of Y might like other things that human X has enjoyed]). It works well to this day (ablf.io) but the codebase is so ugly I've not had the bravery to update its data in a couple of years.

Synthesizing 500 words at a time into digestible topics is significantly less prone to error. You're giving it a lot of info and asking for an organized subset. It's good at following such direction.

In your example, you're doing the inverse (give me a lot of text based on a little), and that's where LLMs have no problem hallucinating the new information.

Yes AI subsumes edge cases to produce a very uniform optimal writing (what we call AI slop). I am assuming this is a book you were reading for knowledge work, not for fun? Not heard about people recreationally using AI for consumer content that's a bridge too far for me lol.