frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

The Art of Multiprocessor Programming 2nd Edition Book Club

https://eatonphil.com/2025-art-of-multiprocessor-programming.html
67•eatonphil•1h ago•8 comments

Compressing Icelandic name declension patterns into a 3.27 kB trie

https://alexharri.com/blog/icelandic-name-declension-trie
132•alexharri•4h ago•41 comments

We may not like what we become if A.I. solves loneliness

https://www.newyorker.com/magazine/2025/07/21/ai-is-about-to-solve-loneliness-thats-a-problem
142•defo10•4h ago•203 comments

WebGPU enables local LLM in the browser. Demo site with AI chat

https://andreinwald.github.io/browser-llm/
20•andreinwald•1h ago•5 comments

Unikernel Guide: Build and Deploy Lightweight, Secure Apps

https://tallysolutions.com/technology/introduction-to-unikernel-2/
15•Bogdanp•1h ago•1 comments

6 Weeks of Claude Code

https://blog.puzzmo.com/posts/2025/07/30/six-weeks-of-claude-code/
69•mpweiher•3h ago•30 comments

The case for having roommates (even when you can afford to live alone)

https://supernuclear.substack.com/p/the-case-for-having-roommates-even
20•surprisetalk•1h ago•30 comments

The Rubik's Cube Perfect Scramble

https://www.solutionslookingforproblems.com/post/the-rubik-s-cube-perfect-scramble
15•notagoodidea•1h ago•3 comments

Caches: LRU vs. Random

https://danluu.com/2choices-eviction/
42•gslin•2d ago•6 comments

Microsoft is open sourcing Windows 11's UI framework

https://www.neowin.net/news/microsoft-is-taking-steps-to-open-sourcing-windows-11-user-interface-framework/
90•bundie•7h ago•88 comments

Cerebras Code

https://www.cerebras.ai/blog/introducing-cerebras-code
406•d3vr•17h ago•156 comments

Palo Alto Networks agrees to buy CyberArk for $25B

https://techcrunch.com/2025/07/30/palo-alto-networks-agrees-to-buy-cyberark-for-25-billion/
50•vmatsiiako•2d ago•39 comments

ThinkPad designer David Hill spills secrets, designs that never made it

https://www.theregister.com/2025/08/02/thinkpad_david_hill_interview/
60•LorenDB•2h ago•8 comments

OpenAI's "Study Mode" and the risks of flattery

https://resobscura.substack.com/p/openais-new-study-mode-and-the-risks
77•benbreen•2d ago•64 comments

Financial Lessons from My Family's Experience with Long-Term Care Insurance

https://www.whitecoatinvestor.com/financial-lessons-father-long-term-care-insurance/
5•wallflower•1h ago•1 comments

Aerodynamic drag in small cyclist formations: shielding the protected rider [pdf]

http://www.urbanphysics.net/2025_Formation_Paper_Preprint_v1.pdf
44•PaulHoule•3d ago•14 comments

Coffeematic PC – A coffee maker computer that pumps hot coffee to the CPU

https://www.dougmacdowell.com/coffeematic-pc.html
250•dougdude3339•17h ago•76 comments

Why leather is best motorcycle protection [video]

https://www.youtube.com/watch?v=xwuRUcAGIEU
136•lifeisstillgood•2d ago•110 comments

At 17, Hannah Cairo solved a major math mystery

https://www.quantamagazine.org/at-17-hannah-cairo-solved-a-major-math-mystery-20250801/
403•baruchel•23h ago•163 comments

This Month in Ladybird

https://ladybird.org/newsletter/2025-07-31/
333•net01•9h ago•104 comments

Palo Alto Networks closing on over $20B acquisition of CyberArk

https://www.calcalistech.com/ctechnews/article/hksugkiwxe
8•tomashertus•3d ago•0 comments

JavaScript retro sound effects generator

https://github.grumdrig.com/jsfxr/
119•selvan•4d ago•21 comments

Cadence Guilty, Pays $140M for Exporting Semi Design Tools to PRC Military Uni

https://www.justice.gov/opa/pr/cadence-design-systems-agrees-plead-guilty-and-pay-over-140-million-unlawfully-exporting
25•737min•2h ago•12 comments

Weather Model based on ADS-B

https://obrhubr.org/adsb-weather-model
224•surprisetalk•3d ago•35 comments

I couldn't submit a PR, so I got hired and fixed it myself

https://www.skeptrune.com/posts/doing-the-little-things/
300•skeptrune•22h ago•183 comments

Yearly Organiser

https://neatnik.net/calendar/
86•anewhnaccount2•4d ago•23 comments

Hardening mode for the compiler

https://discourse.llvm.org/t/rfc-hardening-mode-for-the-compiler/87660
140•vitaut•13h ago•44 comments

Ask HN: Who is hiring? (August 2025)

211•whoishiring•1d ago•239 comments

Ethersync: Peer-to-peer collaborative editing of local text files

https://github.com/ethersync/ethersync
163•blinry•4d ago•31 comments

A.I. Researchers Are Negotiating $250 Million Pay Packages

https://www.nytimes.com/2025/07/31/technology/ai-researchers-nba-stars.html
66•jrwan•4h ago•79 comments
Open in hackernews

Tim Cook rallying Apple employees around AI efforts

https://www.bloomberg.com/news/articles/2025-08-01/apple-ceo-tells-staff-ai-is-ours-to-grab-in-hourlong-pep-talk
65•andrew_lastmile•17h ago

Comments

andsoitis•17h ago
Apple Intelligence? Or Artificial Intelligence?
andrew_lastmile•17h ago
Apple Intelligence™ is most definitely "theirs to grab"
bpmct•17h ago
https://archive.is/iBtcQ
tiahura•13h ago
Deep breath. There’s no sense in trying to outcompete Google in burning cash. They’ve got time to wait until there’s the beginning of commodification of the tech, and a large profitable market to be had.
bigyabai•13h ago
Isn't a larger concern that Tim "Services" Cook failed to skate where the puck was headed on this one? 15 years ago the Mac had Nvidia drivers, OpenCL support and a considerable stake in professional HPC. Today's Macs have none of that.

Every business has to make tradeoffs, it's just hard to imagine that any of these decisions were truly worthwhile with the benefit of hindsight. After the botched launch of Vision Pro, Apple has to prove their worth to the wider consumer market again.

jleyank•13h ago
Their X/OpenGL support has also been in stasis for 10 years or more. There’s not enough money taking over for SGI to move their needle.
orionblastar•13h ago
Don't abandon Intel Macs, then and call them Mac AI systems with NVIDIA chips. Sell them for more than the Apple Silicon Macs.
seanmcdirmid•1h ago
No one would buy slower hotter computers for more money. Most people who own Apple computers today are extremely satisfied with Apple silicon, and AI enthusiasts are an increasing large slice of those people (since there really isn’t anything else and getting a 3090/4099/5090 is still hard and expensive).
seanmcdirmid•13h ago
Silicon Mac’s are great for running LLMs. Unified memory and memory bandwidth of the Max and Ultra processors is very useful in doing inference locally.
bigyabai•12h ago
Great news, but entirely lost on commercial hyperscalers and much of the PC market. Apple's recalcitrance towards supporting Nvidia drivers basically killed their last shot at real-world rackmount deployment of Apple Silicon. Now you can go buy an ARM Grace CPU that does the same thing, but cheaper and with better software support.
seanmcdirmid•9h ago
You really can’t. NVIDIA’s arm chip still looks nerfed compared to apple’s offering, and…I can run 40GB sized LLMs on the plane with no internet…it’s not something that you can do with any other platform.
paulpauper•12h ago
Macs are basically a dead business. The key is somehow creating the AI equivalent of an App Store or something
unsigner•1h ago
Dead business?

The Mac is something like 30 billion in revenue per year, and 10 billion in profit.

The entire "generative AI" "industry" is struggling to reach 30 billion in revenue even with their creative accounting (my free Perplexity that comes with Revolut is somehow counted at full price, even though I never paid anything, and I'm sure Revolut doesn't pay full price), and gross profit is deep in the negative.

stockresearcher•11h ago
> Isn't a larger concern that Tim "Services" Cook failed to skate where the puck was headed on this one?

Doesn't somebody (not named Nvidia) need to make a serious profit on AI before we can say that Tim Cook failed?

OpenAI and Anthropic aren't anywhere close. Meta? Google? The only one I can think of might be Microsoft but they still refuse to break out AI revenue and expenses in the earnings reports. That isn't a good sign.

Fade_Dance•10h ago
I certainly don't think that profit would be required. Many of the massive tech companies that exist today went through long periods of time were they focused on growth and brand no profits for many years even post IPO.

I won't pretend to know exactly how the AI landscape will look in the future, but at this point it's pretty clear that there's going to be massive revenue going to the sector, and Moore's law will continue to crank.

I see what you're saying though. In particular is first generation gigs data centers might be black holes of an investment, considering in the not too distant future AI compute will be fully commoditized and 10x cheaper.

stockresearcher•1h ago
Yeah, I think we're on the same page on this one.

"failed to skate where the puck was headed" assumes that we know where the puck is going to be. We don't.

Everyone is skating towards that same spot while Apple is over by the blue line practicing their swizzles. They sure look like they're doomed. But large groups of people have skated to the "wrong spot" thousands of times. That's the entire point that Gretzky was making with his quote. He's not big enough, strong enough, fast enough to get in that scrum. They're all fighting it out and the puck slides away. To him. All alone.

Maybe that is Apple, maybe it's not. I mean, they're still learning to skate while everyone else is playing hockey.

aunty_helen•13h ago
Or, apples just so bad at this they’re fumbling the bag. Billions in cash on hand each quarter but don’t have the balls that zuck has to pay unreasonable money. They have their own hardware like google does but are talking about perplexity??? They have all data but can’t seem to get an llm that can set an alarm and be a chatbot at the same time?

Sometimes company’s just don’t do good enough.

paulpauper•12h ago
Undercut the competitors by charging less. Apple can afford to run its product at a loss.
anon7000•11h ago
> Billions in cash on hand each quarter but don’t have the balls that zuck has to pay unreasonable money

It remains to be seen whether this was a smart move, or just flailing money at the wall

aunty_helen•11h ago
The difference is it’s a move. Actually doing something rather than putting out internal PR.

Zuck tried and flailed with the metaverse. That was a huge waste, but he can afford it and fortune favours the brave.

lotsofpulp•2h ago
You don’t think Apple makes moves?

Not everyone has to make the same move at the same time.

SebFender•1h ago
Apple did many - Just not the right or good ones in the past decade.
lotsofpulp•1h ago
Services (icloud and music and tv)/airpods/watch/M processors and the new modem seem like good ones.

If those don’t seem like right or good moves, I can’t imagine much will impress you in this world.

xnx•1h ago
> They have all data but can’t seem to get an llm that can set an alarm and be a chatbot at the same time?

This does seem like an embarrassing fail, but even Google has not completed replacing Assistant with Gemini. There have also been lost functionality (maybe temporary) in the process.

unsigner•1h ago
they are not talking about perplexity; the endless rumor mill talks about perplexity. The same that has them buying everything from Disney to Porsche to Nike for decades.
potatolicious•1h ago
> "They have all data but can’t seem to get an llm that can set an alarm and be a chatbot at the same time?"

This is actually one of the hardest frontier problems. The "general purpose" assistant is one of the singular hardest technical problems with LLMs (or any kind of NLP).

I think people are easily snowed by LLMs' apparent linguistic fluency that they impute that to capability. This cannot be further from the truth.

In reality a LLM presented with a vast array of tools has extremely poor reliability, so if you want a thing that can order delivery and remember your shopping list and remind you of your flight and play music you're radically exceeding the capabilities of current models. There's a reason successful (anything that isn't demoware/vaporware) uses of agentic LLMs tend to narrow-domain use cases.

There's a reason Google hasn't done it either, and indeed nor has anyone else: neither Anthropic nor OpenAI have a general purpose assistant (defined as being able to execute an indefinite number of arbitrary tools to do things for you, as opposed to merely converse with you).

benoau•12h ago
They don't really have much time to wait, they could be forced to allow default voice assistants and access to private APIs by the DOJ antitrust, the App Store Freedom Act, the Open Markets Act, if any of those come through then OpenAI and Gemini will quickly end up entrenched.
mips_avatar•12h ago
I wish apple would provide a decent model to apple intelligence and let developers build on it. Like sure it would lose a lot of money right now, but it would mean that app developers making AI agents on the iphone could still charge modest amounts if they aren't responsible for the inference costs.
burnt-resistor•12h ago
Chief Bean Counter Cook doesn't do cool, goodwill, or long term strategy. Only making the same set of products incrementally better and more expensive, and increasingly prone to expensive repair.
dangus•12h ago
Apple may be greedy but they can’t be accused of bungling long term strategy.

While OpenAI sells $2 bills for $1, Tim Cook was out there increasing service revenue and profitability so that it was larger than Macs and iPads combined.

Tim Cook presided over some incredibly lucrative product launches like AirPods, TV+, Apple Music, moved chip design in house which doubled Mac market share and has made the iPhone continually dominant, they’ll even drop third party 5G models soon. These are all incredibly shrewd long term strategy moves.

burnt-resistor•11h ago
Lucrative but not cool. There's no long term strategy if there's no new, category-defining, cool products. People will eventually sour on the same-old, same-old and expensive, fragile, unrepairable products.

Excessive greed obliterates goodwill.

You don't seem to understand sj placed cool first, while TC was the bean counter and continues to optimize this while jumping the shark.

Current Apple has a deep, systemic lack of cool and lack of entrepreneurial leadership with good taste that will ultimately lose the crown.

Fade_Dance•10h ago
Teen iPhone ownership is up to 88% (!) in the US.

I generally agree with what you're saying, and unlike Cook I don't find it "hard to imagine" life without iPhones (an augmented reality future isn't that far out to consider), but they have a long runway. Gen Z and Gen Alpha define "cool", and they are committed to the ecosystem.

pjmlp•9h ago
Except the little issue teens don't appear on the same rate as iPhones hit the store shelves.
nunez•8h ago
Misinformed take IMO.

M-series chips are insanely cool (literally and figuratively) and have no competition even five years in. Same with the W-series SoCs on watches.

Are there any third-party haptic vibration motors yet?

Shoot, even their trackpads, which already stood alone, have gotten _better_ over the years.

Nobody else will have their own vertically-integrated modem out in production. This will make budget iPhones (maybe all iPhones) so much cheaper once they show Qualcomm the door.

That's before the advances they've made in software, like their camera processing pipeline (which only gets better; their video stack still has no equal) and differential privacy.

Oh, yeah, and the Vision Pro, which basically everyone who has tried it has said that it is the most advanced technology they've ever used.

Almost all of this happened under Cook's tenure.

Apple is still in the business of building insanely cool shit.

SturgeonsLaw•4h ago
Vertically-integrated modems are not cool. I mean, they are to me and likely other geeks, but to the general public? Their eyes would glaze over.

Cool is the kind of thing people envy. It's the kind of product that gets namedropped in a music video.

MrDarcy•1h ago
You’re not thinking two or three orders out. It’s not about the modem, it’s about reducing the resource allocation of a boring component to make space for increasing the resources of other exciting components, taking the whole into account.

iPhone will get some new exciting feature and everyone will wonder how they managed to do it at the price point nobody else can.

financltravsty•1h ago
You talk like a nerd.

Cool is a vibe, not tech specs and little things. It's a whole aura.

Apple is not cool.

gabriel666smith•45m ago
Agree. There are probably more than a million tiny contextual data points that make a person look at something (whether it’s a tech product or a musician) and go: “cool, man.”

But those millions of data points can (rarely, briefly,) coalesce around a product or company, even though that’s mostly out of the control of those building the product or company.

EG if you asked someone in 1965 if a Jaguar E-Type was cool, or someone in 2000s London whether the Fruityloops DAW was cool, they’d say “yeah”.

I’m mostly agreeing, and it’s a super minor point, but tech specs are part of the unknowable, constantly-shifting constellation of symbols that produce “cool”, and there isn’t a reason an Apple product couldn’t, in the future, align the stars. They did before! The white iPod earbud wire did, briefly, signify cool.

baal80spam•45m ago
> Apple is not cool.

Says who, exactly? It is very cool for most.

pcdoodle•1h ago
I'm on a M4 with nano texture display. It's IMO the pinnacle of what a computer should be. I hate apple and their anti repair practices but wanted to chime in about how good the hardware has become.

If they push forward with local AI that can be somewhat trusted, it would be a huge win.

jkmcf•2h ago
Apple has a ton of problems, but your comments don't address them (primarily the perceived decline in software quality and app store developer gouging).

While cool is subjective, what new, mass-market products should they create? Which product market should they "re-invent"? I wish they'd buy Sonos and fix that shit show, but that's not a profitable market to enter.

Barring the price jump around the iPhone 7, their smartphones have stayed about the same price. [1]

Over half of all smartphone repairs are battery replacements, which implies people don't take care of their batteries or are keeping their phones long enough to wear the battery down normally. [2] Additionally, Apple ranks very well in repairability. [3] They also support their phone's software longer on older devices than the competition.

[1] https://www.androidauthority.com/iphone-price-history-322149... [2] https://www.businessresearchinsights.com/market-reports/smar... [3] https://www.ifixit.com/repairability/smartphone-repairabilit...

elpakal•2h ago
They announced this at wwdc this year in “Foundation Models”. Developers have access to an on-device llm (not sure how good it is yet).
cgearhart•2h ago
I mean, they’ve released a framework to access the on device model https://developer.apple.com/documentation/FoundationModels
paulpauper•12h ago
Some problems cannot be fixed with more money (unless to buy a stake , as Microsoft did with open ai). See Microsoft's endless failed efforts to compete with Google search or iPhone. Although looking at the recent stock price since 2020, MSFT stock was the winner anyway.
pjmlp•9h ago
Because of Azure, Office, Game Pass, Github,....

To detriment of Windows, XBox hardware, .NET team shooting into all directions.

viraptor•8h ago
> there was a smartphone before the iPhone; there were many tablets before the iPad; there was an MP3 player before iPod

That's the biggest shift I've heard from Apple. They were either "first" or ignored the existence of competing features/products for ages. I'm really surprised by this quote.

Compare "smartphones before iPhone" to the original announcement:

> iPhone also ushers in an era of software power and sophistication never before seen in a mobile device, which completely redefines what users can do on their mobile phones. (...) iPhone is a revolutionary and magical product that is literally five years ahead of any other mobile phone,

troad•4h ago
That's actually very consistent for Apple. Apple doesn't generally claim to be the first to do something, but have always taken the line that they're the first to execute it well. Hence their fondness for words like 'reimagine', 'revolutionise', etc.
Isamu•2h ago
Yes, dominant smartphones before iPhone were BlackBerry inspired, full physical keyboard with small screen.

When the iPhone launched, the Android project changed direction toward a full screen phone and that form became much more dominant and popular than the BlackBerry form.

Apple made the bet that they could make the full screen experience much more compelling that people would accept the trade off of losing the keyboard.

SebFender•1h ago
True. But in this case Apple isn't only a bit late but completely lost with AI - Efforts driven towards AR for years absolutely killed their game.
politelemon•1h ago
> Apple doesn't generally claim to be the first to do something

Strongly disagree with this. Their marketing often claims inventing things that have existed.

spacedcowboy•28m ago
> Strongly disagree with this. Their marketing often claims inventing things that have existed

Strongly disagree with this. Their marketing often claims reinventing things that have existed, or revolutionising them, or reimagining them, but rarely claims to be the first, ever, without qualification.

bni•2h ago
And what exactly in your iPhone announcement quote was untrue?
actinium226•2h ago
It's pretty on brand for Apple, I'm surprised they hadn't pushed this narrative harder earlier.

There were smartphones before the iPhone. Consider the IPAQ and Windows Mobile 6.0.

And of course plenty of MP3 players before iPod.

unsigner•1h ago
I've heard the phrase "through Apple new technologies achieve their final form", possibly not official Apple but one of the Apple choir bloggers (Gruber?).

There were smartphones before iPhone, now all smartphones are black featureless rectangles. There were printers before LaserWriter, then for 20 years all printers became this. (And later disappeared.) There were wireless heaphones before Airpods, now the difference is in the shape of the stubs. There were laptops before the Macbook Air... etc

brcmthrowaway•2h ago
Very sad results; the end of America's once greatest company?
lapcat•1h ago
On Thursday, Apple reported yet another record financial quarter.
WhereIsTheTruth•21m ago
That doesn't seem to help Apple achieve AGI
bni•2h ago
What other companies have successfully integrated LLM tech in their mainstream products?

To be clear, just having a chatbot website/app does not count.

FirmwareBurner•2h ago
Adobe

Nvidia

SebFender•2h ago
I have to admit - outside of cost and core values differences - Adobe has been an early player and often overlooked - but the company smells so bad that I guess they're a little looking for it.
kibwen•1h ago
It looks like the parent was asking about LLMs specifically, in which case I don't think those two count. AFAIK Adobe's image-generation stuff is a diffusion model, not an LLM, and Nvidia's DLSS isn't an LLM either.
seanmcdirmid•1h ago
Aren’t LLMs distinct from image generation and manipulation models?
cma•1h ago
Google AI summaries isn't a chatbot exactly, but probably has been successful in staving off migration to chatgpt search, at least once it improved a lot.
koakuma-chan•1h ago
Are you implying Apple integrated AI successfully? How do you use AI in Apple products?
potatolicious•1h ago
- If you get into a car crash with your iPhone a ML model detects this and automatically calls emergency services.

- If you are wearing an Apple Watch a ML model is constantly analyzing your heart rhythm and will alert you to (some types of) irregularities. It's so computationally efficient it can literally do this in the background all day long.

- When you take any picture on any iPhone a whole array of ML models immediately run to improve the image. More models are used when manually editing images.

- After you save the photo ML models run to analyze and index the photo so it's easily searchable later. That's why you can search for "golden retriever" and get actual results.

- When you speak at your device (for example, to dictate a text message) there's a ML model that transcribes that into text. Likewise, when you're hands-free and want to hear an incoming text message, an ML model converts it to audio. All on-device and available offline at that.

Or are we playing that stupid game where "AI === LLM"?

xdennis•48m ago
> Or are we playing that stupid game where "AI === LLM"?

Well, the original question was specifically about LLMs. ("What other companies have successfully integrated LLM tech in their mainstream products?")

AnonymousPlanet•1h ago
Does the amazing OCR in the macOS Preview app count? As someone who sometimes gets tracebacks sent to as screenshots, I'm really happy about it.
xnx•1h ago
My prediction: Always on video/ audio recording in wearable form factor (probably glasses).

This seems "creepy" now, but people thought that about Google "reading" all your email too. The benefits of an ever present and aware assistant are just to great to ignore.

Apple's angle is that they are well liked and trusted (much more so than Facebook which people already think is eavesdropping on them to show ads) and will do all processing on device.

indy•1h ago
Tim Cook is a "Keep things ticking along" CEO, not a "Change course to a new destination" CEO. Initiatives like this will probably require different leadership to succeed.
reactordev•1h ago
Exactly, Tim Cook is a finance guy - he knows the numbers and how to keep Apple profitable. What he lacks is product vision. His one opportunity (ironically, vision) fell flat.
ljlolel•1h ago
Don’t downvote this guy! Interesting to note that he probably constantly got criticized for not having vision, so he took that literally and called the product Vision Pro.

It’s the kind of mistake an LLM would make. Very Lacanian.

reactordev•21m ago
I wouldn’t go that far. Tim Cook doesn’t really care what people think of him, he only cares about one group of people - shareholders. It was probably marketing that came up with it. Tim is totally fine being the finance guy and the vision he sold the board was “I’ll keep Apple from imploding” which he has been very successful at.

Like Nadella, you need someone from the early years, who knows the business, to run it.

Cook lacks product vision because 1) he’s no Steve Jobs. He was hired. He didn’t create. 2) He doesn’t have an Jonny Ive to make something as boring as a computer be as sexy as an Italian vase. Or as sleek as a pencil. Or as flat as paper. Or whatever metaphors were used during the Ive years to describe his design process.

But he has been there long enough to know how it works.

nojito•1h ago
>not a "Change course to a new destination" CEO.

Based on what exactly? He led the overhaul of a massive amount of Apple under his tenure.

lapcat•1h ago
Serious question: Did Apple employees need rallying?

Also, it sounds like Cook and Federighi just repeated talking points the public has already heard, so I'm not sure what the point of this was.

If there are any current Apple employees here, maybe they can weigh in.

andrewstuart•1h ago
I feel like Steve Jobs would have designed what real androids should be.

Instead Apple can’t even manage to implement speech to text that works in safari and can’t manage to make Siri not suck.

pretext-1•16m ago
Text to speech doesn’t work correctly either. It often highlights different words than what is currently spoken.
qrios•48m ago
Current "good enough" models like Mistral Small require GPUs like the RTX 6000 to achieve user-friendly response times. The model quality is good enough, especially for narrow-scope tasks like summarization and classification. If Moore's Law holds for a few more years, a mobile device will be able to run it on-device in around 8 years (Apple's A11: 410 GFLOPS vs. RTX 6000: 16 TFLOPS [1]).

This is under the assumption that we don't see any significant optimization in the meantime. Looking back over the last eight years, the probability of no progress on the software side is near zero.

For a breakthrough in the consumer market, running LLM on-device with today's capabilities requires solving one key topic: "JIT learning" [2]. We can see some progress here [3, 4]. Perhaps the transformer architecture is not the best for this requirement, but it is hard to argue that it is impossible for Generative AI.

Due to today's technical limitations, we don't have real personal assistants. This could be the Mac for Apple in the AI era.

[1] https://gadgetversus.com/graphics-card/apple-a11-bionic-gpu-...

[2] Increasing context size is not a valid option for my scenario as it also increases the computation demand linear.

[2] https://arxiv.org/abs/2311.06668

[3] https://arxiv.org/abs/2305.18466

[Edit: decimal separator mess]

mingus88•25m ago
Apple’s answer to that is Private Compute Cloud
hollerith•15m ago
I thank Tim Cook for this information. I was leaning towards buying a Mac, but now I won't because I want to do what (little) I can to slow down AI or at least not encourage AI development.

Switching to Windows would also clearly be encouraging the AI juggernaut, so I will stay with Linux.