frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Yt-dlp: Upcoming new requirements for YouTube downloads

https://github.com/yt-dlp/yt-dlp/issues/14404
593•phewlink•4h ago•296 comments

That Secret Service SIM farm story is bogus

https://cybersect.substack.com/p/that-secret-service-sim-farm-story
695•sixhobbits•8h ago•360 comments

SedonaDB: A new geospatial DataFrame library written in Rust

https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-da...
17•MrPowers•34m ago•2 comments

US Airlines Push to Strip Away Travelers' Rights by Rolling Back Key Protections

https://www.travelandtourworld.com/news/article/american-joins-delta-southwest-united-and-other-u...
414•duxup•4h ago•379 comments

Python on the Edge: Fast, sandboxed, and powered by WebAssembly

https://wasmer.io/posts/python-on-the-edge-powered-by-webassembly
31•baalimago•47m ago•4 comments

Learning Persian with Anki, ChatGPT and YouTube

https://cjauvin.github.io/posts/learning-persian/
83•cjauvin•3h ago•30 comments

How to Lead in a Room Full of Experts

https://idiallo.com/blog/how-to-lead-in-a-room-full-of-experts
86•jnord•3h ago•17 comments

Who Funds Misfit Research?

https://blog.spec.tech/p/who-funds-misfit-research
31•surprisetalk•1h ago•5 comments

Smartphone Cameras Go Hyperspectral

https://spectrum.ieee.org/hyperspectral-imaging
26•voxadam•2h ago•9 comments

The Lambda Calculus – Stanford Encyclopedia of Philosophy

https://plato.stanford.edu/entries/lambda-calculus/
21•lordleft•1h ago•1 comments

EU age verification app not planning desktop support

https://github.com/eu-digital-identity-wallet/av-doc-technical-specification/issues/22
279•sschueller•4h ago•184 comments

How to Be a Leader When the Vibes Are Off

https://chaoticgood.management/how-to-be-a-leader-when-the-vibes-are-off/
26•mooreds•1h ago•3 comments

How HubSpot Scaled AI Adoption

https://product.hubspot.com/blog/context-is-key-how-hubspot-scaled-ai-adoption
49•zek•1h ago•24 comments

New bacteria, and two potential antibiotics, discovered in soil

https://www.rockefeller.edu/news/38239-hundreds-of-new-bacteria-and-two-potential-antibiotics-fou...
12•PaulHoule•31m ago•3 comments

Better Curl Saul: a lightweight API testing CLI focused on UX and simplicity

https://github.com/DeprecatedLuar/better-curl-saul
5•jicea•18m ago•0 comments

Zed's Pricing Has Changed: LLM Usage Is Now Token-Based

https://zed.dev/blog/pricing-change-llm-usage-is-now-token-based
13•meetpateltech•21m ago•2 comments

Rights groups urge UK PM Starmer to abandon plans for mandatory digital ID

https://bigbrotherwatch.org.uk/press-releases/rights-groups-urge-starmer-to-abandon-plans-for-man...
152•Improvement•4h ago•109 comments

S3 scales to petabytes a second on top of slow HDDs

https://bigdata.2minutestreaming.com/p/how-aws-s3-scales-with-tens-of-millions-of-hard-drives
133•todsacerdoti•6h ago•41 comments

My Ed(1) Toolbox

https://aartaka.me/my-ed.html
49•mooreds•4h ago•13 comments

Preparing for the .NET 10 GC

https://maoni0.medium.com/preparing-for-the-net-10-gc-88718b261ef2
57•benaadams•5h ago•34 comments

Just Let Me Select Text

https://aartaka.me/select-text.html
183•ayoisaiah•2h ago•186 comments

Everyone's trying vectors and graphs for AI memory. We went back to SQL

74•Arindam1729•2d ago•32 comments

The DHS has been harvesting DNA from Americans for years

https://www.wired.com/story/dhs-has-been-collecting-us-citizens-dna-for-years/
45•righthand•1h ago•5 comments

Exploring GrapheneOS secure allocator: Hardened Malloc

https://www.synacktiv.com/en/publications/exploring-grapheneos-secure-allocator-hardened-malloc
66•r4um•6h ago•1 comments

The Data Commons Model Context Protocol (MCP) Server

https://developers.googleblog.com/en/datacommonsmcp/
3•meetpateltech•45m ago•0 comments

Huntington's disease treated for first time

https://www.bbc.com/news/articles/cevz13xkxpro
201•_zie•4h ago•59 comments

My game's server is blocked in Spain whenever there's a football match on

https://old.reddit.com/r/gamedev/comments/1np6kyn/my_games_server_is_blocked_in_spain_whenever/
308•greazy•6h ago•144 comments

Identity Types

https://bartoszmilewski.com/2025/09/22/identity-types/
5•ibobev•2d ago•0 comments

I Spent Three Nights Solving Listen Labs Berghain Challenge (and Got #16)

https://kuber.studio/blog/Projects/How-I-Spent-Three-Nights-Solving-Listen-Labs-Berghain-Challenge
39•kuberwastaken•3d ago•10 comments

Find SF parking cops

https://walzr.com/sf-parking/
791•alazsengul•22h ago•434 comments
Open in hackernews

Yt-dlp: Upcoming new requirements for YouTube downloads

https://github.com/yt-dlp/yt-dlp/issues/14404
589•phewlink•4h ago

Comments

progbits•3h ago
Can anyone explain specifically what the YT code does that the existing python interpreter is unusable and apparently quickjs takes 20 minutes to run it?

Is it just a lot of CPU-bound code and the modern JIT runtimes are simply that much faster, or is it doing some trickery that deno optimizes well?

progbits•3h ago
From https://github.com/ytdl-org/youtube-dl/issues/33186

> Currently, a new style of player JS is beginning to be sent where the challenge code is no longer modular but is hooked into other code throughout the player JS.

So it's no longer a standalone script that can be interpreted but it depends on all the other code on the site? Which could still be interpreted maybe but is a lot more complex and might need DOM etc?

Just guessing here, if anyone knows the details would love to hear more.

zenmac•3h ago
Yeah that is guess google using spaghetti code to keep their yt moat.
Chris2048•2h ago
Could something like tree-shaking be used to reduce the player code to just the token generating bit? Or does the whole player js change for each video?
zelphirkalt•2h ago
Sounds like a really silly way to engineer things, but then again Google has the workforce to do lots of silly things and the cash to burn, so they can afford it.
ACCount37•3h ago
YouTube is mining cry-

I mean, running some unknown highly obfuscated CPU-demanding JS code on your machine - and using its results to decide whether to permit or deny video downloads.

The enshittification will continue until user morale improves.

adzm•3h ago
I was surprised they went with Deno instead of Node, but since Deno has a readily available single-exe distribution that removes a lot of potential pain. This was pretty much just a matter of time, though; the original interpreter in Python was a brilliant hack but limited in capability. It was discussed a few years ago for the YouTube-dl project here https://news.ycombinator.com/item?id=32793061
nicce•3h ago
Node does not have the concept of security and isolation like the Deno has. There is maintainer comment in the same thread.
doctorpangloss•11m ago
What evidence is there that Deno's "security and isolation" works?

It's their application, yt-dlp can use whatever it wants. But they made their choices for stylistic/aesthetic reasons.

arbll•3h ago
The sandboxing features of Deno also seem to have played a role in that choice. I wouldn't overly trust that as a security layer but it's better than nothing.
CuriouslyC•3h ago
Deno sandboxing is paper thin, last time I looked they had very simple rules. It's a checkbox feature. If you want isolation use WASM.
ndjddirbrbrbfi•3h ago
It doesn’t have granularity in terms of what parts of the code have what permission - everything in the same process has the same permission, but aside from that I’m not sure what you mean about it being paper thin. Certainly WASM is a great option, and I think it can facilitate a more nuanced capabilities model, but for cases like this AFAIK Deno should be secure (to the extent that V8 is secure, which Chrome’s security depends on).

It being a checkbox feature is a weird way to frame it too, because that typically implies you’re just adding a feature to match your competitors, but their main competitors don’t have that feature.

In what ways does it fall short? If there are major gaps, I’d like to know because I’ve been relying on it (for personal projects only myself, but I’ve recommended it to others for commercial projects).

CuriouslyC•2h ago
Last I looked it was just very basic pattern matching allow/deny with no real isolation, and there have been multiple real escapes already. It's better than nothing, and probably good enough for bush league security, but I wouldn't pitch it to my milspec customers.
hyperrail•1h ago
This is the first time I've heard of Deno so I'm only going by their Security & Permissions doc page [1], but it looks like the doc page at the very end recommends using system-level sandboxing as a defense in depth. This suggests that Deno doesn't use system sandboxing itself.

To me this is a bit alarming as IIRC most app runtime libraries that also have this in-runtime-only sandboxing approach are moving away from that idea precisely because it is not resistant to attackers exploiting vulnerabilities in the runtime itself, pushing platform developers instead toward process-level system kernel-enforced sandboxing (Docker containers or other Linux cgroups, Windows AppContainer, macOS sandboxing, etc.).

So for example, .NET dropped its Code Access Security and AppDomain features in recent versions, and Java has now done the same with its SecurityManager. Perl still has taint mode but I wonder if it too will eventually go away.

[1] https://docs.deno.com/runtime/fundamentals/security/

Analemma_•1h ago
Keep in mind that yt-dlp doesn't just support YouTube, which-- notwithstanding the claims of "all DRM is malware" etc.-- probably won't download actively harmful code to your computer: it also supports a huge number of video streaming sites, including some fairly obscure and sketchy ones. Sandboxing in the interpreter that's at least as good as what you'd get in a browser is a must, because by design this is doing untrusted code execution.
trilogic•3h ago
No requirements for me. I don´t use YT at all :) There are plenty of better alternatives.
frizlab•3h ago
until someone shares a video with you
jamesnorden•3h ago
Thanks for letting us know!
VladVladikoff•3h ago
My brother sent me a long talk on YouTube and pleaded with me to listen to it. Watching was pointless the video was just talking heads sitting in chairs. However you can’t just play a video and turn off your phone while listening to the audio on headphones. The mobile browser sleeps and the audio stops. So I used yt-dlp to rip the audio and dropped it into my Plex server to listen to with Prologue. It wasn’t even about the ads, I just wanted to do some gardening and listen to something on headphones while I worked, without my phone screen on.
4gotunameagain•3h ago
https://newpipe.net/

You're welcome

ndriscoll•3h ago
Firefox Mobile has an extension "Video Background Play Fix" to disable the Page Visibility API anti-feature.
jaffa2•2h ago
on iphone, if you use youtube in the browser in stead of the app (as you should), then you can do background listening if you play the video, lock the phone, unlock the phone, play th video again, lock the phone, unlock the phone, resume play with the media controls, lock the phone.
exitb•3h ago
It's obviously not about YT the product, but about YT the content library. I don't think there are better alternatives to that content library.
blacklion•3h ago
I'm watching not youtube but video creators. There is no even worse alternative if person you want to watch doesn't publish video on other site.

Maybe, for watching "recommended" stream without any subscriptions there are alternatives (which? I cannot name good ones, anyway), but if you watch your subscription you are bound to platform which contain this subscription. And no, content creators are not interchangeable.

exe34•42m ago
any recommendations?
piyuv•3h ago
I’m a paying YouTube premium subscriber. Last weekend, I wanted to download something so I can watch it on my way in the train. The app got stuck at “waiting for download..” on my iPad. Same on iPhone. Restart did not work. I gave up after an hour (30 mins hands on trying stuff, 30 mins waiting for it to fix itself). Downloaded the video using yt-dlp, transferred it to my USB c flash drive, and watched it from that.

Awaiting their “premium cannot be shared with people outside household” policy so I can finally cancel. Family members make good use of ad-free.

beerandt•3h ago
Canceled mine after ad-free stopped working on YouTube Kids of all things (on ShieldTV). Was probably a bug, but with practically no customer service options, no real solutions besides cancel.

I was also a holdover from a paying Play Music subscriber, and this was shortly after the pita music switchover to youtube, so it was a last straw.

masklinn•3h ago
Even more hilariously, if you upload to YouTube then try to download from your creator dashboard thing (e.g. because you were live-streaming and didn’t think to save a local copy or it impacts your machine too much) you get some shitty 720p render while ytdlp will get you the best quality available to clients.
shantara•3h ago
I’m another Premium user in the same position. I use uBlock Origin and Sponsorblock on desktop and SmartTube on my TV. I pay for Premium to be able to share ad-free experience with my less technical family members, and to use their native iOS apps. If they really tighten the rules on Premium family sharing, I’ll drop the subscription in an instant.
al_borland•2h ago
I’m a Premium user and primarily watch on AppleTV. A little while ago they added a feature where if I press the button to skip ahead on the remote when a sponsor section starts, it skips over the whole thing. It skips over “commonly skipped” sections.

While it doesn’t totally remove it, it lets me choose if I want to watch or not, and gets me past it in a single button press. All using the native app. I was surprised the first time this happened. I assume the creators hate it.

cactusplant7374•3h ago
Why not use Brave browser and their playlist feature for offline downloads?
piyuv•3h ago
I’m not using brave browser so did not know it could download videos
QuantumNomad_•1h ago
I’m using Brave, but didn’t know either :p
femtozer•3h ago
I also pay for YouTube Premium, but I still use ReVanced on my smartphone just to disable auto-translation. It’s absolute madness that users can’t configure this in the official app.
piyuv•3h ago
It’ll be fixed when some product manager can offer it as a promotion project
ChocolateGod•24m ago
and removed when that person who is promoted doesn't work on it again
the_af•2h ago
The auto-dub feature is madness. I noticed it first a couple of days ago, I'm crossing my fingers that few authors choose to enable it, and that YouTube makes it easy to disable as a default in settings (not currently possible, you have to do it as you watch, every time).

I'm in a Spanish speaking country, but I want to watch English videos in English.

Auto-generated subtitles for other languages are ok, but I want to listen to the original voices!

OJFord•1h ago
I don't want it dubbed whether I speak the language or not.
zahlman•1h ago
> Auto-generated subtitles for other languages are ok, but I want to listen to the original voices!

The first time I saw this feature, it was on a cover of some pop song in a foreign language. Why on Earth... ?

LtdJorge•45m ago
What about the auto translated titles? It also happens for chapters in the video...

Sames languages as you. It drives me nuts because the translations are almost always wrong.

pjc50•44m ago
Comments are quite good at pointing out when the creator has accidentally left it on (it is of course enabled by default and authors have to actively disable it).
ac29•3h ago
> Awaiting their “premium cannot be shared with people outside household” policy so I can finally cancel

That's been a policy for a while, the sign up page prominently says "Plan members must be in the same household".

No idea if its enforced though.

phkahler•8m ago
I have 2 homes. Every time I "go up north" I have to switch my Netflix household and then back again when I return. This sounds like that won't even be possible.
beala•3h ago
I'm also a premium subscriber, and have struggled with the same issues on the iPad app. I try to keep some shows downloaded for my toddler, and the download feature never seems to work on the first try.

I finally got so fed up, I bought a Samsung Galaxy Tab A7 off ebay for $50 and flashed it with LineageOS. I can now load whatever media I want onto the 1 TB sdcard I've installed in it. The 5 year old hardware plays videos just fine with the VLC app. And, as a bonus, I discovered that NewPipe, an alternative YouTube client I installed through the F-Droid store, is actually much more reliable at downloading videos than the official client. I was planning on using yt-dlp to load up the sdcard, but now I don't even need to do that.

moralestapia•51m ago
Tangential.

The TIDAL app is absolute trash, it has this same issue all the time; not just that, but also, if a download fails it just hangs there and does not download the rest of the album/playlist.

Also, why would you want to download things in the first place? To watch them offline, right? Well, guess what happens when you open the app w/o an internet connection ... it asks you to login, so you cannot even access your music. 900k/year TOC genius work there.

The only reason why I haven't canceled is because I'm too lazy to reset my password in order to login and cancel, lol. Might do it soon, though.

aeyes•38m ago
I use yt-dlp inside of a-shell on iOS, then play files using VLC.
maplethorpe•2h ago
What video did you watch?
piyuv•2h ago
Nintendo Direct. Download issue persisted with all videos though
meindnoch•2h ago
>Awaiting their “premium cannot be shared with people outside household” policy so I can finally cancel.

Then I have good news for you! https://lifehacker.com/tech/youtube-family-premium-crackdown

In fact, I've got an email from them about this already. My YT is still ad-free though, so not sure when it's going to kick in for real.

yolo_420•2h ago
I am a premium subscriber so I can download via yt-dlp in peace without any errors or warnings.

We are not the same.

gjsman-1000•1h ago
For anyone here who runs a startup, I propose two lifestyle benefits you should add:

1. Unlimited YouTube Premium

2. Unlimited drink reimbursement (coffee, tea, smoothies, whatever)

The psychological sense of loss from those two things would be larger than any 5% raise.

whatshisface•1h ago
I personally wouldn't want to hire a startup employee who couldn't figure out how to install a browser extension. ;-)
gjsman-1000•1h ago
You're assuming startups are all tech. At my job, tech is not even 1/3 of employees.
nharada•1h ago
Roku tho
posterguy•31m ago
ah yes, let me just install a browser extension on the kids ipad
whatshisface•12m ago
FYI for next time you're buying, you can install Firefox on Android, although this is perhaps threatened by Google's planned changes to user's ability to install software.
edoceo•1h ago
I don't like that math, rather have the 5% than $8k in perks.
gjsman-1000•1h ago
The pitch is for the employer: This would likely be both cheaper and simultaneously stickier.
hysan•1h ago
I also have YouTube premium and watch mostly on my iPad and TV. YouTube constantly logs me out at least once per day. I notice because I’ll randomly start seeing ads again (I open videos from my rss reader, never their site). This never happened when I wasn’t on premium. I don’t get what they’re doing, but my impression after almost a year is that it’s only slightly less annoying than getting ads. At this point, I might as well not renew and just use ad block.
stronglikedan•1h ago
> Awaiting their “premium cannot be shared with people outside household” policy

I recently got paused for "watching on another device" when I wasn't. I don't think that policy you mention is too far off.

observationist•1h ago
ReVanced and other alternatives exist.

So long as they are broadcasting media to the public without an explicit login system, so as to take advantage of public access for exposure, it will remain perfectly legitimate and ethical to access the content through whatever browser or software you want.

After they blitzed me with ads and started arbitrarily changing features and degrading the experience, I stopped paying them and went for the free and adblocking clients and experience.

I may get rid of phones from my life entirely if they follow through with blocking third party apps and locking things down.

mschuster91•1h ago
the problem is, you cannot be sure what Google does if they catch you violating their ToS. They have killed off entire google accounts for YT copyright strikes with no recourse.
realusername•56m ago
That's why I'm not using Google accounts for anything important, I left gmail in 2014 and I really advise everybody to do the same.

You never know when the hammer can drop.

bornfreddy•21m ago
This. I simply don't understand why some people rely on Google given the risk level, impact and their no-recourse-except-maybe-public-shaming policy.
Akronymus•16m ago
Yeah, same. I still have a gmail account that just forwards emails, and I update the email on services as they come on. Being on your own domain for email is just better.Though, I use a service provider to handle the mail server itself
paxys•47m ago
I'm constantly baffled by how bad the implementation of YouTube Premium downloads is. Videos will buffer to 100% in a matter of seconds but get endlessly stuck when I hit the download button. Why? All the bytes are literally on my device already.
jerf•33m ago
The whole YouTube app is weird. Sometimes it lets you do 1.0x-2.0x. Sometimes it lets you range from .25x-4x. Sometimes it pops up a text selection box with every .05x option from .1 to 4.0. Sometimes it has a nicer UI with shortcut selections for common choices and a sliding bar for speed. It recently picked up a bug where if you're listening to a downloaded video, but turn the screen off and on again, the video playback seems to crash. A few months ago it became very, very slow at casting, all manipulations could take 30 seconds to propagate to the cast video (pause, changing videos, etc)... but they didn't usually get lost. (It would be less weird if they did just get lost sometimes.) You aggressively can't cast a short to a TV, in a way that clearly shows this is policy for some incomprehensible reason, but if you use the YouTube app directly on your set top box it'll happily play a short on your TV. Despite its claims in small text that downloads are good for a month without being rechecked, periodically it just loses track of all the downloads and has to redownload them. It also is clearly trying to reauthorize downloads I made just 30 minutes ago sometimes when I'm in a no-Internet zone, defeating the entire purpose. When downloads are about 1/4th done it displays the text "ready to watch on the download screen" but if you try to watch it it'll fail with "not yet fully downloaded".

Feels like the app has passed the complexity threshold of what the team responsible for it can handle. Or possibly, too much AI code and not enough review and testing. And those don't have to be exclusive possibilities.

lukan•17m ago
Because they want to control the bytes on your devices.

Giving you the bytes would be easy, the hard part is preventing the free flow of information. And those bugs are the side effects.

N0isRESFe8GXmqR•42m ago
I run into that download issue all the time. I need to pause downloading each video. Force close the youtube app. Then unpause the downloads to get them downloading again. It has been happening for years and is still unfixed.
dostick•28m ago
YouTube’s “Download” is not really a download, it’s actually “cache offline” within YouTube app.
arbll•3h ago
I wonder if we're going to see JS runtime fingerprinting attempt from google now
jeroenhd•3h ago
I doubt it'd be difficult for Google to detect if the client is a browser or not. They already need to check for signals of abnormal use to detect things like clickfarms and ad scams.
rs186•1h ago
Ah, JavaScript Run-time Integrity checks!
tomalaci•3h ago
Looks like this runtime is written in Rust. Really does seem like Rust is rapidly swallowing all kinds of common tools and libraries. In this case a single compiled binary for multiple architectures is quite convenient for something like yt-dlp.
jeroenhd•3h ago
Deno itself is written mostly in Rust, but it also leverages [1] Google's V8 Javascript engine which is written in C++.

[1]: https://choubey.gitbook.io/internals-of-deno/architecture/v8

est•3h ago
I really appreciate the engineering effort went into this "JavaScript interpreter"

https://github.com/yt-dlp/yt-dlp/blob/2025.09.23/yt_dlp/jsin...

stevage•3h ago
heh, that's pretty cool.
supriyo-biswas•3h ago
Heh, now I wonder how much JavaScript it actually interprets and given that it’s < 1000 lines, whether it could be used towards an introductory course in compilers.
kccqzy•1h ago
Obviously not. An introductory course would introduce concepts like lexers, parsers, AST, etc, instead of working on strings.

Here are lines 431 through 433:

    if expr.startswith('new '):
        obj = expr[4:]
            if obj.startswith('Date('):
Too•7m ago
There’s a famous presentation by David Beazley where he implements a WASM interpreter in Python in under an hour. Highly recommended.
LordShredda•3h ago
I'm on mobile, this seems like an actual js interpreter that only does objects and arithmetic. Impressive that it went that far
jollyllama•2h ago
I wonder how long until it gets split off into its own project. For the time being, it could do with a lot more documentation. At least they've got some tests for it!
CaptainOfCoit•1h ago
> I wonder how long until it gets split off into its own project

The submission is literally about them moving away from it in favor of Deno, so I think "never" probably gets pretty close.

zahlman•1h ago
Aside from the fact that the point of the announcement is that they're dropping it entirely, this "interpreter" is a hack that definitely is nowhere near capable of interpreting arbitrary JS. For example, the only use of `new` it handles is for Date objects, which it does by balancing parens to deduce the arguments for the call, then treating the entire group of arguments as a string and applying regexes to that.
sirbranedamuj•2h ago
This is the buried lede in this announcement for me - I had no idea they were already going to such lengths. It's really impressive!
codedokode•1h ago
I decided just to look at the code for a moment and discovered ChainMap in Python.
XnoiVeX•1h ago
It's a subset of Javascript. HN discussion here https://news.ycombinator.com/item?id=32794081
Aurornis•1h ago
This is perfect for the problem they were solving. Really cool that they took it this far to avoid adding further overhead.
jokoon•5m ago
Wait I thought they were running an entire browser engine
m_ke•3h ago
I used to work on video generation models and was shocked at how hard it was to find any videos online that were not hosted on YouTube, and YouTube has made it impossibly hard to download more than a few videos at a time.
fibers•3h ago
you have to feed it multiple arguments with rate limiting and long wait times. i am not sure if there have been recent updates other than the js interpreter but ive had to spin up a docker instance of a browser to feed it session cookies as well.
m_ke•2h ago
Yeah we had to roll through a bunch of proxy servers on top of all the other tricks you mentioned to reliably download at a decent pace
trenchpilgrim•2h ago
What are your thoughts on the load scrapers are putting on website operators?
raincole•1h ago
> YouTube has made it impossibly hard to download more than a few videos at a time

I wonder why. Perhaps because people use bots to mass-crawl contents from youtube to train their AI. And Youtube prioritizes normal users who only watch a few videos at most at the same time, over those crawling bots.

Who knows?

m_ke•50m ago
I wonder how Google built their empire. Who knows? I’m sure they didn’t scrape every page and piece of media on the internet and train models on it.

My point was that the large players have monopoly hold on large swaths of the internet and are using it to further advantage themselves over the competition. See Veo 3 as an example, YouTube creators didn’t upload their work to help Google train a model to compete with them but Google did it anyways, and creators didn’t have a choice because all eye balls are on YouTube.

raincole•31m ago
> how Google built their empire. Who knows

By scraping every page and directing the traffic back to the site owners. That was how Google built their empire.

Are they abusing the empire's power now? In multiple ways, such as the AI overview stuff. But don't pretend that crawling Youtube and training video generation models is the same as what Google (once) brought to the internet. And it's ridiculous to expect Youtube to make it easy for crawlers.

Andrews54757•3h ago
Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code.

PoToken - Proof of origin token which Google has lately been enforcing for all clients, or video requests will fail with a 403. On android it uses DroidGuard, for IOS, it uses built in app integrity apis. For the web it requires that you run a snippet of javascript code (the challenge) in the browser to prove that you are not a bot. Previously, you needed an external tool to generate these PoTokens but with the Deno change yt-dlp should be capable of producing these tokens by itself in the near future.

SABR - Server side adaptive bitrate streaming, used alongside Google's UMP protocol to allow the server to have more control over buffering, given data from the client about the current playback position, buffered ranges, and more. This technology is also used to do server-side ad injection. Work is still being done to make 3rd party clients work with this technology (sometimes works, sometimes doesn't).

Nsig/sig extraction example:

- https://github.com/yt-dlp/yt-dlp/blob/4429fd0450a3fbd5e89573...

- https://github.com/yt-dlp/yt-dlp/blob/4429fd0450a3fbd5e89573...

PoToken generation:

- https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide

- https://github.com/LuanRT/BgUtils

SABR:

- https://github.com/LuanRT/googlevideo

EDIT2: Addeded more links to specific code examples/guides

ACCount37•3h ago
If you ever wondered why the likes of Google and Cloudflare want to restrict the web to a few signed, integrity-checked browser implementations?

Now you know.

codedokode•3h ago
There could be valid reasons for fighting downloaders, for example:

- AI companies scraping YT without paying YT let alone creators for training data. Imagine how many data YT has.

- YT competitors in other countries scraping YT to copy videos, especially in countries where YT is blocked. Some such companies have a function "move all my videos from YT" to promote bloggers migration.

toomuchtodo•3h ago
- Enforce views of ads

(not debating the validity of this reason, but this is the entire reason Youtube exists, to sell and push ads)

baxuz•3h ago
Then they should allow a download API for paying customers.
dylan604•2h ago
It's not YT's content though.
codedokode•1h ago
Music labels publish the music on YT in exchange for ad revenue, they won't be happy if someone would download their music for free, and making music is expensive, google how much just a single drum mic costs and you need lot of them.
baxuz•1h ago
> for paying customers
codedokode•52m ago
YT shares income from subscriptions with music labels? I didn't hear about this, and even if they shared the download must be paid much higher than a view because after downloading a person could potentially listen for a track hundred times in a row.
diet_mtn_dew•4m ago
Youtube premium includes Youtube Music, which is alphabet's streaming service, and I assume that they are paying the same fees as everyone else.
balder1991•42m ago
But even if you’re a paying customer, the creator is only paid if you watch it on the platform.
transcriptase•3h ago
>AI companies

Like Google?

>scraping YT without paying YT let alone creators for training data

Like Google has been doing to the entire internet, including people’s movement, conversations, and habits… for decades?

codedokode•2h ago
> Like Google?

Like Google competitors obviously.

> Like Google has been doing to the entire internet, including people’s movement, conversations, and habits… for decades?

Yes, but if you allowed to index your site (companies even spent money to make site better indexable), Google used to bring customers and AI companies bring back nothing. They are just freeloaders.

Chris2048•3h ago
Who says these are valid?
supriyo-biswas•2h ago
Why is this being downvoted? Are people really gonna shoot the messenger and fail to why a company may be willing to protect their competitive position?
supriyo-biswas•3h ago
At least for YouTube, viewbotting is very much a thing, which undermines trust in the platform. Even if we were to remove Google ads from the equation, there’s nothing preventing someone from crafting a channel with millions of bot-generated views and comments, in order to paid sponsor placements, etc.

The reasons are similar for Cloudflare, but their stances are a bit too DRMish for my tastes. I guess someone could draw the lines differently.

rwmj•2h ago
I'm sure that's a problem for Youtube. What does it have to do with me rendering Youtube videos on my own computer in the way I want?
pwg•2h ago
> What does it have to do with me rendering Youtube videos on my own computer in the way I want?

It doesn't. That interferes with google's ad revenue stream, which is why YT continues to try to make it harder and harder to do so.

bitwize•1h ago
You don't have that right. When you view copyrighted content, you do so at the pleasure of the licensor.
rwmj•37m ago
How you watch copyrighted content has never been something that copyright has controlled.
ForHackernews•2h ago
> which undermines trust in the platform

What? What does this even mean? Who "trusts" youtube? It's filled with disinformation, AI slop and nonsense.

supriyo-biswas•2h ago
I provided an example is given right after that sentence. Trustworthiness of the content is an entirely separate thing.
ACCount37•2h ago
If any of this was done to combat viewbotting, then any disruption to token calculation would prevent views from being registered - not videos from being downloaded.
supriyo-biswas•2h ago
From my perspective both problems are effectively the same. I want to count unique users by checking for asset downloads and correlating unique session IDs. People can request the static assets directly, leading to view booting and waste of egress bandwidth.

The solution: have clients prove they are a legitimate client by running some computationally intensive JS that interacts with DOM APIs, etc. (which is not in any way unique to big tech, see Anubis/CreepJS etc.)

The impact on the hobbyist use case is, to them, just collateral damage.

ACCount37•2h ago
No, the difference is: if I'm fighting viewbots, I want zero cues to be emitted to the client. The client should NEVER know whether its view is being counted or not, or why.

Having no reliable feedback makes it so much harder for a viewbotter to find a workaround.

If there's a visible block on video downloads? They're not fighting viewbots with that.

supriyo-biswas•48m ago
For general spam deterrence I agree, but how do you prevent paying for the bandwidth in this case?
sporkxrocket•2h ago
As a viewer, this is not even remotely my problem.
wzdd•2h ago
Youtube has already accounted for this by using a separate endpoint to count watch stats. See the recent articles about view counts being down attributed to people using adblockers.

Even if they hadn't done that, you can craft millions of bot-sponsored views using a legitimate browser and some automation and the current update doesn't change that.

So I'd say Occam's razor applies and Youtube simply wants to be in control of how people view their videos so they can serve ads, show additional content nearby to keep them on the platform longer, track what parts of the video are most watched, and so on.

imiric•2h ago
Like another comment mentioned: that's a problem for YouTube to solve.

They pay a lot of money to many smart people who can implement sophisticated bot detection systems, without impacting most legitimate human users. But when their business model depends on extracting value from their users' data, tracking their behavior and profiling them across their services so that they can better serve them ads, it goes against their bottom line for anyone to access their service via any other interface than their official ones.

This is what these changes are primarily about. Preventing abuse is just a side benefit they can use as an excuse.

jasode•2h ago
>If you ever wondered why the likes of Google and Cloudflare want to restrict the web

I disagree with the framing of "us vs them".

It's actually "us vs us". It's not just us plebians vs FAANG giants. The small-time independent publishers and creators also want to restrict the web because they don't want their content "stolen". They want to interact with real humans instead of bots. The following are manifestations of the same fear:

- small-time websites adding Anubis proof-of-work

- owners of popular Discord channels turning on the setting for phone # verification as a requirement for joining

- web blogs wanting to put a "toll gate" (maybe utilize Cloudflare or other service) to somehow make OpenAI and others pay for the content

We're long past the days of colleagues and peers of ARPANET and NFSNET sharing info for free on university computers. Now everybody on the globe wants to try to make a dollar, and likewise, they feel dollars are being stolen from them.

johnebgd•2h ago
It’s like we are living in an affordability crisis and people are tired of 400 wealthy billionaires profiting from peoples largess in the form of free data/tooling.
skydhash•2h ago
> small-time websites adding Anubis proof-of-work

Those were already public. The issue is AI bot ddos-ing the server. Not everyone has infinite bandwith.

> owners of popular Discord channels turning on the setting for phone # verification as a requirement for joining

I still think that Discord is a weird channel for community stuff. There's a lot of different format for communication, but people are defaulting to chat.

> web blogs wanting to put a "toll gate" (maybe utilize Cloudflare or other service) to somehow make OpenAI and others pay for the content

Paid contents are good (Coursera, O'Reilly, Udemy,...). But a lot of these services wants to have free powered by ads (for audience?).

---

The fact is, we have two main bad actors: AI companies hammering servers and companies that want to centralize content (that they do not create) by adding gatekeeping extension to standard protocols.

jrochkind1•2h ago
i want my content borrowed/shared, and I still need to be engaged in this stuff because the poorly behaved distributed bots that have arisen in the past year are trying to take boundless resources from my site(s), that I cannot afford.
bayindirh•2h ago
> Now everybody on the globe wants to try to make a dollar, and likewise, they feel dollars are being stolen from them.

I'm not in it for the dollar. I just want the licenses I put on my content/code to be respected, that's all. IOW, I don't what I put out there to be free forever (as in speech and beer) to be twisted and monetized by the people who re in this for the dollar.

greenavocado•1h ago
When Nixon slammed the gold window shut so Congress could keep writing blank checks for Vietnam and the Great Society, it wasn't just some monetary technicality. It was the moment America broke its word to the world and broke something fundamental in us too. Suddenly money wasn't something you earned through sweat or innovation anymore. It became something politicians and bankers could conjure from thin air whenever they wanted another war, another corporate bailout, another vote-buying scheme.

Fast forward fifty years and smell the rot. That same fiscal recklessness Congress spending like drunken sailors while pretending deficits don't matter has bled into every pore of society. Why wouldn't it? When BlackRock scoops up entire neighborhoods with Fed-printed cash while your kid can't afford a studio apartment, people notice. When Tyson jacks up chicken prices to record profits while diners can't afford bacon, people feel it. And when some indie blogger slaps a paywall on their life's work because OpenAI vacuumed their words to train ChatGPT? That's the same disease wearing digital clothes.

We're all living in Nixon's hangover. The "us vs us" chaos you see Discord servers demanding your phone number, small sites gatekeeping against bots, everyone scrambling to monetize scraps that's what happens when trust evaporates. Just like the dollar became Monopoly money after '71, everything feels devalued now. Your labor? Worth less each year. Your creativity? Someone's AI training fuel. Your neighborhood? A BlackRock asset on a spreadsheet.

And Washington's still at it! Printing trillions to "save the economy" while inflation eats your paycheck alive. Passing trillion-dollar "infrastructure bills" that somehow leave bridges crumbling but defense contractors swimming in cash. It's the same old shell game: socialize the losses, privatize the gains. The factory worker paying $8 for eggs understands this. The nurse getting lectured about "wage spirals" while hospital CEOs pocket millions understands this. The teenager locking down their Discord because bots keep spamming scams? They understand this.

Weimar happened when money became meaningless. 1971 happened when promises became meaningless. What you're seeing now the suspicion, the barriers, the every-man-for-himself hustle is what bubbles up when people realize the whole system's running on fumes. The diner owner charging $18 for a burger isn't greedy. The blogger blocking AI scrapers isn't a Luddite. They're just building levees against a flood Washington started with a printing press half a century ago.

The tragedy is that we're all knee-deep in the same muddy water, throwing sandbags at each other while the real architects of this mess the political grifters, the Fed bankers, the extraction-engine capitalists watch dry-eyed from their high ground. Until we stop accepting their counterfeit money and their counterfeit promises, we'll keep drowning in this rigged game. The gold window didn't just close in '71. The whole damn social contract rusted shut.

chrisweekly•1h ago
Wow. That was eloquent, and coherent, and depressing. I'd be grateful for someone to counter with something less dismal. Good things are still happening in the world. A positive future remains possible -- but we have to be able to imagine it to bring it into being.
sillyfluke•12m ago
Well on the bright side blood avocados are still green. Which the poster also seems to appreciate.
zahlman•9m ago
What does any of this have to do with yt-dlp?
btown•1h ago
But this, too, skips over some nuance. There are a few types of actors here:

- small content creators who want to make their content accessible to individuals

- companies that want to gobble up public data and resell it in a way that destroys revenue streams for content creators

- gatekeepers like Cloudflare who want to ostensibly stop this but will also become rent-extractors in the process

- users who should have the right to use personal tools like yt-dlp to customize their viewing experience, and do not wish to profit at the expense of the creators

We should be cautious both that the gatekeepers stand to profit from their gatekeeping, and that their work inhibits users as well.

If creators feel this type of user (often a dedicated fan and would-be promoter) is a necessary sacrifice to defend against predatory data extractors… then that’s absolutely the creator’s choice, but you can’t say there’s a unified “us” here.

bitwize•1h ago
Duh. I've known this for decades. The biggest advocates for DRM I've known are small-time content creators: authors, video producers, musicians. They've been saying the same thing since the 90s: without things like DRM, their stuff would be pirated, and they'd like to earn a living doing what they love instead of grinding at a day job to support themselves while everybody benefits from their creative output. In addition, major publishers and record labels won't touch stuff that's been online because of the piracy risk. They don't want to make an investment in smaller creators without a return in the form of sales of copies. That last bit is less true of music now than it used to be because of streaming and stuff, but the principle still applies.

This is why the DMCA will never be repealed, DRM will never go away, and there is no future for general purpose computing. People want access digital content, but the creators of that content wouldn't release it at all if they knew that it could be copied endlessly by whomever receives it.

mschuster91•57m ago
> The small-time independent publishers and creators also want to restrict the web because they don't want their content "stolen"

... or just keep their site on the Internet. There hasn't been any major progress on sanctioning bad actors - be it people running vulnerable IoT crap that ends up being taken over by a botnet, cybercriminals and bulletproof hosters, or nation state actors. As long as you don't attack targets from your own geopolitical class (i.e. Russians don't attack Russians, a lot of malware will just quit if it spots Russian locale), you can do whatever the fuck you want.

And that is how we end up with darknet services where you can trivially order a DDoS taking down a website you don't like or, if you manage to get your opponent's IP leaked during an online game, their residential IP address. Pay with whatever shitcoin you have, and no one is any wiser who the perpetrator is.

pryelluw•44m ago
I don’t feel like dollars are stolen from me. It’s more of companies abusing my goodwill to publish information online. From higher bills as a result of aggressive crawling, to copying my work and removing all copyright/licensing from the code. Sure, fair use and all, but when they return the same exact code it just makes me wonder.

Nowadays, producing anything feels like being the cows udder.

mtrovo•2h ago
I don't know, it's really hard to blame them. In a way, the next couple of years are going to be a battle to balance easy access to info with compensation for content creators.

The web as we knew it before ChatGPT was built around the idea that humans have to scavenge for information, and while they're doing that, you can show them ads. In that world, content didn't need to be too protected because you were making up for it in eyeballs anyway.

With AI, that model is breaking down. We're seeing a shift towards bot traffic rather than human traffic, and information can be accessed far more effectively and, most importantly, without ad impressions. So, it makes total sense for them to be more protective about who has access to their content and to make sure people are actually paying for it, be it with ad views or some other form of agreement.

SV_BubbleTime•1h ago
Don’t worry!

Ads are coming to AI. The big AI push next will be context, your context all the time. Your phone will “help” and get all your data to OpenAI…

“It looks like you went for a run today? Good job, you deserve a treat! Studies show a little ice cream after a long run is effectively free calories! It just so happens the nearest Dairy Queen is running a promotion just for the next 30 minutes. I’m getting you directions now.”

bitwize•1h ago
This is why contra Louis Rossman, Clippy was not a good thing for humanity.
codedokode•1h ago
It would not be that much of a problem if ads promoted healthy and tasty food but they will probably promote an ice-cream made from a powder and chemicals emulating taste of berries rather than from milk and fresh-picked berries.
Noumenon72•19m ago
"I'm calling the user analysis tool... it seems this user is health conscious. I'll suggest a trail app for their next run instead of ice cream."
jonas21•16m ago
But that's a very different scenario for creators, right? When YouTube shows an ad, they take a cut of the revenue, and the creator gets the rest. When an ad is shown on top of AI-generated content, creators get nothing.
chrisweekly•1h ago
I think your point is valid, but FTR the "shift" happened long before ChatGPT; bot traffic has exceeded that of humans for over a decade.
gjsman-1000•2h ago
Everything trends towards centralization on a long enough period.

I laugh at people who think ActivityPub or Mastodon or BlueSky will save us. We already had that, it was called e-mail, look what happened once everyone started using it.

If we couldn't stop the centralization effects that occurred on e-mail, any attempt to stop centralization in general is honestly a utopian fool's errand. Regulation is easier.

numpad0•1h ago
e-mail can't handle 24/7 1k posts/sec traffic which Twitter was about. A more appropriate analogue is IRC.
toomuchtodo•23m ago
I am a big supporter of AT Protocol, and I contribute some money to a fund to build on it. Why laugh at running experiments? Nothing will "save us," it is a constant effort as long as humans desire to use these systems to connect. Email exists today, and is very usable still as a platform that cannot be captured. The consolidation occurred because people do not want to run their own servers, so we should build for that! Bluesky and AT Protocol are experiments in building something different, with different use cases and capabilities, that also cannot be captured. Just like email. You can run your own PDS. You can run your own stack from PDS to users "end to end" if you so choose. You can pay to do both of these tasks. No one can buy this or take it away from you, if it is built on protocols instead of a platform someone can own and control.

Regulation would be great. The EU does it well. It is lacking in the US, and will be for some time. And so we have to downgrade to technical mitigations against centralization until regulation can meet the burden.

th0ma5•46m ago
Weird people talking about small time creators wanting DRM I've never seen that... Usually they'd be hounding for any attention? I don't know why multiple accounts are seemingly independently bringing this up, but maybe it is trying to muddy the waters? This concept?
dylan604•2h ago
> For the web it requires that you run a snippet of javascript code (the challenge) in the browser to prove that you are not a bot.

How does this prove you are not a bot. How does this code not work in a headless Chromimum if it's just client side JS?

Andrews54757•2h ago
Good question! Indeed you can run the challenge code using headless Chromium and it will function [1]. They are constantly updating the challenge however, and may add additional checks in the future. I suppose Google wants to make it more expensive overall to scrape Youtube to deter the most egregious bots.

[1] https://github.com/LuanRT/BgUtils

toomuchtodo•1h ago
LLMs solve challenges. Can we not solve these challenges with sufficiently advanced LLMs? Gemini even, if you're feeling lulz-y.
balder1991•41m ago
Yes, by spending money.
toomuchtodo•34m ago
I agree, in some cases and depending on LLM endpoint, some money may need to be spent to enable ripping. But is it cheaper than paying Youtube/Google? That is the question.
Beretta_Vexee•2h ago
Once JavaScript is running, it can perform complex fingerprinting operations that are difficult to circumvent effectively.

I have a little experience with Selenium headless on Facebook. Facebook tests fonts, SVG rendering, CSS support, screen resolution, clock and geographical settings, and hundreds of other things that give it a very good idea of whether it's a normal client or Selenium headless. Since it picks a certain number of checks more or less at random and they can modify the JS each time it loads, it is very, very complicated to simulate.

Facebook and Instagram know this and allow it below a certain limit because it is more about bot protection than content protection.

This is the case when you have a real web browser running in the background. Here we are talking about standalone software written in Python.

dylan604•1h ago
why can a bot dev not just get all of these values from the laptop's settings and hardwire the headless version to have the same values?
Beretta_Vexee•48m ago
Because the expected values are not fixed, it is possible to measure response times and errors to check whether something is in the cache or not, etc.

There are a whole host of tricks relating to rendering and positioning at the edge of the display window and canvas rather than the window, which allow you to detect execution without rendering.

To simulate all this correctly, you end up with a standard browser, standard execution times, full rendering in the background, etc. No one wants to download their YouTube video at 1x speed and wait for the adverts to finish.

Aperocky•53m ago
And barely a few days after google did it the fix is in.

Amazing how they simply couldn't win - you deliver content to client, the content goes to the client. Could be the largest corporation of the world and we still have yt-dlp.

That's why all of them wanted proprietary walled gardens where they would be able to control the client too - so you get to watch the ads or pay up.

sphars•3h ago
This will be interesting to see how it affects the numerous Android apps on F-Droid that are essentially wrappers around yt-dlp to create a YouTube Music clone.
BoredPositron•3h ago
Ugh... Deno. After they started to extort the JS community for money to fund their PR stunt against Oracle and the resulting "last chance" trademark dispute, I stay as far away from it as I can.
MangoToupe•3h ago
At some point we’re going to need a better place to put videos than YouTube. The lack of any democratization of bulk storage is beginning to be a real problem on the internet.

Yes, we have archive.org. We need more than that, though.

I’m sure there’s some distributed solution like IPFS but I haven’t seen any serious attempt to make this accessible to every day people.

zenmac•3h ago
There are: peertube, odysee, minds, rumble, bitchute web torrent)...

It is the same reason why people just can't get off IG. Network effect and in YT case a lot of disk space and bandwidth.

MangoToupe•1h ago
I don’t think network effect matters much if you’re not trying to advertise the content. Organizations can just link to it from their site.

I admit I haven’t looked into peertube, and I didn’t think that rumble was any better than YouTube. I don’t recognize the others. Thank you; I’ll resurvey.

coldpie•3h ago
> The lack of any democratization of bulk storage is beginning to be a real problem on the internet.

There are many thousands of paid hosting services, feel free to pick one. It turns out hosting TB of data for free is a pretty tricky business model to nail down.

superkuh•2h ago
There have been plenty of free distributed hosting services for the web that worked perfectly (popcorn time, etc, etc). It's just that every time they become popular they are attacked legally and shut down. The problem is not technical, or even resource based, the problem is legal. Only a mega-corp can withstand the legal attacks.

And even if the legal attacks could be mitigated most people would still use youtube because they're there for the money (or for people who are there for the money). They are not there for a video host. Youtube enables distribution of money and there's no way that any government would let any free system distribute money without even more intense legal, and indeed physically violent, attacks.

reaperducer•3h ago
I keep seeing ads on TV for Photobucket (Which I thought was dead) for 1TB of storage for either free, or $5, depending on the ad.

Maybe there is an opportunity for that company to expand.

bob1029•2h ago
If you want to compete with YT you need to basically build AWS S3 in your own data centers. You'd have to find a way to make your service run cheaper than google can if you wanted to survive. You'd have to get very scrappy and risky. I'd start with questions like: how many 9s of durability do we actually need here? Could we risk it until the model is proven? What are the consequences for losing cat videos and any% speed runs of mario64? That first robotic tape library would be a big stepwise capex event. You'd want to make sure the whole thing makes sense before you call IBM or whoever for a quote.
ndriscoll•2h ago
Games Done Quick has raised 10s of millions for charity. I suspect they could raise a few thousand for a few dozen TB of nvme storage if they wanted to host a speedrun archive.
warkdarrior•1h ago
YouTube get 700,000 hours of video uploaded every day. That's 4.3 PB added per day. You may need more than a few dozen TB... https://www.reddit.com/r/AskProgramming/comments/vueyb9/how_...
ndriscoll•3m ago
They don't get 700,000 hours of any particular niche though, so it's easy enough for small groups to compete with youtube for their needs.
jsheard•29m ago
> If you want to compete with YT you need to basically build AWS S3 in your own data centers. You'd have to find a way to make your service run cheaper than google can if you wanted to survive.

YouTube's economy of scale goes way beyond having their own datacenters, they have edge caches installed inside most ISP networks which soak up YT traffic before it even reaches a Google DC. You can't compete with them on price without a staggeringly huge buildout.

pmdr•1h ago
> I’m sure there’s some distributed solution like IPFS

Almost 25 years on the internet and I have not been able to download anything from IPFS. Does one need a PhD to do so?

mschuster91•26m ago
The problem with bulk storage is that it will be abused at large scale.

CSAM peddlers, intellectual property violators, unconsensual sexual material ("revenge porn"), malware authors looking for places to exfiltrate stolen data, propagandists and terrorists, the list of abusers is as long as it is dire.

And for some of these abuser classes, the risk for any storage service is high. Various jurisdictions require extremely fast and thorough responses for a service provider to not be held liable, sometimes with turnaround times of 24 hours or less (EU anti terrorism legislation), sometimes with extremely steep fines including prison time for responsible persons. Hell, TOR exit node providers have had their homes raided and themselves held in police arrest or, worse, facing criminal prosecution and prison time particularly for CSAM charges - and these are transit providers, not persistent storage.

And all of that's before looking on the infrastructure provider side. Some will just cut you off when you're facing a DDoS attack, some will bring in extortionate fees (looking at you, AWS/GCE/Azure) for traffic that may leave you in personal bankruptcy. And if you are willing to take that risk, you'll still run the challenge of paying for the hardware itself - storage isn't cheap, 20TB of storage will be around 200€ and you want some redundancy and backups, so the actual cost will rather be 60-100€/TB plus the ongoing cost of electricity and connectivity.

That's why you're not seeing much in terms of democratization.

MangoToupe•23m ago
Maybe that’s true, but YouTube is just absolutely miserable to use in every way. There’s got to be better options.
apetresc•3h ago
The writing is on the wall for easy ripping. If there's any YT content you expect you'll want to preserve for a long time, I suggest spinning up https://www.tubearchivist.com/ or something similar and archiving it now while you still can.
lyu07282•2h ago
They already had the proper-DRM tech for youtube movies for years, why didn't they already turn that on for all content?
Mindwipe•2h ago
YouTube's delivery scale is enormous and adding additional complexity if they don't have to is probably considered a no no.

But if they decide they have to, they can do it fairly trivially.

trenchpilgrim•1h ago
It would break many millions of old consumer devices that no longer receive updates, like old smart TVs. They are waiting for that old device traffic to drop low enough before they can force more robust measures.

You already need such things for certain formats.

advisedwang•34m ago
YT probably HAD to put the DRM on in order to get the license deal with the studios. Nobody is twisting their arm as much so other interests (wider audience, less server side resources, not getting around to it) can prevail.
wintermutestwin•2h ago
I agree and feel that the time is now to archive all of the truly valuable cultural and educational content that YT acquired through monopolistic means.

This solution looks interesting, but I am technical enough to know that this looks like a PITA to setup and maintain. It also seems like it is focused on downloading everything from a subbed channel.

As it is now, with a folder of downloaded videos, I just need a local web server that can interpret the video names and create an organized page with links. Is there anything like this that is very lightweight with a next next finish install?

feverzsj•3h ago
That's why youtube is so buggy and slow.
ivanjermakov•3h ago
Can we remove heartdropping mystery from the title? My first thought is that Google makes it more difficult to download from YouTube.

"yt-dlp moves to Deno runtime"

Fabricio20•2h ago
Google is making it harder to download from Youtube. Your first thought is correct! Every other website that yt-dlp supports doesn't require this change. Additionally, yt-dlp is still written in python, it has not moved to deno. They are only adding a deno dependency for the javascript challenges added by youtube.
ivanjermakov•1h ago
I get that, but still title is too "loud".
zelphirkalt•3h ago
What I found much more annoying, and so far have not been able to work around, is that yt-dlp requires you to have a YouTube account, something that I have not had for a decade or so, and am unwilling to create again.

What tool can I use to simply store what my browser receives anyway, in a single video file?

skydhash•3h ago
It must be a pretty recent (as in added yesterday) addition, as I was watching youtube with mpv+yt-dlp.
degamad•3h ago
When did it start requiring one? It didn't require one the last time I used it a few months ago...
ACCount37•2h ago
Google started using IP range blocks recently. If they decide that your IP stinks, they'll block YouTube viewing and demand that you log in.

It's inconsistent as fuck, and even TOR exit nodes still work without a log in sometimes.

astroflection•32m ago
I can confirm this. I guess they didn't like me using Invidious.
ACCount37•28m ago
That's bad enough for normal VPN users who use VPN for privacy reasons. But a lot of countries have heavily censored web, and not using a VPN is simply not an option there.

Good on Google for kicking people while they're down.

zelphirkalt•2h ago
I think for me it has been this way for a year or so. Maybe it is because I am on a VPN. I also cannot view YouTube videos on YouTube any longer, because it always wants me to log in, to "prove I am not a bot". So I have switched to only using invidious instances, and if they don't work, then I just cannot watch the video.

I wish content creators would think of their own good more, and start publishing on multiple platforms. Are there any terms that YouTube has for them, that reduce revenue, if they publish elsewhere as well? Or is it mostly just them being unaware?

2OEH8eoCRo0•2h ago
> What tool can I use to simply store what my browser receives anyway, in a single video file?

This. I'm interested in such a tool or browser extension.

crtasm•2h ago
I'm using it right now without a youtube account.
wraptile•3h ago
Days of just getting data off the web are coming to an end as everything requires a full browser running thousands of lines of obfuscated js code now. So instead of a website giving me that 1kb json that could be cached now I start a full browser stack and transmit 10 megabytes through 100 requests, messing up your analytics and security profile and everyone's a loser. Yay.
SV_BubbleTime•1h ago
Do you know what Accelerate means?

I want them to go overboard. I want BigTech to go nuts on this stuff. I want broken systems and nonsense.

Because that’s the only way we’re going to get anything better.

nananana9•1h ago
If you showed me the current state of YouTube 8 years ago - multiple unskippable ads before each video, 5 midrolls for a 10 minute video, comments overran with bots, video dislikes hidden, the shorts hell, the dysfunctional algorithm, .... - I would've definitely told you "Yep, that will be enough to kill it!"

At this point I don't know - I still have the feeling that "they just need to make it 50% worse again and we'll get a competitor," but I've seen too many of these platforms get 50% worse too many times, and the network effect wins out every time.

encom•32m ago
It's classic frog boiling. I want them (for whatever definition of "them") to just nuke the frog from orbit.
jdiff•1h ago
Accelerationism is a dead-end theory with major holes in its core. Or I should say, "their" core, because there's a million distant and mutually-incompatible varieties. Everyone likes to say "gosh, things are awful, it MUST end in collapse, and after the collapse everyone will see things MY way." They can't all be right. And yet, all of them with their varied ideas still think it'll be a good idea to actively push to make things worse in order to bring on the collapse more quickly.

It doesn't work. There aren't any collapses like that to be had. Big change happens incrementally, a bit of refactoring and a few band-aids at a time, and pushing to make things worse doesn't help.

exe34•47m ago
I'm not waiting for the collapse to fix things - I'm waiting for it so that I won't have any more distractions and I can go back to my books.
jdiff•3m ago
As I said, there aren't any collapses like that to be had. Heaven and Earth will be moved to make the smallest change necessary to keep things flowing as they were. Banks aren't allowed to fail. Companies, despite lengthy strings of missteps and billions burned on dead ends, still remain on top.

You can step away from the world (right now, no waiting required). But the world can remain irrational longer than you can wait for it to step away from you, and pushing for more irrationality won't make a dent in that.

nananana9•1h ago
On the bright side, that opens an opportunity for 10,000 companies whose only activity is scraping 10MB worth of garbage and providing a sane API for it.

Luckily all that is becoming a non-issue, as most content on these websites isn't worth scraping anymore.

judge2020•1h ago
*and whose only customers are using it for AI training
pmdr•1h ago
> Days of just getting data off the web are coming to an end

All thanks to great ideas like downloading the whole internet and feeding it into slop-producing machines fueling global warming in an attempt to make said internet obsolete and prop up an industry bubble.

The future of the internet is, at best, bleak. Forget about openness. Paywalls, authwalls, captchas and verification cans are here to stay.

dpedu•1h ago
And it's all to sell more ads.
bjourne•54m ago
For now, yes, but soon CloudFlare and ever more annoying captchas may make that option practically impossible.
daemin•41m ago
This 1kb os json still sounds like a modern thing, where you need to download many MB of JavaScript code to execute and display the 1kb json data.

What you want is to just download the 10-20kb html file, maybe a corresponding css file, and any images referenced by the html. Then if you want the video you just get the video file direct.

Simple and effective, unless you have something to sell.

pjc50•27m ago
The main reason for doing video through JS in the first place, other than obfuscation, is variable bitrate support. Oddly enough some TVs will support variable bitrate HLS directly, and I believe Apple devices, but not regular browsers. See https://github.com/video-dev/hls.js/

> unless you have something to sell

Video hosting and its moderation is not cheap, sadly. Which is why we don't see many competitors.

Zopieux•20m ago
And by "not many" you really mean zero competitors.

(before you ask: Vimeo is getting sold to an enshitification company)

pjc50•30m ago
I think this is just another indication of how the web is a fragile equilibrium in a very adversarial ecosystem. And to some extent, things like yt-dlp and adblocking only work if they're "underground". Once they become popular - or there's a commercial incentive, like AI training - there ends up being a response.
sharperguy•2h ago
Why can youtube not just give a micropayments backed API? Just charge a few cents per video download and be done with it.
eitau_1•2h ago
meanwhile Youtubers: a penny per view would be 10x what Youtube pays us

https://www.youtube.com/watch?v=3nloigkUJ-U&t=4851s

kccqzy•1h ago
The YouTube RPM (revenue per mille) strongly depends on the location of the audience and the topic of the video. It could be anywhere from $0.5 to $20. That 10x figure could very well be true for that YouTuber, but it's also true that other YouTubers already earn more than a penny per view.
trenchpilgrim•1h ago
They do. It's called YouTube Premium.
pmdr•1h ago
AFAIK Premium allows you to download to persistent browser storage. But is it DRM-free/open or usable format?
Telaneo•3m ago
It's DRM-ed and somewhat broken.
yreg•1h ago
It's not though. You can't download an mp4 to use however you wish with YouTube Premium. And definitely not via an API.
trenchpilgrim•59m ago
None of that was mentioned in the comment?
exe34•45m ago
did you miss the word "API"? it was there.
rcarmo•2h ago
So, instead of using something lightweight and embeddable like QuickJS, they opted for Deno? Nothing specifically against it, just seems... overkill
Waraqa•2h ago
See this comment:

https://news.ycombinator.com/item?id=45359626

rcarmo•1h ago
There are other embeddable JS engines out there.
zb3•2h ago
Fortunately the community is not alone in this fight, because many AI companies need to be able to download YT videos. But they should sponsor yt-dlp more directly..
zelphirkalt•2h ago
How will this JS execution be contained/isolated? Do we have to run it inside a VM, or containers?
BolexNOLA•2h ago
What are folks thoughts on jdownloader2 these days? Hell is that still kicking?
tommy92•6m ago
Yeah my go to for youtube still. Working as good as ever for that so far.
nikcub•2h ago
Just the other day there was a story posted on hn[0][1] that said YouTube secretly wants downloaders to work.

It's it's always been very apparent that YouTube are doing _just enough_ to stop downloads while also supporting a global audience of 3 billion users.

If the world all had modern iPhones or Android devices you'd bet they'd straight up DRM all content

[0] https://windowsread.me/p/best-youtube-downloaders

[1] https://news.ycombinator.com/item?id=45300810

trenchpilgrim•1h ago
More specifically, yt-dlp uses legacy API features supported for older smart TVs which don't receive software updates. Eventually once that traffic drops to near zero those features will go away.
Aurornis•1h ago
That conspiracy theory never even made sense to me. Why would anyone think that a payment and ad-supported content platform secretly wants their content to be leaked through ad and payment free means?
judge2020•1h ago
Mainly the theory that, if you can’t use downloaders to download videos, then people will no longer see YT as the go-to platform for any video hosting and will consider alternatives.

And I call that a theory for a reason. Creators can still download their videos from YT Studio, I'm not sure how much importance there is on being able to download any video ever (and worst case scenario people could screen recording videos)

elcapitan•2h ago
Is there an official name for this endless uphill battle? Counter-Enshittification?
layer8•1h ago
Cleaning the Augean stables.
phplovesong•2h ago
Youtube is the real monopoly. Creators are also slaves, as they cant monetize elsewhere, and also they cant let their users download their own content. And the icing on the cake is youtube is unbearable without an ad-blocker, and even with that youtube has started throttling ad-block users.

Its such a shithole, with no real replacement, sad state of affairs.

mavhc•2h ago
why can't they monetize elsewhere?
layer8•1h ago
Much, much, much smaller audience elsewhere.
pessimizer•1h ago
And if the audiences got larger on a site, governments around the world would decide together to drag them into court and keep them there until they closed down or sold to Ellison's kid.
trenchpilgrim•1h ago
> Here's the problem (and it's not insurmountable): right now, there's no easy path towards sustainable content production when the audience for the content is 100x smaller, and the number of patrons/sponsors remains proportionally the same.

https://www.jeffgeerling.com/blog/2025/self-hosting-your-own...

HankStallone•52m ago
Some do, and those who are able to make the move to patronage or subscriber monetization seem much happier for it. But that's most viable for creators who have already built up a viable customer base, which usually started on YouTube. It's much harder if you start out somewhere else.
random29ah•2h ago
It's almost funny, not to mention sad, that their player/page has been changed, filling it with tons of JS that makes less powerful machines lag.

For a while now, I've been forced to change "watch?v=" to "/embed/" to watch something in 480p on an i3 Gen 4, where the same video, when downloaded, uses ~3% of the CPU.

However, unfortunately, it doesn't always work anymore.

https://www.youtube.com/watch?v=xvFZjo5PgG0 https://www.youtube.com/embed/xvFZjo5PgG0

While they worsen the user experience, other sites optimize their players and don't seem to care about downloaders (pr0n sites, for example).

skydhash•2h ago
Put that next to GitHub. The app is nearly unusable on an i5 8th, often I just download a snapshot to browse locally.
bArray•2h ago
Personally I am looking to get away from Youtube and looking towards some form of PeerTube/peer-based platform.
jhatemyjob•2h ago
Might as well start an effort to rewrite the whole project in Javascript at this point
DrStartup•2h ago
none of it will matter soon. anything you want to see or watch will be dynamically generated just for you. enders game is here.
HelloUsername•1h ago
What if I want to rewatch it, offline?
AbuAssar•2h ago
on why they chose Deno instead of node:

"Other JS runtimes (node/bun) could potentially be supported in the future, the issue is that they do not provide the same security features and sandboxing that deno has. You would be running untrusted code on your machine with full system access. At this point, support for other JS runtimes is still TBD, but we are looking in to it."

codedokode•48m ago
While deno has sandboxing, it also has potential access to hundreds of dangerous functions, it might be better just to write a tiny wrapper around JS engine that adds only the function to write to stdout.
nromiun•2h ago
TIL that you can run frontend Javascript with a package like Deno. I thought you need a proper headless browser for it.
skydhash•2h ago
I think you only need something like `jsdom` to have the core API available. The DOM itself is just a tree structure with special nodes. Most APIs are optional and you can provide stubs if you're targeting a specific websites. It's not POSIX level.
bob1029•1h ago
I was thinking the same walking into this thread. I figured DOM/CSS/HTML would be part of the black box magic, but I suppose from the perspective of JS all of that can be faked appropriately.
bArray•2h ago
Ronsor [1] and reply by seproDev:

> Why can't we embed a lightweight interpreter such as QuickJS?

> @Ronsor #14404 (comment)

The linked comment [2]:

> @dirkf This solution was tested with QuickJS which yielded execution times of >20 minutes per video

How on earth can it be that terrible compared to Deno?

[1] https://github.com/yt-dlp/yt-dlp/issues/14404#issuecomment-3...

[2] https://github.com/yt-dlp/yt-dlp/issues/14404#issuecomment-3...

jlokier•1h ago
> How on earth can it be that terrible [>20 minutes] compared to Deno?

QuickJS uses a bytecode interpreter (like Python, famously slow), and is optimised for simplicity and correctness. Whereas Deno uses a JIT compiler (like Java, .NET and WASM). Deno uses the same JIT compiler as Chrome, one of the most heavily-optimised in the world.

That doesn't normally lead to such a large factor in time difference, but it explains most of it, and depending on the type of code being run, it could explain all of it in this case.

QuickJIT (a fork of QuickJS that uses TCC for JIT) might yield better results, but still slower than Deno.

ynx•1h ago
JIT is still banned by policy on a LOT of mobile devices, meaning that previous usage of yt-dlp on mobile is now effectively unsupportable.
SpaghettiCthulu•9m ago
I haven't tested this, but in theory running deno with `--v8-flags='--jitless'`[^1][^2] will disable the JIT compiler.

[^1]: https://v8.dev/blog/jitless

[^2]: https://docs.deno.com/runtime/getting_started/command_line_i...

ronsor•1h ago
It's horrifying and Google must've worked very hard to kill the performance in other interpreters.
darknavi•1h ago
That is interesting. We use QuickJS in Minecraft (Bedrock, for modding) and while it's much slower than V8 it's not _that_ much slower.
beyondcompute•2h ago
Why won’t they use my browser for downloads, for example through TestCafe? That would also allow downloading premium quality (for subscribers) and so on.
rfl890•2h ago
I think you can get premium formats through the --cookies-from-browser flag
phendrenad2•2h ago
Good to see the mice are still winning the cat-and-mouse game. Selfishly, I kind of want the cat to start to win, to satisfy my curiosity. I predict that if YouTube ever actually blocked downloading, a YouTube competitor that supports downloading would start to immediately gain popularity. I want to know if I'm right about that, and there's no way to test unless Google actually starts to win. Go, Google, go! I believe in you!
codedokode•46m ago
Those who download videos are a minority and targeting minorities will never give you exponential growth. Furthermore, the same minority probably abuses ad blockers so it would be difficult to squeeze a single cent from these freeloaders.
Alifatisk•1h ago
The length Youtube have gone to make it impossible to download videos. At the same time, Tiktok allows anyone to download a video with just right click
doublerabbit•1h ago
With the recent forced buy of TikTok with Rupert, Larry and co, I doubt that's going to be a thing for much longer; they will want to make money some how.
benoau•1h ago
Came across this the other day - set of yt-dlp helper scripts loaded up with flags:

https://github.com/TheFrenchGhosty/TheFrenchGhostys-Ultimate...

novoreorx•1h ago
Surprisingly, Deno was chosen as the first JavaScript runtime due to its security features. I thought it was almost dead, as Bun is growing very quickly among developers.
RiverCrochet•1h ago
A friend of mine recorded a YouTube video using OBS. She had to do some minor edits on it and could not use her system during the recording, but it worked. I told her to stop it, as that is infringing on the creator's copyright and is an assault on the nation's digital economy. She hasn't recorded a video since, at least not that I know about. I feel good about making sure YouTube can reasonably profit off of creators' content since they give away the storage and access for free.
smiley1437•1h ago
Instructions on Vine-glo grape concentrate during prohibition: "Do not place the liquid in this jug and put it away in the cupboard for twenty-one days, because then it would turn into wine."
RiverCrochet•1h ago
I had another friend that simply recorded YouTube videos from their smartphone. As a zealous law abiding citizen, I immediately smacked the phone out of his hand and lectured on how copyright law is the foundation of the Information Age, which is the future, and disregarding it is an affront to modern life and civilization. I made him delete all his videos, and even made him hand write letters of apologies to the YouTube creators. These creators don't reveal their home addresses, but I'm sure they appreciated the emails containing the scan of the handwritten letters.

We have an old SCSI scanner, so it took about as long to scan it as it did to write it.

sombragris•1h ago
Many Linux distros have Firefox's JavaScript (SpiderMonkey?) runtime independently packaged and available. Can it be used for this?
Havoc•1h ago
Really feels like the somewhat open nature of yt is running on borrowed time
hackingonempty•1h ago
First you spend money to create something people really want and build a big user base.

Then you open it up to third party businesses and get them tied to your platform, making money off your users.

Once locked in you turn the screws on the businesses to extract as much money from them as possible.

Finally you turn the screws on the users to extract every last bit of value from the platform before it withers and fades into irrelevance.

akudha•36m ago
What you say is true for most companies/software, but YouTube can play a nasty game for a very long time before it withers into irrelevance (if at all). They have enormous moat, one would need enormous resources to take on YouTube, I don't think anyone has that kind of patience or resources to even attempt. Like it or not, we are stuck with YT for a while.

I have learned so much from YouTube - I wish it was more open and friendly to its creators and users :(

In the meantime, all we can do is support smaller alternatives like https://nebula.tv/

buyucu•1h ago
I was scared this morning when yt-dlp did not work, but a git pull fixed it.

A huge thank you to the yt-dlp folks. They do amazing work.

guywithahat•1h ago
It's incredible how much work goes into these open source projects for downloading youtube videos, especially since youtube keeps breaking them. There are nearly 1500 contributors
jsheard•39m ago
To be fair yt-dlp supports a lot more than just YouTube

https://github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extracto...

zahlman•1h ago
Noteworthy to me: deno is MIT licensed, but PyPI distributions (at least the ones I checked) include neither license nor source code. It's normal for pre-built distributions ("wheels") to contain only the Python code (which for a project like this is just a small bootstrap used to find the compiled executable — it doesn't appear to be providing any Python API), but they should normally still have a LICENSE file.

It's also common to have the non-Python (here, Rust) source in source distributions ("sdists"), but this project's sdist is only a few kilobytes and basically functions as a meta-package (and also includes no license info). It "builds" Deno by detecting the platform, downloading a corresponding zip from the GitHub releases page, extracting the standalone Rust executable, and then letting Hatchling (a popular build tool in the Python ecosystem) repackage that in a wheel.

Update: It turns out that the Python package is published by a third party, so I submitted an issue (https://github.com/manzt/denop/issues/1) to ask about the licensing.

abaa88_•57m ago
Viva Revanced!
oybng•55m ago
more dependency bloat just to deobfuscate some randomly generated bs that's increasingly complex for no reason and has no value existing in the first place, much like its creators
dandiep•50m ago
I've been using yt-dlp to download transcripts. Are there alternatives that don't require going through all these hoops? I'm guessing no.
zeristor•46m ago
I did download YouTube videos a few years ago, I did value that YouTube could keep your place.

But it’s a real mess it keeps crashing, something I might too humbly put down to me having too many files, but passive aggressively put it down to YouTube on iPad not having a limited amount of storage space.

On the other hand there’s a number of amazing videos I’ve downloaded to watch which have been remotely wiped. Grrr

syrusakbary•40m ago
I wonder if they could use Wasmer to execute Javascript under the hood without limitations.
porphyra•37m ago
At this rate they are just gonna have to ship a whole web browser with it lol.
scosman•21m ago
Great ad for deno. I hit a similar one the other day from pydantic. They make a MCP server for running sandboxed python code and the they did that… Python to WASM, and wasm running in deno.
anthk•20m ago
I refuse to run JS on my n270 netbook; even less with a propietary license. Thus, I will just use some invidious mirror.
jokoon•6m ago
Youtube is victim of its success

I don't promote piracy, but it seems that it's easier to download music from youtube than using torrents, which is quite surprising.

Who expected that such a big company would contribute to piracy?