news newest ask show jobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Ask HN: How does one scrape a website?

1•terry_hc•2h ago

I wish to scrape and preserve the blog of a dead author, before their domain and writings expire. What would be the most accurate wget invocation for obtaining all their articles, all images, et cetera, that reside under the website's domain, such that the whole site (bar external content) can be browsed locally?

Comments

reliefcrew•2h ago

Just ask AI how to mirror w/ wget. But, beware that if the site relies on javascript, wget may not be enough. In that case you'll need to program some kind of headless browsing. Didn't the internet archive (archive.org) take care of everything for you already though?

Clicks Communicator

https://www.clicksphone.com/en/communicator

1•microflash•1m ago•0 comments

Usury FAQ (2014)

https://zippycatholic.wordpress.com/2014/11/10/usury-faq-or-money-on-the-pill/

1•danielam•2m ago•0 comments

The rise of AI tools that write about you when you die

https://www.washingtonpost.com/technology/2025/08/03/ai-obituaries-funeral-homes/

1•_tk_•3m ago•0 comments

A post-American, enshittification-resistant internet

https://pluralistic.net/2026/01/01/39c3/

3•weliveagain•3m ago•0 comments

Ukrainian soldier's M1 MacBook Air takes direct shrapnel and still works

https://www.tomshardware.com/laptops/macbooks/ukrainian-soldiers-m1-macbook-air-takes-direct-shra...

1•speckx•3m ago•0 comments

Show HN: I made presets for Darktable which mimic Fujifilm's Film Simulations

https://jssfr.de/dtsolve/2026-01-02-darktable-styles-fujifilm.html

2•jssfr•3m ago•0 comments

Taming the internet: From horizontal to vertical, from keyboards to screens

https://www.northsouthnotes.org/p/taming-the-internet

1•pje•5m ago•0 comments

Online Planning Method Integrating LLMs into Nested Rollout Policy Adaptation

https://arxiv.org/abs/2511.21706

1•PaulHoule•6m ago•0 comments

AI can feel you now?

https://synheart.ai

1•henok_ademtew•6m ago•0 comments

Show HN: 100% free data extraction tool for invoices

https://www.digiparser.com/free-tools/data-extraction/invoice-parser

1•thepantales•6m ago•0 comments

The Relational Substrate of Reflective Consciousness

https://zenodo.org/records/18037522

1•llamataboot•9m ago•0 comments

Show HN: Arctic – a terminal-first TUI aggregating multiple AI coding plans

https://github.com/arctic-cli/interface

1•femtobusa•9m ago•0 comments

Clicks Communicator smartphone

https://clicksphone.com/en/communicator

1•ChrisArchitect•13m ago•0 comments

Kotlin 2.3.0

https://blog.jetbrains.com/kotlin/2025/12/kotlin-2-3-0-released/

1•andrewstetsenko•14m ago•0 comments

Show HN: Previewcn – a Shadcn/UI theme previewer directly inside your NextJS app

https://github.com/taishikato/previewcn

2•taishikato•14m ago•0 comments

Tesla's fourth quarter sales fell more than expected, 15.6 percent drop

https://www.theverge.com/news/852649/tesla-q4-2025-sales-production-delivery-elon-musk

1•randycupertino•14m ago•1 comments

The Capybaras of Florida

https://storymaps.arcgis.com/stories/a029f491389241fd87ab7396862447e8

1•ohjeez•15m ago•0 comments

If You Meet ET in Space, Kill Him (2024)

https://nautil.us/if-you-meet-et-in-space-kill-him-917243/

1•amarble•17m ago•0 comments

Florida county introduces self driving patrol cars

https://www.cbsnews.com/video/florida-police-department-tests-nations-first-self-driving-patrol-car/

5•iamronaldo•17m ago•3 comments

Clicks Communicator, a Second Phone Built for Communication, Not Consumption

https://financialpost.com/globe-newswire/clicks-introduces-communicator-a-second-phone-built-for-...

1•ksec•18m ago•0 comments

Show HN: Software to calculate NAV of investment funds

https://navquant.com/

2•navquant•18m ago•0 comments

The Kabul Conversations That Could Rewire South Asia's Geopolitical Map

https://ajmals.substack.com/p/back-through-kabuls-gates-khalilzad

1•Gym-Berlin•19m ago•0 comments

Show HN: The Modern AI SEO Tech Stack: 7 Tools You Need to Rank in ChatGPT(2026)

https://www.genrankengine.com/blog/7-tools-you-need-to-rank-in-AI-searches/

1•arunkumars91•19m ago•0 comments

I wanted a camera that doesn't exist – so I built it

https://medium.com/@cristi.baluta/i-wanted-a-camera-that-doesnt-exist-so-i-built-it-5f9864533eb7

1•cyrc•19m ago•0 comments

The Analog Manifesto – For digital people who crave the analog era

https://theanalogmanifesto.com/

2•heshiebee•23m ago•0 comments

Do you know why your users are churning?

1•j_mao•24m ago•0 comments

Using LLMs to compare 500 Pages of Macro Research (with citations)

https://2026macro.vercel.app/about.html

1•OxfordOutlander•24m ago•0 comments

Show HN: CryDecoder – On-device ML for classifying baby cries (Swift, Core ML)

https://apps.apple.com/us/app/crydecoder-baby-translator/id6756557492

3•evanjusttrying•26m ago•0 comments

The rsync algorithm (1996) [pdf]

https://www.andrew.cmu.edu/course/15-749/READINGS/required/cas/tridgell96.pdf

2•vortex_ape•27m ago•0 comments

Memos – An open-source, self-hosted note-taking service

https://github.com/usememos/memos

1•the-mitr•27m ago•0 comments