frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Show HN: Env-shelf – Open-source desktop app to manage .env files

https://env-shelf.vercel.app/
1•ivanglpz•2m ago•0 comments

Show HN: Almostnode – Run Node.js, Next.js, and Express in the Browser

https://almostnode.dev/
1•PetrBrzyBrzek•3m ago•0 comments

Dell support (and hardware) is so bad, I almost sued them

https://blog.joshattic.us/posts/2026-02-07-dell-support-lawsuit
1•radeeyate•3m ago•0 comments

Project Pterodactyl: Incremental Architecture

https://www.jonmsterling.com/01K7/
1•matt_d•4m ago•0 comments

Styling: Search-Text and Other Highlight-Y Pseudo-Elements

https://css-tricks.com/how-to-style-the-new-search-text-and-other-highlight-pseudo-elements/
1•blenderob•6m ago•0 comments

Crypto firm accidentally sends $40B in Bitcoin to users

https://finance.yahoo.com/news/crypto-firm-accidentally-sends-40-055054321.html
1•CommonGuy•6m ago•0 comments

Magnetic fields can change carbon diffusion in steel

https://www.sciencedaily.com/releases/2026/01/260125083427.htm
1•fanf2•7m ago•0 comments

Fantasy football that celebrates great games

https://www.silvestar.codes/articles/ultigamemate/
1•blenderob•7m ago•0 comments

Show HN: Animalese

https://animalese.barcoloudly.com/
1•noreplica•7m ago•0 comments

StrongDM's AI team build serious software without even looking at the code

https://simonwillison.net/2026/Feb/7/software-factory/
1•simonw•8m ago•0 comments

John Haugeland on the failure of micro-worlds

https://blog.plover.com/tech/gpt/micro-worlds.html
1•blenderob•8m ago•0 comments

Show HN: Velocity - Free/Cheaper Linear Clone but with MCP for agents

https://velocity.quest
2•kevinelliott•9m ago•2 comments

Corning Invented a New Fiber-Optic Cable for AI and Landed a $6B Meta Deal [video]

https://www.youtube.com/watch?v=Y3KLbc5DlRs
1•ksec•10m ago•0 comments

Show HN: XAPIs.dev – Twitter API Alternative at 90% Lower Cost

https://xapis.dev
2•nmfccodes•11m ago•0 comments

Near-Instantly Aborting the Worst Pain Imaginable with Psychedelics

https://psychotechnology.substack.com/p/near-instantly-aborting-the-worst
2•eatitraw•17m ago•0 comments

Show HN: Nginx-defender – realtime abuse blocking for Nginx

https://github.com/Anipaleja/nginx-defender
2•anipaleja•17m ago•0 comments

The Super Sharp Blade

https://netzhansa.com/the-super-sharp-blade/
1•robin_reala•18m ago•0 comments

Smart Homes Are Terrible

https://www.theatlantic.com/ideas/2026/02/smart-homes-technology/685867/
1•tusslewake•20m ago•0 comments

What I haven't figured out

https://macwright.com/2026/01/29/what-i-havent-figured-out
1•stevekrouse•21m ago•0 comments

KPMG pressed its auditor to pass on AI cost savings

https://www.irishtimes.com/business/2026/02/06/kpmg-pressed-its-auditor-to-pass-on-ai-cost-savings/
1•cainxinth•21m ago•0 comments

Open-source Claude skill that optimizes Hinge profiles. Pretty well.

https://twitter.com/b1rdmania/status/2020155122181869666
3•birdmania•21m ago•1 comments

First Proof

https://arxiv.org/abs/2602.05192
5•samasblack•23m ago•2 comments

I squeezed a BERT sentiment analyzer into 1GB RAM on a $5 VPS

https://mohammedeabdelaziz.github.io/articles/trendscope-market-scanner
1•mohammede•24m ago•0 comments

Kagi Translate

https://translate.kagi.com
2•microflash•25m ago•0 comments

Building Interactive C/C++ workflows in Jupyter through Clang-REPL [video]

https://fosdem.org/2026/schedule/event/QX3RPH-building_interactive_cc_workflows_in_jupyter_throug...
1•stabbles•26m ago•0 comments

Tactical tornado is the new default

https://olano.dev/blog/tactical-tornado/
2•facundo_olano•28m ago•0 comments

Full-Circle Test-Driven Firmware Development with OpenClaw

https://blog.adafruit.com/2026/02/07/full-circle-test-driven-firmware-development-with-openclaw/
1•ptorrone•28m ago•0 comments

Automating Myself Out of My Job – Part 2

https://blog.dsa.club/automation-series/automating-myself-out-of-my-job-part-2/
1•funnyfoobar•28m ago•1 comments

Dependency Resolution Methods

https://nesbitt.io/2026/02/06/dependency-resolution-methods.html
1•zdw•29m ago•0 comments

Crypto firm apologises for sending Bitcoin users $40B by mistake

https://www.msn.com/en-ie/money/other/crypto-firm-apologises-for-sending-bitcoin-users-40-billion...
1•Someone•30m ago•0 comments
Open in hackernews

Show HN: CrawlerCheck – A tool to check if a site is blocking crawlers

https://crawlercheck.com/
2•bogozi•7mo ago
Hi HN,

I'm a long-time developer and SEO consultant. Over the years, I've seen clients suffer from a simple, costly mistake: accidentally blocking Googlebot or other important crawlers with a misplaced rule in robots.txt or a noindex tag.

Manually checking the robots.txt file, then the page's meta tags, then the X-Robots-Tag HTTP header is a tedious process. I wanted a tool that would do it all in one shot and give me a clear answer.

So, I built CrawlerCheck. You give it a URL, and it checks all three sources of crawler directives to tell you if a page is accessible.

The backend is written in Go, and the frontend is a lightweight Svelte app. The goal was to make it as fast and reliable as possible.

It's a brand new project, and I'd love to get some honest feedback from the HN community. Thanks for taking a look.

Comments

8organicbits•7mo ago
Looks pretty helpful, thanks for building this.

Minor suggestion. Consider sorting the checks by status, or adding a summary at the top. I needed to scroll to find if anything was blocked.

I don't know enough about the SEO space, but would a llms.txt check also help?

bogozi•7mo ago
Thanks so much for the great feedback!

That's a fantastic point about adding a summary. I agree, it would make the results much easier to scan, especially because I'm planning to add even more rules. I'll definitely work on adding that.

Good point on llms.txt too. I'm watching that standard evolve and plan to add it once it's more established. Appreciate you bringing it up!

bogozi•6mo ago
Hi. I have implemented the summary and indeed it looks and feels a lot quicker this way. I have also added scrolling functionalities so it's a bit easier to navigate as well. "but would a llms.txt check also help" - not sure about it as it is not standardized yet. Big players still need to adapt it or come up with a better solution. Will wait and see until then.

I hope you'll find the updates useful. Thanks for the great feedback.

bogozi•7mo ago
Just wanted to post a quick follow-up on this project for anyone who might see it.

After the initial launch, I spent some time hardening the core logic and polishing the UI. I've just pushed a significant update with several improvements that make the tool much more reliable.

Here’s what’s new:

Smarter robots.txt parsing: It now correctly handles group precedence (so the most specific rule wins), wildcards (*), end-anchors ($), and rules with query strings (?) to better match how Googlebot actually interprets these files.

Better error handling: I've polished the error messages on both the frontend and backend, so it's much clearer what's happening if a URL can't be fetched or a robots.txt file is malformed.

Mobile & accessibility fixes: Pushed a few improvements for the experience on smaller screens and for better overall accessibility.

Just wanted to log the progress here as I continue to work on the tool.

Cheers.

bogozi•7mo ago
In the meantime I have implemented a publicly available changelog

Here is what's new in v1.1.1

- Backend HTTP client now mimics real browsers more closely for improved compatibility with strict sites.

- Automatic cookie handling and support for HTTP/2 enabled.

- Referer header now transparently set to crawlercheck.com for all requests.

- Improved diagnostics: HTTP status code and response body are logged for non-2xx responses to help troubleshoot blocks.

- All previous features and compatibility with simpler sites are preserved.