frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

AI search engine – How to prevent bots?

3•chaztaubelman•1h ago
Hi, I'm launching an AI search engine (ex Perplexity like). I don't want to force people to sign up to use it. I want free visitors be able to discover it and use it. However, I've had issues in the past with bots spamming usage, which exploded my costs.

What are the best methdos to prevent those bots, while also having a frictionless UX ? I've heard of Cloudflare. Will that pop for every user or only for those who are trully suspicious ?

Thanks

Comments

reliefcrew•1h ago
Depends on the "widget mode" you choose:

https://developers.cloudflare.com/turnstile/concepts/widget/...

timshell•1h ago
Check out a demo of a similar tool we created (https://model-guessr.com/) that was bot-gated by Roundtable Proof of Human.

Happy to talk more details about PoH (disclaimer: I'm a cofounder and this is my YC S23 company)

reliefcrew•1h ago
Can you comment on the notion that Turnstile's primary goal isn't to keep bots out 100% but instead to slow them down to "human" speeds.

Asking because as a dev I hate when sites don't allow bots... however can appreciate that automation should be rate-limited. IOW, isn't preventing bot access actually an anti-pattern since rate-limiting is sufficient?

I see a lot of marketing which bashes Turnstile [detection] rates and tries to leverage this misunderstood nuance. And, it seems to be a dishonest point of contention but am willing to hear opposing arguments.

Thanks.

timshell•52m ago
Yup! It depends on your use case.

Cloudflare is really good at network bot detection. Rate-limiting is super helpful here, for example during DDoS attacks.

Our customers are a little different. They sometimes struggle with high-volume bot attacks (e.g. SMS toll fraud in ticketing marketplaces), but we specifically focus on online platforms that want to verify a human is on the other side of the screen. For example, survey pollsters and labor marketplaces want to stop a slow agent that can complete traditional CAPTCHA even if it's solving it a human speed

reliefcrew•36m ago
I see. I'll have to read the marketing more closely next time, lol. The cynic in me only notices the detection rate comparisons, which I'm sure the marketing folks don't mind much ;-)
n1xis10t•32m ago
Another option to consider (which marginalia-search.com uses) is Anubis (anubis.techaro.lol). The operator of Marginalia told me that he was getting lots of people spamming the same queries over and over, which he thought might be them trying to influence suggested searches. He put Anubis in place and the query volume dropped to much more reasonable levels. It works by running some sort of complex calculation in javascript, so it won’t get rid of all bots, but it should slow them all down.

The downside is that their silly anime girl mascot is displayed whenever the challenge is running, which I think some people might find off-putting.

Edit: Are you going to announce the search engine on hacker news?

2nd edit: If you are making a search engine, this is probably a good article to read: https://archive.org/details/search-timeline It talks about various search engines that have disappeared mysteriously over the years.

Reinforcing Private-Public Investments

https://parthchopra.substack.com/p/on-reinforcing-private-public-investments
1•probe•53s ago•0 comments

Abusing x86 instructions to optimize PS3 emulation [RPCS3] [video]

https://www.youtube.com/watch?v=40tyEVx_umY
2•davikr•3m ago•0 comments

The Oscars Moving to YouTube Beginning in 2029, Will Stream Free Worldwide

https://variety.com/2025/film/news/oscars-youtube-2029-1236610989/
1•Risse•3m ago•0 comments

Exclusive-How China built its 'Manhattan Project' to rival the West in AI chips

https://finance.yahoo.com/news/exclusive-china-built-manhattan-project-141758929.html
2•WheelsAtLarge•4m ago•0 comments

DB migration tool – For those of us who don't use SQLAlchemy

https://github.com/rodmena-limited/migretti
1•rodmena•4m ago•0 comments

Open source platform for BYOC deployments

https://github.com/nuonco/nuon
1•MorehouseJ09•4m ago•0 comments

Evaluating AI's ability to perform scientific research tasks

https://openai.com/index/frontierscience/
1•Anon84•4m ago•0 comments

Crash clock says satellites in orbit are three days from disaster

https://www.newscientist.com/article/2508752-crash-clock-says-satellites-in-orbit-are-three-days-...
2•Breadmaker•5m ago•0 comments

Yet antoher RAG – for code generation with impressive correctness

https://github.com/rodmena-limited/ragit
1•rodmena•5m ago•0 comments

The quick and dirty genius of Luhn algorithm

https://evgeniipendragon.com/posts/the-quick-and-dirty-genius-of-luhn-algorithm/
1•EPendragon•6m ago•0 comments

Titan Mining Commences Graphite Processing at Empire State Mines in New York

https://www.titanminingcorp.com/news/news-releases/titan-mining-commences-graphite-processing-at-...
1•kotaKat•6m ago•0 comments

Log Structured Merge Trees

http://www.benstopford.com/2015/02/14/log-structured-merge-trees/
2•whatisabcdefgh•6m ago•0 comments

Meta pauses third-party headset program

https://www.roadtovr.com/meta-horizon-os-third-party-headset-cancelled-asus-lenovo/
1•dagmx•7m ago•0 comments

Rust in ClickHouse

https://clickhouse.com/blog/alexey-p99-2025-rust-in-clickhouse
1•Abbit•8m ago•0 comments

MiniMax Agent

https://agent.minimax.io
2•SpyCoder77•9m ago•0 comments

A Roadmap for Federal AI Legislation

https://a16z.com/a-roadmap-for-federal-ai-legislation-protect-people-empower-builders-win-the-fut...
1•kjhughes•9m ago•0 comments

Show HN: Modeling the US Debt as a Healthcare Pricing Failure ($26T Gap)

https://taprootlogic.substack.com/p/the-us-debt-crisis-a-52-trillion
1•kmundy•10m ago•0 comments

Show HN: Bob the Fixer – SonarQube and MCP tools for a fix→test→re-scan loop

https://github.com/andrearaponi/bob-the-fixer
1•andrearaponi12•10m ago•0 comments

Make Me CEO of Mozilla

https://blog.kingcons.io/posts/make-me-ceo-of-mozilla.html
4•phyzome•11m ago•0 comments

Implicit Position-Based Fluids (IPBF)

https://graphics.cs.utah.edu/research/projects/ipbf/
3•ibobev•11m ago•0 comments

Sample Space Partitioning and Spatiotemporal Resampling for Specular Manifolds

https://graphics.cs.utah.edu/research/projects/psms-restir/
2•ibobev•11m ago•0 comments

The Politics of Superintelligence

https://www.noemamag.com/the-politics-of-superintelligence/
1•buellerbueller•12m ago•0 comments

Device Logs Anywhere

https://blog.golioth.io/device-logs-anywhere-with-golioth-pipelines/
1•hasheddan•12m ago•0 comments

A2UI: An open project for agent-driven interfaces

https://developers.googleblog.com/introducing-a2ui-an-open-project-for-agent-driven-interfaces/
1•jarmitage•13m ago•1 comments

Containrrr/watchtower is now archived

https://github.com/containrrr/watchtower/discussions/2135
1•wooben•14m ago•0 comments

Anchored Diffusion Language Model

https://anchored-diffusion-llm.github.io
1•nathan-barry•15m ago•0 comments

Project Everest

https://project-everest.github.io/
1•whatisabcdefgh•15m ago•0 comments

Show HN: Voice-to-text for macOS using Groq's free Whisper API

https://github.com/bokan/stt
1•bbokan•17m ago•0 comments

AI, AI Oh

https://thinkhuman.com/aiaioh/
2•jamesgill•20m ago•0 comments

Finland is in midst of racist firestorm

https://www.bbc.com/news/articles/cde657xj3pxo
5•crazybonkersai•21m ago•4 comments