frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

What Is Ruliology?

https://writings.stephenwolfram.com/2026/01/what-is-ruliology/
1•soheilpro•42s ago•0 comments

Jon Stewart – One of My Favorite People – What Now? With Trevor Noah Podcast [video]

https://www.youtube.com/watch?v=44uC12g9ZVk
1•consumer451•3m ago•0 comments

P2P crypto exchange development company

1•sonniya•16m ago•0 comments

Vocal Guide – belt sing without killing yourself

https://jesperordrup.github.io/vocal-guide/
1•jesperordrup•21m ago•0 comments

Write for Your Readers Even If They Are Agents

https://commonsware.com/blog/2026/02/06/write-for-your-readers-even-if-they-are-agents.html
1•ingve•21m ago•0 comments

Knowledge-Creating LLMs

https://tecunningham.github.io/posts/2026-01-29-knowledge-creating-llms.html
1•salkahfi•22m ago•0 comments

Maple Mono: Smooth your coding flow

https://font.subf.dev/en/
1•signa11•29m ago•0 comments

Sid Meier's System for Real-Time Music Composition and Synthesis

https://patents.google.com/patent/US5496962A/en
1•GaryBluto•36m ago•1 comments

Show HN: Slop News – HN front page now, but it's all slop

https://dosaygo-studio.github.io/hn-front-page-2035/slop-news
4•keepamovin•37m ago•1 comments

Show HN: Empusa – Visual debugger to catch and resume AI agent retry loops

https://github.com/justin55afdfdsf5ds45f4ds5f45ds4/EmpusaAI
1•justinlord•40m ago•0 comments

Show HN: Bitcoin wallet on NXP SE050 secure element, Tor-only open source

https://github.com/0xdeadbeefnetwork/sigil-web
2•sickthecat•42m ago•1 comments

White House Explores Opening Antitrust Probe on Homebuilders

https://www.bloomberg.com/news/articles/2026-02-06/white-house-explores-opening-antitrust-probe-i...
1•petethomas•42m ago•0 comments

Show HN: MindDraft – AI task app with smart actions and auto expense tracking

https://minddraft.ai
2•imthepk•47m ago•0 comments

How do you estimate AI app development costs accurately?

1•insights123•48m ago•0 comments

Going Through Snowden Documents, Part 5

https://libroot.org/posts/going-through-snowden-documents-part-5/
1•goto1•49m ago•0 comments

Show HN: MCP Server for TradeStation

https://github.com/theelderwand/tradestation-mcp
1•theelderwand•52m ago•0 comments

Canada unveils auto industry plan in latest pivot away from US

https://www.bbc.com/news/articles/cvgd2j80klmo
3•breve•53m ago•1 comments

The essential Reinhold Niebuhr: selected essays and addresses

https://archive.org/details/essentialreinhol0000nieb
1•baxtr•55m ago•0 comments

Rentahuman.ai Turns Humans into On-Demand Labor for AI Agents

https://www.forbes.com/sites/ronschmelzer/2026/02/05/when-ai-agents-start-hiring-humans-rentahuma...
1•tempodox•57m ago•0 comments

StovexGlobal – Compliance Gaps to Note

1•ReviewShield•1h ago•1 comments

Show HN: Afelyon – Turns Jira tickets into production-ready PRs (multi-repo)

https://afelyon.com/
1•AbduNebu•1h ago•0 comments

Trump says America should move on from Epstein – it may not be that easy

https://www.bbc.com/news/articles/cy4gj71z0m0o
7•tempodox•1h ago•4 comments

Tiny Clippy – A native Office Assistant built in Rust and egui

https://github.com/salva-imm/tiny-clippy
1•salvadorda656•1h ago•0 comments

LegalArgumentException: From Courtrooms to Clojure – Sen [video]

https://www.youtube.com/watch?v=cmMQbsOTX-o
1•adityaathalye•1h ago•0 comments

US moves to deport 5-year-old detained in Minnesota

https://www.reuters.com/legal/government/us-moves-deport-5-year-old-detained-minnesota-2026-02-06/
9•petethomas•1h ago•3 comments

If you lose your passport in Austria, head for McDonald's Golden Arches

https://www.cbsnews.com/news/us-embassy-mcdonalds-restaurants-austria-hotline-americans-consular-...
1•thunderbong•1h ago•0 comments

Show HN: Mermaid Formatter – CLI and library to auto-format Mermaid diagrams

https://github.com/chenyanchen/mermaid-formatter
1•astm•1h ago•0 comments

RFCs vs. READMEs: The Evolution of Protocols

https://h3manth.com/scribe/rfcs-vs-readmes/
3•init0•1h ago•1 comments

Kanchipuram Saris and Thinking Machines

https://altermag.com/articles/kanchipuram-saris-and-thinking-machines
1•trojanalert•1h ago•0 comments

Chinese chemical supplier causes global baby formula recall

https://www.reuters.com/business/healthcare-pharmaceuticals/nestle-widens-french-infant-formula-r...
2•fkdk•1h ago•0 comments
Open in hackernews

Show HN: E-commerce data from 100k stores that is refreshed daily

https://www.searchagora.com/data-connector
15•astronautmonkey•5mo ago
Hi HN! I'm building Agora, an AI search engine for e-commerce that returns results in under 300ms. We've indexed 30M products from 100k stores and made them easy to purchase using AI agents.

After launching here on HN, a large enterprise reached out to pay for access to the raw data. We serviced the contract manually to learn the exact workflow and then decided to productize the "Data Connector" to help us scale to more customers.

The Data Connector enables developers to select any of our 100k stores in the index, view sample data, format the output, and export the up-to-date data. Data can be exported as CSV or JSON.

We've built crawlers for Shopify, WooCommerce, Squarespace, Wix, and custom built stores to index the store information, product data, stock, reviews, and more. The primary technical challenge is to recrawl the entire dataset every 24 hours. We do this with a series of servers that "recrawl" different store-types with rotating local proxies and then add changes to a queue to be updated in our search index. Our primary database is Mongo and our search runs on self-hosted Meilisearch on high RAM servers.

My vision is to index the world's e-commerce data. I believe this will create market efficiencies for customers, developers, and merchants.

I'd love your feedback!

Comments

amcunicorns•5mo ago
Nice idea! Sounds like a lot of servers are needed to pull this off.
astronautmonkey•5mo ago
Thank you! And yes, the number of servers needed to scale from 100k to 1M stores (the next goal) will be significant.
eastbayjake•5mo ago
Couple thoughts for you:

(1) What are the use cases you envision? I can see the value for a really large marketplace in having a ton of pricing data, or the value to a hedge fund etc in having raw data to analyze macro trends... what is the use case for someone paying $200/month for the developer tier? (If I'm a retailer myself I probably only need data on my direct competitors, unless there's something cool you're imagining that I've failed to see.)

(2) You've got some logos on the store splash that don't show up in store search (eg Nike). Is that a data error or a coding error?

(3) You should probably think about how you scrape and categorize marketplace data... the Walmart tab has a lot of products that are clearly third-party sellers transacting via walmart.com, which pollutes quite a bit of the data value if I primarily want to know what a big retailer is doing on products where they actually set the prices.

(4) Have you looked at grocery data? Have wished someone would build a grocery prices API for like a decade now... lots of cool consumer and hedge-fund monetization opportunities if you can show the price of strawberries in every store across the US (and graph the trendlines over time).

astronautmonkey•5mo ago
Thanks for checking it out!

1. Here are the use-cases we've seen so far: marketplaces, search apps, fashion try-on apps, shopping agents, general purpose agents, web search for LLMs, e-commerce aggregators, hedge funds, etc. The most surprising has been new discovery experiences. Here's an example of an app that uses our data: https://www.forbes.com/sites/charliefink/2025/06/04/glance-a...

2. Great catch. We need to make this more clear on the site but we provide ~100k stores out of the box but keep the bigger brands behind an Enterprise paywall. We're working on fixing this.

3. Absolutely. We have purposely separated out search on the home page between our core index vs searching on Amazon, Walmart, etc. from within Agora. We haven't indexed products from the major marketplaces yet because of this challenge. Generally, we also focus on direct sellers and have filters in place with our crawler to parse out resellers.

4. Haven't looked at this but sounds interesting. And similar to how we think about storing e-commerce data with price history over time.

I'd love to chat more. I'm at param [at] searchagora.com if you want to reach out.