Ask HN: Hardware for 1k RPS?

5•gsky•1d ago

I ran an uncensored model on a CPU server. as expected its dead slow (min or two per query).

What kinda hardware (GPU) do i need to serve 1k RPS?

I could not find APIs for uncensored models that kinda forced me to run locally

Comments

barnabee•1d ago

https://venice.ai claim to offer uncensored models (I’ve not tested that claim)

gsky•1d ago

Thanks, I give it a try.

eddythompson80•1d ago

Depends on your model size and how many of it can fit in memory. Multiply the size by 1k and divide by the memory capacity of the hardware for a rough ballpark.

Ask HN: Anyone making a living from a paid API?

Ask HN: What is the best LLM for consumer grade hardware?

Ask HN: What's the best AI for meme creation?

Ask HN: Why reinvent front-end frameworks and static site builders?

Ask HN: What are you working on? (May 2025)

Ask HN: GEtting a Remote Job in the US

Ask HN: How do I learn practical electronic repair?

What motivates you to contribute to open source projects?

Ask HN: Hardware for 1k RPS?

Volume label field can't be longer than "VolumeLabel" when formatting in Windows

Ask HN: How do you improve code for future AI?

How to improve interview skill?

Ask HN: Lisp eval vs. Lisp macros. Are they the same underlying concept?

Tell HN: Namecheap pre-purchasing searched domain names?

Ask HN: Anyone struggling to get value out of coding LLMs?

Ask HN: How do you set up a new dev machine? (2025 edition)

Ask HN: Why doesn't Google offer POS credit card reader like Square?

Ask HN: How are people using ChatGPT to increase productivity in personal life?

Tell HN: eBay doesn't allow changing country

Ask HN: Career Plateau: Looking for Advice on How to Break Through

Tasks Per Day – A minimalist productivity app that works

Ask HN: New Economics of Software Development Lifecycle

Ask HN: Management wants to talk to my Datalake. What's the best way to do this?

Can we take a moment to appreciate what kind of web experience we are building?

Ask HN: How do you stay motivated when hunting for a job?

Ask HN: Why that many more US-based companies are hiring "US-only" remote?

Ask HN: What tools do you use to discover competitors?

Self-Hosted Cloudflare Alternatives

Ask HN: Building LLM apps? How are you handling user context?

Best Buy is selling a $400 "digital Ethernet" cable for "cleaner, clearer sound"

Ask HN: Hardware for 1k RPS?

Comments

Ask HN: Anyone making a living from a paid API?

Ask HN: What is the best LLM for consumer grade hardware?

Ask HN: What's the best AI for meme creation?

Ask HN: Why reinvent front-end frameworks and static site builders?

Ask HN: What are you working on? (May 2025)

Ask HN: GEtting a Remote Job in the US

Ask HN: How do I learn practical electronic repair?

What motivates you to contribute to open source projects?

Ask HN: Hardware for 1k RPS?

Volume label field can't be longer than "VolumeLabel" when formatting in Windows

Ask HN: How do you improve code for future AI?

How to improve interview skill?

Ask HN: Lisp eval vs. Lisp macros. Are they the same underlying concept?

Tell HN: Namecheap pre-purchasing searched domain names?

Ask HN: Anyone struggling to get value out of coding LLMs?

Ask HN: How do you set up a new dev machine? (2025 edition)

Ask HN: Why doesn't Google offer POS credit card reader like Square?

Ask HN: How are people using ChatGPT to increase productivity in personal life?

Tell HN: eBay doesn't allow changing country

Ask HN: Career Plateau: Looking for Advice on How to Break Through

Tasks Per Day – A minimalist productivity app that works

Ask HN: New Economics of Software Development Lifecycle

Ask HN: Management wants to talk to my Datalake. What's the best way to do this?

Can we take a moment to appreciate what kind of web experience we are building?

Ask HN: How do you stay motivated when hunting for a job?

Ask HN: Why that many more US-based companies are hiring "US-only" remote?

Ask HN: What tools do you use to discover competitors?

Self-Hosted Cloudflare Alternatives

Ask HN: Building LLM apps? How are you handling user context?

Best Buy is selling a $400 "digital Ethernet" cable for "cleaner, clearer sound"