"Google Edge TPU devices", 100 comments (2019), https://news.ycombinator.com/item?id=19130896 & https://news.ycombinator.com/item?id=19313813
"Coral Edge TPU review", 100 comments (2020), https://news.ycombinator.com/item?id=24808755
"TPU transformation: 10 years of our AI-specialized chips", 60 comments (2024), https://news.ycombinator.com/item?id=41148532
TPU v4 (2023), https://arxiv.org/abs/2304.01433
this seems hopelessly out of date/confused
This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory.
The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, ...) that help average throughput more than guaranteed latency.
The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. TPUs were purpose-built specifically for AI. TPUs are an application-specific integrated circuit (ASIC), a chip designed for a single, specific purpose: running the unique matrix and vector-based mathematics that’s needed for building and running AI models..
TPU v2.. built an interconnected machine — our first TPU pod — with 256 TPU chips connected with a very high-bandwidth, custom interconnect.. liquid cooling was added with TPU v3 to help address efficiency needs, while TPU v4 introduced optical circuit switches to allow the chips in pods to communicate even faster and more reliably.
TPUs also underpin Google DeepMind’s cutting-edge foundation models, including the newly unveiled Gemini 1.5 Flash, Imagen 3, and Gemma 2, propelling advancements in AI.. Forget about a single chip, or a single TPU pod — we’re building a global network of data centers filled with TPUs. TPUv3 TPUv4
HBM2 BW 900 GB/s 1200 GB/sInference is easier, since the person deploying a model knows the architecture ahead of time and therefore can write custom code for their particular model.
When training you want to be as flexible as possible. The framework and hardware should not impose any particular architecture. This means lots of kernels and combinations of kernels. Miss one and you're out.
well these days since everything is transformer, your pool of choices is less daunting and theres only about four or five places that someone might get clever.
From https://news.ycombinator.com/item?id=42314333 :
>> From "A carbon-nanotube-based tensor processing unit" (2024) https://www.nature.com/articles/s41928-024-01211-2 :
>>> Using system-level simulations, we estimate that an 8 bit TPU made with nanotube transistors at a 180 nm technology node could reach a main frequency of 850 MHz and an energy efficiency of 1 tera-operations per second per watt.
Can QPUs (Quantum Processing Units) built on with electrons in superconducting graphene ever be faster than photons in integrated nanophotonics?
There are integrated parametric single-photon emitters and detectors.
Is there a lower cost integrated nanophotonic coherent light source for [quantum] computing than a thin metal wire?
"Electrons turn piece of wire into laser-like light source" (2022) https://news.ycombinator.com/item?id=33493885
mdaniel•8mo ago
The /forks contained https://github.com/csirlin/OpenTGPTPU which had a commit 3 hours ago but it seems they have not yet updated the FAQ for their version. Anyway, the fact it has commits greater than 8 years ago makes it seem like a more reasonable submission