Google’s eighth-generation TPU now has four models in total.
At the Google Cloud Next conference on April 22 this year, Google launched two models: TPU 8t, which focuses on training, and TPU 8i, which focuses on reasoning. Two more variants were added on Monday local time.
TPU 8tx is derived from 8t and is aimed at confidential computing, which is a scenario where data remains encrypted during the calculation process and cannot even be seen by the cloud vendors themselves. Financial, medical, and government customers care most about this.
TPU 8i Max is derived from 8i and specifically targeted at long-context inference. Million-token-level context is already a standard feature of each flagship model. The memory and bandwidth pressure during inference is very high. The release of a separate chip shows that the demand for this is large enough.
Rhythm is the biggest highlight. Google’s chip department has released a generation of TPUs in the past two years or so, and this generation has been divided into four models in more than half a year. Nvidia's GPU is a general-purpose card, and one card can dominate the world; Google is increasingly slicing it according to workload, and making one for each scenario.
The premise of this style of play is that the software stack is smooth enough. TorchTPU was open sourced a while ago, and PyTorch code can run on TPU without basically changing it, which lowers the migration threshold first. No matter how finely divided the hardware is, it will be in vain if developers are unwilling to move it.
Google did not provide full specifications and launch time this time.