RISC-V Tensor unit will “supercharge AI” says Semidynamics
Based on Semidynamics’ 64bit customisable RISC-V cores, the RISC-V Tensor Unit is designed for fast AI solutions, said the company.
It can address the need for computation power in applications such as the LLaMa-2 or ChatGPT machine learning models. These consist of billions of parameters and require a large degree of computation power – in the order of several trillions of operations per second. There is also the requirement to keep energy consumption low which poses a significant challenge for hardware design, said Semidynamics.
The Tensor Unit provides “unprecedented computation power” for AI applications, continued the company. The bulk of computations in large language models (LLMs) is in fully-connected layers that can be efficiently implemented as matrix multiplication. The Tensor Unit provides hardware specifically tailored to matrix multiplication workloads, resulting in a huge performance boost for AI, explained Semidynamics.
The Tensor Unit is built on top of the Semidynamics’ RVV1.0 vector processing unit and leverages the existing vector registers to store matrices. This enables the Tensor Unit to be used for layers that require matrix multiply capabilities, such as fully connected and convolution, and use the vector unit for the activation function layers (e.g., ReLU, Sigmoid, Softmax), Standalone NPUs can struggle dealing with activation layers, added Semidynamics.
The Tensor Unit leverages both the vector unit capabilities as well as the Atrevido-423 Gazzillion capabilities to fetch the data it needs from memory to keep up the data rate. Other solutions rely on difficult-to-program DMAs, said Semidynamics, adding that it seamlessly integrates the Tensor Unit into its cache-coherent subsystem for simple programming of AI software.
The Tensor Unit uses the vector registers to store its data and does not include new, architecturally-visible state, enabling it to seamlessly work under any RISC-V vector-enabled Linux without any changes.
Further details on the Tensor Unit will be disclosed at the RISC-V North America Summit in Santa Clara, California, USA on November 7th 2023.
