DEEPX CO., LTD.

A neural processing apparatus is disclosed. The apparatus includes a memory array including a plurality of physically distinct memory banks, configurable interconnect circuitry coupled to the plurality of memory banks, and a controller. The controller is configured to dynamically reallocate the plurality of memory banks into variable-sized memory regions corresponding to different data types required for an inference operation of a neural network model. The controller adjusts a number of memory banks allocated to each data type for a current layer of the neural network model based on configuration information derived from a structure of the neural network model. The controller is further configured to enable data access between the reallocated memory banks and a processing engine via the configurable interconnect circuitry.

A neural processing unit for performing inference operations of a large-scale language model based on an artificial neural network is disclosed. The neural processing unit according to the present disclosure includes a processing element core configured to perform an attention mechanism-based operation based on input data in vector format to output an operation result, a special function unit comprising a plurality of arithmetic circuits including at least one vector-dedicated arithmetic circuit that exclusively performs vector operations and at least one mixed arithmetic circuit capable of performing both vector and scalar operations, and configured to perform a special function operation on the operation result, and a controller configured to, upon receiving an RMS normalization operation execution command, activate at least one of the plurality of arithmetic circuits to control the special function unit to perform an operation of converting at least one of the operation result or the input data into a normalized vector whose magnitude is adjusted based on a root mean square (RMS), wherein the operation result may include an attention score for the input data.

A neural processing unit (NPU), a method for driving an artificial neural network (ANN) model, and an ANN driving apparatus are provided. The NPU includes a semiconductor circuit that includes at least one processing element (PE) configured to process an operation of an artificial neural network (ANN) model; and at least one memory configurable to store a first kernel and a first kernel filter. The NPU is configured to generate a first modulation kernel based on the first kernel and the first kernel filter and to generate second modulation kernel based on the first kernel and a second kernel filter generated by applying a mathematical function to the first kernel filter. Power consumption and memory read time are both reduced by decreasing the data size of a kernel read from a separate memory to an artificial neural network processor and/or by decreasing the number of memory read requests.

Provided is a method for performing an aging test on a neural processing unit (NPU) with a capability of a runtime test. The method may comprise: performing an aging test on the NPU which comprises a plurality of functional components. The plurality of functional components may comprise at least one memory and plural processing elements. The performing of the aging test may include: performing a scan test on the NPU to verify whether at least one functional component in the NPU is defective or not; and performing a memory test on the at least one memory. At least one of the scan test and the memory test may be repeatedly performed to put a stress on the NPU for the aging test. The aging test may be repeated by a predetermined number.

A scalable AI system includes a plurality of slots on a main board, at least one AI module mounted in the slots to perform AI operations, and a main processor that controls the entire system. A user can flexibly scale the overall AI computation performance of the system by adjusting the number of AI modules as needed. Each AI module is equipped with a low-power NPU specialized for AI operations and adopts standard form factors (e.g., M.2, E1.S) to improve economy and scalability. This can solve the problems of high-power consumption and cost, which are limitations of GPU-based systems, and can efficiently support various AI applications.

A mobile electronic device may include a main memory for storing weight values of a compiled and quantized large language model, a processing core for performing an inference operation based on an attention mechanism on text based on the large language model, a controller for controlling the operation of the processing core and the main memory and for scheduling the inference operation, and a special function unit for performing vector operations and scalar operations necessary for the inference operation. Quantization and dequantization are performed in the inference operation of a large language model, enabling on-device inference operations with minimal performance degradation.

According to one example of the present disclosure, a system may be provided. The system may comprise at least one processing core configured to perform computation operations of at least one neural network model associated with tensors, at least one memory circuit configured to store the tensors, a plurality of bus circuits operably coupled to the at least one processing core and the at least one memory circuit. The plurality of bus circuits configured to send the tensors from the at least one memory circuit to the at least one processing core responsive to receiving requests for read operations or write operations, and a controller operably coupled to the plurality of bus circuits, the controller configured to determine a priority of each of the tensors for each bus circuit for the read operations or the write operations.

A neural processing unit (NPU) for decoding video or feature map is provided. The NPU may comprise at least one processing element (PE) to perform an inference using an artificial neural network. The at least one PE may be configured to receive and decode data included in a bitstream. The data included in the bitstream may comprise data of a base layer. Alternatively, the data included in the bitstream may comprise data of the base layer and data of at least one enhancement layer. The data of the base layer included in the bitstream may include a first feature map. The data of the at least one enhancement layer included in the bitstream may include a second feature map.

An image processing method is disclosed. The method includes receiving an input image including at least one object, and classifying the at least one object in the input image using a first model based on an artificial neural network trained to classify objects into one of a plurality of predetermined categories. At least one second model corresponding to the classified category of the at least one object is determined from among a plurality of second models, each of which is based on an artificial neural network trained to output a specialized processing applied image specific to a respective category. An output image is obtained by inputting the input image, or a region thereof corresponding to the at least one object, into the determined at least one second model.

A neural processing unit (NPU) is capable of testing a component of the NPU in a running system, i.e., during runtime. The NPU includes a plurality of functional components, each of which includes an electronic circuit; at least one wrapper connected to at least one of the functional components; and an in-system component tester (ICT). The ICT performs a selection of one of the at least one functional component, in an idle state, as a component under test (CUT) and performs a test, via the at least one wrapper, of the selected functional component. The ICT may monitor states of the plurality of the functional components via the at least one wrapper, stop the test based on a detection of a collision due to an access to the selected functional component, and return a connection of the selected functional component to the at least one wrapper according to the stop.