DEEPX CO., LTD.

According to one example of the present disclosure, a post-processing unit may be provided. The post-processing unit may be implemented in register transfer level (RTL) code and designed to interface with a neural processing unit (NPU) configured for object detection computations of a neural network model. The post-processing unit may include a processing unit configured to filter a plurality of bounding boxes transmitted from the NPU and output only those that satisfy a particular condition and one or more input registers configured to store data output from the processing unit.

According to one example of the present disclosure, the neural processing unit may comprise a processing element array configured to perform operations of a neural network model and a post-processing unit configured to process data output from the processing element array. The post-processing unit includes a first computation circuit that extracts a subset of classes for each bounding box by comparing class scores of classes and a second computation circuit configured to extract one or more bounding boxes by comparing a class confidence score of each bounding box with a threshold confidence score.

A mobile artificial intelligence (AI) device includes a camera configured to capture a video of a product, and a neural processing unit (NPU) configured to perform inference with an AI recognition model to recognize product information from the captured video on the mobile AI device. A battery supplies power to the NPU. A transceiver is configured to transmit only the recognized product information to a remote server without transmitting the captured video, and to receive additional commercial information corresponding to the recognized product information. A display is configured to present the additional commercial information.

A mobile artificial intelligence (AI) device includes a camera configured to capture a video of a product at a first frame rate, and an AI recognition model configured to recognize product information from the captured video on the mobile AI device. A processor is configured to operate the AI recognition model at a second frame rate, which is selectively adjustable to manage power consumption. A display is configured to concurrently present the video at the first frame rate and the recognized product information at the second frame rate.

A heterogeneous processor includes a first processor and a second processor of a different type. The heterogeneous processor operates in either a low-power mode or a full-power mode. The first processor is configured to operate in the low-power mode, process sensing data from a sensor using a trained neural network model, and generate a wake-up signal when an output of the trained neural network model satisfies a predefined criterion. The wake-up signal is provided to the second processor during the low-power mode. The second processor remains in a powered-down state during the low-power mode and transitions to the full-power mode in response to the wake-up signal.

An electronic device mounted on a fixed or a movable apparatus is provided. The electronic device may comprise an image signal processor (ISP) for at least one camera; a neural processing unit (NPU), including a plurality of processing elements (PEs), configured to: process an operation of an artificial neural network model trained to detect or track at least one object, based on an input feature map generated from at least one image, which is acquired via the ISP from the at least one camera, and output an inference result; and a signal generator generating a signal applicable to the at least one camera or the ISP.

Embodiments relate to converting functions or function call instructions of a first neural network (NN) model to graph modules. Relationships between inputs and outputs of the of graphing modules are analyzed and a second NN model in the form of a directed acyclic graph (DAG) using the graph modules corresponding to the first NN model. Markers are added to the graph modules in the second NN model. Calibration data is generated by collecting input values and output values of each of the graph modules by using the markers. A scale value and an offset value applicable to the second NN model is determined. Based on the scale value and the offset value, a third NN model including weight parameters quantized in the form of integers are generated and training is performed to update the weight parameters of the third NN model.

A method may comprise: adding a plurality of markers to a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); generating calibration data by collecting input values and output values of each of the plurality of graph modules using the plurality of markers; determining, based on the calibration data, a scale value and an offset value applicable to the first NN model; generating, based on the scale value and the offset value, a second NN model including a weight parameter in integer format through quantization; obtaining first output values of the first NN model with respect to a first retraining data; and updating, based on the first output values of the first NN model, at least one weight parameter included in the second NN model by performing a quantization-aware retraining technique on the second NN model.

According to the present disclosure, a device is provided. The device includes a first memory of a first capacity configured to store a first generative neural network model comprising a first parameters, and a first neural processing unit configured to generate a response corresponding to an input query utilizing the first generative neural network model stored in the first memory, and wherein the first neural processing unit may be configured to store a first execution code of the first generative neural network model compiled to process speculative decoding.

A method may comprise: adding a plurality of markers to a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); generating calibration data by collecting input values and output values of each of the plurality of graph modules using the plurality of markers; determining, based on the calibration data, a scale value and an offset value applicable to the first NN model; generating, based on the scale value and the offset value, a second NN model including a weight parameter in integer format through quantization; and updating at least one parameter included in the second NN model by performing a quantization-aware retraining technique on the second NN model.