GRAPHCORE LIMITED

A computer-implemented method comprising: processing data in a neural network to compute a network tensor comprising a plurality of tensor elements represented in an initial numerical format; computing a histogram of tensor elements; selecting a target numerical format, the target numerical format having a lower precision than the initial numerical format; evaluating a metric based on the histogram of tensor elements and the target numerical format, the metric indicating a degree of accuracy of a representation of the network tensor in the target numerical format; and based on the evaluated metric, converting the plurality of tensor elements from the initial numerical format to the target numerical format.

A computer comprising a plurality of processing units, each processing unit having an execution unit and access to computer memory which stores code executable by the execution unit and input values of an input vector to be processed by the code, the code, when executed, configured to access the computer memory to obtain multiple pairs of input values of the input vector, determine a maximum or corrected maximum input value of each pair as a maximum result element, determine and store in a computer memory a maximum or corrected maximum result of each pair of maximum result elements as an approximation to the natural log of the sum of the exponents of the input values and access the computer memory to obtain each input value and apply it to the maximum or corrected maximum result to generate each output value of a Softmax output vector.

A hardware module is provided in an execution unit and is responsive to execution of multiple instances of a new type of instruction to perform a plurality of reductions in parallel. The hardware module comprises: a first accumulator storing first state associated with a first of the reductions; and a second accumulator storing second state associated with a second of the reductions. Upon execution of each of the multiple instances of the first type of instruction: an input value for the respective instance is provided to a first processing circuit of the hardware module such that the first processing circuit performs a first type of operation to update the first state; and the same input value is provided to the second processing circuit of the hardware module such that the second processing circuit performs a second type of operation to update the second state.

A processor comprising a barrel-threaded execution unit for executing concurrent threads, and one or more register files comprising a respective set of context registers for each concurrent thread. One of the one or more register files further comprises a set of shared weights registers common to some or all of the concurrent threads. The types of instructions defined in the instruction set of the processor include an arithmetic instruction having operands specifying a source and a destination from amongst a respective set of arithmetic registers of the thread in which the arithmetic instruction is executed. The execution unit is configured so as, in response to the opcode of the arithmetic instruction, to perform an operation comprising multiplying an input from the source by at least one of the weights from at least one of the shared weights registers, and to place a result in the destination.

A computer comprising a plurality of processor devices connected in a ring, wherein each of the processor devices is connected to each of two neighbouring ones of the processor devices by a respective physical inter-processor link. Each of a set of external memory device stores a local portion of the externally stored dataset. Each processor device executes instructions to: determine that a synchronisation point has been reached by the plurality of processor devices; responsive to the determination, access from its connected external memory device its local portion of the externally stored dataset stored; record a copy of its local portion of the externally stored dataset in its local memory; transmit its local portion of the externally stored dataset to at least one of its connected neighbouring processing devices; and receive an incoming portion of the externally stored dataset from at least one of its connected neighbouring processing devices.

A method of processing batches of data in a computer comprising a plurality of pipelined stages each providing one or more layers of a machine learning model. The method comprises: processing a first batch of data in the pipeline processing stages, each layer of the model using an activation function and weights for that layer to generate an output activation, wherein an output layer generates an output of the model. The method further comprises, for each layer: computing an estimated gradient of a loss function; generating updated weights by processing the estimated gradient with respect to the weights for the first batch using a learning rate for the model; and storing the updated weights for processing on the next batch of data. Updated weights are generated using a modulation factor based on the number of processing stages between that layer and the output layer.

A new apparatus and method for securely distributing an application to processors of a processing unit. The processing unit is formed as part of an integrated circuit and comprises a plurality of processors (referred to as tiles), each having their own execution unit and storage for storing application data and additional executable instructions. The integrated circuit comprises a hardware module (referred to herein as the autoloader) that is configured to distribute a set of bootloader instructions (referred to herein as a secondary bootloader) to each of at least some of the tiles. Each of the tiles then executes instructions of the received secondary bootloader, which causes each tile to issue read requests to read a set of executable application instructions from a memory external to the integrated circuit. Each tile then performs operations using the received set of executable application instructions so as execute the application using the processing unit.

An execution unit, the execution unit having access to a local memory storing a lookup table with a plurality of entries, each entry comprising an x value and corresponding y value representative of a point on a curve of a cumulative distribution function, CDF, consecutive entries of the plurality of entries forming an interval of the CDF, the execution unit being configured to: receive one or more computer program instructions, and in response: generate a random number using random number generation hardware associated with the execution unit, determine, based on the lookup table, the interval of the CDF in which the generated random number falls, and interpolate between y values of entries forming the interval based on the generated random number to generate a random sample of the CDF.

An execution unit configured to: receive a first computer program instruction to populate a lookup table with a plurality of entries, each entry comprising an x value and corresponding y value representative of a point on a curve of a function, consecutive entries of the plurality of entries forming an interval of the function, populate a lookup table stored in a local memory associated with the execution unit with the plurality of entries, receive a second computer program instruction, the second computer program instruction indicating an input value, determine, based on the lookup table, the interval of the function in which the input value falls, and interpolate between y values of entries forming the interval to generate an output value corresponding to the input value.

A processing device comprising a plurality of operand registers, wherein a first subset of the operand registers are configured to store state information for a plurality of bins, comprising a range of values and a bin count associated with each respective bin, wherein a second subset of the operand registers is configured to store a vector of floating-point values; and an execution unit configured to execute a first instruction taking the state information for the plurality of bins and the vector of floating-point values as operands, and in response to execution of the first instruction, for each of the floating-point values: identify based on an exponent of the respective floating-point value, each one of the plurality of bins for which the respective floating-point value falls within the associated range of values; and increment the bin count associated with the identified bins.