DEEP RENDER LTD

There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.

Lossy or lossless compression and transmission, comprising the steps of: (i) receiving an input image; (ii) encoding it to produce a y latent representation; (iii) encoding the y latent representation to produce a z hyperlatent representation; (iv) quantizing the z hyperlatent representation to produce a quantized z hyperlatent representation; (v) entropy encoding the quantized z hyperlatent representation into a first bitstream, (vi) processing the quantized z hyperlatent representation to obtain a location entropy parameter μy, an entropy scale parameter σy, and a context matrix Ay of the y latent representation; (vii) processing the y latent representation, the location entropy parameter py and the context matrix Ay, to obtain quantized latent residuals; (viii) entropy encoding the quantized latent residuals into a second bitstream; and (ix) transmitting the bitstreams.

A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving an input video at a first computer system; encoding an input frame of the input video to produce a latent representation; producing a quantized latent; producing a hyper-latent representation; producing a quantized hyper-latent; entropy encoding the quantized latent; transmitting the entropy encoded quantized latent and the quantized hyper-latent to a second computer system; decoding the quantized hyper-latent to produce a set of context variables, wherein the set of context variables comprise a temporal context variable; entropy decoding the entropy encoded quantized latent using the set of context variables to obtain an output quantized latent; and decoding the output quantized latent to produce an output frame, wherein the output frame is an approximation of the input frame.

A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising steps including: receiving an input image at a first computer system; encoding the input image using a first neural network and decoding the latent representation using a second neural network to produce an output image; at least one of the plurality of layers of the first or second neural network comprises a transformation; and the method further comprises the steps of: evaluating a difference between the output image and the input image and evaluating a function based on an output of the transformation; updating the parameters of the first neural network and the second neural network based on the evaluated difference and the evaluated function; and repeating the above steps.

There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.

A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; identifying one or more regions of the input image associated with high visual sensitivity; encoding the one or more regions of the input image associated with high visual sensitivity using a second trained neural network to produce one or more region latent representations; performing a quantization process on the latent representation and the one or more region latent representations; transmitting the result of the quantization process to a second computer system; decoding the result of the quantization process to produce an output image, wherein the output image is an approximation of the input image.

A computer-implemented method of training an image generative network f θ for a set of training images, in which an output image {circumflex over (x)} is generated from an input image x of the set of training images non-losslessly, and in which a proxy network is trained for a gradient intractable perceptual metric that evaluates a quality of an output image {circumflex over (x)} given an input image x, the method of training using a plurality of scales for input images from the set of training images. In an embodiment, a blindspot network b α is trained which generates an output image {tilde over (x)} from an input image x. Related computer systems, computer program products and computer-implemented methods of training are disclosed.

There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.

A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving an input video at a first computer system; encoding an input frame of the input video to produce a latent representation; producing a quantized latent; producing a hyper-latent representation; producing a quantized hyper-latent; entropy encoding the quantized latent; transmitting the entropy encoded quantized latent and the quantized hyper-latent to a second computer system; decoding the quantized hyper-latent to produce a set of context variables, wherein the set of context variables comprise a temporal context variable; entropy decoding the entropy encoded quantized latent using the set of context variables to obtain an output quantized latent; and decoding the output quantized latent to produce an output frame, wherein the output frame is an approximation of the input frame.

There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.