OP Solutions, LLC

An encoder for encoding video for machine consumption includes a first feature encoding element and a second feature encoding element. The first feature encoding element receives a set of features extracted from a source video by a backbone of a convolutional neural network and outputs features having reduced dimensions compared to the features input into the first feature encoding element. The second feature encoding element receiving the features having reduced dimensions and outputting an encoded bitstream for a machine video application.

A decoder includes circuitry configured to receive a bitstream, identify a first frame, locate a first independent reference region within the first frame, extract the first independent reference region from the first frame, decode a second frame using the first independent reference region as a reference for the second frame.

Systems and methods for encoding and decoding video for machine consumption (video coding for machines) are provided in which an autoencoder is employed. The autoencoder has an encoder portion, a bottleneck portion, and a decoder portion. The autoencoder being distributed between a VCM encoder and VCM decoder such that the VCM encoder includes the encoder portion and bottleneck portion and the VCM decoder includes the bottleneck portion and the decoder portion.

Methods and systems for hybrid feature video bitstream encoding and decoding are provided. Encoding can be performed in one of a plurality of modes. In one mode, the video components of the encoded bitstream include residual data generated encoding the difference between feature data and input video data. A compatible decoder can operate in a plurality of decoding modes. In one decoding mode, the video components are decoded using the encoded video and encoded feature data in the bitstream. The hybrid bitstream between the encoder and decoder is formed with a plurality of hybrid segments, each having a plurality of components specifying characteristics the feature and video content of the segment.

The present systems and methods for video decoding for machine processing extract features and image statistics from the decoded bitstream and evaluate the image statistics to predict whether augmentation of a frame will enhance task performance and generate at least one parameter for selectively applying an augmentation process. The parameter is applied to selectively alter at least a portion of a frame to enhance task performance by the machine processing the decoded bitstream.

A system and method for video and feature coding of neural-network structures used for multi-task machine learning includes an encoder, decoder and a decoder-compliant bitstream. A task-specific video decoder includes a first decoder receiving a bitstream having at least one feature and a description of a neural network backbone used to generate the bitstream, and a task-specific neural network head. The neural network head recreates the neural network backbone from the description, receives a feature from the bitstream a generates a task-specific output.

A method is provided for processing a video bitstream which includes a first picture and a sequence parameter set containing information indicating a mode is enabled in which a first area of a picture can be extracted and replaced by a second area of another picture. Using the mode, the first picture is modified by extracting the first area and replacing the extracted area with the second area from the other picture. The modified first picture is stored and a predictor is formed using the modified first picture as a reference picture. A block of a subsequent picture is decoded using the predictor.

A method of memory-efficient prediction mode selection includes receiving, by an encoder, a coded bitstream including a current frame, determining, by the encoder, costs of a first prediction mode and a second prediction mode, wherein determining further comprises determining, for the first prediction mode, a first bit cost and a first memory cost and determining, for the second prediction mode, a second bit cost and a second memory cost, selecting, by the encoder, a current prediction mode of the first prediction mode and the second prediction mode as a function of the first bit cost, first memory cost, second bit cost, and second memory cost, and encoding, by the encoder, the current frame using the current prediction mode. The prediction mode may be informed by at least one parameter received from the decoder.

An video signal processor includes circuitry configured to receive a video, determine a plurality of long-term reference frames, determine a long-term reference retention time, and encode the video into a bitstream including the determined plurality of long-term reference fames and the determined retention time. A decoded picture buffer for retaining a plurality of reconstructed pictures, including a long term reference picture, is also provided.

A decoder includes circuitry configured to receive a bitstream; determine, for a current block and using the bitstream, a global motion vector candidate utilized by an adjacent block; construct a motion vector candidate list including adding the determined global motion vector candidate to the motion vector candidate list; and reconstruct pixel data of the current block and using the motion vector candidate list. Related apparatus, systems, techniques and articles are also described.