HiddenLayer, Inc.

An analysis engine receives data characterizing a multimodal prompt for ingestion by a generative artificial intelligence (GenAI) model. The multimodal prompt is processed and fed into a plurality of layers from which an intermediate result of the GenAI model or a proxy of the GenAI model is obtained. The analysis engine, using a classifier and the intermediate result, determines whether the prompt elicits undesired behavior by the GenAI model. Data characterizing the determination is provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

An analysis engine receives data characterizing a prompt for ingestion by a generative artificial intelligence (GenAI) model. The analysis engine, using a prompt injection classifier determines whether the prompt comprises or is indicative of malicious content or otherwise elicits malicious actions. The prompt injection classifier can be trained using a dataset generated by populating benign content and malicious content into a plurality of different prompt attack structures at pre-defined locations. Data characterizing the determination is provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

The inputs and/or outputs of a generative artificial intelligence model are monitored to determine whether they contain or otherwise elicit undesired behavior by the model such as bypassing security measures, leaking sensitive information, or generating or consuming malicious content. This determination can be used to selectively trigger remediation processes to protect the model from malicious actions. Related apparatus, systems, techniques and articles are also described.

An analysis engine receives data characterizing a multimodal prompt for ingestion by a generative artificial intelligence (GenAI) model. The multimodal prompt is processed and fed into a plurality of layers from which an intermediate result of the GenAI model or a proxy of the GenAI model is obtained. The analysis engine, using a prompt injection classifier and the intermediate result, determines whether the prompt comprises or is indicative of malicious content or elicits malicious actions. Data characterizing the determination is provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

An output of a GenAI model responsive to a prompt is received. The GenAI model is configured using one or more system prompts including one or more Easter eggs. The output is scanned to confirm whether an Easter egg is present. In cases in which at least one Easter egg is present, one or more remediation actions can be initiated to thwart an information leak by the GenAI model. Related apparatus, systems, techniques and articles are also described.

A prompt for a generative artificial intelligence (GenAI) model which contains unicode is received. The prompt is then tokenized to result in a plurality of tokens. Token forming part of a repeating sequence are identified and then removed to result in a modified set of tokens. The modified set of tokens are subsequently detokenized to result in a modified prompt. It is then determined, whether ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner. The modified prompt is passed to the GenAI model when it is determined that ingestion of the modified prompt will not result in the GenAI model behaving in an undesired manner. Otherwise, at least one remediation action is initiated when it is determined that ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner.

A prompt for a generative artificial intelligence (GenAI) model is received which includes unicode. Unicode fonts in the prompt are identified and then translated into a plaintext representation. Further, unicode characters in the prompt are identified which each have an associated unicode tag. It is determined, based on the associated unicode tags, whether at least a portion of the unicode characters are valid. When at least a portion of the unicode characters are determined to be valid, the unicode characters in the prompt are converted into a plaintext representation. The prompt with the translated fonts and the converted unicode fonts are passed into the GenAI model. When at least a portion of the unicode characters are not determined to be valid, the unicode characters are removed from the prompt. This prompt with the translated unicode fonts, after the unicode characters are removed, is input into the GenAI model.

An output of a vision-language model (VLM) can be steered by receiving an input that includes both an image and text, and compositing the image with a universal image configured to elicit specific model activations associated with a targeted behavioral change. The modified image, together with the text, is then supplied to the VLM, resulting in an output that reflects the desired behavioral change. This technique enables steering of the VLM's output without modifying its internal states. The generated output is then provided to an application, process, or system for further utilization. Related apparatus, systems, techniques and articles are also described.

The provenance of a machine learning model can determined by receiving at least one file encapsulating the machine learning model. A computational graph corresponding to the machine learning model is extracted from the at least on file. The computational graph is converted from a first format into a normalized computational graph having a second, different format. The normalized computational graph is decomposed into components. These components can include, for example, nodes, edges between nodes, blocks, and edges between blocks. The components of the normalized computational graph can be compared with components of each of a plurality of normalized computational graphs which, in turn, each correspond to a different known machine learning model. Based on the comparison, a model genealogy of the machine learning model is determined. Data characterizing this model genealogy is provided to a consuming application or process.

Vulnerabilities in a machine learning model can be identified by receiving at least one file encapsulating the machine learning model. A computational graph corresponding to the machine learning model is then extracted from the at least on file. The computational graph is converted from a first format into a normalized computational graph having a second, different format. The normalized computational graph is decomposed into components. These components can include nodes, blocks, and edges between blocks. The normalized computational graph is scanned by iterating through the components to identify any backdoors. Data characterizing whether any backdoors were identified can be provided to a consuming application or process.