Sophos Limited

A method includes obtaining a pre-trained LLM that generates a resulting malicious response to a malicious prompt input to the LLM. The method further includes adjusting a respective weight of neurons of the LLM to cause the LLM to generate a known malicious response to a test prompt input to the LLM, where the test prompt includes a plurality of known test tokens. The method further includes identifying a subset of the neurons based on comparing a respective activity level of each neuron in response to the test prompt with a baseline activity level. The method further includes modifying the respective weights of one or more neurons in the subset of the neurons in the LLM such that, after the modifying, a likelihood that the LLM generates the resulting malicious response to the malicious prompt is below a threshold likelihood value.

A security application provides a prompt and content that includes text and one or more images as input to a multimodal large language model (LLM). The security application receives, from the multimodal LLM and responsive to providing the prompt and the content, a summary report of the content, the summary report including a text summary of the content. The security application extracts features from the summary report. The security application provides the extracted features as input to one or more pre-trained lightweight machine-learning models. The security application receives, from the one or more lightweight machine-learning models, a classification of the content, wherein the classification indicates whether the content is suspicious.

Methods and systems are described for developing a malicious content detector to identify new malicious text content, such as phishing messages, malicious documents, and/or malicious web content. A computing device is used to generate input data which contains an instruction, examples of content, and content to be analyzed. The examples include malicious and benign content samples, designed to recognize similar malicious content. The computing device feeds this input into a generative language model, which produces text labels that indicate the maliciousness of the content to be analyzed. The methods and systems enable rapid development of security protection by leveraging a small number of malicious samples, instead of training with a large dataset of new training samples.

Methods and systems are described for developing a malicious content detector to identify new malicious text content, such as phishing messages, malicious documents, and/or malicious web content. A computing device is used to generate input data which contains an instruction, examples of content, and content to be analyzed. The examples include malicious and benign content samples, designed to recognize similar malicious content. The computing device feeds this input into a generative language model, which produces text labels that indicate the maliciousness of the content to be analyzed. The methods and systems enable rapid development of security protection by leveraging a small number of malicious samples, instead of training with a large dataset of new training samples.

In one or more embodiments, a command is repeatedly input a predetermined number of times into a machine learning model to generate a plurality of different natural language (NL) descriptions. The plurality of different NL descriptions are input into the machine learning model to generate a plurality of different check commands. A plurality of similarity metrics are determined by comparing each check command from the plurality of different check commands to the command. A check command from the plurality of different check commands that is most similar to the command is identified based on the plurality of similarity metrics. An NL description from the plurality of different NL descriptions is caused to be displayed, the NL description previously input into the machine learning model to generate the check command.

A large language model (LLM) is trained to classify Uniform Resource Locator (URL) requests using a data that has been labeled with a domain-to-category database of corresponding, suitable categories for web content. This model can then be distilled using a student model trained to reproduce the behavior of the tuned large language model. The resulting student model can be deployed inline as a content filter for, e.g., content-based filtering of web requests from endpoints behind a firewall or other network device. While this disclosure emphasizes filtering of outbound URL's from network endpoints, it will be understood that the techniques described herein may also or instead be used to create inbound content filters using similarly derived student models that have been trained to identify content based on metadata, text content, image content, and so forth.

In some embodiments, a processor receives natural language data for performing an identified cybersecurity task. The processor can provide the natural language data to a first machine learning (ML) model. The first ML model can automatically infer a template query based on the natural language data. The processor can receive user input indicating a finalized query and to provide the finalized query as input to a system configured to perform the identified computational task. The processor can provide the finalized query as a reference phrase to a second ML model, the second ML model configured to generate a set of natural language phrases similar to the reference phrase. The processor can generate supplemental training data using the set of natural language phrases similar to the reference phrase to augment training data used to improve performance of the first ML model and/or the second ML model.

Methods and systems are described for developing a malicious content detector to identify new malicious text content, such as phishing messages, malicious documents, and/or malicious web content. A computing device is used to generate input data which contains an instruction, examples of content, and content to be analyzed. The examples include malicious and benign content samples, designed to recognize similar malicious content. The computing device feeds this input into a generative language model, which produces text labels that indicate the maliciousness of the content to be analyzed. The methods and systems enable rapid development of security protection by leveraging a small number of malicious samples, instead of training with a large dataset of new training samples.

In one or more embodiments, a command is repeatedly input a predetermined number of times into a machine learning model to generate a plurality of different natural language (NL) descriptions. The plurality of different NL descriptions are input into the machine learning model to generate a plurality of different check commands. A plurality of similarity metrics are determined by comparing each check command from the plurality of different check commands to the command. A check command from the plurality of different check commands that is most similar to the command is identified based on the plurality of similarity metrics. An NL description from the plurality of different NL descriptions is caused to be displayed, the NL description previously input into the machine learning model to generate the check command.

In some embodiments, a processor receives natural language data for performing an identified cybersecurity task. The processor can provide the natural language data to a first machine learning (ML) model. The first ML model can automatically infer a template query based on the natural language data. The processor can receive user input indicating a finalized query and to provide the finalized query as input to a system configured to perform the identified computational task. The processor can provide the finalized query as a reference phrase to a second ML model, the second ML model configured to generate a set of natural language phrases similar to the reference phrase. The processor can generate supplemental training data using the set of natural language phrases similar to the reference phrase to augment training data used to improve performance of the first ML model and/or the second ML model.