ASAPP, INC.

A speech processing model may be trained using pseudo tokens. Training a speech processing model with pseudo tokens may allow for training with a smaller amount of labeled training data and accordingly lower costs. A set of pseudo tokens may be determined by computing feature vectors from unlabeled training data, clustering the feature vectors, and performing token compression using the clustered feature vectors. A first speech processing model may be trained using unlabeled training data by determining sequences of pseudo tokens corresponding to the unlabeled training data. A second speech processing model may be initialized using the first speech processing model and then trained using labeled training data. The second speech processing model may then be deployed to a speech processing application.

A task, such as task completed using a website, may be automated by submitting prompts to a language model and requesting that that language model provide one or more next actions to be performed to complete the task. The accuracy of the language model in providing correct actions may be improved by using one or few-shot learning where examples of completing a task are provided in a prompt to the language model. The accuracy of the language model may also be improved by breaking a task into subtasks. A prompt may be submitted to the language model to request that the language model indicate a subtask to be performed to complete the task. A prompt may then be submitted to the language model to request that the language model indicate a next action to be performed to complete the subtask.

The extraction of key values from a conversation may be facilitated by using automation techniques. Communications of a conversation may be processed to determine a natural language intent, and the intent may be used to determine one or more target keys. The automated process may be used to obtain values for the one or more target keys. For a target key, a prompt may be presented to a user in the conversation, a response may be received from the user, and the response may be processed with a value extractor corresponding to the target key to determine a value for the target key. The values may then be presented to user in the conversation or used for other processing.

A diffusion model may be used to generate an audio signal from text. The diffusion model may process received text and noise vectors to compute encoded audio vectors that correspond to the text. The encoded audio vectors may be decoded to generate an audio signal of a person speaking the text that may be presented to a user. The diffusion model may process a sequence of byte-encoding vectors corresponding to the text, and the use of the byte-encoding vectors may allow for the generation of higher quality audio signals. In some implementations, prompt audio of a person may also be used to generate an audio signal that resembles the speech of that person.

A communications session with a user may be automated using a language model. The language model may be instructed to select a next action to be performed where the next action may include transmitting a responsive communication to the user or performing an API call. The prompt used to query the language model may include one or more of the following: a representation of text of the communications session, a list of available API calls, instructions to select a next action, a representation of API calls performed, or a representation of API call responses received. The language model may be sequentially queried to continue the communications session by transmitting responsive communications or performing API calls. In some implementations, a prompt template may be used to generate the prompt and a prompt template may be selected using text of the communications session.

A third-party service may be used to assist entities in responding to requests of users by determining a suggested resource corresponding to a received communication. The third party service may receive a request from a first entity, such as via an application programming interface request, that includes a message in a conversation. A conversation feature vector may be computed by processing the message with a first neural network. A suggested resource may be determined using the conversation feature vector. The third-party service may then return the suggested resource for use in the conversation. The third-party service may similarly be used to assist other entities in responding to requests of users.

A message may be suggested to a user participating in a conversation using one or more neural networks where the suggested message is adapted to the preferences or communication style of the user. The suggested message may be adapted to the user with a user embedding vector that represents the preferences or communication style of the user in a vector space. To suggest a message to the user, a conversation feature vector may be computed by processing the text of the conversation with a neural network. A context score may be computed for one or more designated messages, where the context score is computed by processing the user embedding vector, the conversation feature vector, and a designated message feature vector with a neural network. A designated message may be selected as a suggested message for the user using the context scores. The suggestion may then be presented to the user.

For any application that processes speech, improving the quality of the feature vectors may improve the quality of the speech application. The quality of feature vectors may be improved by modifying a neural network architecture for computing feature vectors to allocate computational resources where they are more effective for learning and computing the feature vectors. Contextual feature vectors may be computed from feature vectors by using a parameterized downsampling operation that decreases a vector sequence rate, processing the downsampled vectors with a neural network, and using a parameterized upsampling operation that increases a vector sequence rate. For example, parameterized downsampling may decrease a vector sequence rate by a factor of two, a neural may require fewer computational resources since it operates with a lower vector sequence rate, and parameterized upsampling may then increase the vector sequence rate by a factor of two.

A communications session with a user may be automated using a language model. The language model may be instructed to select a next action to be performed where the next action may include transmitting a responsive communication to the user or performing an API call. The prompt used to query the language model may include one or more of the following: a representation of text of the communications session, a list of available API calls, instructions to select a next action, a representation of API calls performed, or a representation of API call responses received. The language model may be sequentially queried to continue the communications session by transmitting responsive communications or performing API calls. In some implementations, a prompt template may be used to generate the prompt and a prompt template may be selected using text of the communications session.

Natural language communications may be automated using language models and session contexts. A first session context may be created for communications between a language model and a first entity. Prompts may be generated using this session context, and responses from the language model may be used to communicate with the first entity. A second session context may also be created for communications between a language model and a second entity. Context transfer information may be generated by processing the first session context with a language model. The context transfer information may be added to the second session context. The second session context with the context transfer information may then be used to generate a communication to the second entity using a language model.