Pony AI Inc.

Provided herein is a sensor enclosure assembly. The sensor assembly includes a first enclosure positioned at a front of a vehicle roof, and a second enclosure positioned at a rear of the vehicle roof. The first enclosure and the second enclosure each include sensors, which include at least two cameras, and at least one Lidar, cleaning nozzles that direct a fluid towards an exterior cover of the first enclosure or the second enclosure, and a fan disposed within an interior of the first enclosure or the second enclosure and facing towards vents on an exterior of the first enclosure or the second enclosure.

A system trains a model to infer an intent of an entity. The model includes one or more sensors to obtain frames of data, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the system to perform steps. A first step includes determining, in each frame of the frames, one or more bounding regions, each of the bounding regions enclosing an entity. A second step includes identifying a common entity, the common entity being present in bounding regions corresponding to a plurality of the frames. A third step includes associating the common entity across the frames. A fourth step includes training a model to infer an intent of the common entity based on data outside of the bounding regions.

Provided herein is a sensor enclosure assembly. The sensor assembly includes a first enclosure positioned at a front of a vehicle roof, and a second enclosure positioned at a rear of the vehicle roof. The first enclosure and the second enclosure each include sensors, which include at least two cameras, and at least one Lidar, cleaning nozzles that direct a fluid towards an exterior cover of the first enclosure or the second enclosure, and a fan disposed within an interior of the first enclosure or the second enclosure and facing towards vents on an exterior of the first enclosure or the second enclosure.

A computing device triggers a sensor operation. The computing device includes one or more processors and instructions or logic that, when executed by the one or more processors, implements computing functions. The computing device performs receiving timestamps from a sensor, simulating an operation of the sensor, the simulation including predicting orientations of the sensor at different times based on the received timestamps, comparing a latest timestamp of the computing device to a latest timestamp of the sensor, and based on the comparison, triggering a second sensor to perform an operation.

A computing device performs initial processing of sensor data. The computing device performs obtaining sensor data, writing the sensor data to first addresses of a dynamically allocated buffer associated with the computing device, encoding the sensor data, writing the encoded sensor data to second addresses of the dynamically allocated buffer, in response to completing the writing of the encoded sensor data, indicating that the writing of the encoded sensor data has been completed, receiving, from a computing resource, a polling request to read the encoded sensor data, transmitting, to the computing resource, a status that the writing of the encoded sensor data to the second addresses has been completed, reading, to a memory of the computing resource, the encoded sensor data, receiving, from the computing resource, a second status that the encoded sensor data has been read, and removing, from the dynamically allocated buffer, the encoded sensor data.

Described herein are systems, methods, and non-transitory computer readable media for using 3D point cloud data such as that captured by a LiDAR as ground truth data for training an instance segmentation deep learning model. 3D point cloud data captured by a LiDAR can be projected on a 2D image captured by a camera and provided as input to a 2D instance segmentation model. 2D sparse instance segmentation masks may be generated from the 2D image with the projected 3D data points. These 2D sparse masks can be used to propagate loss during training of the model. Generation and use of the 2D image data with the projected 3D data points as well as the 2D sparse instance segmentation masks for training the instance segmentation model obviates the need to generate and use actual instance segmentation data for training, thereby providing an improved technique for training an instance segmentation model.

Provided herein is a sensor enclosure assembly. The sensor assembly includes a first enclosure positioned at a front of a vehicle roof, and a second enclosure positioned at a rear of the vehicle roof. The first enclosure and the second enclosure each include sensors, which include at least two cameras, and at least one Lidar, cleaning nozzles that direct a fluid towards an exterior cover of the first enclosure or the second enclosure, and a fan disposed within an interior of the first enclosure or the second enclosure and facing towards vents on an exterior of the first enclosure or the second enclosure.

A computing device performs initial processing of sensor data. The computing device includes one or more processors and instructions or logic that, when executed by the one or more processors, cause the computing device to perform obtaining sensor data, encoding the sensor data, writing the encoded sensor data to a dynamically allocated buffer, and logging a status of the written encoded sensor data at a static location of the dynamically allocated buffer. The status includes any one or more of memory addresses at which frames of the sensor data begin in the dynamically allocated buffer, valid bit fields corresponding to the frames, and sizes of each of data segments within the frames. The instructions further cause the computing device to perform, in response to receiving a polling request from a computing resource, transmitting the logged status to the computing resource.

Improved calibration of a vehicle sensor based on static objects detected within an environment being traversed by the vehicle is disclosed. A first sensor such as a LiDAR can be calibrated to a global coordinate system via a second pre-calibrated sensor such as a GPS IMU. Static objects present in the environment are detected such as signage. Point cloud data representative of the static objects are captured by the first sensor and a first transformation matrix for performing a transformation from a local coordinate system of the first sensor to a local coordinate system of the second sensor is iteratively redetermined until a desired calibration accuracy is achieved. Transformation to the global coordinate system is then achieved via application of the first transformation matrix followed by application of a second known transformation matrix to transition from the local coordinate system of the second pre-calibrated sensor to the global coordinate system.

Improved calibration of a vehicle sensor based on static objects detected within an environment being traversed by the vehicle is disclosed. A first sensor such as a LiDAR can be calibrated to a global coordinate system via a second pre-calibrated sensor such as a GPS IMU. A static object present in the environment is detected such as signage. A type of the detected object is determined from static map data. Point cloud data representative of the static object is captured by the first sensor and a first transformation matrix for performing a transformation from a local coordinate system of the first sensor to a local coordinate system of the second sensor is iteratively redetermined until a desired calibration accuracy is achieved. Transformation to the global coordinate system is then achieved via application of the first transformation matrix followed by a second known transformation matrix.