Rapsodo Pte. Ltd.

Disclosed are embodiments for determining the position and speed of a projectile in three-dimensional world coordinates using a single camera and radar. In some embodiments, a method comprises: determining a zero radial speed time of the projectile based on radar measurements of the projectile; determining a radial speed of the projectile at a particular time based on the radar measurements; determining a radial speed slope at the particular time based on the radial speed measurements; determining a two-dimensional image point of the projectile at the particular time; determining a depth scale coefficient at the particular time based on the zero radial speed time, the radial speed, the radial speed slope, the image point, and a homographic transformation; and determining a position of the projectile in space at the particular time based on a three-dimensional position of the camera, the depth scale coefficient, the image point and the homographic transformation.

Pixel intensity differences between pixel values of a region in a first image frame and the region in a second image frame are determined, where the region in the first image frame includes an image of a spherical object, with no overlapping spherical object in the second image frame. Based on the pixel intensity differences, the region of the first image frame is thresholded into foreground pixels and background pixels until the background pixels make up more than a predefined percentage of the region of the first image frame. From the background pixels, a subset of the background pixels located along a plurality of paths that radially extend outward from a center of the region in different angular degrees is selected. The subset of the background pixels is filtered by applying a filter, where the filtered subset is detected as edge pixels.

In a first step, a time series of images are resized to a smaller dimension, and the smaller images are fed into a first classifier that is trained to classify as a ball any objects in the smaller images that resemble a ball. In a second step, the smaller images are mapped back to the series of images, and regions in the series of images that contain the mapped ball are cropped from the series of images. The mapped ball is shifted based on a velocity of the mapped ball in the cropped regions, and the second classifier regresses center coordinates and a radius of the shifted ball, classifies whether the shifted ball is the ball based on a confidence score, and updates the shifted ball in the cropped regions based on the regressed center coordinates and radius.

Embodiments are disclosed for determining a spin rate and axis of a ball using deep learning. In some embodiments, a method comprises: training a deep learning network on training images of spinning balls, each spinning ball having at least one feature point in a time series that forms a two-dimensional (2D) ellipse image in a 2D plane; capturing, with at least one camera, a series of images of a ball; predicting, with the trained deep learning network, spin measurements associated with the ball based on the series of images; determining a spin rate of the ball based on the spin measurements; determining coefficients of a 2D ellipse model based on the spin measurements and the spin rate; and determining, with the at least one processor, a spin axis of the ball in 3D space based on the 2D ellipse model and the spin rate.

Embodiments are disclosed for a range-gated imager. In some embodiments, a method comprises: transmitting, with a single-tone continuous wave (STCW) radar, a signal; receiving, with the STCW radar, a return signal from a projectile impinged by the radar signal; counting, with a measuring apparatus, a specified number of periods of non-ambiguity range based on the return signal, performing a flashing operation; and gating or triggering, by the measuring apparatus, an imager to capture an image of the projectile in response to the count reaching the specified number of periods.

Embodiments are disclosed for a range-gated imager. In some embodiments, a method comprises transmitting, with a multi-tone continuous wave (MTCW) radar, a radar signal comprising a first tone and a second tone, where the first tone and the second tone are separated by a frequency gap; receiving, with the MTCW radar, a return signal from a projectile impinged by the radar signal; detecting, with a measuring apparatus, a zero crossing of a phase difference between the first and second tones; and responsive to detecting the zero crossing, gating or triggering, by the measuring apparatus, an imager to capture an image of the projectile.

A launch-monitoring system that models a portion of a golf club, golf swing, and golf ball may include a camera and a radar positioned orthogonally to a swing direction of the golf club. A series of images of the golf ball are collected during and after the golf club contacts the golf ball by the camera. The golf swing is captured by the radar. The images are converted into parameterized motion representations, and the radar signal is converted into time-frequency images, which are sent to a convolutional neural network. The convolutional neural network outputs golf club parameters, golf swing parameters, and golf ball parameters, which generate a visual model of the golf club, golf swing, and golf ball in a virtual space. The parameterized motion representations of the golf ball and the time frequency images of the golf swing are not correlated and operate independently from each other.

A method includes receiving motion data of a user in an environment with respect to a plurality of instances of a first action by the user, determining a kinematic movement based on receiving the motion data, analyzing the kinematic movement using a neural network, obtaining a plurality of outcome types with respect to the first action of the user, correlating the kinematic movement with the at least one indication of the outcome type with respect to the first action, classifying an outcome of the first action as at least one of the plurality of outcome types, determining which of the kinematic movements of the user result in the at least one of the plurality of outcome types, and providing instructions to the user to alter the determined kinematic movements of the user that result in the at least one of the plurality of outcome types.

Disclosed are embodiments for determining the position and speed of a projectile in 3D world coordinates using a calibrated camera and radar. In some embodiments, a method comprises: capturing a sequential set of images of a projectile in motion; determining a radial velocity of the projectile; determining a set of two-dimensional (2D) image points representing respective locations of the projectile in the sequential set of images; determining respective depths of the 2D image points in their respective 2D image planes based on the 2D image points, time intervals between the captured 2D images, the radial velocity and a rotational part of a homographic transformation, the homographic transformation configured to project three-dimensional (3D) world coordinates to the 2D image planes; and determining a set of 3D world points of the projectile in the 3D world coordinates based on the 2D image points, the respective depths thereof and the homographic projection.