METHOD FOR DETECTING A GOLF MOTION USING AN ELECTRONIC DEVICE SUCH AS A WATCH

TECHNICAL FIELD

[0001]The present invention relates to the field of electronic devices, worn on the wrist or on another part of the arm, such as bracelets or electronic watches, also known as “smartwatches”. More specifically, the present invention is aimed at an electronic device worn on the wrist or on another part of the arm, notably a watch, equipped with hardware and software means for detecting golf strokes.

BACKGROUND

[0002]Golfers practicing their sport on a course generally need to keep track of their score, that is, the number of strokes it took them to get the ball in the hole. The accumulated scores per hole give the player's score for the course. By comparing their score with the course par, the player can thus evaluate their performance. In the case of competitions, this allows competitors to be ranked.

[0003]Notably for amateur or non-competitive players, it is tedious for players to write down every stroke played to keep track of their scores.

[0004]There is therefore a need for a method capable of calculating the score of a golfer in real time by automatically detecting each of their golf strokes.

[0005]Document U.S. Pat. No. 10,709,945 B2 discloses a concept for detecting golf strokes using sensors embedded in a watch strap or casing. However, this document is limited to describing expected results, without disclosing a concrete method for achieving this golf motion detection. Additionally, this document outlines a concept requiring an impact or vibration sensor to be embedded on the object (a priori the club or the ball) affected by the impact.

[0006]The invention thus aims to eliminate at least some of these disadvantages. Notably, the invention dispenses with the need for a sensor carried by the club or by the ball.

SUMMARY OF THE INVENTION

[0007]More specifically, the invention relates to a golf-stroke-detecting method configured to operate on an electronic device worn on the wrist or on another part of the arm, such as a bracelet or a watch, comprising one or more angular or linear measuring means, notably in orientation, in speed or in acceleration. In particular, to implement the invention, the electronic device in question may comprise at least a three-axis accelerometer and/or a three-axis gyroscope and/or at least a three-axis magnetometer, cooperating with a computing unit and a memory space.

[0008]A golf stroke being defined by a golf motion performed by a user manipulating a golf club, the club having a face, and causing said club to describe a typical trajectory resulting in an impact between the face of the golf club and a golf ball, the method comprises implementing a golf-motion-detecting function and an impact-detecting function, the motion-detecting function being configured to detect, at any time, a golf motion associated with one golf motion among a set of predefined golf-stroke motions, notably a drive, a swing and a putt, and the impact-detecting function being configured to detect, synchronously, an impact between the golf club and the golf ball, based on measurements carried out by at least one sensor embedded in the electronic device.

[0009]The distinction between, on the one hand, the golf-motion-detecting function and, on the other hand, the impact-detecting function, with impact detection being performed actively, is essential to allow improved detection of all actual golf strokes and to reduce the risk of detecting “false” golf strokes, such as practice repetitions. Impact detection, in particular, makes it possible to distinguish between test strokes and actual golf strokes.

[0010]According to one embodiment, the golf-motion-detecting function and the impact-detecting function are implemented solely from measurements carried out by at least one sensor embedded in the electronic device, such as a wristband or a watch, to the exclusion of sensors embedded in the club or the ball, from at least one three-axis accelerometer and at least one three-axis gyroscope.

[0011]Preferably, the measurements carried out originate from six embedded sensors: accelerometers and gyroscopes along three direct orthogonal axes, respectively.

[0012]According to one embodiment, the measurements are carried out continuously and recorded at a first sampling rate at least equal to 10 Hz, notably about equal to 50 Hz plus or minus 10%, and at a second sampling rate at least equal to 100 Hz for measurements carried out over at least one reduced time window.

[0013]As described in detail below, the use of different sampling rates, and in particular the use of a sampling rate that increases the number of measurements per time unit over a reduced time window, enhances the impact detection while reducing the load on the internal memory of the electronic device.

[0051]The invention also relates to a computer program-type product, comprising at least one sequence of instructions stored and readable by a processor and which, when read by this processor, causes the steps of the method as previously presented to be carried out.

[0052]The invention further relates to a computer-readable medium comprising the computer program product as presented previously.

BRIEF DESCRIPTION OF THE DRAWINGS

[0053]The invention will be better understood upon reading the following description, given by way of example, and referring to the following figures, given as non-limiting examples, in which identical references are given to similar objects.

[0054]FIG. 1 is a graph showing filtered signals from IMU sensors;

[0055]FIG. 2 is a graph showing a peak on a signal from an IMU sensor;

[0056]FIG. 3 shows the characterization of a peak in a signal from an IMU sensor;

[0057]FIG. 4 shows the normalization of signals from IMU sensors;

[0058]FIG. 5 shows graphs representing model envelopes for normalized signals;

[0059]FIG. 6 shows a zoom over a time sub-window applied to normalized and regularized signals, for impact detection;

[0060]FIG. 7 shows a comparison between a normalized and regularized signal, on a time sub-window, with an “expected” sinusoidal signal.

[0061]It should be noted that the figures set forth the invention in detail to implement the invention, said figures of course being capable of being used to further define the invention where appropriate.

DETAILED DESCRIPTION

[0062]The invention relates to a golf-motion-detecting method implemented by means of sensors embedded in an electronic device, such as a bracelet or a watch, worn by the player, in particular on the wrist or optionally on another part of the arm. The embedded sensor means cooperate with computing and display means and a memory which can preferably also be embedded in the electronic device, notably the watch, for greater compactness.

[0063]Alternatively, at least part of the computing and display means and at least part of the memory can be hosted by an external device, such as a smartphone, a tablet or a computer for example.

[0064]To detect a golf motion, the method according to the invention implements two distinct functions, namely the detection of a golf motion, corresponding to a golf motion, and the detection of an impact, between the ball and the club, based notably exclusively on data from sensors embedded in the electronic device worn by the player on the wrist or on another part of the arm, in particular the watch.

[0065]In the following, the present invention is more particularly described in the context of an implementation via a watch worn by the player. However, this is only one example and the invention also applies in cases where the player wears for example a bracelet or another similar electronic device, worn on the wrist or on another part of the arm.

[0066]The embedded sensors used comprise in particular IMU (Inertial Measurement Unit) type sensors, such as accelerometers and/or gyroscopes and/or magnetometers along one to three axes.

[0067]Measurements are carried out continuously at a frequency greater than 100 Hz and recorded with a first sampling rate preferably greater than 10 Hz, notably equal to 50 Hz plus or minus 10%, and, over at least one reduced time window (described below), with a second sampling rate greater than 100 Hz.

[0068]For each function, the invention notably provides for the use of successive and progressive detection layers. Initially not very selective, the detection layers become more discriminating and make it possible to determine with certainty whether a real golf motion has been played.

[0069]In practice, the golf-motion-detecting function thus comprises the continuous recording of data measured by sensors embedded in the electronic device, notably the watch, in particular an IMU sensor integrating for example three accelerometers and three gyroscopes along three respective mutually orthogonal axes.

[0070]According to one embodiment, the accelerometers measure accelerations of the watch, and therefore of the user's wrist, along three mutually orthogonal axes. The gyroscopes measure rotation speeds of the watch, and therefore of the user's wrist, along three mutually orthogonal axes.

[0071]By virtue of the measurements carried out by these sensors, time signals are obtained formed of a plurality of acceleration or angular velocity values based on time.

[0072]Notably, the time signals from these sensors are first filtered to filter out noise and high-frequency residuals. Filtered signals accx, accy, accz (from accelerometers) and gyrx, gyry, gyrz from gyroscopes are then obtained as shown in [FIG. 1].

[0073]A different filter can be implemented for the purposes of implementing the golf-motion-detecting function and, respectively, for the purposes of implementing the impact-detecting function.

[0074]For the golf-motion-detecting function, for example, time signals are filtered using a low-pass filter, notably a first-order low-pass filter.

[0075]Embodiments of the golf-motion-detecting function are detailed hereinafter.

[0076]As shown in [FIG. 2], for each signal, a peak is identified each time the signal amplitude exceeds a simple threshold, designated “POSSIBLE SWING” in [FIG. 2], notably selected as being “permissive”, that is, with little discrimination, and a date of occurrence of the peak of greatest amplitude is recorded in a memory, embedded in the watch or in a separate device. A potential swing type golf motion is thus detected when the “POSSIBLE SWING” threshold is crossed. Below this level, there is no motion (“NO SWING”) and no further analysis is required at this stage.

[0077]When a golf motion is executed, whether it is a real stroke or a test stroke, at least one peak is thus detected, generally two peaks, and potentially three or more peaks. In any case, one to three peaks having the highest amplitudes are stored in memory, together with the date of occurrence of the peak with the highest amplitude, for at least one, preferably several, filtered time signals. According to the invention, as shown in [FIG. 3], these one to three peaks are characterized in terms of height, width and prominence.

[0078]According to one embodiment, each of the height, the width and the prominence of each peak is then compared with a respective threshold for minimum height, minimum width and minimum prominence in order to confirm or deny the existence of a corresponding peak.

[0079]Notably, the respective thresholds can be predefined empirically. For this first part of the method according to the invention, simple, so-called “permissive” thresholds are selected, that is, with little discrimination, as it is important not to miss the detection of a golf motion. The following steps, however, provide sufficient discrimination against “false positives”, in particular by virtue of the impact-detecting function.

[0080]Once the detection of at least one peak has been confirmed, according to the invention, it is determined that the recorded time signals correspond to possible golf motions. From then on, the time signals corresponding to the measurements carried out by the various IMU sensors are then preferably normalized, as shown in [FIG. 4]. Normalization relates in particular to each of the signals delivered by the IMU sensors, corresponding to measurements carried out, in terms of acceleration, based on measurements carried out by at least one accelerometer, and/or, in terms of rotation speed, based on measurements carried out by at least one gyroscope.

[0081]Normalization consists in particular, according to one embodiment, of transposing the signal so that the maximum norm reached is equal to 1. The general principle of signal normalization is known.

[0084]In other words, the common time base for normalized signals is translated and contracted or dilated so as to bring the main peak and any secondary peak to selected reference abscissas. This facilitates further calculations and analyses by working with normalized and regularized signals.

[0085]Said time base dilation or contraction coefficient can be mathematically constrained by imposed bounds and used separately to help assess the credibility of the original signal.

[0086]Then, according to one embodiment, the implementation of the golf-motion-detecting function comprises the execution of a golf motion recognition algorithm comprising, from a golf stroke database, in other words a database comprising a set of signals (accelerations and/or angular velocities along three axes) corresponding to trajectories of the electronic device during confirmed golf strokes, establishing a vector μ and a covariance matrix P of the set of regularized normalized signals, so that said covariance matrix P mathematically represents an ellipsoid from which to evaluate the probability that a candidate trajectory, in other words a possible golf motion, is actually a golf motion.

[0087]To establish the covariance matrix P, the following methods can notably be used.

[0088]According to a first method, a statistical vector variable X is considered, of which each signal in the database is a realization, each time value of the signal being considered as a distinct scalar variable of X, so that the dimension n of X is equal to the product of the number of signals (typically 6 in the case of an IMU sensor measuring the acceleration and angular velocity respectively along three axes) and the number of values forming said signals over the time window considered (typically a hundred or so time values recorded with the first sampling rate). Each trajectory in the database is then considered as a realization of the statistical variable X, whose mean μ and covariance matrix P are then established from the set of realizations.

[0089]According to a second method, the time signals are sub-sampled beforehand to reduce the dimensionality n of X, μ and the covariance matrix P, with the aim of reducing the volume of calculations without losing representativeness. According to one implementation, a simple convolution (averaging or simple linear filtering) is applied to each time signal so that the sub-sampling is as non-destructive as possible.

${\sum }_k{X}_a^2\left(k\right)$

[0101]According to this implementation, the candidate motion is then considered an actual golf motion if the corresponding quantile is less than a set quantile threshold.

[0102]According to another embodiment, the implementation of the golf-motion-detecting function comprises the execution of a golf motion recognition algorithm comprising, from a golf stroke database, comparing each regularized normalized signal with a model signal corresponding to a typical golf motion for each type of golf motion including at least a drive, a swing and a putt, said comparison comprising checking that the regularized normalized signal lies within a probabilistic envelope defined so as to include the possible variability of a real golf motion. This probabilistic envelope is determined from a database of golf motions and the application of appropriate selected margins.

[0103]For example, the golf motion database can be enriched by the collection of new data, notably user data. In particular, according to one embodiment, the database differs between right-handed users and left-handed users and based on whether the watch is worn on the right wrist or on the left wrist.

[0104]Based on the comparison, a type of golf motion is identified.

[0105]The comparison can then implement a learning algorithm capable of improving the model signals used.

[0106]According to one embodiment, to validate the golf motion detection, a normalized error is calculated for each regularized normalized signal with respect to the model signal of the type of stroke identified and checking that the normalized error is less than a threshold and that the sum of the normalized errors over all the regularized normalized signals is less than a threshold.

[0107]For example, the normalized error is calculated according to the equation:

$\begin{array}{cc}{E}_X\left(i\right)=\frac{X\left(i\right)-\mu \left(i\right)}{\sigma \left(i\right)}& \left[\mathrm{Math}.2\right]\end{array}$

[0109]The threshold can be predefined or be adjustable.

[0110]Additionally, it is moreover possible to calculate a normalized error instability for each regularized normalized signal and check that the normalized error instability is less than a threshold for each regularized normalized signal and that the sum of the normalized error instabilities over all regularized normalized signals is less than a threshold. This instability of the normalized error can for example be obtained by calculating the integral or the direct sum of the absolute value of the derivative of the normalized error.

[0111]For example, the instability of the normalized error is calculated according to equation:

$\begin{array}{cc}{\mathrm{Estab}}_x=\frac{1}{N-1}{\sum }_{i=1}^N❘E_X\left(i\right)-E_X\left(i-1\right)❘& \left[\mathrm{Math}.3\right]\end{array}$

[0112]The threshold can be predefined or be adjustable.

[0113]In the foregoing equations, N is the number of measurements present in the time window (for example 0 to 2.5 seconds) after application of the time homothety.

[0114]μ is the average envelope for the signal from a normalized and regularized IMU sensor X.

[0115]σ is the standard deviation of the envelope for the normalized and regularized signal X.

[0121]According to one embodiment, several of these conditions must be met to validate the detection of a golf motion. Notably, all of these conditions may have to be met cumulatively.

[0122]According to one embodiment, several thresholds, notably two thresholds—a low threshold and a high threshold—can be defined for each parameter. Validation of a golf motion is then acquired for example when at least one parameter remains below two of the low thresholds or at least two parameters when the other parameters are above the low thresholds while remaining below the high thresholds.

[0123]The examples given hereinbefore for setting the conditions for validating the detection of a golf motion should not be interpreted restrictively, as they are merely illustrative.

[0124]It should be noted that other signal comparison methods can be implemented. For example, it is possible to define an error function that is always constructed according to the database described previously. Reduced variables are then established, constructed as linear combinations of the individual time values of each normalized and regularized signal at each sampling rate, the coefficients of these linear combinations being established so as to maximize the difference between the values taken by the reduced variable between, on the one hand, a normalized and regularized signal corresponding well to a golf motion to be detected and, on the other hand, a normalized and regularized signal not corresponding to a golf motion. These linear combination coefficients are then applied to the normalized and regularized signals to validate or invalidate the detection of a golf motion. This error function can also be applied to the impact-detecting function.

[0125]For the impact-detecting function, measurements recorded with a higher sampling rate are analyzed. The sampling rate is selected, in particular greater than 100 Hz, for example equal to about 400 Hz plus or minus 10% (notably 416 Hz) or to about 800 Hz plus or minus 10% (notably 833 Hz). Measurements are stored in memory for the duration of a reduced time window.

[0126]In practice, according to the invention, the IMU sensor (preferably accelerometer and/or gyroscope and/or magnetometer) carries out measurements with a high sampling rate, greater than 100 Hz, notably of the order of 400 Hz or of the order of 800 Hz. The measurements carried out are recorded continuously with the first sampling rate, greater than 10 Hz, and the golf-motion-detecting function is implemented on these measurements recorded with the first sampling rate.

[0127]The impact-detecting function is preferably implemented over a reduced time window on measurements recorded with the second sampling rate, which is greater than 100 Hz, for example of the order of 400 Hz or of the order of 800 Hz.

[0128]To this end, as with the motion detection function, continuous checks are carried out on signal amplitude threshold criteria at each time step and at high frequency, that is, at the frequency at which measurements are taken.

[0129]According to one embodiment, the amplitude threshold criterion is notably applied to a derivative of the signal corresponding to the measurements carried out, obtainable by digital filtering, typically using a 1st-order derivative filter, of the signal from the three-axis accelerometer. The time constant of the derivative filter is selected to be particularly low (a few milliseconds) so as not to cut off the frequencies of interest, it being understood that an impact is detected over a reduced time window of between about 15 and 30 ms.

[0130]According to the invention, as shown in [FIG. 6], if the absolute value of this derivative daccz takes values greater than a predefined threshold, it is checked whether a change of sign and an exceeding of the threshold in the opposite direction occurs. If this is the case, it is considered that an impact may have occurred and a fine analysis of the signal is carried out over a reduced time window of a few tens of milliseconds, comprising the portion of signal comprising the two threshold crossings opposite in sign.

[0131]It should be noted that this predefined threshold for identifying a potential impact can be selected dynamically, based on for example the current amplitude of the acceleration or the rotation speed itself. In this case, if the acceleration or the rotation speed corresponding to the golf motion identified in parallel is high, then the threshold for identifying a potential impact will be higher, and conversely.

[0132]The reduced time window dedicated to the fine analysis of an impact lasts for example between 100 ms and 150 ms.

[0133]As shown in [FIG. 7], over this fine analysis window, according to one embodiment of the invention, the aim is to check whether the signal corresponding to the measurements recorded with the second sampling rate is similar to a sinusoid whose period is twice the date difference between the two peaks of opposite sign mentioned hereinbefore.

[0134]According to one embodiment, it is first checked if the period thus defined is realistic, by comparing the period corresponding to twice the date difference between the two peaks with a predefined minimum period and maximum period. For example, the period should be between 15 ms and 30 ms.

[0135]As shown in [FIG. 7], each regularized normalized signal can thus be compared to a sinusoidal signal, designated sinus_approx, whose period corresponds to twice the date difference between the aforementioned peak and the opposite peak.

[0139]Impact detection will then be validated if the squared error rmsd is less than a threshold.

[0140]The threshold is for example predefined, configurable or adjustable.

[0141]If this error is less than a threshold, and, optionally, if its instability (as previously) is less than a threshold, then impact detection is confirmed.

[0142]It is evident that, according to the invention, it is possible to exit the impact-detecting function as soon as a condition is not satisfied.

[0143]According to another embodiment, a DFT (Discrete Fourier Transform) is performed over a sliding time window on the high-frequency signal, that is, on the signal corresponding to the measurements carried out with the high sampling rate used to carry out the measurements. Said sliding time window is selected to be likely to correspond to a nominal sinusoid period as described hereinbefore. This period is typically between 15 ms and 30 ms, for example equal to about 20 ms. Fourier analysis can be limited to the coefficients of interest of the decomposition, that is, those of the lowest possible non-zero frequency over such a sliding time window.

[0144]If the absolute value of the DFT exceeds a certain predefined threshold which, as previously, can be adapted to the signal amplitude, in terms of acceleration or rotation speed, then the analysis continues. Otherwise, it can be considered that there was no impact.

[0145]If the analysis is continued, the length of time during which this threshold is crossed is determined. The maximum reached over this period is then evaluated.

[0146]According to one embodiment, it is also possible to evaluate whether this absolute value of the DFT is consistent with the amplitudes of the acceleration and/or rotation speed signals as recorded with the first sampling rate.

[0147]If this is the case, it is possible to evaluate the time during which the absolute value of the DFT remains greater than or equal to a threshold equal to half the peak of this absolute value. It is then determined whether this duration lies between a predefined minimum threshold and a predefined maximum threshold.

[0148]If this is the case, it is possible to confirm the impact detection. Otherwise, it can be considered that there was no impact.

[0149]According to the invention, it should be noted that the criteria and checks listed hereinbefore can be implemented cumulatively. Alternatively, only some of these criteria and checks are implemented.

[0150]Ultimately, the existence of an impact between the ball and the golf club is detected, or not.

[0151]According to a particular embodiment, “permissive” thresholds are defined to allow more potential golf motions and more potential impacts to be analyzed, so as to avoid failing to detect some thereof. From then on, to avoid the opposite problem of detecting “false positives”, a probability is calculated for each threshold crossing based on whether or not it is close to these thresholds. Thus, an included value far from the thresholds is associated with a high probability, while an included value close to a threshold is associated with a low (or even very low) probability. The final decision of whether a golf motion or an impact is detected is then based on the total probability obtained by aggregating (in other words, by multiplying) the different probabilities thus obtained.

[0152]Alternatively or additionally, the validation of the impact detection can comprise the error and error instability calculation as described by means of the equations [Math. 1], [Math. 2] and [Math. 3].

[0153]According to one embodiment, the confirmed detection of an impact furthermore comprises verifying a consistency between the motion detection and the impact detection. Notably, the motion and the impact must for example be detected substantially simultaneously, typically within a narrow time window, notably of a few tenths of a second, for example 400 ms. Thus, an impact detected for example one second after a motion would be eliminated and would not be identified as a real golf motion.

[0154]Additionally, for example, the amplitude of the time signals of the IMU sensors must be consistent between the detected motion and the detected impact.