US20260183614A1 · App 19/127,406
A SWIM STROKE ANALYSIS METHOD AND SYSTEM
Publication
Application
Classifications
IPC Classifications
CPC Classifications
Applicants
BUTLER 88 PTY LTD
Inventors
Niall WELLS, Brody RAMPONO, Dmytro BYSTRANOVSKYY
Abstract
Swim stroke is analysed by taking measurements from an arm-worn device. The measurements comprise accelerometer measurements with respect to time. The beginning of each stroke is computed with respect to time from the measurements, where the beginning of each stroke is the time of hand entry.
Get a summary, plain-language explanation, or ask your own question.
Figures
Description
BACKGROUND
[0001]The following discussion of the background art is intended to facilitate an understanding of the present invention only. It should be appreciated that the discussion is not an acknowledgement or admission that any of the material referred to was part of the common general knowledge as at the priority date of the application.
[0002]Swimmers from athletes to casual like to, or indeed in some cases need to, monitor their swim stroke performance, particularly when they are seeking to improve their stroke technique.
[0003]Current wrist worn smart devices provide some ability to monitor movement, such as steps, but currently have limited functionality for monitoring swimming. Such devices include Apple™ watches and Garmin™ fitness watches. The present functionality of such devices is to recognise the movement of the wrist is that of performing a swim stroke and then to count the number of strokes in a time period to be able to determine a rate of strokes. It is desirable to determine different phases of the stroke for technique analysis and training. Mere identification of a stroke is insufficient to analyse swim stroke technique.
[0004]The present invention seeks to provide improved or alternative swim stroke analysis to what is currently available.
[0005]Throughout the specification unless the context requires otherwise, the word “comprise” or variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.
[0006]Throughout the specification unless the context requires otherwise, the word “include” or variations such as “includes” or “including”, will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.
[0007]Discussion of any known information or prior patent documents herein is not intended to be regarded as an admission that that information or document forms part of the common general knowledge of a person skilled in the art.
SUMMARY OF THE INVENTION
- [0009]taking measurements from an arm-worn device, wherein the measurements comprise accelerometer measurements with respect to time; and
- [0010]computing the beginning of each stroke with respect to time from the measurements, wherein the beginning of each stroke is the time of hand entry.
[0011]In an embodiment the measurements further comprise gyroscope measurements with respect to time.
[0012]In an embodiment the method further comprises computing segments of each stroke.
[0013]In an embodiment the method further comprises computing stroke kinetics.
[0014]In an embodiment the method comprises computing hand roll angle at hand entry.
[0015]In an embodiment the method further comprises computing stroke crossover.
[0016]In an embodiment the method comprises computing peak acceleration at hand entry and the time thereof.
[0017]In an embodiment the method further comprises computing stroke kinematics.
[0018]In an embodiment the method further comprises computing stroke count and period.
[0019]In an embodiment the method further comprises computing a rest.
[0020]In an embodiment the method further comprises defining a workout and computing a comparison of the strokes to the defined workout.
[0021]In an embodiment computing the beginning of each stroke comprises determining a peak acceleration. Preferably the peak acceleration is the Euclidean norm of measured acceleration vectors.
[0022]In an embodiment determining the peak acceleration comprises removing false positive peaks in acceleration. In an embodiment the removing false positive peaks in acceleration comprises determining the peak within a selected time period. In an embodiment determining the peak acceleration comprises focusing on peaks in a selected time period. In an embodiment focusing on peaks in the selected time period comprises boosting acceleration signal within the selected time period.
[0023]In an embodiment computing the beginning of each stroke comprises determining a negative rotational velocity during a recovery phase of each stroke.
[0024]In an embodiment computing the beginning of each stroke comprises determining a peak in amplitude of a snap signal over time.
[0025]In an embodiment the snap signal is boosted to determine the peak in amplitude.
[0026]In an embodiment the snap signal is boosted by determining a rolling standard deviation of each orthogonal acceleration and scaling the snap signal by the determined rolling standard deviation.
[0027]In an embodiment the snap signal is boosted by a proximity signal. In an embodiment the proximity signal is a triangular wave having a period. In an embodiment the period is determined by identifying a prominent repeating low frequency signal in the acceleration signal. In an embodiment the peak of the triangular wave coincides with the peak in the amplitude of the repeating low frequency signal.
[0028]In an embodiment the stroke start points are determined by determining prominent local maxima of the boosted snap signal.
[0029]In an embodiment stroke period of each stroke is determined by computing the time between stroke start times.
[0030]In an embodiment the method comprises computing temporal phases in each stroke.
[0031]In an embodiment an insweep phase is computed. In an embodiment computing the insweep phase comprises computing the maximum value of acceleration in the y axis during each stroke period.
[0032]In an embodiment a catch phase is computed. In an embodiment computing the catch phase comprises computing jerk. In an embodiment the jerk is applied with a low pass filter. In an embodiment the catch phase ends when the filtered jerk is negative.
[0033]In an embodiment an upsweep phase is computed. In an embodiment computing the upsweep phase comprises computing a peak in acceleration in the x axis.
[0034]In an embodiment a recovery phase is computed. In an embodiment computing the recovery phase comprises computing when the acceleration in the y axis is negative.
[0035]In an embodiment the roll angle entry is computed by determining constituent y and z axis accelerations at the time of the beginning of each stroke and then determining theta and omega angles from the magnitude of the accelerations.
[0036]In an embodiment the crossover angle of entry is computed by determining the amount of radial-ulnar rotation in the z axis from wrist entry to insweep. In an embodiment the radial-ulnar rotation in the z axis is computed by taking the integral of angular velocity in the z axis.
[0037]In an embodiment a splash score is computed. In an embodiment the splash score is computed from absolute acceleration at the time of entry.
[0038]In an embodiment a force score is computed. In an embodiment the force score is computed by determining the maximum magnitude of acceleration after the stroke begins.
[0039]In an embodiment the computed values are displayed to a user. In an embodiment the values are displayed in a graph against time.
[0040]In an embodiment the hand entry angle and/or the cross over angle are graphically displayed by count at each angle.
[0041]In an embodiment the measurements are recorded on a wrist device. In an embodiment the wrist device transfers data comprising the measurement to a cloud based analysis engine. In an embodiment the analysis engine performs at least some of the computations.
[0042]In an embodiment the computations are time-aligned with biometric readings (eg. heart rate).
[0043]In an embodiment the graphical displays are computed by the computation engine and provided as graphical display instructions to a viewing device.
- [0045]taking measurements from an arm-worn device, wherein the measurements comprise accelerometer measurements with respect to time; and
- [0046]computing the time of hand entry during each stroke from the measurements.
- [0048]a receiver of measurements with respect to time from an arm-worn device, wherein the measurements comprise accelerometer measurements and preferably gyroscope measurements;
- [0049]a processor for computing the beginning and end of each stroke with respect to time from the measurements, wherein the beginning of each stroke is the time of hand entry.
- [0051]an arm-worn device comprising an accelerometer coupled to an arm of a user for taking acceleration measurements over time and preferably a gyroscope for taking angular velocity measurements over time;
- [0052]a transmitter of the measurements;
- [0053]a receiver of the measurements;
- [0054]a processor for computing the beginning and end of each stroke with respect to time from the measurements, wherein the beginning of each stroke is the time of hand entry.
[0055]According to an embodiment of the present invention there is provided a computer program embodied in a non-transient storage form comprising instructions for controlling a processor to perform the method defined above or to operate as or as part of the system as defined above.
DESCRIPTION OF FIGURES
[0056]In order to provide a better understanding of the present invention example embodiments will now be described in which:
[0057]
[0058]
[0059]
[0060]
[0061]
[0062]
[0063]
[0064]
[0065]
[0066]
[0067]
[0068]
[0069]
[0070]
[0071]
[0072]
[0073]
[0074]
[0075]
[0076]
[0077]
[0078]
DESCRIPTION OF EXAMPLES
[0079]Referring to
[0080]The watch 12 is able to connect to a smartphone 14, such as an Iphone™ which in turn communicates with an online server, such as a cloud server 20. The watch 16 may be able to communicate over the internet with a dedicated/proprietary server 18, in some embodiments via an intermediary device 17, 17′ (such as a smartphone 17 or personal computer 17′), from which the data may be obtained and sent to or retrieved by the cloud server 20. The server 20 comprises a webserver to display information based on the collected data on a display device 22, such as a web browser running on a computer. The information may also be displayed on the phone 14 or another device (such as device 17, 17′).
[0081]Any of devices 12, 14, 16, 17, 17′, 18, 20, 22 could perform all or some of the computations described herein, however it is preferred and is described herein as the server 20 performs the computations as programmed by computer program instructions in order to handle performing such computations from multiple devices 12, 16 or their respective users 30, 32, scaled accordingly so that the computational load of many user's devices can be centralised and performed computationally efficiently. The computer program instructions are typically stored in a non-volatile manner, such as on a hard disk drive or solid state drive but may be stored on other media. The computer program instructions control the or each processor of the server 20 to perform some or all of the computations described herein and in an embodiment to prepare instructions (such as a website) for display on the phone 14, device 17, 17′, or display device 22.
[0082]As described below, the computations allow for analysis of the swimming technique at macro and micro levels. This enables the user to better understand their technique and improve it. The invention enables self analysis, but also can be used for computational analysis, such as identification of a deficiency in technique (for example hand entry angle is too great), and can thus enable a market place 34 for either computational diagnostic tools, or self improvement instructions (such as instructional videos) and can provide the analysis to a coach 36 for instruction during training.
[0083]The starting point of the analysis is to identify the beginning of each stroke (not merely that a stroke is being performed or of the type of stroke being performed). As shown in
[0084]In the present invention, each stroke is regarded as beginning at hand entry (into the water), which in freestyle transitions into the glide phase of the stroke once the hand has entered the water. The impact of the hand with the water creates acceleration changes that are measurable. Once water entry is completed the glide occurs when the outstretched hand moves through the water and turbulence from the hand entry subsides.
[0085]As each stroke repeats, it is possible to recognise a stroke by the pattern of repetition of the acceleration experienced by the device 12, 16. However, using that technique will only identify the duration and start/finish of each stroke.
[0086]Also shown in
[0087]This feedback is valuable to coaches and swimmers to understand how long they are spending in each phase of the stroke and for the stroke overall. Integrated with data labelling the user can see the affect of swim equipment like fins and paddles and provide more detailed analysis about or removing these laps and strokes from analysis.
[0088]In order to segment each stroke into these phases the accelerometer measurements and preferably gyroscope measurements require complex analysis, as described by example below.
[0089]On a WatchOS™ device 12, an application is installed to create a motion file from the watch's accelerometer and gyroscope (if available) sensors. This file is distinct from the device operating system's activity data, so that raw data can be processed using the present invention, instead of or in addition to using the device's interpretation of the raw data. The motion file is compressed and transferred to the phone 14. The watch application can also be used to provide instruction information to the user 30 during a training session.
[0090]On an iOS device (such as phone 14), an application is installed to receive the file from the watch 12 to interface with the cloud server 20 and to provide the user interface on the phone 14.
[0091]On a Garmin device 16, raw data may not be able to be accessed. Instead, the raw data is obtained from the Garmin cloud server 18. Instruction information can be provided to the user 32 by interfacing with Garmin's training API to push workouts to the Garmin device 16.
[0092]Other brand devices having at least a 3 axis accelerometer could be used if the raw accelerometer data (or acceptably usable data) is able to be accessed.
[0093]Lap beginning/end (in time) can be calculated by the detection of the turnaround of the swimmer between laps. Alternatively, a button push can be used to identify a lap transition.
[0094]A rest can be identified by using a combination of device activity, motion and swimmer's historical pace to estimate and automatically apply a rest. The rest sensitivity can be tuned based on the pace of the swimmer's functional threshold pace (FTP), set in the application or defaulted to 2:00/100 m. At default, rests need to be greater than ˜7 seconds for rest identification for those with faster FTPs, rest segmentation can be as small as ˜4 seconds. This significantly improves the application experience for faster swimmers who will often use sub 5 second rest as part of training programs.
[0095]The application on the phone 14 or the display device 22 can show kinetic and kinematic information.
[0096]In
[0097]In
[0098]The kinematic information such as pull force and entry splash can also be displayed to the user 30, 32/coach 36. The stroke count per lap/displace and person of each stroke (or average stroke period) can also be displayed, as exemplified in
- [0100]ax: proximal to distal.
- [0101]ay: medial (ulnar) to lateral (radial).
- [0102]az: palmar to dorsal.
- [0104]gx: wrist pronation to supination.
- [0105]gy: shoulder extension to flexion.
- [0106]gz: shoulder adduction to abduction.
Stroke Start Point
[0108]In the example described, the point of entry into the water of the user's hand is defined as the start point of each stroke.
[0109]Define the magnitude of acceleration a*(t) as the Euclidean norm of the component acceleration vectors by:
[0110]
[0111]A peak detection algorithm is used to find the prominent local maxima of a*(t), which are plotted in
[0112]This technique alone can be insufficient for finding all strokes. For example, swimmers with gentle hand entry may not produce acceleration peaks large enough to be detected, or sharp, forceful propulsions through the water may be incorrectly identified as a stroke start point. Thus false positive acceleration peaks might be detected. To address this, additional methods can be used to limit the range in which acceleration peaks are searched for. In embodiments, the method used depends on whether gyroscope signal is available on the device (as is the case for device 12, but may not be the case for device 14).
Where Angular Velocity is Available from a Gyroscope
[0113]Angular velocity gy(t) follows an oscillating signal with a period IT of one stroke length, with positive rotational velocity during the propulsive phase of the stroke, and negative rotational velocity during the recovery phase of the stroke. Example gy(t) data is demonstrated in
[0114]A Butterworth low-pass filter is applied to the gy(t) to produce g*(t). An example of which is shown in
[0115]Stroke start points S are found by finding the maximum of a*(t) in a window of length & following these local minima, as follows:
[0116]An example of the computed beginning of each stroke determined by the maximum acceleration following a peak in negative rotational velocity (during a recovery phase of each stroke) is shown in
Where Angular Velocity is not Available
[0117]In the absence of a gyroscope signal, the snap signal (second derivative of the acceleration signal) can be used to find stroke start points. Define the snap signal s(t) as the second derivative of the acceleration signal a*(t) with respect to time, as follows:
[0118]The signal s(t) is shown alongside a*(t) in
[0119]The snap signal has high amplitude during periods of rapidly changing acceleration, such as the hand entry into the water. It is preferred to boost this signal to aid in estimation of stroke start points.
Signal Boosting with S(t)
[0120]Fluctuation in the a*(t) is lowest in the period immediately after hand entry, where the hand glides into the water. For each time t and window size &, define the set Tt,ε as the set of all discrete time points in the signal a*(t) within a window of size centred on t:
[0121]For each axis, the rolling standard deviation σ(t) centred on time t with window size of ε is given by:
[0122]Define S(t) as the sum of the σ(t) in each axis:
[0123]An example of the signal S(t) is shown in
[0124]The signal S(t) is then scaled to a range of [0, 1] using:
Sscaled(t) is shown in
[0125]The boosted snap signal, sboost(t), is then found by multiplying (elementwise) the snap signal by Sscaled(t):
[0126]The signal sboost(t) is shown alongside a*(t) in
Signal Boosting with P(t)
[0127]Strokes are expected to occur with a consistent frequency within a lap. This is used to further boost the snap signal in times where the next stroke would be expected. This is particularly effective at removing outliers due to irregular timing.
[0128]A fourier transformation is applied to the a*(t) signal, and the prominent repeating low frequency signal is used to estimate the stroke period πstroke.
[0129]Define the proximity signal P(t) as the triangle wave spanning the range [0, 1], with period πstroke, and peaks fit to the prominent peaks of the sboost(t). The signal P(t) is shown alongside sboost(t) in
[0130]The double-boosted snap signal is then found by multiplying (elementwise) the boosted snap signal by P(t):
[0131]The signal sdouble-boost(t) is shown alongside a*(t) in
[0132]Stroke start points are determined as the prominent local maxima of sbouble-boost(t) using a peak-detection algorithm. These peaks are shown in
Stroke Period
[0133]Stroke period is calculated as the time between adjacent stroke start times. This is in contrast to simply dividing stroke count by distance.
Stroke Temporal Phases
Insweep
[0135]The insweep phase is characterised by elbow flexion and acceleration in the y-axis towards the midline. The start of insweep tinsweep is found by taking the maximum value of ay(t) during each stroke period as follows:
[0136]An example of the maximum value of acceleration in the y axis during each stroke period is shown in
Catch
[0137]The catch phase is characterised by a period of reduced acceleration after hand entry, followed by an increase in acceleration during the propulsive phase of the stroke. Jerk, the derivative of acceleration magnitude with respect to time j(t), is found as follows:
[0138]A low-pass filter is applied to j(t) to produce j*(t). The last point between the stroke start and insweep start in which j*(t) is negative is labelled as tcatch; the point of the catch phase ending.
[0139]The points at which j*(t) are negative are shown in
Upsweep
[0140]Following insweep, the upsweep phase is characterised by proximal movement of the wrist towards the shoulder as the hand pushes out behind the swimmer. The local minimum in ax(t) after insweep is labelled as the beginning of the upsweep phase tupsweep, which is found using a peak-detection algorithm. The detected peaks, and thus the beginning of the upsweep phase, are shown in
Recovery
[0141]Following upsweep, the recovery phase is characterised by the shift from elbow flexion to elbow extension as it emerges from the water. The first point in which ay(t) is negative after tupsweep is labelled as the beginning of the recovery phase trecovery.
[0142]The detected points of negative ay(t) are shown in
Kinetics
Roll Angle of Entry of Hand
[0143]The peak in a*(t) at the start of each stroke is a result of the water exerting a normal force on the accelerometer in the device 12, 16 orthogonal to the water's surface. Assuming the water surface is stable, this force is parallel to the direction of gravity. The roll angle φ(t) of the wrist at hand entry can be determined by considering the constituent vectors ay(t) and az(t), as follows:
Crossover Angle of Entry
[0144]Assuming that the wrist is pointing downwards during the insweep phase, we can estimate the crossover angle ψ(t) (wrist yaw) at hand entry by calculating the amount of radial-ulnar rotation in the z axis from wrist entry tstart to insweep tinsweep. This is found through the integral of gz, as follows:
Kinematics
Splash Score
[0145]The peak in a*(t) at the point of hand entry t0 is caused by the deceleration of the accelerometer caused by impact with the water. The splash-score Fsplash is given by the value of a*(to), and measures how smooth hand entry is.
Force Score
[0146]The maximum value of a*(t) during the stroke (after hand entry) reflects a peak in acceleration during the propulsive phase of the stroke. The force score Fpropulsion is a measure of this maximum, and reflects the force exerted during propulsion.
Use of Phase Segmentation, Kinetics and Kinematics
[0147]In an embodiment the method further comprises defining a workout and computing a comparison of the strokes to the defined workout. Stroke phases that take longer or shorter than the defined stroke workout can be highlighted for stroke improvement. Likewise measured kinematics and kinetics can be compared to workout defines kinematics and kinetics can be used for stroke technique coaching.
[0148]In an embodiment the computed values are displayed to a user. In an embodiment the values are displayed in a graph against time.
[0149]In and embodiment the hand entry and cross over angle are graphically displayed by count at each angle and may be compared graphically to the defined angles desired for the specified technique.
[0150]In an embodiment the computations are time-aligned with biometric readings (eg heart rate). This allows assessment of and thus coaching of the work rate, swim pace and stamina of the swimmer, amongst other things. Beginning of the swim can be recorded by a button press on the device 12, 16 or by the hand slapping a hard surface (such as the edge of a pool) or by palm slapping the surface of the water, which will produce a significantly greater acceleration change due to the slap in comparison to the hand entry acceleration change.
[0151]In an embodiment the graphical displays are computed by the computation engine and provided as graphical display instructions to a viewing device. The display may provide aggregated report where the longitudinal view of the swim is displayed and can show changes over time. For example, a swim from 3 months ago had a cross over angle of 15 degrees, which has been progressively worked on and shows progression to a current cross over angle of 10 degrees. This view will be of benefit to show improvement in stroke technique as well as being able to see benefits in different coaching techniques.
[0152]Coaching services, such as videos can be provided and coach matching/finding can be provided in the phone or web application.
[0153]In some freestyle swimming guides hand entry is part of the recovery phase at the end of a stroke before the glide commences. However, since the phases must be computed from measurements, the present invention does not regard the hand entry as a part of the recovery phase. Instead, hand entry is regarded as the beginning of the stroke which starts the glide phase. In practical terms this distinction is irrelevant as the end of one stroke is followed by the beginning of the next, with an insignificant timing difference between the transition between the strokes. However, computationally this provides an unexpected advantage because hand entry is detectable with good accuracy as described herein. For example, in WO2018045211 glide is determined by pitch being near zero and acceleration is 10% of maximum, which is believed to be less accurate.
[0154]Aspects of the invention may be applicable to swim stroke types where the hand is out of the water and then enters the water, in addition to freestyle stroke, such as butterfly and back stroke, where the computation of hand entry is used to determine the beginning of a stroke with respect to time. Other freestyle applicable aspect
[0155]Modifications may be made to the present invention within the context of that described and shown in the drawings. Such modifications are intended to form part of the invention described in this specification.
Claims
1. A swim stroke analysis method comprising:
taking measurements from an arm-worn device, wherein the measurements comprise accelerometer measurements with respect to time; and
computing a time of each hand entry from the accelerometer measurements and from the time of each hand entry determining the beginning of each stroke with respect to time.
2. A swim stroke analysing system comprising:
a receiver of measurements with respect to time from an arm-worn device, wherein the measurements comprise accelerometer measurements and preferably gyroscope measurements;
a processor for computing a time of each hand entry from the accelerometer measurements and from the time of each hand entry determining the beginning and end of each stroke with respect to time.
3. A swim stroke analysing system comprising:
an arm-worn device comprising an accelerometer coupled to an arm of a user for taking acceleration measurements over time and preferably a gyroscope for taking angular velocity measurements over time;
a transmitter of the measurements;
a receiver of the measurements;
a processor for computing a time of each hand entry from the accelerometer measurements and from the time of each hand entry determining the beginning and end of each stroke with respect to time.
4. (canceled)
5. (canceled)
6. (canceled)
7. The method according to
8. The method according to
9. The method according to
10. The method according to
11. The method according to
12. The method according to
13. (canceled)
14. The method according to
15. (canceled)
16. The method according to
17. The method according to
18. (canceled)
19. The method according to
20. The method according to
21. The method according to
22. (canceled)
23. The method according to
24. The method according to
25. The method according to
26. The method according to
27. The method according to
28. The method according to
29. The method according to
30. (canceled)
31. The method according to
32. The method according to
33. (canceled)