Digital Surgery Limited

An aspect includes model merging for surgical workflow mapping. Cases with one or more rare phases can be identified and removed. Remaining cases without the one or more rare phases can be clustered to form principal workflows. The principal workflows can be used to create a flow diagram for display via a user interface.

A location of an anatomical structure in image(s) from a surgical procedure is predicted using machine learning. An image/video capture device such as an endoscope, a wearable camera, a stationary camera, etc., can be used to capture the image(s). A confidence score of the prediction of the machine learning is determined. A surgeon can be provided an augmented visualization of the surgical procedure by displaying one or more graphical overlays based on the findings of the machine learning to enhance the surgeon-s information.

A robotic surgical simulation system for clinical workplace simulation includes a virtual reality headset configured to display the clinical workplace simulation, a processor, and a memory. The memory includes instructions stored thereon, which, when executed by the processor, cause the robotic surgical simulation system to: generate a virtual operating room within a three-dimensional coordinate space. The virtual operating room includes a virtual surgical console. The instructions when executed by the processor further cause the system to compute, based on the captured image, a position of the user within the three-dimensional coordinate space relative to the virtual surgical console; and determine whether a user is engaged with or disengaged from the virtual surgical console based on the computed position.

Data captured during a surgical procedure can include multiple video streams, such as from an endoscopic or microscopic camera, an external camera, etc., along with data from one or more instruments used during the surgical procedure. Transferring the surgical data over communication networks, and storing the surgical data, is, therefore, resource-intensive and a technical challenge. Technical solutions are described to optimize the surgical data by intelligent compression methods that can adaptively reduce the size of certain procedural workflows. In some aspects, the compression is tunable so that different surgical phases or surgical actions have variable fidelity in any use of the data after compression.

Data captured during a surgical procedure can include multiple video streams, such as from an endoscopic camera, an external camera, etc., along with data from one or more instruments used during the surgical procedure. Transferring the surgical data over communication networks, and storing the surgical data, is therefore resource-intensive, and a technical challenge. Technical solutions are described to optimize the surgical data by intelligent compression methods that can adaptively reduce the size of certain procedural workflows. In some aspects, the compression reduces the storage/transmission resources required for a video catalogue by reducing redundancy from across the several videos stored in the video catalogue. The compression also enables faster querying and searching through a catalogue or collection of archived data.

An aspect includes a computer-implemented method that accesses input data including spatial data and/or sensor data temporally associated with a video stream of a surgical procedure. One or more machine-learning models predict a state of the surgical procedure based on the input data. The one or more machine-learning models detect one or more surgical instruments at least partially depicted in the video stream based on the input data. A state indicator and one or more surgical instrument indicators temporally correlated with the video stream are output.

An aspect includes a computer-implemented method that identifies variations in surgical approaches to medical procedures. Surgical videos documenting multiple cases of a medical procedure are analyzed to identify different surgical approaches used by service providers when performing the medical procedure. According to some aspects surgical phases are identified in each surgical video and groups of similar surgical phase sequences are grouped into surgical approaches.

Aspects include low latency video capture and overlay through video capture circuitry. Video input associated with a surgical procedure can be provided to one or more processing devices of a video processing system to analyze the video input. An overlay received from the one or more processing devices is combined with the video input to create a blended video output of the surgical procedure. The blended video output is output to a primary display using an interface having a lower latency than a secondary display output of the video processing system.

Techniques are described for improving computer-assisted surgical (CAS) systems, particularly, to recognize surgical phases in a video of a surgical procedure. A CAS system includes cameras that provide video stream of a surgical procedure. According to one or more aspects the surgical phases are automatically detected in the video stream using a machine learning model. Particularly, the machine learning model includes a boundary aware cascade stage network to perform surgical phase recognition.