Dr Changyan He

Vitreoretinal surgery is among the most delicate surgical tasks during which surgeon hand tremor may severely attenuate surgeon performance. Robotic assistance has been demonstrated to be beneficial in diminishing hand tremor. Among the requirements for reliable assistance from the robot is to provide precise measurements of system states, e.g., sclera forces, tool tip position, and tool insertion depth. Providing this and other sensing information using existing technology would contribute toward development and implementation of autonomous robot-assisted tasks in retinal surgery such as laser ablation, guided suture placement/assisted needle vessel cannulation, among other applications. In this article, we use a state-estimating Kalman filtering (KF) to improve the tool tip position and insertion depth estimates, which used to be purely obtained by robot forward kinematics (FWK) and direct sensor measurements, respectively. To improve tool tip localization, in addition to robot FWK, we also use sclera force measurements along with beam theory to account for tool deflection. For insertion depth, the robot FWK is combined with sensor measurements for the cases where sensor measurements are not reliable enough. The improved tool tip position and insertion depth measurements are validated using a stereo camera system through preliminary experiments and a case study. The results indicate that the tool tip position and insertion depth measurements are significantly improved by 77% and 94% after applying KF, respectively.

In the Ophthalmic microsurgery, as the contact forces between tool and tissue are too small to perceive and the excessive operating force might lead to tissue damage, an FBG-based two-dimensional micro-force sensor is developed. After the force-wavelength relationship is determined and the temperature effect to the wavelength is discussed, an algorithm is developed to cancel the temperature effect. A calibration experiment is performed on a self-developed micro-force sensor calibration platform (the measuring accuracy is 0.42 mN). The sensor is integrated to the end of an ophthalmic surgical forceps, which was used to perform continuous curvilinear capsulorhexis (CCC) on isolated pig eyeballs to obtain the force-time curves. By analyzing 19 sets of data, the average value of the maximum capsular force was 22.43 mN. The research laid the foundation for precise operation of ophthalmic surgery and micro-force control of surgical robots.

Eye surgery, specifically retinal micro-surgery involves sensory and motor skill that approaches human boundaries and physiological limits for steadiness, accuracy, and the ability to detect the small forces involved. Despite assumptions as to the benefit of robots in surgery and also despite great development effort, numerous challenges to the full development and adoption of robotic assistance in surgical ophthalmology, remain. Historically, the first in-human-robot-Assisted retinal surgery occurred nearly 30 years after the first experimental papers on the subject. Similarly, artificial intelligence emerged decades ago and it is only now being more fully realized in ophthalmology. The delay between conception and application has in part been due to the necessary technological advances required to implement new processing strategies. Chief among these has been the better matched processing power of specialty graphics processing units for machine learning. Transcending the classic concept of robots performing repetitive tasks, artificial intelligence and machine learning are related concepts that has proven their abilities to design concepts and solve problems. The implication of such abilities being that future machines may further intrude on the domain of heretofore "human-reserved" tasks. Although the potential of artificial intelligence/machine learning is profound, present marketing promises and hype exceeds its stage of development, analogous to the seventieth century mathematical "boom" with algebra. Nevertheless robotic systems augmented by machine learning may eventually improve robot-Assisted retinal surgery and could potentially transform the discipline. This commentary analyzes advances in retinal robotic surgery, its current drawbacks and limitations, and the potential role of artificial intelligence in robotic retinal surgery.

Planar flow casting (PFC) is a major method for industrial preparing amorphous ribbon. The key factors that affect ribbon formation in PFC are temperature and thermal stable time of cooling roller. This study proposes a new approach to investigate the transient three-dimensional temperature of the cooling roller using variable heat flux as the boundary condition. Firstly, the heat flux acted on the roller outer wall is calculated by two-dimensional heat transfer model. Employing the heat flux as the boundary condition, the three-dimensional transient temperature of the cooling roller is then analyzed. This study also examines the effects of the roller thickness and diameter on the temperature and stable time. Results show that the heat conduction in the roller not only in the radial direction but also in the width direction. The temperature is unevenly distributed, reaching the peak value in the middle width position, and the roller achieves thermal equilibrium after about 120 s of casting. In addition, the roller thickness mainly affects the stable time and the temperature of the inner wall; the diameter affects the stable time as well as the temperatures of both the inner and outer walls. Finally, the reliability of the simulation approach and results are verified by measuring the roller transient evolution temperature on the spot with the infrared thermal imager. This study provides the reference for the selection of roller parameters and the prediction of roller stable time during the PFC process.

Retinal surgery continues to be one of the most technical demanding surgeries for its high manipulation accuracy requirement, small and constrained workspace, and delicate retinal tissue. Robotic systems have the potential to enhance and expand the capabilities of surgeons during retinal surgery. Thus, focusing on retinal vessel bypass surgery, a master-slave robot system is developed in this paper. This robotic system is designed based on characteristics of retinal vascular bypass surgery and analysis of the surgical workspace in eyeball. A novel end-effector of two degrees of freedom is designed and a novel remote center of motion mechanism is adopted in the robot structure. The kinematics and the mapping relationship are then established, the gravity compensation control strategy and the hand tremor elimination algorithm are applied to achieve the high motion accuracy. The experiments on an artificial eyeball and an in vitro porcine eye are conducted, verifying the feasibility of this system.

Planar flow casting (PFC) is a primary method for preparing an amorphous ribbon. The qualities of the amorphous ribbon are significantly influenced by the temperature and thermal expansion of the cooling roller. This study proposes a new approach to analyze the three-dimensional temperature and thermal expansion of the cooling roller using variable heat flux that acted on the cooling roller as a boundary condition. First, a simplified two-dimensional model of the PFC is developed to simulate the distribution of the heat flux in the circumferential direction with the software FLUENT. The resulting heat flux is extended to be three-dimensional in the ribbon¿s width direction. Then, the extended heat flux is imported as the boundary condition by the CFX Expression Language, and the transient temperature of the cooling roller is analyzed in the CFX software. Next, the transient thermal expansion of the cooling roller is simulated through the thermal¿structural coupling method. Simulation results show that the roller¿s temperature and expansion are unevenly distributed, reach the peak value in the middle width direction, and the quasi-steady state of the maximum temperature and thermal expansion are achieved after approximately 50¿s and 150¿s of casting, respectively. The minimum values of the temperature and expansion are achieved when the roller has a thickness of 45¿mm. Finally, the reliability of the approach proposed is verified by measuring the roller¿s thermal expansion on the spot. This study provides theoretical guidance for the roller¿s thermal expansion prediction and the gap adjustment in the PFC.

Scleral flap preparation is one of the basic operations in glaucoma surgery including trabeculectomy, glaucoma drainage implants (GDIs), and canaloplasty. In most cases, scleral flap preparation is performed manually by surgeons based on experience, which is various from person to So far, few robot-Assisted scleral flap operations are applied. In this paper, a scleral flap drilling system is developed by integrating a force/torque sensor and an arc-shaped drilling tool. The drilling tool is designed based on a corneal trephine. The maximum force and torque are derived theoretically. Scleral flap preparation experiments are then performed on cadaveric porcine eyes. The maximum vertical cutting force and the difference of two continues peaks of torque are recorded and analyzed under various linear and angular velocities, and an empirical polynomial model is used to fit the correlation between the penetration force and the cutting depth. The results show that the maximum vertical cutting force and the penetration force occurred at different location. About 95% of the penetration force is observed below 2 N.

Retinal surgery is a complex activity that can be challenging for a surgeon to perform effectively and safely. Image guided robot-Assisted surgery is one of the promising solutions that bring significant surgical enhancement in treatment outcome and reduce the physical limitations of human surgeons. In this paper, we demonstrate a novel method for 3D guidance of the instrument based on the projection of spotlight in the single microscope images. The spotlight projection mechanism is firstly analyzed and modeled with a projection on both a plane and a sphere surface. To test the feasibility of the proposed method, a light fiber is integrated into the instrument which is driven by the Steady-Hand Eye Robot (SHER). The spot of light is segmented and tracked on a phantom retina using the proposed algorithm. The static calibration and dynamic test results both show that the proposed method can easily archive 0.5 mm of tip-To-surface distance which is within the clinically acceptable accuracy for intraocular visual guidance.

Retinal vein cannulation is a promising approach for treating retinal vein occlusion that involves injecting medicine into the occluded vessel to dissolve the clot. The approach remains largely unexploited clinically due to surgeon limitations in detecting interaction forces between surgical tools and retinal tissue. In this paper, a dual force constraint controller for robot-assisted retinal surgery was presented to keep the tool-to-vessel forces and tool-to-sclera forces below prescribed thresholds. A cannulation tool and forceps with dual force-sensing capability were developed and used to measure force information fed into the robot controller, which was implemented on existing Steady Hand Eye Robot platforms. The robotic system facilitates retinal vein cannulation by allowing a user to grasp the target vessel with the forceps and then enter the vessel with the cannula. The system was evaluated on an eye phantom. The results showed that, while the eyeball was subjected to rotational disturbances, the proposed controller actuates the robotic manipulators to maintain the average tool-to-vessel force at 10.9 mN and 13.1 mN and the average tool-to-sclera force at 38.1 mN and 41.2 mN for the cannula and the forcpes, respectively. Such small tool-to-tissue forces are acceptable to avoid retinal tissue injury. Additionally, two clinicians participated in a preliminary user study of the bimanual cannulation demonstrating that the operation time and tool-to-tissue forces are significantly decreased when using the bimanual robotic system as compared to freehand performance.

A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina in order to perform the tool-navigation task, which can be prone to human error. To alleviate such uncertainty, prior work has introduced ways to assist the surgeon by estimating the tooltip distance to the retina and providing haptic or auditory feedback. However, automating the tool-navigation task itself remains unsolved and largely unexplored. Such a capability, if reliably automated, could serve as a building block to streamline complex procedures and reduce the chance for tissue damage. Towards this end, we propose to automate the tool-navigation task by learning to mimic expert demonstrations of the task. Specifically, a deep network is trained to imitate expert trajectories toward various locations on the retina based on recorded visual servoing to a given goal specified by the user. The proposed autonomous navigation system is evaluated in simulation and in physical experiments using a silicone eye phantom. We show that the network can reliably navigate a needle surgical tool to various desired locations within 137 µm accuracy in physical experiments and 94 µm in simulation on average, and generalizes well to unseen situations such as in the presence of auxiliary surgical tools, variable eye backgrounds, and brightness conditions.

Retinal vein cannulation is a promising treatment for retinal vein occlusion that involves the injection of an anticoagulant directly into the occluded vein to dissolve the blockage. However, excessive forces applied by the injection tool during the procedure, at either the scleral incision or injection site, can result in injury to the eye. Furthermore, the force required to puncture retinal veins (around 10 mN) is well below human sensing ability and an order of magnitude smaller than those that can be safely applied at the sclera (around 100 mN). Detection and management of tool-to-tissue forces on these different scales are some of the most challenging aspects of the cannulation procedure. This work describes the development of a sensorized cannulation tool capable of detecting both tool-to-vein puncture forces and tool-to-sclera contact forces. By combining two materials, nitinol alloy for the tool tip and stainless steel for the tool shaft, to achieve dual stiffness, the tool possesses a flexible tip to capture small vein puncture forces and a stiffer shaft to maintain straightness during use. Three segments of fiber Bragg grating sensors are calibrated to measure the transverse forces at both the tool tip and sclerotomy, as well as to determine the tool insertion depth within the eye. The results of the validation experiments show that the root mean square error of the measurements for the force at the tip, the force at the sclerotomy, and the tool position are 0.70 mN, 1.59 mN, and 0.69 mm, respectively.

To overcome the surgeon's manual operation limitations in vitreoretinal microsurgery, many kinds of robots are developed to enhance the operation accuracy, including Master-Slave Teleoperation Robot (MSTR), Hand-Held Robot (HHR), Cooperative Robot (CR) and Micro Robot (MR). With the introduction of vitreoretinal surgical assisted robots (VRSARs), the surgeon's operation skills can be improved through various functions of the robots, such as depth information perception, improvement of the surgical precision, and micro-force perception and feedback. After description and classification of the latest research progress of VRSARs, the current problems of each type of VRSARs were discussed and future works summarized for further research.

One of the significant challenges of moving from manual to robot-assisted retinal surgery is the loss of perception of forces applied to the sclera (sclera forces) by the surgical tools. This damping of force feedback is primarily due to the stiffness and inertia of the robot. The diminished perception of tool-to-eye interactions might put the eye tissue at high risk of injury due to excessive sclera forces or extreme insertion of the tool into the eye. In the present study therefore a 1-dimensional adaptive control method is customized for 3-dimensional control of sclera force components and tool insertion depth and then implemented on the velocity-controlled Johns Hopkins Steady-Hand Eye Robot. The control method enables the robot to perform autonomous motions to make the sclera force and/or insertion depth of the tool tip to follow pre-defined desired and safe trajectories when they exceed safe bounds. A robotic light pipe holding application in retinal surgery is also investigated using the adaptive control method. The implementation results indicate that the adaptive control is able to achieve the imposed safety margins and prevent sclera forces and insertion depth from exceeding safe boundaries.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. During surgery the tool needs to be inserted into the eyeball while maintaining constant contact with the sclera. Any unexpected manipulation could cause extreme tool-sclera contact force (scleral force) thus damage the sclera. The introduction of robotic assistance could enhance and expand the surgeon's manipulation capabilities during surgery. However, the potential intra-operative danger from surgeon's mis-operations remains difficult to detect and prevent by existing robotic systems. Therefore, we propose a method to predict imminent unsafe manipulation in robot-assisted retinal surgery and generate feedback to the surgeon via auditory substitution. The surgeon could then react to the possible unsafe events in advance. This work specifically focuses on minimizing sclera damage using a force-sensing tool calibrated to measure small scleral forces. A recurrent neural network is designed and trained to predict the force safety status up to 500 milliseconds in the future. The system is implemented using an existing 'steady hand' eye robot. A vessel following manipulation task is designed and performed on a dry eye phantom to emulate the retinal surgery and to analyze the proposed method. Finally, preliminary validation experiments are performed by five users, the results of which indicate that the proposed early warning system could help to reduce the number of unsafe manipulation events.

Surgeon hand tremor limits human capability during microsurgical procedures such as those that treat the eye. In contrast, elimination of hand tremor through the introduction of microsurgical robots diminishes the surgeons tactile perception of useful and familiar tool-to-sclera forces. While the large mass and inertia of eye surgical robot prevents surgeon microtremor, loss of perception of small scleral forces may put the sclera at risk of injury. In this paper, we have applied and compared two different methods to assure the safety of sclera tissue during robot-assisted eye surgery. In the active control method, an adaptive force control strategy is implemented on the Steady-Hand Eye Robot in order to control the magnitude of scleral forces when they exceed safe boundaries. This autonomous force compensation is then compared to a passive force control method in which the surgeon performs manual adjustments in response to the provided audio feedback proportional to the magnitude of sclera force. A pilot study with three users indicate that the active control method is potentially more efficient.

Vitreoretinal surgery is among the most challenging microsurgical procedures as it requires precise tool manipulation in a constrained environment, while the tool-tissue interaction forces are at the human perception limits. While tool tip forces are certainly important, the scleral forces at the tool insertion ports are also important. Clinicians often rely on these forces to manipulate the eyeball position during surgery. Measuring sclera forces could enable valuable sensory input to avoid tissue damage, especially for a cooperatively controlled robotic assistant that otherwise removes the sensation of these familiar intraoperative forces. Previously, our group has measured sclera forces in phantom experiments. However, to the best of our knowledge, there are no published data measuring scleral forces in biological (ex-vivo/in-vivo) eye models. In this paper, we measured sclera forces in ex-vivo porcine eye model. A Fiber Bragg Grating (FBG) based force sensing instrument with a diameter of ~900 µm and a resolution of ~1 mN was used to measure the forces while the clinician-subject followed retinal vessels in manual and robot-assisted modes. Analysis of measured forces show that the average sclera force in manual mode was 133.74 mN while in robot-assisted mode was 146.03 mN.

Retinal surgery involves manipulating very delicate tissues within the confined area of eyeball. In such demanding practices, patient involuntary head movement might abruptly raise tool-to-eyeball interaction forces which would be detrimental to eye. This study is aimed at implementing different force control strategies and evaluating how they contribute to attaining sclera force safety while patient head drift is present. To simulate patient head movement, a piezoelectric-actuated linear stage is used to produce random motions in a single direction in random time intervals. Having an eye phantom attached to the linear stage then an experienced eye surgeon is asked to manipulate the eye and repeat a mock surgical task both with and without the assist of the Steady-Hand Eye Robot. For the freehand case, warning sounds were provided to the surgeon as auditory feedback to alert him about excessive slclra forces. For the robot-assisted experiments two variants of an adaptive sclera force control and a virtual fixture method were deployed to see how they can maintain eye safety under head drift circumstances. The results indicate that the developed robot control strategies are able to compensate for head drift and keep the sclera forces under safe levels as well as the free hand operation.

One of the major yet little recognized challenges in robotic vitreoretinal surgery is the matter of tool forces applied to the sclera. Tissue safety, coordinated tool use and interactions between tool tip and shaft forces are little studied. The introduction of robotic assist has further diminished the surgeon's ability to perceive scleral forces. Microsurgical tools capable of measuring such small forces integrated with robotmanipulators may therefore improve functionality and safety by providing sclera force feedback to the surgeon. In this paper, using a force-sensing tool, we have conducted robotassisted eye manipulation experiments to evaluate the utility of providing scleral force feedback. The work assesses 1) passive audio feedback and 2) active haptic feedback and evaluates the impact of these feedbacks on scleral forces in excess of aboundary. The results show that in presence of passive or active feedback, the duration of experiment increases, while the duration for which scleral forces exceed a safe threshold decreases.

The performance of retinal microsurgery often requires the coordinated use of both hands. During bimanual retinal surgery, dominant hand performance may be negatively impacted by poor non-dominant hand assistance. Therefore understanding bimanual latent determinants, and establishing safety criteria for bimanual manipulation is relevant to robotic development and to eventual patient care. In this paper, we present a preliminary study to quantitatively evaluate one aspect of bimanual tool use in retinal surgery. Two force sensing tools were designed and fabricated using fiber Bragg grating sensors. Tool-to-sclera contact force is measured using the developed tools and analyzed. The tool forces were recorded during five basic surgical maneuvers typical of retinal surgery. Two subjects are involved in experiments, including one clinician and one engineer. For comparison, all manipulations were replicated under robot-assisted conditions. The results indicate that the average tool-to-sclera force recorded from the dominant hand tool is significantly higher than that from the non-dominant hand tool (\pmb p=0.004). Moreover, the average forces under robot-assisted conditions with the present steady hand robot is notably higher than freehand conditions (\pmb p=0.01). The forces obtained from the dominant and not-dominant hand instruments indicate a weak correlation.

Retinal microsurgery is technically demanding and requires high surgical skill with very little room for manipulation error. The introduction of robotic assistance has the potential to enhance and expand a surgeon's manipulation capabilities during retinal surgery, i.e., improve precision, cancel physiological hand tremor, and provide sensing information. However, surgeon performance may also be negatively impacted by robotic assistance due to robot structural stiffness and nonintuitive controls. In complying with robotic constraints, the surgeon loses the dexterity of the human hand. In this paper, we present a preliminary experimental study to evaluate user behavior when affected by robotic assistance during mock retinal surgery. In these experiments user behavior is characterized by measuring the forces applied by the user to the sclera, the tool insertion/retraction speed, the tool insertion depth relative to the scleral entry point, and the duration of surgery. The users' behavior data is collected during three mock retinal surgery tasks with four users. Each task is conducted using both freehand and robot-assisted techniques. The univariate user behavior and the correlations of multiple parameters of user behavior are analyzed. The results show that robot assistance prolongs the duration of the surgery and increases the manipulation forces applied to sclera, but refines the insertion velocity and eliminates hand tremor.

Concentric-tube robots, which consist of several pre-curved tubes, can achieve dexterous motion through axial rotation and translation of each component tube. Aiming at equilibrium conformation modeling of externally loaded concentric-tube robots, an equivalent conservative system is proposed to translate the force balance problem into the minimum potential energy configuration problem of the conservative system. Then, the optimal control theory is used to derive the differential equations for the equilibrium conformation. Finally, this model is visually evaluated through the simulation of a loaded two-tube robot, and the effects of the external loads on the vital parameters of the equilibrium conformation are analyzed.