Professor Alex Zelinsky

Since 2005, the IEEE Robotics and Automation Society and International Federation of Robotics (IFR) have cooperated to host a joint Forum of Innovation and Entrepreneurship in Robotics and Automation to foster cooperation between the scientific community and industry. As part of the forum, an award for Invention and Entrepreneurship in Robotics and Automation (IERA) is made. This year the IEEE--IFR form was held in Kobe, Japan, in conjunction with the IEEE International Conference on Robotics and Automation (ICRA) 2009.

Biology often offers valuable example of systems both for learning and for controlling motion. Work in robotics has often been inspired by these findings in diverse ways. Though, the fundamental aspects that involve visual landmark learning and motion control mechanisms have almost exclusively been approached heuristically rather than examining the underlying principles. In this paper we introduce theoretical tools that might explain how the visual learning works and why the motion is attracted by the pre-learnt goal position. Basically, the theoretical tools emerge from the navigation vector field produced by the visual behaviors. Both the learning process and the navigation scheme influence the motion field. We apply classical mathematical and dynamic control to analyze the efficiency of our method.

This paper introduces a theory of formally and practically analyze the robustness issues of visual guidance methods for robot navigation. The first aspect is related to the convergence of the navigation system to the goal. It will be shown how the dynamic system which drives the strategies can be analyzed by using classical concepts such as the Lyapunov functions. The second aspect concerns the conservativeness of the resulting navigation vector fields. It will be shown how this deals with the repeatibility of the trials. Furthermore, the selection of the best landmarks to perform the navigation processes strongly affects the conservativeness thus providing a formal way to do landmark learning. The theory has been tested with two different visual methods that have been derived from the biological world: the snapshot model and the landmark model. The former considers a portion of the full panorama taken by a color camera to accomplish navigating actions. The latter is a more sophisticated approach which uses the most suitable visual landmarks to calculate navigation movements.

Our interaction with machines is always severely constrained by unnatural interfaces such as mouse, keyboard and joystick. Such interfaces make it difficult for us to convey our ideas in order for computers to understand and perform our intended tasks. In this research, our aim is to build systems that use natural human actions as interfaces. In particular we exploit gaze to track a person's focus of attention. We present an active gaze tracking system that enables a user to instruct a robot arm to pick up and hand over an object placed arbitrarily in 3D space. Our system determines the precise 3D position of the object of unknown size, shape and color by following the person's steady gaze. © 2005 IEEE.

We have incorporated interactive skills into an active gaze tracking system. Our active gaze tracking system can identify an object in a cluttered scene that a person is looking at. By following the user's 3-D gaze direction together with a zero-disparity filter, we can determine the object's position. Our active vision system also directs attention to a user by tracking anything with both motion and skin color. A Particle Filter fuses skin color and motion from optical flow techniques together to locate a hand or a face in an image. The active vision then uses stereo camera geometry, Kalman Filtering and position and velocity controllers to track the feature in real-time. These skills are integrated together such that they cooperate with each other in order to track the user's face and gaze at all times. Results and video demos provide interesting insights on how active gaze tracking can be utilized and improved to make human-friendly user interfaces. Copyright 2003 ACM.

© 2003 IEEE. One of the more startling effects of road related accidents is the economic and social burden they cause. Between 750,000 and 880,000 people died globally in road related accidents in 1999 alone, with an estimated cost of US$518 billion. One way of combating this problem is to develop Intelligent Vehicles that are self-aware and act to increase the safety of the transportation system. This paper presents the development and application of a novel multiple-cue visual lane tracking system for research into Intelligent Vehicles (IV). Particle filtering and cue fusion technologies form the basis of the lane tracking system which robustly handles several of the problems faced by previous lane tracking systems such as shadows on the road, unreliable lane markings, dramatic lighting changes and discontinuous changes in road characteristics and types. Experimental results of the lane tracking system running at 15 Hz will be discussed, focusing on the particle filter and cue fusion technology used.

© 2003 IEEE. This paper presents a method for accurately tracking the position of a mobile robot which moves between places in a previously learned topological map. Places in the map are represented by sets of visual landmarks extracted from panoramic images. Probabilistic localisation methods and the landmark representation enable position tracking within places. A sensor model is presented which improves the accuracy of local position estimates, and is robust in the presence of occlusion and data association errors. Position tracking between places requires the recognition of place transition events and the passing of local position estimates between places. This paper presents such a system and reports real world position tracking results from paths through topological maps.

The use of landmarks is a natural and instinctive method to determine the whereabouts of a location or a means to proceed to a particular location. Results provided in this paper indicate that landmark-based navigation possesses a corrective or feedback trait that produces a convergence bound on the movements to the goal position, in contrast to the odometry-based movements, which leads to the drift between successive navigation movements. Experiments show that the vector field approach can be used to explain the convergence property of landmark-based guidance tasks. Experiments have been carried out operating with a Nomad mobile robot equipped with real-time visual landmark tracking system.

© 2000 IEEE. Virtual environments provide a whole new way of viewing and manipulating 3D data. Current technology moves the images out of desktop monitors and into the space immediately surrounding the user. Users can literally put their hands on the virtual objects. Unfortunately techniques for interacting with such environments have yet to mature. Gloves and sensor based trackers are unwieldy, constraining and uncomfortable to use. A natural, more intuitive method of interaction would be to allow the user to grasp objects with their hands and manipulate them as if they were real objects. We are investigating the use of computer vision in implementing a natural interface based on hand gestures. A framework for a gesture recognition system is introduced along with results of experiments in colour segmentation, feature extraction and template matching for finger and hand tracking and hand pose recognition. Progress in the implementation of a gesture interface for navigation and object manipulation in virtual environments is discussed.

Computer systems which analyse human face/head motion have attracted significant attention recently as there are a number of interesting and useful applications. Not least among these is the goal of tracking the head in real time. A useful extension of this problem is to estimate the subject's gaze point in addition to his/her head pose. This paper describes a real-time stereo vision system which determines the head pose and gaze direction of a human subject. Its accuracy makes it useful for a number of applications including human/computer interaction, consumer research and ergonomic assessment. © 2000 IEEE.

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are deeply related with his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. We describe our real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Since the 3D coordinates of the features on a face can be directly measured in our system, we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. Consequently we achieved a non-contact, passive, real-time, robust, accurate and compact measurement system for head pose and gaze direction. © 2000 IEEE.

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are deeply related with his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. In this paper, we describe a behavior recognition system based on the real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Since the 3D coordinates of the features on a face can be directly measured in our system, we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. Consequently we achieved a non-contact, passive, real-time, robust, accurate and compact measurement system for head pose and gaze direction. The recognition of attentions and gestures of a person is demonstrated in the experiments.

In this paper we present a paradigm for robot control, Supervised Autonomy. Supervised Autonomy is a framework, which facilitates the development of human robot systems. Each of these components have been devised to augment users in accomplishing their task. Experimental results of this framework in applying to the use of a teleoperation system are presented. Our current progress and planned future work is also presented.

We have been developing an autonomous robotic agent that helps people in a real world environment, such as in an office. When a robotic agent works by cooperating with person in a real world environment, it must manage a lot of information and deal with the knowledge and languages that people usually use. Therefore it is important for the agent to recognize what people request as soon as possible. To realize common communication with people, the agent should provide robust communicative functions to obtain information from people. In this paper, we discuss communicative functions of our robotic agent called Jijo-2. Especially we focus on the problem of detecting human faces, and discuss how a method of detecting a human face can be robustly archived.

When a person instructs operations to a robot, or performs a cooperative task with a robot, it is necessary to inform the robot of the person's intention and attention. Since the motion of a person's face and the direction of the gaze is deeply related with person's intention and attention, detection of such motions can be utilized as a natural way of communication in human- robot interaction. In this paper, we describe our real-Time stereo face tracking system. The key of our system is the use of a stereo vision. Since the 3D coordinates of the features on the face can be directly measured in our system, we can drastically simplify the algorithm of the 3D model fitting to obtain the full 3D pose of the head compared with convention-Al system with monocular camera. Consequently we achieved a non-contact, passive, real-Time, robust, accurate and compact measurement system of human's head pose and gaze direction.

When a person instructs operations to a robot, or performs a cooperative task with a robot, it is necessary to inform the robot of the person's intention and attention. Since the motion of a person's face and the direction of the gaze is deeply related with person's intention and attention, detection of such motions can be utilized as a natural way of communication in human-robot interaction. In this paper, we describe our real-time stereo face tracking system. The key of our system is the use of a stereo vision. Since the 3D coordinates of the features on the face can be directly measured in our system, we can drastically simplify the algorithm of the 3D model fitting to obtain the full 3D pose of the head compared with conventional system with monocular camera. Consequently we achieved a non-contact, passive, real-time, robust, accurate and compact measurement system of human's head pose and gaze direction.

Modelling and reducing uncertainty are two essential problems with mobile robot localisation. Previously we developed a robot localisation system, namely the Gaussian Mixture of Bayes with Regularised Expectation Maximisation (GMB-REM), using sonar sensors. GMB-REM allows a robot's position to be modelled as a probability distribution, and uses Bayes' theorem to reduce the uncertainty of its location. In this paper, a new system for performing sensor fusion is introduced, namely an enhanced form of GMB-REM. Empirical results show the new system outperforms GMB-REM using sonar alone. More specifically, it is able to constrain the error of robot's positions even when sonar signals are noisy.

This paper presents a biologically-inspired method for navigating using visual landmarks which have been self-selected within natural environments. A landmark is a region of the grabbed image which is chosen according to its reliability measured through a phase (Turn Back and Look - TBL) that mimics the behavior of some social insects. From the self-chosen landmarks suitable navigation information can be extracted following a well known model introduced in Biology to explain the bee's navigation behavior. The landmark selection phase affects the conservativeness of the navigation vector field thus allowing us to explain the navigation model in terms of a visual potential function which drives the navigation to the goal. The experiments have been performed using a Nomad200 mobile robot equipped with monocular color vision.

According to the behaviour-based philosophy, the structure of an agent's internal representations of the environment should not be explicitly imposed by the designer; they should be grounded in its sensor-action space. This paper presents a scheme in which the agent's action selection mechanism gives rise to an integrated spatial and topological navigation and mapping capability. The navigation behaviour emerges from the notion of location feature detectors and homogeneous action selection. The scheme is demonstrated using two autonomous mobile robots in a multi-robot cooperation scenario.

This paper introduces a new approach to the control of robot manipulators in a way that is safe for humans in the robot's workspace. Conceptually the robot is viewed as a tool with limited autonomy. The limited perception capabilities of automatic systems prohibits the construction of failsafe robots with the capabilities of people. Instead, the goal of our control scheme is to make the interaction with a robot manipulator safe by making the robots actions predictable and understandable to the human operator. At the same time the forces the robot applies with any part of its body to its environment have to be controllable and limited. Experimental results are presented of a human-friendly robot controller that is under development for a Barrett Whole Arm Manipulator robot.

© Springer-Verlag Berlin Heidelberg 1997. Real-time computer vision combined with robust gesture recognition provides a natural alternative to traditional computer interfaces. Human users have plenty of experience with actions and the manipulation of objects requiring finger movement. In place of a mouse, users could use their hands to select and manipulate data. This paper presents a first step in this approach using a finger as a pointing and selection device. A major feature of a successful tracking system is robustness. The system must be able to acquire tracked features upon startup, and reacquire them if lost during tracking. Reacquisition should be fast and accurate (i.e. it should pick up the correct feature). Intelligent search algorithms are needed for speedy, accurate acquisition of lost features with the frame. The prototype interface presented in this paper is based on finger tracking as a means of input to applications. The focus of the discussion is how the system can be made to perform robustly in real-time. Dynamically distributed search windows are defined for searching within the frame. The location and number of search windows are dependent on the confidence in the tracking of features. Experimental results showing the effectiveness of these techniques are presented.

We present a scheme for specifying and executing purposive navigation tasks for a behaviour-based mobile robot. A user specifies the robot's navigation task in general and qualitative terms using a graphical resource called the Purposive Map (PM). The robot navigates using the incomplete and approximate information stored in the PM as an aid to achieve the specified mission. The robot is able to augment the knowledge provided in the PM with environment information learnt by the robot. Using the augmented PM, the robot learns how to perform efficient obstacle avoidance. We present experimental results using a real robot to show our scheme is robust. Our robot can escape from dead-ends, can deduce that goals are unreachable and can withstand disturbances to the environment between missions.

This paper presents a new scheme for purposive navigation for mobile agents. The new scheme is robust, qualitative and provides a mechanism for combining mapping, planning and mission execution for a mobile agent into a single data structure called the Purposive Map (PM). The agent can navigate using incomplete and approximate information stored the PM. We present a novel approach to finding how to perform obstacle avoidance for a behaviour based robot. Our approach is based on using a physically grounded search while monitoring and co-ordinating behaviours. The physically grounded search exploits stagnation points (local minima) to guide the search for the shortest path to a target. This scheme enables our robot to escape from dead-end situations and allows it to deduce that a target location is unreachable. Simulation results are presented.

© Springer-Verlag Berlin Heidelberg 1990. This paper describes a method of producing high resolution maps of an indoor environment with an autonomous mobile robot equipped with sonar range finding sensors. This method is based upon investigating obstacles in the near vicinity of a mobile robot. The mobile robot examines the straight line segments extracted from the sonar range data describing obstacles near the robot. The mobile robot then moves parallel to the straight line sonar segments, in close proximity to the obstacles, continually applying the sonar barrier test. The sonar barrier test exploits the physical constraints of sonar data, and eliminates noisy data. This test determines whether or not a sonar line segment is a true obstacle edge or a false reflection. Low resolution sonar sensors can be used with the described method. The performance of the algorithm is demonstrated using a Denning Corp. Mobile Robot, equipped with a ring of Polaroid Corp. Ultrasonic Rangefinders.