Publications

TRI Author: Hongkai Dai

All Authors: Bernardo Aceituno-Cabezas, Carlos Mastalli, Hongkai Dai, Michele Focchi, Andreea Radulescu, Darwin G. Caldwell, Jose Cappelletto, Juan C. Grieco, Gerardo Fernandez-Lopez, Claudio Semini

Traditional motion planning approaches for multilegged locomotion divide the problem into several stages, such as contact search and trajectory generation. However, reasoning about contacts and motions simultaneously is crucial for the generation of complex whole-body behaviors. Currently, coupling theses problems has required either the assumption of a fixed gait sequence and flat terrain condition, or nonconvex optimization with intractable computation time. In this letter, we propose a mixed-integer convex formulation to plan simultaneously contact locations, gait transitions, and motion, in a computationally efficient fashion. In contrast to previous works, our approach is not limited to flat terrain nor to a prespecified gait sequence. Instead, we incorporate the friction cone stability margin, approximate the robot's torque limits, and plan the gait using mixed-integer convex constraints. We experimentally validated our approach on the HyQ robot by traversing different challenging terrains, where nonconvexity and flat terrain assumptions might lead to suboptimal or unstable plans. Our method increases the motion robustness while keeping a low computation time. Read More

Citation: Aceituno-Cabezas, Bernardo, Carlos Mastalli, Hongkai Dai, Michele Focchi, Andreea Radulescu, Darwin G. Caldwell, José Cappelletto, Juan C. Grieco, Gerardo Fernández-López, and Claudio Semini. "Simultaneous contact, gait, and motion planning for robust multilegged locomotion via mixed-integer convex optimization." IEEE Robotics and Automation Letters 3, no. 3 (2017): 2531-2538.

TRI Authors: Kuan-Hui Lee, Yusuke Kanzawa, Matthew Derry

All Authors: Kuan-Hui Lee, Yusuke Kanzawa, Matthew Derry, and Michael R. James

This paper proposes the Permutation Matrix Track Association (PMTA) algorithm to support track-to-track, multi-sensor data fusion for multiple targets in an autonomous driving system. In this system, measurement data from different sensor modalities (LIDAR, radar, and vision) is processed by object trackers operating on each sensor modality independently to create the tracks of the objects. The proposed approach fuses the object track lists from each tracker, first by associating the tracks within each track list, followed by a state estimation (filtering) step. The eventual output is the unified tracks of the objects provided for further autonomous driving processing, such as path and motion planning. The permutation matrix track association (PMTA) algorithm considers both spatial and temporal information to associate object tracks from different sensor modalities. Experimental results show that the proposed approach improves not only the performance of the multipletarget track-to-track fusion, but also stability and robustness in the resulting speed control and decision making in the autonomous driving system. Read More

Citation: Lee, Kuan-Hui, Yusuke Kanzawa, Matthew Derry, and Michael R. James. "Multi-target track-to-track fusion based on permutation matrix track association." In 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 465-470. IEEE, 2018.

TRI Author: Wadim Kehl All Authors: Miroslava Slavcheva, Wadim Kehl, Nassir Navab, Slobodan Ilic We tackle the task of dense 3D reconstruction from RGB-D data. Contrary to the majority of existing methods, we focus not only on trajectory estimation accuracy, but also on reconstruction precision. The key technique is SDF-2-SDF registration, which is a correspondence-free, symmetric, dense energy minimization method, performed via the direct voxel-wise difference between a pair of signed distance fields. It has a wider convergence basin than traditional point cloud registration and cloud-to-volume alignment techniques. Furthermore, its formulation allows for straightforward incorporation of photometric and additional geometric constraints. We employ SDF-2-SDF registration in two applications. First, we perform small-to-medium scale object reconstruction entirely on the CPU. To this end, the camera is tracked frame-to-frame in real time. Then, the initial pose estimates are refined globally in a lightweight optimization framework, which does not involve a pose graph. We combine these procedures into our second, fully real-time application for larger-scale object reconstruction and SLAM. It is implemented as a hybrid system, whereby tracking is done on the GPU, while refinement runs concurrently over batches on the CPU. To bound memory and runtime footprints, registration is done over a fixed number of limited-extent volumes, anchored at geometry-rich locations. Extensive qualitative and quantitative evaluation of both trajectory accuracy and model fidelity on several public RGB-D datasets, acquired with various quality sensors, demonstrates higher precision than related techniques. Read more

Citation: Slavcheva, Miroslava, Wadim Kehl, Nassir Navab, and Slobodan Ilic. "Sdf-2-sdf registration for real-time 3d reconstruction from rgb-d data." International Journal of Computer Vision 126, no. 6 (2018): 615-636.

TRI Author: Ryan M. Eustice

All Authors: Ross Hartley, Maani Ghaffari Jadidi, Ryan M. Eustice, Jessy W. Grizzle

This paper derives a contact-aided inertial navigation observer for a 3D bipedal robot using the theory of invariant observer design. Aided inertial navigation is fundamentally a nonlinear observer design problem; thus, current solutions are based on approximations of the system dynamics, such as an Extended Kalman Filter (EKF), which uses a system's Jacobian linearization along the current best estimate of its trajectory. On the basis of the theory of invariant observer design by Barrau and Bonnabel, and in particular, the Invariant EKF (InEKF), we show that the error dynamics of the point contact-inertial system follows a log-linear autonomous differential equation; hence, the observable state variables can be rendered convergent with a domain of attraction that is independent of the system's trajectory. Due to the log-linear form of the error dynamics, it is not necessary to perform a nonlinear observability analysis to show that when using an Inertial Measurement Unit (IMU) and contact sensors, the absolute position of the robot and a rotation about the gravity vector (yaw) are unobservable. We further augment the state of the developed InEKF with IMU biases, as the online estimation of these parameters has a crucial impact on system performance. We evaluate the convergence of the proposed system with the commonly used quaternion-based EKF observer using a Monte-Carlo simulation. In addition, our experimental evaluation using a Cassie-series bipedal robot shows that the contact-aided InEKF provides better performance in comparison with the quaternion-based EKF as a result of exploiting symmetries present in the system dynamics. Read More

Citation: Hartley, Ross, Maani Ghaffari, Ryan M. Eustice, and Jessy W. Grizzle. "Contact-aided invariant extended Kalman filtering for robot state estimation." The International Journal of Robotics Research 39, no. 4 (2020): 402-430.

TRI Authors: Jeff Walls, Ryan Eustice

All Authors: Hartley, R., Mangelson, J. Gan, L., Ghaffari Jadidi, M., Walls, J., Eustice, R., Grizzle, J.

State-of-the-art robotic perception systems have achieved sufficiently good performance using Inertial Measurement Units (IMUs), cameras, and nonlinear optimization techniques, that they are now being deployed as technologies. However, many of these methods rely significantly on vision and often fail when visual tracking is lost due to lighting or scarcity of features. This paper presents a state-estimation technique for legged robots that takes into account the robot's kinematic model as well as its contact with the environment. We introduce forward kinematic factors and preintegrated contact factors into a factor graph framework that can be incrementally solved in real-time. The forward kinematic factor relates the robot's base pose to a contact frame through noisy encoder measurements. The preintegrated contact factor provides odometry measurements of this contact frame while accounting for possible foot slippage. Together, the two developed factors constrain the graph optimization problem allowing the robot's trajectory to be estimated. The paper evaluates the method using simulated and real sensory IMU and kinematic data from experiments with a Cassie-series robot designed by Agility Robotics. These preliminary experiments show that using the proposed method in addition to IMU decreases drift and improves localization accuracy, suggesting that its use can enable successful recovery from a loss of visual tracking.  Read more 

Citation: Hartley, Ross, Josh Mangelson, Lu Gan, Maani Ghaffari Jadidi, Jeffrey M. Walls, Ryan M. Eustice, and Jessy W. Grizzle. "Legged robot state-estimation through combined forward kinematic and preintegrated contact factors." In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1-8. IEEE, 2018.

TRI Author: Simon Stent

All Authors: Pablo F. Alcantarilla and Simon Stent and German Ros and Roberto Arroyo and Riccardo Gherardi

We propose a system for performing structural change detection in street-view videos captured by a vehicle-mounted monocular camera over time. Our approach is motivated by the need for more frequent and efficient updates in the large-scale maps used in autonomous vehicle navigation. Our method chains a multi-sensor fusion SLAM and fast dense 3D reconstruction pipeline, which provide coarsely registered image pairs to a deep Deconvolutional Network (DN) for pixel-wise change detection. We investigate two DN architectures for change detection, the first one is based on the idea of stacking contraction and expansion blocks while the second one is based on the idea of Fully Convolutional Networks. To train and evaluate our networks we introduce a new urban change detection dataset which is an order of magnitude larger than existing datasets and contains challenging changes due to seasonal and lighting variations. Our method outperforms existing literature on this dataset, which we make available to the community, and an existing panoramic change detection dataset, demonstrating its wide applicability.  Read More

Citation: Alcantarilla, Pablo F., Simon Stent, German Ros, Roberto Arroyo, and Riccardo Gherardi. "Street-view change detection with deconvolutional networks." Autonomous Robots 42, no. 7 (2018): 1301-1322.

TRI Author: Muratahan Aykol

All Authors: Daniel P Tabor, Loïc M Roch, Semion K Saikin, Christoph Kreisbeck, Dennis Sheberla, Joseph H Montoya, Shyam Dwaraknath, Muratahan Aykol, Carlos Ortiz, Hermann Tribukait, Carlos Amador-Bedolla, Christoph J Brabec, Benji Maruyama, Kristin A Persson, Alán Aspuru-Guzik

The discovery and development of novel materials in the field of energy are essential to accelerate the transition to a low-carbon economy. Bringing recent technological innovations in automation, robotics and computer science together with current approaches in chemistry, materials synthesis and characterization will act as a catalyst for revolutionizing traditional research and development in both industry and academia. This Perspective provides a vision for an integrated artificial intelligence approach towards autonomous materials discovery, which, in our opinion, will emerge within the next 5 to 10 years. The approach we discuss requires the integration of the following tools, which have already seen substantial development to date: high-throughput virtual screening, automated synthesis planning, automated laboratories and machine learning algorithms. In addition to reducing the time to deployment of new materials by an order of magnitude, this integrated approach is expected to lower the cost associated with the initial discovery. Thus, the price of the final products (for example, solar panels, batteries and electric vehicles) will also decrease. This in turn will enable industries and governments to meet more ambitious targets in terms of reducing greenhouse gas emissions at a faster pace. Read More

Citation: Tabor, Daniel P., Loïc M. Roch, Semion K. Saikin, Christoph Kreisbeck, Dennis Sheberla, Joseph H. Montoya, Shyam Dwaraknath et al. "Accelerating the discovery of materials for clean energy in the era of smart automation." Nature Reviews Materials 3, no. 5 (2018): 5-20.

TRI Authors: Simon Stent, Luke Fletcher All Authors: John Gideon, Simon Stent, Luke Fletcher We present a system for the detection of elevated levels of driver alertness in driver-facing video captured from multiple viewpoints. This problem is important in automotive safety as a helpful feedback signal to determine driver engagement and as a means of automatically flagging anomalous driving events. We generated a dataset of videos from 25 participants overseeing an hour each of driving sequences in a simulator consisting of a mixture of normal and near-miss driving events. Our proposed system consists of a deep neural network which fuses information from three driver-facing cameras to estimate moments of elevated driver alertness. A novel aspect of the system is that it learns to actively re-weight the importance of camera inputs depending on their content. We demonstrate that this approach is not only resilient to dropped or occluded frames, but also has significantly improved performance compared to a system trained on any single stream. Read more

Citation: Gideon, John, Simon Stent, and Luke Fletcher. "A Multi-Camera Deep Neural Network for Detecting Elevated Alertness in Drivers." In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2931-2935. IEEE, 2018.

TRI Author: Katherine M. Tsui

All Authors: Suresh Kumaar Jayaraman, Chandler Creech, Lionel P. Robert Jr., Dawn M. Tilbury, X Jessie Yang, Anuj K. Pradhan, Katherine M. Tsui

Autonomous vehicles (AVs) have the potential to improve road safety. Trust in AVs, especially among pedestrians, is vital to alleviate public skepticism. Yet much of the research has focused on trust between the AV and its driver/passengers. To address this shortcoming, we examined the interactions between AVs and pedestrians using uncertainty reduction theory (URT). We empirically verified this model with a user study in an immersive virtual reality environment (IVE). The study manipulated two factors: AV driving behavior (defensive, normal and aggressive) and the traffic situation (signalized and unsignalized). Results suggest that the impact of aggressive driving on trust in AVs depends on the type of crosswalk. At signalized crosswalks the AV»s driving behavior had little impact on trust, but at unsignalized crosswalks the AV»s driving behavior was a major determinant of trust. Our findings shed new insights on trust between AVs and pedestrians. Read more 

Citation: Jayaraman, Suresh Kumaar, Chandler Creech, Lionel P. Robert Jr, Dawn M. Tilbury, X. Jessie Yang, Anuj K. Pradhan, and Katherine M. Tsui. "Trust in AV: An uncertainty reduction model of AV-pedestrian interactions." In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pp. 133-134. 2018.

TRI Authors: Astrid Jackson, Brandon D. Northcutt

All Authors: Astrid Jackson, Brandon D. Northcutt, Gita Sukthankar

One of the advantages of teaching robots by demonstration is that it can be more intuitive for users to demonstrate rather than describe the desired robot behavior. However, when the human demonstrates the task through an interface, the training data may inadvertently acquire artifacts unique to the interface, not the desired execution of the task. Being able to use one»s own body usually leads to more natural demonstrations, but those examples can be more difficult to translate to robot control policies. This paper quantifies the benefits of using a virtual reality system that allows human demonstrators to use their own body to perform complex manipulation tasks. We show that our system generates superior demonstrations for a deep neural network without introducing a correspondence problem. The effectiveness of this approach is validated by comparing the learned policy to that of a policy learned from data collected via a Sony Play Station~3 (PS3) DualShock 3 wireless controller. Read more

Citation: Jackson, Astrid, Brandon D. Northcutt, and Gita Sukthankar. "The Benefits of Teaching Robots using VR Demonstrations." In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pp. 129-130. 2018.