According to the HD map production pipeline, production workers will label lane boundaries, crosswalks, obstacles, road connectivity, and so on. We then need to plan paths in the road network for autonomous vehicles. As for non-intersections, such paths can be generated as the geometrical centerline of lanes, while for intersections, such virtual lanes are labeled manually, which results in low efficiency, inconsistent style, and kinematic infeasibility.

To address these problems, I developed an automation tool for path planning at intersections. Given the information around an intersection, the tool will output a smooth virtual lane to connect the previous lane and the next lane, respecting kinematic constraints, obstacle avoidance, and traffic rules. More importantly, the tool can accurately forewarn users of potential risks, which makes it easier for users to locate such constraint-inconsistent virtual lanes, without checking all results.

The algorithms planned 9,625 intersection paths in Shanghai: 99% met the required standards, while the remaining 1% were correctly flagged for review. Deployment to the HD-map labeling production platform increased labeling efficiency by over 1,000%.

The problem of optimal control in the presence of noise occurs in many areas of science and engineering. Examples are the control of movement in biological systems, robotics, financial investment policies, and robot path planning. Dealing with real systems, we often have state-input inequality constraints. For instance, we want to keep the contact force inside the friction cone.

In stochastic sense, the cost function would thus be the expectation of nominal cost. After elegant manipulation and log transform, the optimality condition for the cost can be expressed in the linear form. Based on the path integral formulation, I investigated the possibility to find an analytic and a numerical optimizer. The sampling policy is finally modified to respect the inequality constraints. As a result, the robot can safely circumvent the barriers by applying the optimal control corrupted by the noise.

For higher dimensions, the state-input constrained feasible region may be complex and the computation cost grows fast. Further work would thus be applying parallel computation to improve the computation efficiency.

Depth map fusion is an essential part of both stereo and RGB-D based 3D reconstruction pipelines. Thanks to the research mode in HoloLens, it’s possible to get direct access to the raw data obtained from the time-of-flight (ToF) camera. The goal of this research is to implement an algorithm that fuses multiple depth maps into a single point cloud by checking for visibility, geometric and appearance consistency. The input is a stream of depth, and intensity images from ToF Camera of HoloLens, together with a sequence of corresponding camera poses. The resulting output is a single point cloud containing the same amount of 3D information but in a more compact and consistent representation.

The outcome of this fusion algorithm is a combined and consistent point cloud derived from a stream of depth and intensity images produced by HoloLens. Tests were performed on public data sets as well as real data recorded from HoloLens. With appropriate parameter setting, the result is satisfying, as shown on the left side.

3D models of crops contain rich information such as the branch number and the branch angle. However, the measurement of a single crop in the fields is typically hard to perform due to poor working conditions and multiple inter-plant shades. Therefore, we aim to develop a robot, controlled by human-machine interaction, to create optimal measurement conditions. To remove obstacles before the object of interest, we apply wearable human motion sensors to facilitate precise remote control (so that agricultural scientists do not have to go to fields). Specifically, we collect human motion data via motion capture equipment produced by Noitom. Then, joint angles are computed accordingly by the kinematic model so that human motion is mapped to the robot motion.

To avoid damage to the robot itself caused by misoperation, I designed a self-collision avoidance algorithm to ensure safety. The algorithm will first read input human motion data and compute the desired joint angles. If implementing these joint angles cause a link to enter the safety zone of other parts, the algorithm will prevent the potential collision.

Traditional 2D pictures lose important spatial information such as spatial angles. Therefore, I implemented the 3D reconstruction algorithm based on linear structured light. A calibrated camera and laser projector are carried by the robot to fulfill the reconstruction task.

With these two features combined, one can control the robot remotely to 3D reconstruct the crops of interest for further biological study.

Linear actuators available in the market are mostly expensive with limited types. The most common ones are voice coil actuators and classical linear actuators, which are expensive and big in shape respectively. However, there are a lot of application scenarios where linear actuators outperform rotary actuators. Thus, we wanted to design a system with movable coil arrays as linear actuators. The potential applications include the military sand table, rhythm visualization and compliant interaction.

The key idea is exploiting the properties of the magnetic field induced by a solenoid. After careful calculation and simulation, we decided on magnetic material (NdFeB) and coil parameters to develop a prototype.

Result: we failed to come up with a feasible design to meet all required specifications, namely, excellent dynamic response characteristics, adequate heat-sinking capability and large maximum current carrying capacity. The three most significant barriers are the limited magnetic force, the high cost of the feedback loop and significant magnetic flux leakage. Despite the failure, I learnt a lot about field simulation and enjoyed the time when I tried to exploit every possibility.

Coagulation monitoring is essential for perioperative care and thrombosis treatment. However, existing assays for coagulation monitoring have limitations such as a large footprint and complex setup. In this work, we developed a miniaturized device for point-of-care blood coagulation testing by measuring dynamic clot retraction force development during blood clotting. In this device, a blood drop was localized between a protrusion and a flexible force-sensing beam to measure clot retraction force. The beam was featured with micropillar arrays to assist the deposition of carbon nanotube films, which served as a strain sensor to achieve label-free electrical readout of clot retraction force in real time. We characterized mechanical and electrical properties of the force-sensing beam and optimized its design. We further demonstrated that this blood coagulation monitoring device could obtain results that were consistent with those using an imaging method and that the device was capable of differentiating blood samples with different coagulation profiles. Owing to its low fabrication cost, small size, and low consumption of blood samples, the blood coagulation testing device using carbon nanotube strain sensors holds great potential as a point-of-care tool for future coagulation monitoring.

For crop breeding and variety screening, it is essential to improve the phenotyping throughput for quantitatively evaluating the seed traits. However, existing high-throughput phenotyping facilities for crop seeds generally have some limitations such as high cost, large size and high weight which hinder the usage in the field. Thus, a smartphone-based, low-cost, portable instrument to achieve the function of high-throughput field phenotyping for crop seeds is in high demand.

I use a smartphone as the instrument's main platform, which offers rich functionality for fast phenotyping the crop seeds. First, advanced computing ability and the high-resolution camera offer the conditions for image analysis, which measures morphological parameters by image processing. Second, the Bluetooth module makes communication convenient for device control. Third, user-friendly Human-Machine Interface (HMI) makes it easy for users to find and correct a calculation error. The experiment results showed that the measurement accuracy of phenotypic parameters is over 97%, and this instrument can be applied in the precise measurement of crop seeds phenotypic parameters. More importantly, I introduced cloud storage to my instrument and equipped it with the capability of location-based data collection. I took full advantage of the big data collected via this instrument to provide valuable information for agricultural breeding as well as crop selection. So far, this instrument has been deployed in 24 agricultural technology promotion service centers, and the number continues increasing.

For cutting-edge protected agriculture, it is a leading trend to build an unmanned plant factory that requires robotic machinery as basic autonomous production units. However, the scheduling and cooperation among agricultural machines are rarely investigated. This research focuses on a modular logistics system based on the motion decomposition, and the objective is to develop a fully interrelating intelligent agricultural production system.

Firstly, an unmanned ground vehicle is developed for transferring intelligent agricultural machines such as agricultural robots. In consideration of the complex terrain features, both automatic mode and manual mode are available. The multifunctional smart remote-control mode is designed for users to control the logistic systems accurately and the application of tablet PC provides a good user experience. Secondly, a control and mechanical system is designed to dock and transfer a harvesting robot. The safety and reliability of this system are jointly ensured by its movement pattern, the design of guide rails on the slope and the precise positioning. Third, the automatic charging function is realized by an automatic docking system with a maximum tolerance of 8 cm, which enables the system to work 7/24.