Marine

The needed system should be able to recognize static and dynamic obstacles around the ship, predict their motion (GIM-Observer), and plan a safe and efficient trajectory for the ship. We are experts in a) fusing thermal and color camera data to enable visual vessel detection and tracking in all lighting conditions, b) fusing lidar, camera and radar data to accurately segment, classify and track any objects of interest, c) comparing sea chart and 3D sensor data to automatically extract all moving targets on the surface and d) calibrating multiple sensors into a common coordinate frame, enabling advanced sensor fusion. Natural and built environment around the ship, as well as moored vessels are considered as static features. Vessels moving in the vicinity of the ship are considered as dynamic features. Safe trajectory is a trajectory that does not collide with any other vessel or land, follows traffic rules and COLREG (International Regulations for Preventing Collisions at Sea). In addition, a safe trajectory has movements large enough to signal to the other vessels that the ship is giving way. A safe trajectory is also feasible for the ship to follow, and an efficient trajectory is a trajectory that is as fast as possible without compromising the safety constraints. The vessel sensory system provides interoceptive and exteroceptive information about the vessel and its surroundings. In addition, the system must be able to use Electronic Navigational Charts and perform at least rudimentary route planning.  

When your GNSS+IMU combination works like it should, you will basically have all the tools you need to adjust your vessel correctly while running it in autonomous mode. For berthing you need some additional information about the static environment such as the localization (position and orientation) of the dock. (Dynamic targets are skipped in this context.) However, to rely on the fully autonomous operation, GNSS should not be your only option for the task. The jamming of the GNSS signal, be it intentional or unintentional, can have serious consequences. Other sources for GNSS signal related problems are caused by large and tall waterfront buildings. They can effectively block the signals from the satellites. To be prepared for these common problems, your autonomous ship must be equipped, in addition to a reliable and safe remote-control connection, with an alternative and supportive way of localizing your vessel while inside the harbor area. There are many ways to do this, but because we do not want to rely on additional infrastructure, such as signal beacons or similar, the solution must be based on existing features of the harbor area.  

We need to solve the SLAM (Simultaneous Localization and Mapping) problem and find a way to construct and update a map of the unknown harbor environment (GIM-Mapper) while simultaneously keeping track of our autonomous vessel’s location within it (GIM-Locator). In this case it would mean the fusion of point-clouds provided by various sensors, such as LiDARs, radars and various types of cameras, into INS (Inertial Navigation System) and occasional GNSS provided data streams. Every man-made urban harbor has plenty of topological landmarks, which will be used as features to create the topological map. After you have done this recording once, while arriving at the harbor for the first time after the installation of the system, the map can be used for future localization. This is something we have done in our previous projects worldwide. The same approach can be applied to other locations as well, such as various waterways in the urban areas if the environment has enough features and the vessel has additional information coming from its INS system. Even sporadic GNSS fixes will be naturally useful and will be fused to limit the inherent drift of the INS system.