Using Artificial Intelligence in the Domain of Sailing

Artificial Intelligence (AI) and machine learning are on the rise and have the potential to improve our lives. The question arises how this technology might impact the world of sailing.

In this article I would like to discuss use cases in the domain of sailing where machine learning and similar technologies might be applied. There are university groups, private companies and individuals carrying out research in this field. This article summarizes recent development in the use of AI for sailing and illustrates how machine learning might change the world of sailing.

Autonomous sailing

Currently, there are multiple companies working on self-driving cars and autonomous vehicles. The question arises whether the concept of driver-less vehicles can be transferred to the world of sailing. This would mean that machine learning and artificial intelligence related technologies would steer sailing vessels and yachts. You could think of a robot that is steering a sailing yacht, controlling the ruder and sails, taking tactical decisions about sailing routes and performing docking maneuvers in a port.

How would the development process of autonomous sailing look like?

Machine Learning and artificial intelligence applications rely on a large amount of data. Relevant data needs to be collected by sensors that are installed on the sailing vessel. Example for such sensors are:

cameras
radar
lidar
GPS
wind sensor
accelerometer
depth sensor
sensors for rudder position and sheet position
etc

Once such sensors are installed on the sailing vessel the next step is the data acquisition. The sailing vessel needs to be moved and sailed so that the sensors can collect relevant data. Data acquisition is a time consuming process and it takes many hours of recording to capture critical moments that could impose a challenge for a sailing robot. There might be the possibility to simulate certain scenarios and augment the actual collected data with augmentation strategies.

Next step is the training or the AI algorithm: The process of training involves that the acquired data is fed into a computer algorithm with the goal to understand the data and reason about it. Different algorithms could be applied to process images from the camera: Object detection models could be used to identify other boats or objects in images. There might also be a potential for the use of segmentation models to detect shore lines or separate water from structures. Input from other sensors based on lidar or radar technology can help as well to reconstruct and model the surrounding of the sailing vessel. Once the scene around the vessel is understood, next step is to take appropriate actions. The software which acts as a virtual skipper needs to learn which actions to take. Reinforcement Learning provides an interesting framework to model an agent in a certain environment and reward according to actions that are taken by the agent.

The quality of the algorithm can be assessed via evaluation metrics. Apart from the accuracy of the model there are also other important criteria such as the speed (how many frames per seconds can be processed) and memory usage which might impact how you can run the model.

Once the model is developed the next step is the deployment of the AI model so that a sail boat can navigate and sail autonomously.

Multiple university groups carry out research in the field of autonomous sailing.

Object detection on maritime images — This image illustrates that an object detection algorithm is capable of identifying sailing vessels. Next to the class label the confidence value of the prediction is shown.

Unmanned sailing vessels

Autonomous sailing allows to create sailing vessels that can sail without any crew or humans on board. Such sailing drones have different use cases:

Unmanned sailing vessels could be used for transportation of goods
Sailing drones could be used to access remote areas for ocean data collection.

There are various companies working on the development of sailing drones. Not only do they develop the software that contains the logic to steer the boat but they also build the hardware and structure of the sailing vessel. Examples for this are companies such as saildrone or sailingrobots.

Manned sailing vessels

With autonomous sailing technology it might be possible to equip sailboats with an autopilot. Currently used autopilot systems are typically only able to steer the rudder and keep a certain heading relative to the wind. In a completely autonomous fashion, the technology would steer the sailing yacht without any interference from the crew. It would not only control the rudder but also the sails and take routing decisions depending on the target position that is defined by the crew.

With truly autonomous sailing a crew could go on a sailing trip without having any license or experience. None of the crew would need to take any sailing decisions and people could simply go on a sailing trip without any prior knowledge in how to sail a boat.

Using artificial intelligence to teach sailing

When learning how to sail you typically rely on an instructor. The instructor explains how sailing works and how you should steer a sailing vessel on the water. The idea behind using AI to teach sailing is that the process of teaching is automated. The automated sailing instructor would analyse the situation of the student and give appropriate instructions such as "keep the rudder straight", "ease the main sheet", etc.

How to develop an automated sailing instructor?

The approach is very similar to the development of autonomous sailing mentioned above. Sensors would need to be installed to not only monitor the sailing vessel and its environment but also the students and their actions and behavior. From the acquired sensor data such as camera images, lidar, etc., the artificial intelligence model needs to be able to understand the scene and give appropriate instructions to the student to safely steer the boat and provide good feedback.

The automated sailing instructor could become a handheld device that is mounted on a sailing dinghy or yacht. Compared to truly autonomous sailing solutions less hardware is needed as the device would only record a video and provide audio feedback leaving the control of the boat to the student.

In a separate more technical article we have covered how a machine learning model can be trained to detect rudder and boom of a sailing dinghy in images and derive the displacement. Such a model could be useful for both, teaching new sailors as well as helping competitive sailors to improve their performance.

Using artificial intelligence for sailboat racing

Technology is taking a larger and larger role in the sailing sport. Innovation such as foiling for instance has opened up new possibilities. Artificial intelligence also has the potential to revolutionize racing with sailboats. AI could be used at all stages of sailboat racing starting from the design phase of a racing vessel up to steering the boat over the finish line.

An important aspect of a race is the sailing boat itself: Especially in races which are open to design choices, a good boat design can make the difference and win a race. Artificial intelligence can help with the design and building process of a sailing vessel. Evolutionary algorithms such as genetic programming could be used to develop new sailboat designs. AI models could also be used to replace and approximate hydrostatic simulations.

AI could also be used to support the training process of skipper and crew. Similar to the approach of an automated sailing instructor, machine learning could be used to analyze how the racing boat is sailed and provide feedback in certain situations. With a 3D Human Pose Estimator it might be possible to track the position and actions of a sailor and his crew in great detail. Such information can then be used to analyse and optimize the human factor in sailing.

With machine learning it is also possible to train a model that is capable of understanding the rudder position and boom position from 2D images in case it is not possible to install appropriate sensors. With such model the 3D scene of the boat can be reconstructed and information about tiller movement and sail setting can be understood and further analyzed.

Also in the actual race artificial intelligence can be used to support decision making: During the race a lot of data is available such as weather, current, position of competitors and so on that could be fed into an AI algorithm to provide support with tactical decisions.

A genetic algorithm is used to simulate sailors and their decisions on a simple race track. Although the two solutions shown in the image still differ from the optimal track, the algorithm learned to beat upwind through mutation and cross-over of random tracks.

Using artificial intelligence for the development of sailing boats

Artificial intelligence might also help to design and build better sailing boats. The process of designing a boat involves lots of ideation, drafting proposals and prototypes and testing. Evolutionary algorithms could help with creating new design candidates by using existing design, applying operations such as mutations, selection and cross-over as inspired by natural evolution.

Another aspect of developing a new sailing yacht is running simulations. To give an example: Simulations can help to check hydrostatic properties of the design which tells you how the boat would sit and behave on the water. Depending on planned materials and use cases, more complex simulations might be necessary which are computationally expensive and take a longer amount of time. AI based models could provide an alternative to such simulation work.

Using artificial intelligence for the production of sailing boats

Whilst statistical process control and simulations can help you to improve quality, better manage production resources and produce boats on time, there is also potential for the use of artificial intelligence in manufacturing of sailing vessels.