Journal of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

Ships have long been used for maritime transportation and traffic, and various transportation regulations, standards, and information systems have been developed for this purpose. Cargo ships transport dry and liquid cargo, such as crude oil, liquefied petroleum gas (LPG), and liquefied natural gas (LNG), and passenger ships transport passengers by roundtrip on specific routes. Special ships are used for various purposes in marine research, construction, and exploration [1]. Various international regulations for maritime traffic of these ships are established and managed by the International Maritime Organization (IMO) [2]. Safety of life at sea (SOLAS) stipulates minimum standards for ship structures, equipment, and navigation safety, and the International Convention for the Prevention of Pollution from Ships (MARPOL) prevents pollution from ships to protect the marine environment. In addition, the Standards of Training, Certification, and Watchkeeping (STCW) stipulate crew training and qualification standards, and the International Regulations for Preventing Collisions at Sea (COLREGs) help ships avoid collisions at sea. Generally, ships are equipped with various information systems to comply with these regulations and quantitatively analyze the navigation context [3]. For example, the electronic chart display and information system is an information system that displays the ship’s location, route, obstacles, etc. on an electronic chart in real time. In addition, the automatic identification system automatically transmits and receives various information, such as the ship's location, speed, route, etc., with other ships at sea and is an important information system for preventing collisions and effectively managing marine traffic through communication with other ships and port authorities. These information systems provide useful information for navigation based on signals measured from various sensors, such as radar, LiDAR, GPS, and echo sounders installed on the ship.

Along with technological advancements in other means of transportation, such as automobiles and aircraft, AI-based digital navigation technologies, such as autonomous navigation, are also being developed for ships. However, various obstacles exist to the digital transformation of existing technologies. After Statheros et al. first published their research on autonomous ships in 2008 [4], Chae, [5] analyzed the technological requirements for autonomous ship systems and Gu et al. [6] discussed the potential opportunities in transport and logistics with autonomous ships. Moreover, numerous research teams have proposed techniques for path planning, as well as for detecting and avoiding other ships in autonomous navigation [7,8,9]. On the one hand, although the Maritime Safety Committee has approved guidelines for the trials of autonomous ships in 2019, the details of the guidelines still need to be further defined [10]. One of the issues with the detailed guidelines is the integration of various information systems installed onboard the ship. Modern ships are equipped with various information systems that collect and process signals from dozens of sensors, and provide information to navigators. However, these information systems operate individually for different purposes, and navigators acquire the information necessary for navigation from each system. Therefore, to transform current ships into autonomous systems, integrated management of current fragmented information systems is necessary.

It is necessary to develop an integrated navigation information system that effectively and efficiently provides navigation-related information to navigators to realize an onboard decision support ship, which is the first stage of autonomous ships defined by the IMO, Therefore, numerous studies have been conducted for this purpose. The IMO defines autonomous ships in six stages. In the first stage, human operators are responsible for all actions with minor support from digital systems. Navigators navigate with the help of information systems, and the responsibility for this is given to navigators. The characteristics of navigation tasks must be analyzed in detail, and accurate information required for each step of task performance must be provided to a digital information system to effectively support navigation tasks [11]. In addition, even if digital systems that process information exist individually, information must be provided in a user-friendly manner and in the right place for navigators to efficiently acquire information from digital information systems [12]. Furthermore, to advance to the sixth stage of autonomous navigation (fully autonomous), no human actions are involved, and integrated management of information provided by each information system is essential. Referring to previous studies on integrated navigation information systems, Li et al. [13] proposed a user interface for visualizing various types of information on smart ships. However, the proposed user interface is focused on systematically organizing ship information so that outside staff can easily search for information remotely and was found to be unsuitable for navigators who need to search for information directly on the ship. Oh et al. [14] proposed an AR-based navigation support system that visualizes various types of information required for navigation and actual images on a monitor. This system can be utilized by providing various types of information needed by navigators on the ship in an integrated system; however, it has a limitation in that the gaze must inevitably move from looking outside through the windshield in the bridge to checking information on the monitor.

In this study, we develop a user interface for a HUD-based integrated navigation information system that supports safe and efficient navigation tasks for navigators. The most important task of a navigator is the lookout, and any information system that interferes with the lookout is contrary to its essence. COLREGs also recommends that every vessel maintain a proper lookout by sight and hearing to avoid the risk of collision. Therefore, it is necessary to develop a navigation information system that allows navigators to check navigation support information efficiently while focusing on lookouts. This study developed a user interface for a HUD-based navigation information system that can check and operate information provided by various navigation information systems installed in the bridge and sensor data installed in various locations on the ship in one place.

First, this study investigated educational materials for obtaining navigator qualifications and reviewed various research papers on international standards and HUD-based information system designs that should be considered when designing navigation information systems to understand the overall tasks of navigators onboard ships. Educational materials [15] for taking the ship officer license exam managed by the Korean Institute of Maritime and Fisheries Technology (KIMFT) were reviewed to investigate the types and procedures of overall tasks on board ships. In addition, the IEC 62288 document was examined to identify the international standards that should be considered when designing a user interface for navigation information systems. This standard provides guidelines on the screen display method, information layout, and warning system of navigation information system user interfaces, so that information related to ship navigation can be clearly conveyed to navigators. Finally, research papers related to changes in performance according to the color and arrangement of HUD design elements were investigated to identify considerations when designing HUD-based information systems for vehicles and aircraft.

We conducted expert interviews to confirm the consistency of the information identified in the literature and identify additional considerations according to the characteristics of the field. Two third-class officers participated in the expert interview, and information from the literature on the procedures for various tasks performed by navigators onboard ships and the types of information required at each stage was reviewed. For example, an explanation for the types of major tasks and their procedures was investigated, and then the information systems involved in each step of the procedure were surveyed. In addition, practical design requirements that should be considered and the preferred information design layouts from an expert’s perspective were collected.

Based on the collected information, prototypes of the user interfaces for the integrated navigation information system were designed to fit a transparent display that will be used in future usability testing experiments. The figure specializes in user interface prototyping and was used to design a screen for the integrated navigation information system. The GUI of the integrated navigation information system is designed to be optimized for a 55-inch 3840 px × 2160 px resolution transparent LCD with plans to install it on a ship and verify its usability in the future.

The overall ship configuration and main tasks were investigated to identify the duties of various officers on the ship and the types and procedures of the main tasks during ship operation. Stakeholders on the ship were divided into deck, engine, and communication departments centered around the captain, and the deck department was identified as the department that directly contributes to navigation. The deck department is composed of a navigator and helmsman, and the navigator is the main person performing the tasks for ship movement. The main tasks during ship operation include unberthing, berthing, propulsion, turning, deceleration, and anchoring, as listed in Table 1. The general procedures for each task were identified.

Table 1 . General navigation tasks in a ship.

Task	Description
Unberthing (departure)	The process of a ship leaving the berth or dock
Berthing (arrival)	The process of bringing a ship into a designated berth at the port or dock
Propulsion	The process of using a ship’s propulsion system to move it forward or backward
Turning	Changing the direction of a ship
Deceleration	The process of slowing down a ship’s speed
Anchoring	The process of securing a ship in place by lowering an anchor to the seabed

The international standards for designing navigation information systems and literature related to the design of HUD-based information systems for vehicles and aircraft have identified important considerations when designing integrated navigation information systems. Pečečnik et al. [16] recommended providing an appropriate amount of information for better driving performance and safety, as displaying too much information on the HUD can increase cognitive load. Li et al. [17] found that a composite design is more effective for task efficiency and performance through a flag visual display designed only with symbols, a perspective visual display expressed only with perspective, and a flag and perspective visual display that provides symbols in a general form and expresses distance information with perspective. These were utilized as useful information when designing the screen in this study. In addition, regarding the colors of design elements, Gabbard et al. [18] suggested that blue, green, and yellow AR colors are relatively strong in HUD graphics, but red and brown are not. Zhong et al. [19] applied seven colors (red, green, blue, yellow, pink, yellowred, and white) to digital characters displayed on a HUD in a vehicle, and the results showed that the yellow and green characters were the most visible during the day and the red and yellow characters were the most visible at night [19]. Finally, Yamin et al. [20] confirmed that the colors of the buttons and backgrounds of the HUD interface while driving significantly affect the driver’s accuracy and response time, and the most effective button color is orange and the least effective button color is blue [20], and thus the recommended colors to be applied to design elements when developing HUDs were identified.

Through expert interviews with navigators, detailed procedures for various ship operation tasks and the characteristics of the practical environment that should be considered when designing an integrated navigation information system were identified. The detailed procedures for major tasks during ship operations were identified from the perspective of practitioners. Fig. 1 shows the procedure for collision avoidance maneuvers when another ship is discovered during various ship operational tasks. For such collision avoidance maneuvers, ships with an avoidance duty and ships with a maintenance duty are determined by the international regulation COLREGs, and intership communication during maneuvers is possible using communication equipment. However, in reality, many small ships do not comply with COLREGs regulations during navigation, and some ships do not even have proper communication equipment, which requires continuous and faithful surveillance along with quick judgment and execution. Navigation continues even at night; therefore, it is necessary for navigators to continuously maintain a darkadapted state to discover and avoid obstacles at sea in the dark. Therefore, not only are the lights outside the ship that can be directly observed from the wheelhouse turned off, but the lights generated by the information systems inside the wheelhouse are also blocked or set as dark as possible. Regarding the new HUD-based navigation information system, opinions were gathered that because there are currently too many information systems that navigators must check and operate, which distracts them from focusing on the lookout, an alternative system that can integrate and manage existing systems is needed rather than applying a new system.

Figure 1. Task procedure for collision avoidance against other ships.

Based on the design considerations identified through the literature review and expert interviews, three concepts (lookout-focused, information-focused, and nighttime-supportive) of integrated navigation information system screens were designed, as shown in Fig. 2. The lookout-focused type minimizes information in the field of view and provides information at the top of the horizon and bottom of the display, which are peripheral visions, so that the navigator can focus on the most important task of the lookout. Informationfocused types provide information gathered from various information systems onboard a ship.

Figure 2. Three concepts of user interface for integrated navigation information system.

When a navigator needs it, it can quickly return to the lookout task without switching between various systems, providing as much information as possible on a single screen. The nighttime-supportive type maintains a minimum brightness level to support the navigator's physical capabilities (vision and hearing) during nighttime navigation, which significantly relies on the navigator's human capabilities while providing supportive information that is difficult for the navigator to observe directly.

This study proposed a user interface for a HUD-based navigation information system that provides various types of information inside and outside the ship to enable navigators of autonomous ships that satisfy onboard decision support to make efficient and effective decisions. The proposed HUD-based navigation information system is designed to provide information from various information systems installed on a ship in a single information system. Various systems are installed on ships, such as a system that provides information on the ship’s speed, direction, and passage plan related to the ship’s operation, a system that provides information on obstacles and other ships at sea, and equipment for communication. Navigators must move among various information systems to collect information. However, because continuous and close surveillance of the sea situation is essential for safe navigation, the use of various information systems has the disadvantage of distracting navigators. The HUD-based navigation information system proposed in this study is designed to enable navigators to acquire essential information necessary for navigation while maintaining surveillance. To design a user interface useful for navigation tasks, the detailed procedures of the navigation task were analyzed, and the types and source systems of information required at each step of a task were identified. The user interface of the proposed HUD-based navigation information system is designed to be easily switched between three modes – lookout-focused, information-focused, and nighttime-supportive – so that it can be selectively used depending on the context of the navigator. Through the lookout-focused user interface, the navigator can check essential information that must be checked frequently while focusing on the most important task, lookout. When detailed information is required depending on the situation, the navigator can switch to the information-focused user interface and check detailed information provided by other navigation information systems without leaving the lookout location. In addition, at night, when visibility is limited, the nighttime-supportive user interface can provide information using highly sensitive sensors without obstructing the navigator’s vision. The proposed navigation information system with a user interface is expected to be a useful trigger for the introduction of autonomous ships. However, this study requires additional analyses and design applications for the various types of information provided by existing information systems.

The proposed HUD-based user interface derives the overall direction of the user interface based on the overall characteristics of the navigation tasks and pain points of the navigators. However, it is necessary to analyze in detail the types of information provided by various information systems installed on existing ships, and to integrate and reprocess information through middleware to develop an advanced navigation information system. In addition, a detailed design must be developed to effectively provide the identified detailed information elements to navigators. For example, in the case of ships, the direction of the bow and the actual direction of the ship are almost always different owing to factors such as wind direction and ocean currents. Therefore, it is necessary to provide information on the direction of the bow and the direction of the ship; however, detailed design standards or guides for effectively conveying specialized information for such navigation tasks still require much research. Therefore, we plan to develop a prototype of an HUD-based navigation information system and conduct follow-up studies to evaluate the usability and usefulness of the user interface proposed in this study to support navigation tasks.

The prototype of the HUD-based navigation information system proposed in this study can be implemented using a large transparent display, and various design alternatives can be developed to verify its usefulness for providing information during navigation. Further research is expected to be conducted to derive design specifications that can provide information most efficiently during navigation through variations in the size and color of the information elements.

This results are supported by “Regional Innovation Strategy (RIS)” through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (MOE) (2021RIS-003).