Introduction
The airport is the most important strategic facility, linking a number of air, railways, and highways into a single network. It forms a transnational hub, handling huge flows of passengers through its terminals. The airport plays the role of an air gate not only for a single city but sometimes for an entire region. The airport as a transport hub looks the most attractive to terrorists. An act of terrorism at an airport or on an airplane has the most profound effect on the minds of people. Due to their special importance and attractiveness to terrorists, airports have long been equipped with security systems, and this is a continuous process requiring constant modernization.
Augmented reality is one of the promising areas of technological development. It has the potential to make people’s relationships with information related to security more ergonomic. Data will be automatically delivered to users in the required context for various everyday situations, thus, the technology raises the interaction of a person with information to a fundamentally different level, which is critically important in security systems of such strategic objects as an airport.
State of the Art
The vast majority of airport borders are protected by multiple layers of security, which consist of walls, fences, barbed wire fences, video surveillance systems, security structures, and patrols. These measures can be effective, but they are mostly passive barriers that can be easily bypassed by attackers and criminals. For the most part, perimeter security remains fairly low, and as a result, many airports try to unsuccessfully detect perimeter violations and fail to stop intruders. Despite the combination of different technologies in airport security strategies, it would be right to say that an airport needs to further digitize its perimeters in order to defend its borders more effectively. Thus, the main focus of airport security operations should be on the investment, implementation, and integration of technologies such as artificial intelligence systems, distributed acoustic scanning, and other innovations in digital technologies to improve airport management and security.
Augmented Reality Technology: Specifics, Possibilities, and Advantages
Augmented reality (AR) is a technology for real-time superimposing text, audio, or graphics information and other virtual objects on real objects. It is interesting to note that the authorship of the term “augmented reality” belongs to Thomas Preston Codell, an engineer at the Boeing Research Laboratory. In 1992, he applied the principles of the technology in a system designed to help workers install electrical cables on airplanes (Aukstakalnis, 2016). The augmented reality technology market is young and still small; it is currently dominated by startups pushing this innovation forward. However, the market has high potential and will be characterized by high growth rates over the next 5-10 years (Deloitte, 2019). The global AR market is projected to reach $60.5 billion in 2023 (Tromp et al., 2020). In AR, a person can interact with a three-dimensional, computerized environment, as well as manipulate objects or perform specific tasks. Achieving the effect of complete immersion in virtual reality to a level where the user cannot distinguish between visualization and the real environment is the task of technology development.
The general scheme for creating augmented reality in all cases is as follows: the camera of a computer device takes an image of a real object, the software scans and identifies the resulting image, and creates its virtual model using various sensors and databases. Thus, it builds a visual complement of a real object, combining its real image with a visual complement on the screen of a visualization device – smart glasses, smartphones – making the change in the completed visual image – a digital “superstructure” – dependent on changes in the characteristics of the physical object in real time (Schmalstieg & Hollerer, 2016, p. 26). Since the virtual and real worlds coexist harmoniously in the digital space, users are able to perceive and interact with a more informative version of reality, in which virtual information is used as an additional tool to support the user. Moreover, almost any modern smartphone or tablet can become an augmented reality device; one has to install an appropriate application that allows recognizing objects using QR markers, generated points, logos, and using computer vision and face recognition.
Augmented Reality Technology in Safety Sector
AR is becoming an increasingly popular trend in various fields and the security industry is not an exception. The emergence of a hardware basis for AR solutions and the provision of application programming interfaces by augmented reality hardware manufacturers made it possible to introduce AR technologies in the security industry. In particular, for system integrators, modern AR developments can be very useful. Security experts predict AR in the safety and security industry as a promising tool for improving effective response time (Zhu & Li, 2021). For example, S3 Security and Defense Consultants recently purchased VuZix M100 goggles for testing during security assessments, for example, to call up the floor plan of a building, as well as to build the fastest and safest route to travel around the facility (Tromp et al., 2020). In addition, the use of augmented reality provides advantages in the installation and maintenance of security systems. Experts also mention the advantages of quick photo and video capture, in order to reduce the response time to an alarming event (Chen et al., 2019). Experts cite “X-ray vision” and accurate GPS positioning as the technology development potential in the security industry (Shinde et al., 2020). Augmented reality can be used to reduce incident response times. The effect can be achieved by faster assessment of the situation at the scene of the incident and the elimination of false alarms. Decision-makers in the security management center will be able to quickly send employees to the scene of an incident.
Augmented Reality Technology at the Airports
Unlike many IT inventions, AR does not work independently, but works well with a person, and supplements the human mind with valuable and accurate information, which can only be remembered and reproduced without error by highly qualified specialists with extensive work experience. The user of the AR device sees the virtual and the real at the same time. A prerequisite for combining the real and virtual worlds is knowledge of the spatial position of the observer, which makes it possible to form images of virtual objects with the required angle and scale. Several main features can be distinguished with the help of which it is advisable to classify AR systems for use at airports in order to control passenger flows and continuous monitoring of the situation (Eschen et al., 2018):
A way of positioning the observer and virtual objects
- Positioning method using special positioning systems. Such systems allow obtaining 3-‑linear and 3‑angular coordinates of the object’s spatial position. They are built on the basis of different physical principles of functioning – electromagnetic, inertial, acoustic, optical, using navigation satellites; often the end result is obtained by combining several subsystems of different types. Their main disadvantage is the need to use additional equipment with stationary placement and its high cost.
- Positioning method using graphic markers, when special graphic markers are used, the image of which is entered using a video camera. Further, the marker image is highlighted in the general video image and processed to determine its position in space, and the obtained coordinates are used to bind the virtual object. The disadvantages of the system are that it is operational only if the markers are in the field of view of the camera and they are clearly distinguishable; in addition, there is the need for special placement of markers on real-world objects.
- A positioning method by recognizing images of real objects. Determination of the position of the observer in space is carried out according to the same scheme as in the previous case, however, instead of special graphic markers, ordinary objects known to the system are used, which complicates the recognition algorithms and requires much higher processor performance. The approach is viewed as promising for AR systems.
- Positioning method by combining data from built-in sensors of mobile devices without using markers. This method is used in mobile devices, which are characterized by a relatively low processor power, which does not allow the use of advanced pattern recognition algorithms. For example, tracking points are selected in the image, and then the position of the camera is determined, taking into account the accelerometer data; another version involves the use of the accelerometer and the camera’s autofocus mechanism to determine the distance to the object (Eschen et al., 2018). The approach does not require any preparation of the environment but does not always provide reliable and accurate positioning.
A way of displaying the real world
- Display method using video cameras. The simplest and most widespread method today is to display the real world using a video camera built into a computer. Stereoscopicity is achieved by using a virtual reality helmet and two video cameras placed in front of each user’s eye, the signals from which are transmitted to the corresponding helmet microdisplays. The main disadvantage is associated with the cumbersomeness of such a solution and the problems of compact video cameras. This method is of little use for effective airport security control.
- A way of displaying with transparent or translucent panels used in helmets or special glasses. A large number of developments in this area with the use of projections onto translucent surfaces, holographic and other methods have been known for more than 20 years, but none of them has been accepted by the market due to shortcomings, both technical and economic (Tromp et al., 2020). However, since 2011, several proposals for compact devices of this type with acceptable consumer properties have appeared on the market.
- A display method by projecting an image of virtual objects directly onto the retina of the user’s eye. This method is often used in mobile military applications where high-quality photorealistic images are not required.
- A way of displaying using special contact lenses. Experimental samples of such contact lenses include built-in means of displaying virtual objects.
The work of markerless systems capable of recognizing real objects is implied in a similar way. The main difference is that the problem of recognizing the image of a real object is much more complicated than recognizing a highly characteristic image of a graphic marker. In particular, the selection of image features becomes a very difficult stochastic task, however, Sony’s SmartAR technology, announced in 2011, looks promising.
A wide and intensively developing class is composed of AR algorithms that provide positioning and scaling based on built-in sensors of mobile devices, which do not require any special preparation of the environment in the form of markers placement or computationally cumbersome image recognition procedures. The three-axis accelerometer, which has already become the standard for smartphones, in principle, allows linear coordinates to be obtained by double integration of its data during movement, and the existence of gravity makes it possible to calculate the roll and pitch angles (Zhu & Li, 2021). The yaw angle can be obtained from the digital magnetic compass and, in the latest smartphones, from the built-in laser gyroscope.
However, the accuracy of the coordinates obtained in this way turns out to be insufficient – due to the accumulation of the zero drift error during double integration, the influence of the magnetic field of the external environment, and the reactivity of gyroscopic devices. In this context, one can mention the Japanese development, where positioning is implemented using an automatically generated database of environmental objects and a laser rangefinder. Such developments are the most promising for ensuring security at airports, since they allow monitoring passenger traffic with high accuracy and unnoticed by others, identifying suspicious persons, and continuously assessing the situation in the buildings and premises of the airport and on the runways.
The task of the display device in AR systems is to combine the real and virtual worlds in the picture presented to the user; therefore, such a device has a dual character, which includes means for reproducing real and virtual objects. The simplest and most widespread version of such a device is a combination of a built-in video camera and a display of modern mobile computers in the form of laptops, tablets, and smartphones. AR display devices using contact lenses are actively being carried out at the University of Washington in conjunction with the research division of Microsoft (Shinde et al., 2020). This solution involves the use of contact lenses, in the center of which there is a small area that transmits and focuses only the image from the display and is surrounded by an area that filters this signal, but at the same time transmits the image of the surrounding world.
The expediency of using such systems is also justified by the need to monitor the airfield. Air traffic controllers working at the aerodrome tower receive information about what is happening at the aerodrome mainly through direct observation of the airfield, while instrument data play an auxiliary role. It should be noted that experiments are underway on the less obvious application of AR. For example, to improve passenger safety and comfort, replacing aircraft portholes with flexible OLED displays is proposed (Shinde et al., 2020). They can broadcast current views overboard, supplemented by flight parameters and entertainment content.
Conclusion
The most obvious advantage of augmented reality is the ability to ‘decouple’ the operator from the workplace and the potential abandonment of centralized control posts for video surveillance systems. AR allows binding any digital data to objects of physical reality and controlling their output using intuitive gestures. At the same time, the data output is carried out on a personal basis, which simplifies the implementation of security measures for data access. An important feature of augmented reality hardware is the tracking of the user’s gaze direction with special sensors. The data of this tracking directly affects the content and form of presentation of the data provided to the user.
The fact described above itself significantly affects the organization of security processes: for example, it makes no sense for a security guard wearing an augmented reality helmet to inform the central post about his location by radio communication. The operator of the central post can transmit operational orders in the form of text messages, which is extremely important for quick response that is invisible to an attacker in the event of a threat of a terrorist attack at an airport. In addition, machine vision, on which augmented reality systems are based, lies on the transmission of spatial images – this allows not only to effectively embed elements of virtual reality into the operator’s field of view but also to transmit the “picture” to other users, for example, for expert assessment of the situation. Thus, AR technologies represent a promising area of airport security in today’s environment of growing threats from national and international terrorism.
References
Aukstakalnis, S. (2016). Practical augmented reality: A guide to the technologies, applications, and human factors for AR and VR. Addison-Wesley Professional.
Chen, Y., Wang, Q., Chen, H., Song, X., Tang, H., Tian M. (2019). An overview of augmented reality technology. Journal of Physics: Conference Series, 1237(2). Web.
Deloitte (2019). Virtual, augmented, and mixed reality for defence and the public sector. Web.
Eschen, H., Kotter, T., Rodeck, R., Harnish, M. (2018). Augmented and virtual reality for inspection and maintenance processes in the aviation industry. Procedia Manufacturing, 19, 156-163.
Shinde, G. R., Dhotre, P. S., Mahalle, P., Dey, N. (2020). Internet of things integrated augmented reality. Springer.
Schmalstieg, D., & Hollerer, T. (2016). Augmented reality: Principles and practice. Addison-Wesley Professional.
Tromp, J., Le, D., & Le, C. (2020). Emerging extended reality technologies for Industry 4.0: Early experiences with conception, design, implementation, evaluation and deployment. Wiley-Scrivener.
Zhu, Y., & Li, N. (2021). Virtual and augmented reality technologies for emergency management in the built environments: A state-of-the-art review. Journal of Safety Science and Resilience, 2(1), 1-10.