HUMAN-MACHINE INTERFACE RELEVANCE AND NEW REQUIREMENTS
The adoption of IoT, AI, Robotics and Unmanned Vehicles has resulted in an increase in the range, complexity and number of automated tasks conducted by machines. This is does not imply the full replacement of human labour, but rather a change of human jobs to provide direction, and to collaborate with machines, which requires a new and improved human-machine interaction.
KETTL enabled machines in warehousing, logistics and manufacturing bring new workflows that include both physical, and also digital work information. Their use requires constant and intense exchange of information, increasing the complexity of the interaction between human workers and machines.
Traditional human-machine interfaces are insufficient in facillitating enough context and efficient interaction between human and advanced machines. For this reason, a wave of new interface technologies powered by AI and IoT and pioneered in the creative industries, has already been applied to manufacturing and adoption, and it is rapidly extending to Transport and Logistics, bringing operational efficiency to a level unimagined a decade ago.
EXTENDED REALITY, DEFINITION AND CATEGORIES
This new wave interface technology is included under the umbrella of Extended Reality. These technologies enhance human senses, providing additional information, either about the actual world or through simulated worlds for humans to experience, with the objective of monitoring, managing and making decisions working with advanced machines. It includes Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) technologies.
Virtual Reality (V.R.)
VR implies a complete immersion experience in a computer-generated environment. Even as VR has been used to simulate some physical operations or environments, it has limited applicability in Transport and Logistics, and its development and adoption exists mainly in the creative sector, and to a minor extent in retailing.
Augmented Reality (A.R.)
AR Is the next-generation human-machine interface to bridge the workflow gaps in tasks and maximise efficiencies of human and machine interactions. Unlike VR, which requires the user to inhabit a virtual environment, AR applications superimpose digital information like digital 3D, computer generated graphics and images of the equipment on top of the user’s physical environment. AR thus provides a more natural environment for workers to instruct advanced machines to actuate, often through human physical motion. Additionally, when machine tasks have to be altered, AR offers a virtual, immersive, and interactive dashboard that avoids costly reprogramming stops.
Mixed Reality (M.R.)
Mixed Reality is an emerging technology that combines VR and AR, facilitating working in a real-world environment using some virtual objects. For example, an immersive rendering of the internal components and functioning of a device that the worker is using.
The VR, AR and Mixed Reality elements powered by AI and IoT can be combined in different proportions, allowing for specificities of the machine, workflow and tasks being conducted. Also, proportions are adjusted for the tasks related to the machine being conducted by the human operator, such as monitoring, scenario testing, task modifications and overall real-time optimisation. For example, recently improved VR simulations are used to make proactive, dynamic, and computerised adjustments without written reports, nor direct physical work on the machines.
NEW EXTENDED REALITY ADOPTION AND EFFECTS
The following table summarises some of the uses of AR and VR in Transport and Logistics by companies, operators, planners, and administrations. Also, the use of advanced human-machine interfaces is a requirement for the optimal use of other KETTLs already deployed, like IoT enabled machinery, unmanned vehicles and drones and advanced robotics.
Data Access and Environment Interaction
Enhance Human Capabilities
Equipment assembly, use and maintenance, Human training
Remote support, Fulfillment Claims
Navigation by virtual display in autonomous vehicles and drones
Functional design optimisation in warehouses, urban spaces, factories and public use infrastructures, Inventory and in-transit monitoring, Disruption and safety response, Consumer virtual navigation of facilities or travel
Space and environment monitoring, Buildings and corporate or public facilities inspection
Spatial optimisation for order packing, and shipment organisation, Order picking
Product, construction, urban and infrastructure design
Operations monitoring, Production line optimisation, Quality Control, Equipment monitoring and diagnostics
Operator instruccions for large equipment and also precision processes,Task sequence and assembly work instructions, Asset installation, Maintenance instruction and guidance
CETMO Analysis, adapted from Boston Consulting Group (1)
Regarding its maturity and adoption, creative and entertainment sectors are at the forefront, followed by retail. Also, the adoption of other KETTLs has generated the need for improved interfaces and accelerated its adoption by the Manufacturing and Transport and Logistics sectors.
While these new human-machine interfaces are a logical continuation of the deployment of other KETTLs, they also have some potential strong effects in customer relationship, adding a degree of empathy absent from automated digital interactions, and the possibility to offer new customer experiences, particularly in passenger transportation.
Regarding social and Country Disparity, AR, VR and MR are potent tools to improve economic, urban, agricultural and health care Planning. In addition, they make it possible for the first time, the management of infrastructures and services in underdeveloped and remote areas by specialised human operators, located in distant central or regional operational centers.
CETMO Analysis, adapted from McKinsey & Company (2)
Yusuf, Lukic, Boston Consulting Group-BCG. Unleashing the Power of Data with IoT and Augmented Reality. 2020
McKinsey & Company. Ashutosh, Hastings, Murnane, Neuhaus, Automation in Logistics: Big Opportunity, Bigger Uncertainty, 2019. link
HUMAN INTERFACE, AUGMENTED AND VIRTUAL REALITY EFFECTS ON TRANSPORT PROCESSES
These technologies, which are less mature than those presented before, are starting to reduce the risk of work accidents, improve human resource conditions and increase their skills. In passenger transport, it allows for the offering of new services to users and the improvement of their travel experience. Additionally, the simulation of real conditions and situations opens the door to new possibilities in planning, operations, business relations and support. Some effects include:
Increased Use of Information and Efficiency: Augmented Reality facilitates the interaction between employees and machines and allows to locate, extract relevant information, finalise tasks and solve problems more efficiently. For example, more agile physical search for cargo and merchandise, with less risk of losing merchandise, less time spent searching department stores, and less time for cargo operations.
Error Reduction: Particularly, in picking errors, and in general, any documentary and physical errors in shipments. It also allows an increase in quality, efficiency and safety, in tasks such as maintenance or supervision.
Staff Training: simulating real situations in areas or contexts that are risky or expensive to reproduce or deal with in another way. It also offers the possibility of optimised cost-efficient training in all areas.
Service Quality Increase: by creating simulated situations which allow for the identification of inefficiencies in processes and protocols, plus generating scenarios and situations not contemplated previously. Additionally, simulations can be created to optimise operations and services by virtual trial and error at a near insignificant additional cost.
Increased Risk Management Capabilities: simulations allow for the generation of risk situations not contemplated before, that facilitate the creation of contingency and emergency plans.
Increased Use of Information and Efficiency: Similar to Freight Transportation, augmented reality facilitates the interaction between employees and machines, to extract relevant information, finalise tasks and solve problems more efficiently. For exemple, in Passenger transport, it will facilitate and improve the process of identification and location of vehicles, etc., allowing for more speed and greater control, both for the operating company and for the user.
Marketing, Pre-testing and Sales: Smart HMI changes the way the products are sold (both from the supplier company, to the operator and, from the company to the user), as it will recreate reality in a more accurate way (for example, showing the simulation of future conditions, or the visual comparison with the current ones).
Staff Training: similar case to Freight Transport.
Customer Experience Improvement: can improve the service experience for the user, facilitating the recreation and simulation of processes related to the different stages of the trip, such as the arrival at the door or platform, vehicle access and installation, vehicle exit and station access, or station exit. Additionally, it can be achieved while reducing support tasks and cost.
Increased Risk Management Capabilities: same case as Freight Transport.
Sources: CETMO and “Impacte de les KETs en la digitalització dels diferents àmbits del transport”, CENIT-CINESI – December 2020