May 2017

Human-machine interaction in MANTIS project

Proactive, collaborative and context-aware HMI

One of the objectives of the MANTIS project is to design and develop the human-machine interface (HMI) to deal with the intelligent optimisation of the production processes through the monitoring and management of its components. MANTIS HMI should allow intelligent, context-aware human-machine interaction by providing the right information, in the right modality and in the best way for users when needed. To achieve this goal, the user interface should be highly personalised and adapted to each specific user or user role. Since MANTIS comprises eleven distinct use cases, the design of such HMI presents a great challenge. Any unification of the HMI design may impose the constraints that could result in the HMI with a poor usability.

Our approach, therefore, focuses on the requirements that are common to most of the use cases and are specific for proactive and collaborative maintenance. A generic MANTIS HMI was specified to the extent that does not introduce any constraints for the use cases, but at the same time describes the most important features of the MANTIS HMI that should be considered when designing the HMI in individual use cases.

MANTIS HMI specifications are the result of refinement of usage scenarios provided by the industrial partners, taking the general requirements of MANTIS platform into account. Functional specifications describe the HMI functionalities, present in most use cases and abstracted from the specific situation of every single use case.

We describe a generic static model that can be used together with the requirement specifications of each individual use case to formalize the structure of the target HMI implementation. The model has been conceived, in particular, with two ideas in mind: (i) to provide means that would help to identify the HMI content elements and their relationships of a given use case and (ii) to unify (as much as possible) the HMI structure of different use cases, which is useful for comparison of implementations and exchange of good practices. When setting up the model structure we follow the concepts of descriptive models applied in task analysis and add specifics of MANTIS, denoted as MANTIS high-level tasks. For each of these high-level tasks, we provided a list of functionalities supporting them.

MANTIS human-machine interaction comprises five main aspects:

  • User interfaces;
  • Users;
  • MANTIS platform;
  • Production assets; and
  • Environment.

Through their user interfaces, several different users within the use case communicate with MANTIS platform, which in term communicates with production assets. Interaction can take place in both directions. Users can not only access the information, retrieved from production assets and stored on the platform but provide an input to the MANTIS system as well. They can initiate an operation which is then carried out by the platform, such as rescheduling maintenance task, or respond to a system triggered operation, for example, alarms. On the other hand, through the MANTIS platform, users can also communicate among themselves. In addition to the straightforward communication in terms of the textual or video chat functions, the users can also communicate via established workflows.

The last but not least main part of the interaction is also the environment. Although it can be treated neither as a direct link between the user and the system nor as a part of communication among the users, the environment can influence the human-machine interaction through the context-aware functionalities.

From the users’ point of view, the human-machine interaction within the MANTIS system supports five main high-level user tasks associated with proactive and collaborative maintenance:

  • Monitoring production assets;
  • Data analysis;
  • Maintenance tasks scheduling;
  • Reporting; and
  • Communication.

While monitoring production assets, data analysis and maintenance task scheduling are vital for proactive maintenance, reporting and communication enable collaboration among different user roles. Each of these tasks is carried out by a number of MANTIS specific functionalities that can be classified as user input, system output, user- or system- triggered operation. These functionalities should cover all the main aspects of MANTIS human-machine interaction and should also be general enough to be applicable to any MANTIS as well as potential future use case.

MANTIS HMI demonstrator

At the MANTIS meeting in Helsinki, the first version of the web based HMI demonstrator, developed in with Angular Dashboard Framework and other Javascript libraries by XLAB, was presented to the MANTIS consortium. Currently it is connected to the MIMOSA database and is demonstrating live data from the FORTUM use case. The HMI is designed as a customizable, user-dependent, responsive multi-widget dashboard, comprising basic read-only widgets, such as graphs and tables. The features of the demonstrator follow the HMI functional specifications and it is designed in the way that can be applied to any use case with MIMOSA database.

The first version of the web-based HMI demonstrator
The first version of the web-based HMI demonstrator

In the near future, many other features will be implemented, including more widget types, dashboard navigation, search function and sharing of data views. Some context-aware features, such as hidden widgets that appear when needed and suggestions of further user actions based on the usage history, will be implemented as well. In addition, general visual design recommendations such as colours, fonts and widgets positioning, described earlier in the project, will be applied.

Helsinki Consortium Meeting in May 2017, and the Conventional Energy Production Use-Case

The second (sixth overall) full consortium meeting of 2017 was held between 8th and 10th of May. This time it was hosted by VTT at their new Center for Nuclear Safety located in Espoo, Finland. The three day event gathered 65 participants from all of the participating countries. The program was more technologically oriented and contained a long open space session, where partners could present their work within the project. The tight program allowed some time to enjoy the wonderful Finnish spring weather.

The wonderful Finnish spring
The wonderful Finnish spring

Finnish use-case was prominently on display at the Open Space session at the MANTIS consortium meeting. The floor in the first open space room was dedicated to the Finnish use case. Presented were Nome, Wapice, Fortum, VTT and Lapland University of Applied Sciences (UAS). Each partner presented their work done in the Finnish use case. Wapice and Fortum presented their HMIs (IoT Ticket and TOPi respectively). Nome and VTT presented their measurement systems (NMAS and the affordable sensor research respectively) and finally Lapland UAS provided the database and REST interface that allows each partner to share and access data beyond organizational boundaries. The second room had most use cases represented. Of note was XLABs common MANTIS user interface demo that can be connected to the Finnish use case platform.

Open space session at Helsinki Consortium meeting
Open space session at Helsinki Consortium meeting

The Finnish use case is centered on a flue gas recirculation blower located in Fortum’s Järvenpää power plant. The blower is classified as a critical component in the energy production process and is monitored closely. In this use case Wapice, Nome and VTT have all provided their own sensors or virtual sensors to monitor the performance and condition of the blower. In addition, Lapland UAS has a few Wzzard sensors, made by B+B Electronics/Advantech, provide some additional measurement data bulk. However these are not related to the Järvenpää case. This measurement data is stored, using the REST interface developed by Lapland UAS, in the MANTIS database that is based on the MIMOSA data model.

Flue gas recirculation blower in Fortum’s TOPi Proview browser
Flue gas recirculation blower in Fortum’s TOPi Proview browser

The REST interface and MIMOSA database mapper provides a simple an interface, which is both easy to use and to integrate, between different applications and systems. It provides basic CRUD –functionalities and contains a mapper that maps measurement system specific data formats and structures into MIMOSA compliant data structures to ensure interoperability and compatibility with the MIMOSA data model. It is widely in use in the Finnish use case and research partners from both Slovenia and Hungary have shown interest towards utilizing MIMOSA in their use case.

A diagram of the Finnish use case
A diagram of the Finnish use case

SmartG presentation in Hannover Messe

Goizper and IK4-TEKNIKER will be present at the Hannover Messe from 24 to 28 April 2017‎ presenting Smart G, a data acquisition module for clutch-brake monitoring.

Clutch-brake systems produced by Goizper are key components in cutting, forming, folding and press machines.

The aim of this presentation is to show how incorporation of the Smart G module can convert a clutch-brake system into a monitorable smart component, which includes self-diagnostics capabilities that can provide information about the current state of the component and predict failures before they occur.

Communications modules incorporated in the Smart G component provides capabilities to:

  • Remotely monitor the component
  • Send the data to a cloud platform where all historic data are stored.

Having the data of Goizper’s clutch-brakes fleet on a cloud platform will provide the possibility to use more advanced techniques and algorithms in order to predict failures and/or remaining useful life of key components of the system.

Benefits are two-fold: Goizper will drastically improve the knowledge about their equipment to improve reliability of their products and the maintenance services provided to customers, while customers will benefit from a reduced downtime of the machines and more cost-effective maintenance strategy.

Goizper and Tekniker’s work on failure prediction and diagnosis, as well as cloud platform development, have received funding from the European Union under the MANTIS project.


Smart G concept block diagram
Figure 1. Smart G concept block diagram.


Figure 2. General status of the machine.




Figures 3 and 4. Braking and Clutching processes performance

Figure 5. Active alarms and alarms history


Figure 6. Product pictures