December 2016

Analyzing maintenance log data to predict system failures

Cyber-Physical Systems (CPS) are often very complex and require a tight interaction between hardware and software. As it happens in almost any software systems, also CPS  generate different kinds of logs of the activities performed, including correct operations, warnings, errors, etc. Frequently, the logs generated are specific to the different subsystems and are generated independently. Such logs contains a wealth of information that needs to be extracted and that can be analyzed in different ways to understand how the single subsystem behaves and even retrieve information about the behavior of the overall system. In particular, considering the generated logs, it is possible to:

  1. Analyze the behavior of a single subsystem looking at the data generated by each one in an independent way;
  2. Analyze the overall behavior of the system looking at the correlations among the data generated by the different subsystems

Such data are very useful to understand the behavior of a system and are often used to perform post-mortem analysis when some failures happen. However, such data could also be used to understand in a more comprehensive way how the system behaves through a real-time analysis able to monitor continuously the different subsystems and their interactions. In particular, it is possible to focus on preventing failures through predictive maintenance triggered by specific analysis.

Making predictions about system failures analyzing log files is possible but such predictions are strictly related to some characteristics of such files. In particular, some very important characteristics are: data generation frequency, information details, history.

The data generation frequency needs to be related to the prediction time and the time required to take proper actions. For example, if we need to detect a failure and take proper action in a few minutes, we need to use data generated with a higher frequency (e.g., in the scale of the seconds) and we cannot use data generated with a lower one (e.g., in the scale of the hours). This requirement affects the ability to make predictions and their usefulness to implement proper maintenance actions.

The information details provided need to include proper granularity and meaningful massages. In particular, it is important to get detailed information about errors, warnings, operations performed, status of the system, etc. The specific details required are tightly connected to the specific predictions that are needed. Moreover, the finer the granularity of the information, the higher are the chances of being able to create a proper prediction model.

High quality data history is required to build proper prediction models. However, just having a large dataset is not enough. Historical data need to be representative of the operating environment and include all the possible cases that may happen during  operations. In particular, it is required to have information about the log entries and the actual behavior of the system to create a reliable model of the reality.

The requirements described are just a first step towards the definition of a proper predictive maintenance model but they are essential. Moreover, the proper approaches and algorithms need to be selected based on the specific system and the related operating conditions.

Xtel Wireless ApS

Xtel-MANTIS

Xtel Wireless is a Danish company specialized in ultra-low power solutions and technologies. With a specialized knowledge and competences in developing state-of-the-art IoT (Internet of Things) products and wireless core technologies, Xtel Wireless has an expertise in developing innovative products. Xtel Wireless employs 15 engineers with an average experience of 15 years in developing high-tech and embedded solutions, as well as having design and innovation competences in the team.

Relevant Expertise for the project:

Xtel Wireless has an expertise in developing innovative products. In the IoT business segment, Xtel Wireless has developed a platform for wireless sensor modules. This platform has unique features in matter of performance, price level and size.

Role in the project:

Xtel Wireless will be involved in developing sensors for typical domestic households to provide the project with data, monitoring humidity, CO2-levels, temperature and VTO in the residential buildings. The aim is to collect and distribute the data, as well as present the data motivationally to the residents hereby encouraging changed energy behavior.

Sataservice Oy

Sataservice MANTIS

Sataservice (Group) Oy provides industrial maintenance services and projects for customers to reach high level of performance. The Group includes following companies, Sataservice Oy and Rauman Sähkökonehuolto Oy.

Both companies have perfect service portfolio and co-operation. Sataservice was founded in 2003. The headquarter is in Rauma, Finland. The company has been growing every year and for the moment we have approximately 350 employees. The turnover 2016 is around 35 M€. We are ISO 9001:2008 certified by Bureau Veritas.

The company have expertise of demanding environments which means that mainly our customers are in businesses that have demanding regulations and laws that have to be followed, for instance the food industry where we are involved with living animals and in the end of the process we handle ready made food. We are also involved in the medical business with its own regulations. We mainly have customers that have outsourced their maintenance, either production or facility maintenance or both. We do also lot of projects such as modernizations of old equipment that ensures that the lifecycle can be extended and the productivity is high for the customer.

We have expertise in mechanical, electrical, automation, HVAC and cranes. Rauman Sähkökonehuolto is specialized in electrical motors, pumps and gears and helps us to keep up the customers productivity, while we can manage ourselves most of all maintenance areas and can minimize unnecessary waiting time. At the moment most of our customers are in Finland but we are aiming to expand abroad when we find the right partners and customers.

Relevant Expertise for the project:

We have a really broad expertise in maintenance of production equipment, especially machining, packing machines, electrical motors, automation and so on. We have divided our business in different technologies and all of those have a dedicated leader for instance for cranes, HVAC and automation.

Role in the project:

We have a good possibility to provide environments to test use-cases. We want to develop ourselves our IoT and CBM strategy and clarify what we can and should offer our customers. The aim is that everyone, meaning our customers, ourselves and possibly third parties, sees it as a win-win situation. We also want to find good long term partners to execute the strategy.

Neogrid Technologies ApS

Neogrid MANTIS

Neogrid Technologies ApS, is an entrepreneur company working with intelligent energy visualisation, monitoring and control, utilising knowhow within wireless communication technology to develop Smart Grid solutions.

Neogrid Technologies develop intelligent forecast based energy management systems for both consumers, energy companies and 3rd party actors, enabling home energy management capabilities for house owners and large scale monitoring – allowing optimised individual and aggregated controlling.

This consists of a data acquisition and IoT platform and online control-interface. It includes advanced analytics, which are able to forecast energy consumption and flexibility, based on individual house modelling and advanced model predictive control.

The aim is to enable the user to monitor, plan and even shift energy consumption based on knowledge of price, average consumption patterns and conditions, allowing for cost reduction and increased control and overview. The integration between hardware and software goes well with the background and experience of the people behind Neogrid Technologies.

Furthermore, Neogrid Technologies develop and deliver a cloud-based system for monitoring and optimized individual and aggregated control of heat pumps and district heating buildings based on requirements from the energy system actors and 3rd party actors.

Relevant Expertise for the project:

Neogrid Technologies have by participation in selected research project gained more than 6 years’ experience developing control strategies and cloud-applications for heat pumps and district heating focusing on the power and district heating system and end user requirements. Through this work, Neogrid Technologies have gained deep knowledge and experience regarding practical challenges arises when 80 live heat pump installations are centrally controlled.

Role in the project:

Neogrid will Participate in WP7, task 7.3, where an integrated energy system comprising of energy production, distribution and energy consumption of buildings (electricity and district heating) can be forecasted, optimized and moved according to various needs in the energy system.

Via aggregated control of a number of buildings (district heating and heat pumps), besides what is mentioned above, the predicted energy consumption can be taken into account when planning maintenance outage/interruptions so the influence or disadvantage of the heating service towards the consumers is minimized.

The above mentioned IoT platform Neogrid will develop and deliver services where the Product PreHeat and aggregator among other will be used to simulate and analyze the energy system with respect saving energy, monitoring and predicted maintenance.

Reference Architecture of the Portuguese Mantis Pilot

Introduction

The MANTIS Steel Bending Machine pilot aims at providing the use case owner – ADIRA – a worldwide remote maintenance service to its customers. The main goal is to improve its services by making available new maintenance capabilities with reduced costs, reduce response time, avoiding rework and allowing for better maintenance activities planning.

To this purpose existing ADIRA’s machines (starting with their high end machine model – the Greenbender) will be augmented with extra sensors, which together with information collected from existing sensors will be sent to the cloud to be analyzed. Results made available by the analysis process will be presented to machine operators or maintainers through a HMI interface.

adira greenbender gb-22040 MANTIS
Adira –  Greenbender GB-22040

A number of partners are involved into the development and testing of the modules, which regard the communication middleware (ISEP, UNINOVA), data  processing  and  analytics activities (INESCISEP), the HMI applications (ISEP), and a stakeholder providing a machine to be enhanced with the MANTIS innovations (ADIRA).

System Architecture

The distributed system being built responds to a reference architecture that is composed by a number of modules, the latter grouped into 4 logical blocks: the Machine under analysis, Data Analysis module, Visualization module, and the Middleware supporting inter-module communications.

architechture of the maintenance system for MANTIS
architecture of the maintenance system for MANTIS

Machine

Data regarding the machine under analysis are collected by means of sensors, which integrate with the machine itself. This logical block consists thus of data sources that will be used for failure detection, prognosis and diagnosis. This set of data sources comprises an ERP (Enterprise Resource Planning) system, data generated by the machine’s Computer Numerical Controller (CNC) and the safety programmable logical controller (PLC).

Middleware

This logical block operates through two basic modules. The first is the MANTIS Embedded PC, which is basically an application that can run on a low cost computer (like a Raspeberry Pi) or directly on the CNC (if powerful enough). This module is responsible for collecting the data from the CNC I/O and transmitting it to the Data Analysis engine for processing and is implemented as a communication API. When based on an external computer, this module also connects to the new wireless MANTIS sensors placed on the machine using Bluetooth Low Energy protocol (BLE). Communications are then supported by the RabbitMQ message oriented middleware, which takes care of proper routing of messages between peers. This middleware handles both AMQP and MQTT protocols to communicate between nodes.

The I/O module is used in order to extract raw information from the machine sensors which is collected by the existing PLC, made available on the Windows-based numerical controller through shared memory and then written to files. Our software collects sensor data from these files, thus completely isolating the MANTIS applications from the numerical controller’s application and from the PLC.

Data Analysis

This logical block takes care of Data Analysis and Prediction, and it exploits three main modules. The first is a set of prediction models used for the detection, prognosis and diagnosis of the machine failures. The second is an API that allows clients to request predictions from the models, and that can respond to different paradigms such as REST or message-queue based. Finally, the third module is a basic ETL subsystem (Extraction, Transformation and Loading) that is responsible for acquiring, preparing and recording the data that will be used for model generation, selection and testing. This last module is also used to process the analytics request data as the same model generation transformations are also required for prediction.

Visualization

This logical block consists of two modules, the human machine interface (HMI) and the Intelligent Maintenance DSS. The HMI is designed to be a web-based mobile application, and to be accessed via the network from any computer or tablet. The HMI is developed to work in two different modes, depending on which kind of user is accessing it. In fact, the HMI is developed to support two user types, the data analyst and the maintenance manager, allowing both of them to analyze the machine’s status, record failure and diagnostics related data. Moreover, the data analysis HMI provides an interface with the data analyst, allowing the consultation and analysis of data and results. On the other hand, the maintenance management HMI allows for consulting predicted events and suggested maintenance actions.

The second module is an Intelligent Maintenance DSS, which uses a Knowledge Base that uses diagnosis, prediction models and the data sent by sensors. On top of this Knowledge Base there is a Rule based Reasoning Engine that includes all the rules that are necessary to deduce new knowledge that helps the maintenance crew to diagnose failures.

Ongoing work

The work performed so far is well advanced and an integration event will occur in the near future where the interconnection between all systems will be tested and validated.

The demonstrator being built, will be evaluated according to the following criteria: prediction model performance (live data sets will be compared to model generation test   sets) and the applications usability (the user should access the required information easily, in order to facilitate failure detection and diagnosis).