English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
Metallography studies microscopically the structure of a metal or an alloy (Newell, 2016). It determines the grain size, form and distribution of various phases and inclusions that have a great effect on the metal mechanical properties. The microstructure reveals the mechanical and thermal treatment of the metal and under a given set of conditions; one can predict the metal behavior. The material exhibits metallographic behavior patterns that are diagnosed by metallographic experts based on the image of the material's microstructure. The quality of the diagnosis to detect the metallographic pattern present in the material depends to a large extent on the experience of the human expert (Barroso, 2014). This paper proposes an alternative solution to make a wear diagnostic system using an Artificial Neural Network (ANN) classification scheme based on a pattern-recognition system employing the multilayer perceptron in conjunction with digital image processing to classify patterns in the image from the metal microstructure. Digital image processing is used first to improve the quality of the image, thus defining an appropriate input signal to the ANN.
Research has been carried out where the benefits of the integration of networks in the field of materials were verified. One of the main activities of artificial neural networks within the area of materials is the estimation. In the work presented by Chokshi (2017), a phase prediction model was developed by means of a multi-layer artificial neural network, the model is used in the experiments carried out on the 22MnB5 steel in order to apply a process called hot stamping with which the microstructures of the material are controlled. The artificial neural network obtained an error of 7,7 % making it a reliable mechanism. In Chakraborty (2017), an artificial neural network model was developed for predicting the phase transformation of steel by austenite, and thus the construction of the continuous cooling transformation diagram. Within the research is used a multi-layer artificial neural network and its weights are defined by a genetic algorithm.
In the research carried out by Jiao (2015), it is demonstrated that the use of a radial-based neural network is capable of analyzing and predicting the electrical resistivity of the aluminum alloy Al-Zn-Mg-Cu experimental data were taken in situ using an optical microscopy, scanning electronics and transmission electronics. The results show that the model is able to predict electrical resistivity with remarkable success reaching a high level of generality. The correlation coefficient between the expected results and the experimental data is 0,9958 and the relative error is 0,33 %.
Another work that relates artificial neural networks and materials science is presented by Powar (2015). In this research, it was proposed to develop a methodology to predict the mechanical properties and microstructure of thermally treated materials, the prediction is done by means of an artificial neural network and the application case was carried out on a shaft made of 30CrMoNiV5-11 steel. The data set for modeling the neural network was generated by studying the microstructural parameters and mechanical properties. In the study, a correlation coefficient (R) of more than 90 %> was obtained to predict the mechanical properties and microstructural behavior of the thermally treated steel. In Guo (2016), the ability of the backpropagation artificial neural network to estimate the stress-deformation relationship of the Ti-6Al-4V alloy was demonstrated; experimentation was carried out by thermal compression tests varying the temperature range. Due to reliable performance in tracking and predicting data, the model was used to expand the volume of stress-strain data.
In the investigation developed by Singh (2016), also the effect of the materials due to the temperature is studied, in this work the behavior of hot deformation of the steels with high phosphorus content was defined through thermo-mechanical simulations for temperatures between 750 °C and 1 050 °C. The behavioral estimation was performed through a multi-layered neural network, the inputs of the neural network were: temperature, deformation and strain rate. In the work presented by Dae (2014), the deterioration under the phenomenon of oxidation at a high temperature in Ni-Cr-W-Mo alloys is studied. A database was constructed from cyclic oxidation experiments performed between 400 °C and 1 150 °C of 27 alloys with different Cr, W and Mo contents. The Bayesian neural network technique was used for the modeling of oxidation experiments cyclic. The model shows an accurate prediction of the oxidation property of Ni-Cr-W-Mo alloys (R = 0,999).
An important characteristic of artificial neural networks is the ability to solve tasks of recognition and classification of patterns, an application where this task is solved in the metallographic area is presented by (Hou, 2016), in the research, a system was developed capable of identify the type of corrosion suffered by carbon steel in aqueous media. The types of damages studied are: uniform corrosion, stinging and passivation.
In the investigation presented by Pouraliakbar (2015), an artificial neural network model with feed forward topology and back propagation algorithm was developed to predict the toughness of high strength low alloy steels. The inputs of the neural network included the weight percentage of 15 alloying elements and the tensile test results such as yield strength, ultimate tensile strength and elongation. The model was developed whit 118 different steels from API X52 to X70 grades. The model was validated with 26 steels. The developed model was very accurate and had the great ability for predicting the toughness of pipeline steels. In Pouraliakbar (2015), a neural network with feedforward topology and back propagation algorithm was used to predict the effects of chemical composition and tensile test parameters on hardness of heat affected zone (HAZ) in X70 pipeline steels. The paper developed by Khalaj (2014), presents results of a research connected with the development of new approach based on the artificial neural network (ANN) of predicting the transformation start temperature of the phase constituents occurring in steels after continuous cooling. The training and testing results in the ANN show a strong potential for prediction of effects of chemical compositions and heat treatments on phase transformation of microalloyed steels. In the research realized by Khalaj (2014), the ultimate tensile strength of steel made using thermomechanically controlled processing has been modeled by means of gene expression programming. To build the model, training and testing using experimental results from 104 specimens were conducted. The training and testing results in gene expression programming models have shown a strong potential for correlating the ultimate tensile strength to yield strength and chemical composition of X70 pipeline steels.
According to the review, it is observed that the use of artificial neural networks is viable as a method of estimation and recognition of the deterioration present in the materials. The proposed work uses the artificial neural network multilayer preceptron as a mechanism for recognition and classification of metallographic patterns, and incorporates digital image processing as a tool that aims to generate detailed input signals to the artificial neural network.
The study presented in this article combines the use of neural networks with digital image processing where the network is trained by the histogram of an image that has been previously segmented and binarized; this procedure provides efficient results in the classification-recognition of carbon steel states (decarburization, graphitization and spheronization) that, to the best knowledge of the authors, have not been addressed from the viewpoint of Artificial Intelligence. The maintenance program within a power plant includes the health monitoring of the metal structures including the steam and water conducts that work at high pressures and temperatures. These conducts come from the steam turbine and generator as it is illustrated in Figure 1. Routinely, metal samples are taken and stored in a database with the purpose of being analyzed by a material expert with associated high costs.
The development of the wear pattern recognition system rises from the need to corroborate the diagnostic made by a metallographic expert and also to automate the process since the human expert may not be available or there can be a shortage of such highly qualified specialist. The importance of the system lies on the recommendation of whether the material is still in working conditions or needs to be replaced. In current practice within the power plant, a failure to diagnose correctly the state of the material can lead to high costs in the case of early replacement or potentially deadly consequences if the defective material continues in operation.
The material under study is the carbon steel grade A210 C, because it is one of the most used for the construction of conductors of water and steam under high temperatures in thermoelectric plants in Mexico materials. The Contact with high temperatures causes damage to the material throughout its useful life and transforms its microstructure over time. According to ASTM 210-02 the steel has the chemical specifications described in Table 1.
The proposed wear recognition system is intended to facilitate the plant condition monitoring in the steam and water metal conducts. Currently, several samples are taken from the conducts and stored in a database to later on, be analysed by a failure specialist. The basic idea is to use this image knowledge base to train an ANN in order to learn to recognize and classify the three principal condition states avoiding the requirement of a human expert. This procedure is illustrated in Figure 2.
Digital image processing
The process starts obtaining a metallographic image consisting of consecutive operations. The first operation is to cut the affected material in order to remove small sections to reveal information about the cause and the range of the damage. The next step is the manufacture of a test tube with epoxy resin, sanded and polished with silicon carbide solution to get a mirror finish. Next, a Nital solution with concentration of 4 % is applied for five seconds to disclose the microstructure. Finally, the specimen is observed by the human expert using an inverted metallographic microscope (Olympus-GX71) and an image analyzer (Analysis five). Once the image sample is taken this image can be stored and transmitted for further processing. The techniques of digital image processing are used in order to improve the visual appearance for an observer and conveniently prepare photographic content in the face of machine perception; in this case, the artificial neural network. A flowchart indicating operations in the process of extraction and preprocessing of metallographic images is illustrated in Figure 3.
This is the part of the image pre-processing stage for the input image to lower the spatial resolution. After trying different spatial resolutions such as: [1024 χ 1024] [512 χ 512] and [256 χ 256], it was concluded that best results were achieved with [512 χ 512] pixels defining also the length of the histogram. The histogram is the input to the neural network and if it is obtained from higher resolution images then the computing efficiency of the neural network can be compromised. On the other hand, having a lower resolution can result in a misclassification. In Figure 4, the image resizing is illustrated using patterns related to graphitization and decarburization.
Source: Authors
Figure 4 Image Resizing. (a) Original graphitization image [3072 χ 4080 pixels]. (b) Resized image [512 χ 512 pixels]. (c) Original decarburization image [602 χ 800]. (d) Resized image (c) [512 χ 512 pixels].
Image filtering
Filtering is applied to enhance or attenuate spatial detail in order to enhance the visual interpretation or facilitate further processing and it is one of the techniques involved in image enhancement. A low-pass median filter is used employing a [3x3] mask, which basically replaces the central pixel value to the value determined by the multiplication. In Figure 5 and Figure 6, the result of the median filtering is shown in the metallographic images with their respective intensity histogram, calculated from the accumulated sum in the image columns.
Source: Authors
Figure 5 Filtered graphitization image and histogram. (a) Filtered image (b) Histogram.
Source: Authors
Figure 6 Filtered decarburation image and histogram. (a) Filtered image (b) Histogram.
It is important to make the difference between the histogram used as input signal to the ANN and a histogram representing the intensity levels. The first is constructed from the sum of the columns of the matrix that makes up the image digitally processed. The second is a graph representing the levels of color intensity in a certain image, concerning the number of pixels present in the image with each color intensity.
The intensity histogram after applying the median filter to the image for graphitization does not show in detail the affected areas. For instance, through inspection we can observe that the affected zone 2 does not appear in the given histogram. See Figure 5(a) and 5(b). The pattern behavior for decarburization in the affected zone 1 (Figure 6(a)) does not show in a clear way inside the generated histogram from the metallographic image. See Figure 6(b). Considering these results, it is not possible to provide an adequate input signal to the ANN for the classifying-recognition task, therefore image segmentation is recommended.
Image Segmentation
It is a stage of digital image processing that locates and separates different areas within an image based on a defined intensity threshold. This step allows the expert to determine the presence of any abnormal element. There are several methods to appropriately select a threshold.
However, some of them do not work well when working with real-world images due to the presence of noise and varying lighting. Considering this, we decided to borrow some ideas from Otsu's method (Wakaf, 2016) to automatically perform histogram shape-based image thresholding, reducing a gray level image to a binary image. The algorithm assumes that the image to be thresholded contains two classes of pixels (e.g. foreground and background) then calculates the optimum threshold separating those two classes so that their combined intra-class variance is minimal. Let us define the global and intra-class variances as in Equation (1).
Respectively, where L is the number of gray levels (typically 256), mG is the average gray level in the image and Pi is the probability of the i
th
gray level in the image. The probabilities P1 and P2 of the two potential classes C1 and C2 are defined respectively as 
and P2 (k) = 1- P1 (k) with 0 < k < L - 1. The average probabilities m1 and m2 of C1 and C2 are defined respectively as 
and 
.
Note that the optimal classes C1 and C2 are separated by the kth gray level value that makes maximal. The relationship between global and intra-class variances is given by 
. This means that the intra-class variability must be evaluated for every gray level as 
, with m (k) being the cumulative average probability up to the kth gray level.
The optimal threshold is then defined as k satisfying. Equation (2).
For the total number of changes in light source intensity, the criterion is shown in Equation (3).
is recorded for every nth intensity variation, where N is the number of variations. The 
image maximizing the criterion is then selected using the equation 
. Finally, the optimal threshold k* is used to segment the selected image, (Gonzalez, 2009). Using this approach, the affected zones are clearly visible as it is shown in Figure 7, where the graphitization pattern and decarburization pattern are shown after binarization and segmentation.
Neural Network
An artificial neural network is a massively parallel processor which consists of simple processing units that have a natural property to store experimental knowledge, making it viable for use. The neural network resembles the brain in two respects:
Knowledge is acquired by the network from its environment through a learning process.
The strength of the connections between neurons, known as synaptic weights are used to store knowledge.
A neural network architecture is the organization and arrangement of neurons in the network forming layers or groups of neurons. The basic parameters are: number of layers, number of neurons per layer, degree of connectivity and type of connection between neurons.
Multilayer networks
Multilayer neural networks are formed with a group of simple cascading layers. The output of one layer is the input of the next layer. It is known that multilayer networks have superior learning performance compared to networks of a single layer. Since these networks have several layers, the connections between neurons can be feed forward type (forward connection) or feedback type (back connection). Feed forward networks are particularly useful in applications for pattern recognition or classification. The Figure 8, shows the general structure of these types of networks. Figure and nomenclature are taken from (Haykin, 2009).
The input to the network is the vector p whose length is equal to R, W is the weight matrix with dimensions SxR. a and b are vectors of length S which represents the number of neurons in the network.
The ability of the multilayer perceptron network to learn from a set of examples, to approximate nonlinear relationships, filter noise in the data, etc. makes it a suitable model to address real problems and appropriate tools as universal approximators (Gamarra, et al., 2013). The design of the network architecture involves defining the activation function, the number of neurons and the number of layers in the network. Choosing the activation function is usually done based on the required smoothness for the output signal in the hidden layers, though its election does not affect the ability of the network to solve the problem (Benitez, 2014).
As for the number of neurons and layers, some of these parameters are given by the problem and others should be chosen and defined by the designer. The number of neurons in the input layer and in the output is defined by the number of input and output variables which characterize the problem. The number of hidden layers and number of neurons in these layers must be chosen by the designer. There is not a method to determine the optimal number of hidden layers and neurons to solve a given problem. The use of the ANN in the wear pattern recognition system is described by a flow chart given in Figure 9.
The adjustment relates to the action of modifying parameters that make up the topology of the ANN to achieve a combination of parameters that yields an acceptable solution to the classification presented to the network. These parameters are: number of neurons in the hidden layer, learning rate, training time and allowed error. See Table 4.
Experimental methodology
The study case for this research was carried out in the Laboratory of Equipment and Materials Testing (LAPEM) of the Federal Electricity Commission (CFE). Typical spheronization (a), decarburization (b) and graphitization (c) image patterns are presented in Figure 10.
The pattern set employed to train and test the neural network was a database of four hundred and sixteen metallographic images affected by patterns previously recognized and identified by experts at LAPEM. The training matrix was generated from the process shown in Figure 3, defining a final input matrix [124 χ 512] incorporated to the neural multilayer perceptron network which was defined according to the number of images selected for the training phase (124) leaving (292) for the testing phase. The dimension of the training and testing matrix are shown in Table 2.
Table 2 Training and Testing matrix
| Pattern | Training | Testing |
|---|---|---|
| Spheronization | 56 χ 512 | 122 χ 512 |
| Decarburization | 29 χ 512 | 84 χ 512 |
| Graphitization | 39 χ 512 | 86 χ 512 |
Source: Authors
The activation function in the architecture of the ANN is a hyperbolic tangent, the value of (-1) to spheronization, (0) Decarburization and (1) graphitization, are assigned. The topologies of ANN's proposals in Table 3, are adequately conform the 124 target values corresponding to metallographic pattern in the training phase, which is why was resorted to a statistical parameter as the root mean square error (MSE) to determine which topology should be selected and then used in the test phase with the remaining 139 images. The values of the ANN were rounded to the closest integer and then compared against the target values.
Results
In order to verify the proper operation of the network and the ability to solve the task of wear pattern detection, a validation phase was developed introducing 292 new images of spheroidization, decarburization and graphitization patterns for classification and recognition. The metallographic images were previously recognized and identified by human experts (targets). The results for the validation phase in each of the topologies developed with the parameters described in Table 3 are given in the Figure 11. The code was developed in the Matlab® 2015. It is possible to validate a correct classification-recognition through the following indicators: sensitivity Equation (4), specificity Equation (5) and accuracy Equation (6). The probability of not making a type I error, (1-α), is known as test efficiency, equivalent to specificity. The probability of not making a type II error, (1-β), is known as the power of the test, equivalent to sensitivity. The degree of accuracy of a test is defined as the percentage of cases in which the result is correct.
Where: VP - True-Positive; FN - False-Negative; VN - True-Negative; FP - False-Positive
From the indicators of sensitivity, specificity and accuracy it was demonstrated that using the histogram generated from the digital processing of metallographic image, generates a signal efficient entry to the ANN and thus be able to perform the classification task-recognition through architecture presented in Table 4. The topology of the selected artificial neural network was the number 1.
The experimental results were obtained using a computer with AMD A10 Processor @3.7 GHz with 16 GB RAM. The time average to reach the allowed error during the learning stage was 31s.
Conclusions
A novel approach to address the problem for structure health monitoring in power plants has been proposed. The methodology involves the use of image information coming from intensity histograms to train an artificial neural network (ANN) to automate the detection of three wear metal states (spheronization, decarburization and graphitization). Obtained results revealed the feasibility of using the system in an automated process to reduce the vast amount of data that a failure expert would have to analyze.
The result of this research will be useful to generate or to confirm a diagnostic accomplished by an expert which is necessary to determine the causes of material failure, maintenance periods or parts replacement. The methodology presented is efficient according to indicators of sensitivity (97,22 %), specificity (96 %) and average accuracy (96,83 %>). An important aspect is due to the fact that the digital image becomes a wear pattern using the histogram of intensity to produce a pattern input to the ANN. Another important point is that in order to increase the robustness of the segmentation process in different lighting conditions, the Otsu's method was employed to select the best threshold value. Under the above it is possible to think of establishing this procedure in companies that require analysis of materials in use and do not have an expert immediately.
