IAES International Journal of Artificial Intelligence (IJ-AI)

Sethu Selvi Selvan, Sharath Delanthabettu, Menaka Murugesan, Venkatraman Balasubramaniam, Sathvik Udupa, Tanvi Khandelwal, Touqeer Mulla, Varun Ittigi Department of Electronics and Communication, M.S. Ramaiah Institute of Technology, Bengaluru, India Centre for Imaging Technology, M.S. Ramaiah Institute of Technology, Bengaluru, India Department of Chemistry, M.S. Ramaiah Institute of Technology, Bengaluru, India Safety, Quality and Resource Management Group, Indira Gandhi Center for Atomic Research, Kalpakkam, India


INTRODUCTION
In nuclear, process and petrochemical industries regular inspection of in-service components is important to detect and characterize service induced defects.This helps in enhancing the life cycle of the component and ensures the safety of the components and the workers.Non-destructive evaluation (NDE) methods evaluate the inherent properties of materials and identify any defect or irregularity without damaging it.This not only helps in detecting defects but also predicts whether a defect is probable to occur in future or not which is a crucial factor in preventing major crisis in an industry.Deep learning approaches are widely used in the field of non-destructive testing (NDT) to enhance the signal-to-noise ratio (SNR) and thus improving the defect visibility and to measure the size and depth of defects automatically.
Pulsed thermography (PT) is one of the advanced NDE methods in which the front side of the object under inspection is exposed to a short and high energy optical pulse [1].The front surface of the object absorbs the optical energy and converts it to thermal energy resulting in increase of surface temperature.As the rear end of the object is at ambient temperature, thermal waves diffuse from front surface to rear end causing decrease in front surface temperature.Any defect in the material alters this diffusion rate and the surface temperature above it, which is easily detected by an infrared camera [2].It allows materials to be inspected very quickly for near-surface defects and bonding weakness.Compared to conventional techniques  ISSN: 2252-8938 Int J Artif Intell, Vol.11, No. 3, September 2022: 949-960 950 like radiography and ultrasound testing, PT has advantages like non-contactless measurement and fast inspection rate, whereas the limitation is that it is confined to surface and subsurface defect detection.
Noise is an unwanted but integral part of a signal.In PT, noise is associated with the temperature response recorded in infrared (IR) camera.This noise can affect the detection of deeper and smaller defects.Reduction of such noise is essential, which can be considered as a post processing step.In PT numerous methods have been reported for reduction of noise and thus improve SNR, an important parameter which indicates the effectiveness of noise reduction.Some of the important methods based on statistical and regression-based algorithms are higher order statistics (HOS) [3], thermal signal reconstruction (TSR) [4], principal component thermography (PCT) [5], and partial least square thermography (PLST) [6].In recent years the trend has been shifted towards neural network (NN)-based algorithms for signal enhancement.Extensive work has been reported in the field of automated detection and characterization of defects in PT [7]- [12], [13]- [18].Various neural networks like multilayer back-propagation NN [7]- [10], [12], [13], Kohonen and perceptron based NN [11], convolutional neural network (CNN) [14] and deep feed forward NN [15]- [17] have been used for defect detection and depth quantification in materials like plastics, composites and aluminium.These algorithms provide reasonable accuracy in defect detection and depth estimation.Apart from defect depth and size estimation, improving the defect visibility by enhancing the SNR is also important.SNR enhancement using neural networks has not been explored in PT.In [18] stacked auto-encoder (AE) method was reported for enhancing the delamination visibility in composites using PT.In this approach, pre-processed temporal pixel information is used to train the neural network.The study showed that this method significantly improved the delamination contrast.
In this paper, an unsupervised AE based neural network approach is explored to enhance SNR in PT experiments for AISI 316 L Stainless Steel material and the performance is compared with other approaches, which gives one or few images as output, namely HOS, PCT, and PLST in terms of SNR and defect visibility.Similar to HOS, the output is a single image obtained from 3D raw PT data.This saves on the time for inspection, as one need not go through the complete image sequence to locate the defect.
The paper is organized as follows: section 2 provides a detailed theoretical background on PT and the existing algorithms for signal enhancement.Section 3 elaborates the proposed algorithm for enhancing PT signal and section 4 provides the comparison of the performance of the proposed algorithm with the existing algorithms.Finally, the paper is concluded and future directions for the proposed work is provided in section 5.

THEORETICAL BACKGROUND AND EXISTING ALGORITHMS 2.1. Pulsed thermography (PT)
In PT, a short and high-power pulse is impinged on the surface of the object under inspection.The absorption of the optical pulse by the surface of the object results in instantaneous increase in its surface temperature.The diffusion of heat then results in decrease in the surface temperature which is monitored using an IR camera.Any interference, like a defect, alters the diffusion rate, which causes a change in the surface temperature picked up by the IR camera.The signal acquired by PT is of three dimensions with spatial information (camera pixels) recorded as a function of time.If   ×   is the resolution or the number of pixels captured by the IR camera and   =   ×  is the total number of thermal images, where   is the frame rate and  is the duration of the PT experiment, then the raw PT data set is of size   ×   ×   .The schematic diagram of the PT set-up is shown in Figure 1.

Higher order statistics (HOS)
HOS analysis [3] is employed to process IR images and to compress the most useful information into a unique image for inspection.PT response with respect to time is described through its statistical behaviour.This statistical behaviour is used to analyze the different characteristics of thermal images as higher order statistical parameters have a relation to thermal conductivity in the longitudinal direction of a material.Various statistical moments such as skewness, kurtosis, hyper-skewness and hyper-flatness are considered, and these moments are then combined to form one image for each of the statistical moment.The  ℎ standardized central moment is calculated using (1) as skewness is 3 rd order central moment, kurtosis is 4 th order central moment, hyper-skewness is 5 th order and hyper-flatness is 6 th order central moment: where  is the data distribution,  is the standard deviation and [] is the mean of .Skewness represents a measure of symmetry, or the lack of symmetry of a distribution.Kurtosis characterizes the heaviness of the tail of the distribution, compared to the normal distribution.Hyper-skewness measures the symmetry of the tails, while hyper-flatness measures similarly but with heavier focus on outliers than the fourth moment.Odd order moments quantify relative tailedness and even order moments quantify total tailedness.Standardized central moments of higher order provide larger values due to the higher power terms and cannot be defined physically as they are associated with the presence of outliers.Kurtosis and skewness are used for enhancing the defect visibility and SNR in PT.

Principal component thermography (PCT)
The temperature response signals of PT tend to be monotonic and, so oscillatory basis functions are inappropriate.This method [5] constructs a set of empirical orthogonal functions (EOF) which are statistical modes that provide the strongest projection for the data and offers a very compact representation.The thermal behavior associated with underlying defects of a material is compactly described through a singular value decomposition (SVD) of the data matrix.In general, any matrix  of size   ×   can be decomposed as in (2): where the three matrices are:  whose columns comprise the set of EOF that models spatial variations,  is a diagonal matrix with singular values on its diagonal and rows of   are the principal component vectors which describe the characteristic time behaviour.The first few columns of matrix  are used to reconstruct the data to reduce the redundancy in the original data set.Principal component analysis (PCA) applied to PT data in the form of thermogram sequence is called PCT [5] and cannot be applied directly to a 3-D thermographic data matrix.A pre-processing step of data unfolding is necessary which converts the 3D matrix of size   ×   ×   to a 2D expanded matrix ((  ×   ) ×   ).The unfolded data matrix is normalized by subtracting the mean from each column divided by standard deviation of that column.The normalisation ensures that pixel to pixel variations does not influence the decomposition.PCT yields high levels of thermal contrast for underlying defects in composite materials which results in defect detection compared to conventional thermographic algorithms.

Partial least square thermography (PLST)
In contrast to PCA where the matrix  describes the variance, partial least squares regression (PLSR) [6] computes loadings () and scores () vectors that are correlated with the predicted matrix , which describes the variation in matrix  similar to principal component regression.The matrix  is the surface temperature matrix, while  is the observation time of the thermal images.The result of the bilinear decomposition is a new set of thermal images and observation time vector composed of latent variables in a new subspace which considers only the most important variations.
PLSR is based on the basic latent component decomposition of  and  matrices into a combination of loadings, scores and residuals [6].Mathematically, the PLS model is expressed as in (3) and (4).The scores are orthogonal and are expressed as linear combinations of the original variables  with the coefficients  as expressed in (5): where  is the score matrix,  and  are loading or coefficient matrices and describe how the variables in  relate to the original matrices  and . and  are residual matrices and represent noise or irrelevant variability in  and  respectively.Once the scores matrix  is obtained, the loading matrices  and  are estimated by regression of  and  onto .The residual matrices  and  are found by subtracting the estimated versions of  and  from  and , respectively.The regression coefficients are obtained using (6) and the regression model is given as in (7).

Deep learning based algorithms
Infrared imaging-based PT is used to automatically inspect, detect, and analyse infrared images.Passive thermography is based on visible light and is a imaging tool for self-heating objects such as the human body and electrical power devices.Active thermography is a NDT method for quality and safety evaluation of non-self-heating objects.The rapid development of deep learning makes PT more intelligent and highly automated, thus considerably increasing its range of applications.This paper [19] reviews the principle, cameras, and PT data to discuss the applications of deep learning.
In this study [20], artificial intelligence was applied in combination with infrared thermography to detect and segment defect on laminates.Segmentation was performed on both mid-wave and long-wave infrared sequences obtained during PT experiments through a deep neural network for each wavelength.The F1-score for mid-wave images based model is 92.74%, while for long-wave images is 87.39%.
In this paper [21], synthetic data from the standard finite element models (FEM) are combined with experimental data to build large datasets with mask region based convolutional neural networks (Mask-RCNN), learn essential features of objects of interest and achieve defect segmentation automatically.The results prove the efficiency of adapting inexpensive synthetic data together with the experimental dataset for training the neural networks to obtain an achievable performance from a limited collection of the annotated experimental data of a PT experiment.
In this paper [22], an artificial neural network (ANN) is employed to detect depth of the defects in composite samples, coupled with PT.The study presents a proof of concept using a Multiphysics FEM simulation model of the inspection process, to generate a training dataset and the proposed NN was further tested experimentally to validate its accuracy and performance.The accuracy of the developed NN for the synthetic data was more than 97% and for the experimental data was around 90%.
Kovács et al. [23] investigate two deep learning approaches to recover temperature profiles from PT images in NDT.In the first method, a deep neural network (DNN) was trained in an end-to-end fashion with surface temperature measurements as input.In the second method, the surface temperature measurements were converted to virtual waves, which was then fed to the DNN.For a dataset of 1,00,000 simulated temperature measurement images both the end-to-end and hybrid approach outperformed the baseline algorithms in terms of reconstruction accuracy.The end-to-end approach requires less domain knowledge and is computationally efficient.The hybrid approach requires extensive domain knowledge and is computationally expensive than the end-to-end approach.The virtual waves are useful features, that yields better reconstructions with the same number of training samples compared to the end-to-end approach.It also provided more compact network architectures and is suitable for NDT in two dimensions.
In [24] Fang et al. propose a specific depth quantifying technique by using gated recurrent units (GRUs) in composite material samples via PT.The proposed GRU model automatically quantified the depth of defects and was evaluated for accuracy and performance of synthetic carbon fiber reinforced polymer (CFRP) data from FEM for defect depth predictions.In [25], Lou et al. propose a spatial and temporal hybrid deep learning architecture, which has the capability to significantly minimize the uneven illumination and enhance the detection rate.The results show that Visual Geometry Group-Unet (VGG-Unet) significantly improves the contrast between the defective and non-defective regions.

AUTOENCODER BASED DEFECT VISUALIZATION
Neural network architectures have seen tremendous success in the past few years.CNN is one of the key architectures which performs well on a lot of tasks such as image classification and object detection.It does so by capturing the inherent features of the images in the convolution layers.The convolution layers Int J Artif Intell ISSN: 2252-8938  A deep learning approach based defect visualization in pulsed thermography (Sethu Selvi Selvan) 953 perform convolution operations over the image with a set of kernels, and in this case, convolution refers to Hadamard product.These layers are stacked together many times and additional methods are incorporated in between such as pooling and activation functions.These set of layers are then optimised towards a target and the loss encured is backpropagated.This process modifies the kernel weights until the loss converges.Once the architecture is optimised, the kernels capture the subtle features in the dataset and represents them in a very high dimension.The integration property of CNNs aids in feature representation.
Autoencoder is an unsupervised learning algorithm which tries to learn the function ℎ() ≈ .A well-trained function ℎ() generates a new thermal image with better representation of features compared to any single input image obtained from input images used for training.The structure of the autoencoder is as shown Figure 2. In the encoder section, multiple convolution operations are applied to learn a large number of feature maps.These feature maps contain predominant information from the input images.These feature maps are reconverted back into the shape of input images in the decoder section.Training this network over the input images optimizes a loss function to output a new image with considerable SNR improvement.
The architecture consists of convolution layers denoted by   ,  = 1, 2, 3 and transpose convolution layers   ,  = 1, 2, 3.After each convolution layer an activation function () = () is used.In the last layer of transpose convolutions, another function () = () is used.The convolution operation in each layer produces a feature map after it is multiplied with predefined number of kernels as shown in (8), where   is the feature map of layer ,   is the dimension of the input to  ℎ layer and 'weight' and 'bias' are the parameters learnt by the network.During convolution, rectified linear units (ReLu) defined in (9) are inserted as activation functions to include non-linearity and to help in loss convergence.
output(  ) = bias(  ) + ∑ weight(  )input() () = max (0, ) In the decoder, different transpose convolution operations are stacked which upsamples the image back to its original size.A sigmoid activation function as shown in ( 10) is included at the end of last layer, which squashes the values between 0 and 1 providing the probability values.
To train this network, mean squared error as in ( 11) is used as a pixel level loss function between original and reconstructed images which employs Adam optimizer based gradient descent algorithm.Hyperparameters are chosen for loss convergence and which provides a performance exceeding a pre-defined SNR threshold.
The proposed algorithm consists of 3 layers of convolution followed by 3 layers of transpose convolution.The number of input channels vary as 1-8-16-8-1 during this process.Convolution filters of size 11, 22 and 33 is considered.A batch size of 4 was considered as there were less number of images for training and learning rate was set to 1 × 10 −4 .For the study, AISI 316L grade stainless steel plate of dimension 150×100×3.54mm with artificially induced defects of size 10 mm, 8 mm, 6 mm, 4 mm and 2 mm at depths 0.4 mm, 1.13 mm, 1.78 mm, 2.48 mm, 3.17 mm, and 3.36 mm was used.The front surface of the sample was coated with black paint to improve the emissivity and light absorption.The photograph of the sample is given in Figure 3.

Experimental set-up
CEDIP Silver 420 infrared camera was used for the experiment.The camera has a focal plane array of 320×256 pixels which is made up of Indium Antimonide (InSb) detector with Stirling cooling system.It detects the infrared radiations in 3-5μm region.The maximum achievable temperature resolution is 25 mK with a frame rate of 176 Hz.For PT experiment, two Xenon flash lamps of power 1600 W each were used.The flash duration was less than 2 ms.Experiment was carried out in reflection mode.The non-defective area temperature decreases until stabilization is reached.

Data acquisition
The thermal images acquired are stored as a 3D matrix.The spatial x-and y-coordinates correspond to the pixel locations, respectively, and the z-coordinate represents time.The acquisition frequency (fs) is 125 Hz (Sampling time t = 1/fs = 0.008s), which is the maximum full-frame rate achieved for a 256×320pixel array.Furthermore, a total of N = 250 frames were collected during the cooling regime, where the acquisition window is t = 2s.For analysing the proposed algorithm, different defects of varying sizes and depths are considered as depicted in Table 1 and in the defect name R indicates the defect row number and C indicates the defect column number in the thermal images.

Higher order statistics (HOS)
The PT image is reconstructed using four different statistical moments.The SNR for different defects have been calculated and tabulated in Table 2. From the results, it is observed that maximum SNR is obtained with skewness based reconstruction in most of the defects.Kurtosis based reconstructed image also provides good SNR improvement but lower than skewness.It is also observed that hyper-skewness and hyper-flatness provide smaller SNR values than that of the unprocessed raw image.

Principal component thermography (PCT)
The first five components of the EOF matrix are reconstructed into 5 images along with the raw image for analysis as shown in Figure 4.These five components were extracted from the PCA of the reshaped thermal data.By extracting the most dominant features, the defects are predominantly seen in the feature space.
The principal components provide a very high contrast image with respect to the defects.This helps in clear visibility of the defects and also increases the SNR.The SNR values of six defects are tabulated for the first five principal components and then compared with the SNR values of the defects in the raw image as tabulated in Table 3. From the illustrations, it is observed that the first principal component corresponding to EOF 1 preserves more information about the defects and also visibility and contrast reduces in further components, when compared with the raw image.

Partial least square regression (PLSR)
Figure 5 provides the PLS components of the defects considered for the experiment.The SNR values of six defects were tabulated for the first six PLS components in Table 4 and then compared with the SNR values of the defects in the raw image.It is observed that various defect depths achieve maximum SNR in different PLS components.

Autoencoder based approach
The tradeoff considered in this work is SNR improvement vs defect visibility.The autoencoder architecture is capable of providing output images with higher SNR, though the defects may not be visible always.Depending on the requirements or specifications, the parameters defined for the neural network is varied to maximize either SNR or defect visibility.The reconstructed images for R1C2 defect for varying SNR are shown in Figure 6.The images in Figure 6(a) to Figure 6(i) are the reconstructed outputs from randomly sampled input thermal images.The changes in SNR obtained during different epochs of training the autoencoder is presented in Figure 7 and maximum SNR obtained is tabulated in Table 5.
Neural networks perform very well in optimizing an objective function.In this case, the network is trained to reconstruct the images.The non-linearity of the model and differing statistics of the training data ensure that the reconstructed image is not a copy of the input.This is utilised to select the parameters for which the reconstructed images provide a higher SNR and have good amount of defect visibility.This has a drawback of not providing the best or optimum performance unless fine tuned.

Comparison of SNR for different methods
The performance of defect detection is compared in terms of SNR obtained by different methods and tabulated in Table 6.From the table it is observed that PCT performs better than other signal processing algorithms in terms of SNR increase.The proposed deep learning architecture based autoencoder exhibits the best SNR for all the defects.As shown in the table, reconstructed images from randomly sampled input thermal images exhibit a higher SNR for the proposed autoencoder architecture compared to other signal processing approaches.This observation shows promise on how further research in this area on customised algorithms and objective functions can result in optimal performance without any fine-tuning.

CONCLUSION
The evaluation and extensive analysis of standard signal processing approaches provided a good baseline for SNR improvement and defect visibility.HOS not being used for defect visualization improved the SNR of the defects.PCT displayed considerable contrast enhancement between defect and non-defect regions which led to better defect visualization and higher SNR values as compared to other algorithms.PLSR provided the best performance among all the algorithms in terms of SNR improvement, defect visibility and contrast enhancement.The proposed autoencoder architecture-based algorithm performs better than signal processing approaches with noticeable increase in SNR.It improves defect visibility in some cases and this can be attributed to embedding the input data into higher dimensions using convolution kernels and using non-linear activation functions in this space.During reconstruction, activation function is used only at the last layer to normalise the output.The advantage is that large convolution filters are not necessary for this task.This is due to the fact that representation in a higher dimensional space is more important than learning large dimensions spatial information for this task.This network when optimised can represent and reconstruct any given image effectively.It also helps in data compression with the architecture able to learn the important features.Some of the limitations of this algorithm is that neural network architectures do not perform well with distribution shift, i.e., when the test set is statistically very different from the training set.Normalization or adaptative algorithms may ensure that test data is similar to the data the model is trained on.These models are still in its nascent stage compared to signal processing approaches and neural networks are indeed a high dimensional learnable black-box function, and so the generated models are not explainable.To conclude, signal processing and neural network based algorithms provide a set of tools to increase the SNR, enhance defect visibility and provide data compression in many applications for thermal imaging of stainless steel material with PT approach.Neural network architectures show a lot of promise for this application.Emerging neural network-based approaches such as generative adverserial networks (GANs), variational autoencoders (VAE) and attention-based networks can be considered to represent thermal images for better SNR and defect visibility as compared to autoencoders.

Figure 1 .
Figure 1.Schematic diagram of PT experimental set-up

Figure 3 .
Figure 3. Photograph of a sample

Figure 4 .
Figure 4. Different components obtained through PCT

Figure 5 .
Figure 5. Different PLS components obtained after applying PLSR

Table 1 .
Defects considered for analysis

Table 2 .
SNR in dB for HOS based algorithm ISSN: 2252-8938 A deep learning approach based defect visualization in pulsed thermography (Sethu Selvi Selvan) 955

Table 3 .
Different principal components obtained after applying PCT

Table 4 .
SNR values of different defects for varying PLS components

Table 5 .
Maximum SNR values of different defects

Table 6 .
SNR in dB of different algorithms