%~Mouliné par MaN_auto v.0.32.0 2024-07-10 15:30:19
\documentclass[CRPHYS, Unicode, XML, screen, biblatex, published]{cedram}

\addbibresource{crphys20230592.bib}


\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{setspace}
%\usepackage{tocloft}
\usepackage{lipsum}
\usepackage{adjustbox}
\usepackage{subfigure}
\usepackage{caption}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{soul}


\newcommand{\rmH}{\mathrm{H}}
\newcommand{\rmW}{\mathrm{W}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\graphicspath{{./figures/}}

\newcommand*{\mk}{\mkern -1mu}
\newcommand*{\Mk}{\mkern -2mu}
\newcommand*{\mK}{\mkern 1mu}
\newcommand*{\MK}{\mkern 2mu}

%\hypersetup{urlcolor=purple, linkcolor=blue, citecolor=red}

\newcommand*{\relabel}{\renewcommand{\labelenumi}{(\theenumi)}}
\newcommand*{\romanenumi}{\renewcommand*{\theenumi}{\roman{enumi}}\relabel}
\newcommand*{\Romanenumi}{\renewcommand*{\theenumi}{\Roman{enumi}}\relabel}
\newcommand*{\alphenumi}{\renewcommand*{\theenumi}{\alph{enumi}}\relabel}
\newcommand*{\Alphenumi}{\renewcommand*{\theenumi}{\Alph{enumi}}\relabel}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\RubricEF{Rubric information missing}{Information rubrique manquante}

%\makeatletter
%\def\TITREspecial{\relax}
%\def\cdr@specialtitle@english{Energy in the heart of EM waves: modelling, measurements and management}
%\def\cdr@specialtitle@french{L'énergie au c{\oe}ur des ondes électromagnétiques : modélisation, mesures et gestion}
%\makeatother

\title{Multi-label classification with deep learning techniques applied to the B-Scan images of GPR}


\alttitle{Classification multi-label par des techniques d'apprentissage profond appliquées aux images B-Scan de radars à sondage de sol}


\keywords{\kwd{Ground Penetrating Radar}
\kwd{Image processing}
\kwd{Detection of Buried objects}
\kwd{Deep learning}}

\altkeywords{\kwd{Radar à sondage de sol}
\kwd{traitement d’images}
\kwd{détection d’objets enfouis}
\kwd{apprentissage profond}}

\editornote{This article follows the URSI-France workshop held on 21 and 22 March 2023 at Paris-Saclay.}


\author{\firstname{Soukayna} \lastname{El Karakhi}\IsCorresp}
\address{University of Limoges, XLIM Institute, 123 Av. Albert Thomas, 87000 Limoges, France}
\email[S. El Karakhi]{soukaynaelkar@gmail.com}

\author{\firstname{Alain} \lastname{Reineix}\CDRorcid{0000-0002-9439-6258}\IsCorresp}
\address[1]{University of Limoges, XLIM Institute, 123 Av. Albert Thomas, 87000 Limoges, France}
\email[A. Reineix]{alain.reineix@xlim.fr}
%\urladdr{https://www.xlim.fr/personnel/reineix-alain}

\author{\firstname{Christophe} \lastname{Guiffaut}\CDRorcid{0000-0002-7550-9966}}
\address[1]{University of Limoges, XLIM Institute, 123 Av. Albert Thomas, 87000 Limoges, France}
\email[C. Guiffaut]{christophe.guiffaut@xlim.fr}
%\urladdr{https://www.xlim.fr/personnel/guiffaut-christophe}

\begin{abstract}
The ground penetrating radars (GPR) are now widely used for the detection of buried objects in areas such as: geology, archaeology and civil engineering. It has the advantage of allowing detection by a non-destructive technique. The principle for time domain GPR consists in emitting electromagnetic pulses in the ground, these one are then diffracted by the targets to be detected. A single GPR signal trace captured at a position of the radar is a 1D signal called Ascan. A set of Ascan radar waveforms captured at a certain number of different consecutive positions along a particular direction will form a 2D image called B-scan. They show response shapes of hyperbolic type and their analysis give many characteristics. For example, in the case of buried pipes, a specific processing allows to find their diameter, their nature as well as the electrical characteristics of the ground. However, these approaches often require complex post-processing of the Bscan, which can be time-consuming and therefore makes it difficult to perform real-time characterization at the expense of such methods. With the emergence of deep neural networks and with a learning phase on a large number of Bscan, it becomes possible to extract almost instantaneously the characteristics of GPR radar data. In this study, a multi-label classification (MLC) model based on transfer learning and data augmentation has been developed to generate multiple information elements on the same image and to realize classification. Three deep learning models: VGG-16, ResNet-50 and adapted CNN were used as pre-trained models for transfer learning. The networks were trained on a synthetic dataset created in this study and evaluated on a set of performance metrics.
\end{abstract}


\begin{altabstract}
Les radars à pénétration de sol (GPR) sont aujourd'hui largement utilisés pour la détection d'objets enterrés dans des domaines tels que la géologie, l'archéologie et le génie civil. Il présente l'avantage de permettre une détection par une technique non destructive. Le principe du GPR dans le domaine temporel consiste à émettre des impulsions électromagnétiques dans le sol, celles-ci étant ensuite diffractées par les cibles à détecter. Une seule trace de signal GPR capturée à une position du radar est un signal 1D appelé Ascan. Un ensemble de formes d'ondes radar Ascan capturées à un certain nombre de positions consécutives différentes le long d'une direction particulière formera une image 2D appelée B-scan dans le cas d'un déplacement rectiligne. Elles montrent des formes de réponse de type hyperbolique et leur analyse donne de nombreuses caractéristiques. Par exemple, dans le cas de canalisations enterrées, un traitement spécifique permet de connaître leur diamètre, leur nature ainsi que les caractéristiques électriques du sol. Cependant, ces approches nécessitent souvent un post-traitement complexe du Bscan, ce qui peut être chronophage et rend donc difficile la caractérisation en temps réel au détriment de ces méthodes. Avec l'émergence des réseaux neuronaux profonds et avec une phase d'apprentissage sur un grand nombre de Bscan, il devient possible d'extraire presque instantanément les caractéristiques des données radar GPR. Dans cette étude, un modèle de classification multi-label (MLC) basé sur l'apprentissage par transfert et l'augmentation des données a été développé pour générer des éléments d'information multiples sur la même image et réaliser la classification. Trois modèles d'apprentissage profond : VGG-16, ResNet-50 et CNN adapté ont été utilisés comme modèles pré-entraînés pour l'apprentissage par transfert. Les réseaux ont été formés sur un ensemble de données synthétiques créé dans cette étude et évalués sur un ensemble de mesures de performance.
\end{altabstract}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\dateposted{2024-09-05}
\begin{document}
\maketitle

\section{Introduction}


Ground Penetrating Radar (GPR) \cite{jol2008ground} has become an increasingly significant and effective tool for non-destructive engineering investigations. Over an extended period, GPR has found extensive applications in geological research, civil engineering, agriculture, and environmental studies. In recent years, various approaches have been proposed by researchers to enhance the processing of GPR data. Notably, T.~ Noreen~\cite{noreen2017using} has suggested a machine learning-based approach for hyperbolic signature identification using a Support Vector Machine (SVM) with Histogram of Oriented Gradient (HOG) features. E.~Temlio~\cite{temliouglu2016comparison} has explored diverse techniques for landmine detection, including Binary Robust Independent Elementary Features (BRIEF), Edge Histogram Descriptor (EHD), Histogram of Oriented Gradients (HOG), Scale Invariant Feature Transform (SIFT), and Speeded Up Robust Features (SURF). Additionally, in a separate study, W.~A.~Wahab~\cite{wahab2013interpretation} introduced an innovative hyperbola fitting technique for estimating the radius of buried utilities such as pipes and cables, conducting experiments on pipes with nine different diameter values. Furthermore, B.~Walker and L.~Ray~\cite{walker2019multi} have implemented a feature-based machine learning approach to process GPR data, utilizing feature vectors derived from the Histogram of Oriented Gradients (HOG) in conjunction with a Support Vector Machine (SVM) for the detection of deep and shallow crevices. These methods often involve additional characteristic operators as a preprocessing step, including Sobel's, Wavelet Edge Detection~\cite{giannakis2020machine}, and Canny's operators. While these methods yield precise results, they are computationally intensive. In recent years, several researchers have proposed novel approaches for the automatic identification and localization of buried objects using deep learning models. This has underscored the numerous advantages of deep learning technologies. With the growing volume of GPR data, traditional machine learning techniques are showing limitations in digital image processing. Researchers have increasingly turned to Convolutional Neural Network (CNN)-based methods to learn hyperbolic shapes for classifying and identifying buried objects. Most of the methods mentioned have focused on the identification and positioning of buried objects, classifying them based on a single characteristic. Notably, limited studies have explored multi-label classification models, aiming to extract multiple pieces of information from a single image. It is also crucial to highlight that recent studies in this field have advanced the capabilities of GPR data analysis. For instance, Nairit Barkataki~\cite{barkataki2021classification} introduced a deep Convolutional Neural Network (CNN) model for the automatic classification of soil types based on GPR B-Scan images. Mehmet Sezgin~\cite{sezgin2023classification} achieved nearly 100~\% classification accuracy for the two-class identification of buried objects, distinguishing clutter from surrogate mines, using real GPR datasets. Enver Aydin demonstrated the advantages of deep learning in detecting buried targets considered by GprMax simulation. These recent advancements underscore the potential of deep learning in GPR data analysis. Our paper's contributions are synthesized as follows: we have created a database using the Finite Difference Time Domain (FDTD) simulation, employed gradient operators for edge region detection (Sobel, Canny, and Prewitt) for image processing, and developed a multi-label model for the identification of the diameter of buried pipes, the characterization of their internal spaces, and the determination of the soil medium's composition. The originality of our work primarily lies in the development of deep learning model architectures and the approach to model prediction and analysis for the three models we have developed. What distinguishes our work from existing studies is our innovative approach to multi-label classification in the context of GPR data analysis. While prior research has predominantly focused on single-class or binary-class classification of GPR data, our study takes a significant step forward by simultaneously extracting multiple facets of information from a single GPR image. Unlike the conventional approach that classifies objects based on a single characteristic, our model is designed to identify various attributes concurrently, such as the diameter of buried pipes, the characteristics of their internal spaces, and the composition of the surrounding soil medium. This multi-label approach introduces a new dimension to GPR image classification, allowing for a more comprehensive exploration of the data and providing rich and detailed insights into the subterranean environment. As a result, our research represents a significant advancement in the field of non-destructive engineering investigations, opening up exciting possibilities for applications in geology, civil engineering, and other related fields. In addition to the multi-label approach, our research stands out due to our novel methodology for creating a synthetic database using the Finite Difference Time Domain (FDTD) simulation model. We have further enhanced the quality of our data by implementing gradient operators, such as Sobel~\cite{maas2013using}, Canny, and Prewitt, for edge region detection. To classify the GPR images, we have developed and employed three distinct deep learning models: VGG 16, Resnet 50, and a custom-designed CNN. The subsequent sections of this paper will provide detailed explanations of our methodology, including the creation of the database and the evaluation of our multi-label model. In summary, our work signifies a new era in non-destructive engineering investigations by extending the boundaries of GPR image classification and offering fresh insights into data analysis, with implications spanning from geology to construction.

\section{The creation of the synthetic dataset} 

In order to generate our dataset, we will use the TEMSI-FD software~\cite{TEMREF} developed within the EMC team of the XLIM Institute. It consists of a FDTD-based method software~\cite{taflove2005computational} designed to simulate the propagation of electromagnetic waves in complex media. For the GPR study, the time domain signal emitted, propagating and received by antennas can be calculated. It is useful to represent these recordings in two forms: the A-scan (1D response), the B-scan (2D image resulting of multiple consecutive A-scan).To create a simulation model, a number of settings are required. First, the entire scene has to be designed. The scene is characterized by the propagation medium of the electromagnetic waves. The model of the ground is based on the fractal model~\cite{TEMREF}, but also on homogeneous medium such as dry clay, dry sand and concrete. The dimensions of the scene is 1000 mm x 800 mm x 1200 mm (X x Y x Z), the cell size being 4 mm in each cartesian direction. The soil layer depth is 1 meter. It is the maximum depth value to be probed by the GPR, and consequently, it determines the estimation of the maximum time of observation/simulation, which is 20 ns for this volume. The waveform generated by the Wu-King antenna is a sinusoidal Gaussian, and this pulse is centered at the frequency of 1GHz with a bandwidth of 500 MHz. This duration was calculated for the case of the concrete layer which has the smallest value for the propagation velocity among the media considered: $v_p = \frac{c}{\sqrt{\epsilon_r}}$ with c the velocity of light in vacuum. The GPR antennas system is composed of a fixed emitter and 10 receiving antennas. The receivers are also spaced 20 mm apart and moved in groups of 200 mm along the horizontal scan line on the soil, resulting in a 10-fold reduction in the number of simulations compared to considering the receivers individually throughout the computational volume. The multi-receiver bistatic GPR is illustrated in the Figure~\ref{fig1}. A set of 5400 GPR B-scans database concerning 4 soil types, 50 depth values and 9 different pipe diameters is generated; the current diameters for the simulation data are 16, 24, 32, 40, 48, 64, 72, 80 and 100 mm. The depth values range from 204 mm to 400 mm below the soil surface. Three types of pipes were chosen during the simulation: a perfectly conductive metal pipe, an air-filled pipe, and a water-filled pipe. For the second one, the pipe is not modeled, the sheath dielectric is then neglected. That is also the case for the water-filled configuration.



\begin{figure}[h]
\centering
\includegraphics[width=1\textwidth]{multireceiveir.jpg}
\caption{a) The multi-receiver 10 Rx antennas bistatic GPR with one Tx antenna emitter architecture, b) The way the 10 Rx antennas are moved in one direction during the exploration on the soil.}
\label{fig1}
\end{figure}


\subsection{A-Scan}

An A-scan is a one-dimensional (1D) representation of the amplitude of the collected signal as a function of time after placing the receiving sensor antenna above the point of interest. An example of an A-scan is shown in Figure~\ref{fig2}. It represents the signal captured by the receiving sensor antenna $R_{x1}$ from the group of 10 low coupled receiving antennas. The first reflection generally corresponds to a combination of direct coupling between the antennas $(Tx, R_{x1})$ and reflection off the surface of the ground. The second reflection (at $t = 8.5$ ns), with a lower amplitude, corresponds to the response of the object of interest.

%
\begin{figure}
\centering
\includegraphics[scale=0.6]{ascan1.png}
\caption{Example of the waveform of an A-scan captured by the receiving antenna $R_{x1}$.}
\label{fig2}
\end{figure}

To improve the quality of the recordings, it is often essential to eliminate both surface echoes and direct antenna coupling, retaining only the relevant signal. This operation can be performed using two methods during the simulation. Firstly, by taking a measurement without any objects present and subtracting it from the recording obtained with objects in the scene, effectively removing the various couplings. Alternatively, this can be achieved by disregarding the initial moments of the recording.



\subsection{B-Scan}

The 2D image of the B-scan represents a vertical cross-sectional view of the subsurface. The horizontal axis of this section represents the position on the ground surface, while the vertical axis represents the round-trip time of the EM wave propagation. By combining all the A-scans that make up the B-scan, a matrix is created where each row represents a time sample and each column represents an A-scan trace. Each element of this matrix has a value corresponding to the amplitude of the field for the associated A-scan trace and time sample. Buried objects appear as hyperbolic shapes in the B-scan images. Recordings taken from different positions on the ground are combined in the B-scan image, thus creating these specific hyperbolic shapes. The Figure~\ref{fig4} is illustrating the resulting treatment of B-scan.

\begin{figure}
\centering
\begin{subfigure}
{\includegraphics[scale=0.45]{figurebscan_fractal_72mm_metalicpipe.jpg}}
\end{subfigure}
\\
\begin{subfigure}
{\includegraphics[scale=1.5]{drysand_16mm_waterfilledpipe.jpg}}\end{subfigure}
\hfill
\begin{subfigure}
{\includegraphics[scale=1.5]{concrete100mmmetallictube.jpg}}
\end{subfigure}
\caption{An example of B-scan images obtained using various propagation mediums, different diameters, and types of pipes.}
\label{fig3}
\end{figure}


\subsection{Relevant parameter values}

The parameter values employed in the simulation can be found in Table~\ref{table1}, while Table displays the dielectric parameters corresponding to the soil types.


\begin{table}[htbp]
\centering
%\captionsetup{justification=centering}
\caption{The parameter values of simulations.}
\begin{tabular}{||c c||}
\hline
\textbf{Parameters of simulations} & \textbf{Values} \\ [0.5ex]
\hline\hline Source frequency & 1.00 GHz \\
Bandwidth & 500 MHz \\
Source waveform & Sino-Gaussian pulse \\
Source pulse duration & 1 ns \\
A-scans intervals & 20 mm \\
No of A-scans & 40 \\
Spatial resolution & 4 mm \\
\hline
\end{tabular}
\label{table1}
\end{table}

\break
The Table~\ref{table2} provides information on different soil types along with their respective conductivity and relative permittivity values. The fractal model illustrates that soils are present in distinct layers or geological formations, which are separated by rough interfaces. These layers demonstrate a wide range of characteristics~\cite{ciarletti2017consert}.

\begin{table}[htbp]
\centering
%\captionsetup{justification=centering}
\caption{Soil Conductivity and Relative Permittivity.}
\begin{tabular}{||c c c c||}
\hline SI. No & Soil type & Conductivity (S/m) & {Relative permittivity} ($\varepsilon_r$) \\
\hline\hline 1 & Dry sand & 0.002 & 10.00 \\
2 & Dry clay & 0.001 & 5.53 \\
3 & Concrete & 0.001 & 8.00 \\
4 & Fractal & 0.001 & 6.00 \\
\hline
\end{tabular}
\label{table2}
\end{table}

\vspace*{20pt}

\break
\section{Proposed Deep Learning architecture}

Now, the CNN and the procedure used to apply deep learning for retrieving the soil and pipes characteristics will be described. First, the global GPR data handling workflow in this study involves feature extraction, splitting the dataset into training and test sets. This allows respectively CNN model training and prediction efficiency. The feature extraction pipeline for B-Scan images consists of testing several approaches to edge detection. The goal of this technique is to significantly reduce the amount of data while retaining information that can be considered more relevant. The objective is to focus on regions of the hyperbola where there is a distinct intensity variation. The hyperbola region is detected by a bounding box, the image is cropped to extract the region of interest from each frame. For this purpose, we have focused on Canny’s operators. Canny is a multi-step edge detection algorithm developed by John~F.~Canny in 2001~\cite{reddy2016canny}, which proceeds through the following steps:
\begin{enumerate}
\item noise reduction: Noise is removed from the image with a 5x5 Gaussian filter~\cite{barkataki2022cnn}.
%\vspace{-0.2cm}
\item Determine the gradient intensity of the image using the Canny operator.
%\vspace{-0.2cm}
\item Non-maximum Suppression: this step consists of eliminating those undesirable pixels that may not be edges.
%\vspace{-0.2cm}
\item Hysteresis Thresholding.
\end{enumerate}

\begin{figure}[!h]
\centering
{\includegraphics[scale=0.6]{Fig4-a.jpg}\\
{\hspace*{-30pt}\includegraphics[scale=0.6]{Fig4-b.jpg}}}
\vspace*{-15pt}
\caption{Example of Output from the Canny Edge Detection Operator.}
\label{fig4}
\end{figure}

%\break
The objective of this work is to implement Deep Learning models for extracting features like buried pipes diameters or type of soil layer from B-scan images. We will implement three different architectures: VGG-16, Resnet-50 through learning transfer and a custom CNN model. The proposed architecture is a multi-label model classification in which individual objects can be classified into multiple classes at one time, compared to traditional one-label classification cases involving a single class of objects. Multi-label classification approaches are becoming unavoidable in advanced technology. In this work, in the classification of B-Scan images, each image can belong to several distinct categories. For example, we classify the type of pipe (metal pipe, water-filled pipe and empty pipe), 9 different diameters, so that we distinguish four different mediums. As seen above, the dataset contains 5400 B-Scans, it will be divided in such a way that 80~\% is reserved for model training, and 20~\% for test data. Now we will describe the different topologies.

\subsection{Transfer learning using VGG 16 and Resnet 50}

Residual neural networks, commonly known as ResNet, have revolutionized the field of computer vision by serving as a fundamental architecture for various tasks. The term ``Residual'' refers to the key concept behind ResNet's~\cite{sarwinda2021deep} success. Prior to ResNet, training deep neural networks with over 50 layers was challenging due to the problem of gradient backpropagation~\cite{tan2019vanishing}. However, ResNet overcame this obstacle. The strength of ResNet lies in its implementation of skip connections, which are illustrated in Figure~\ref{fig5}. On the left of Figure~\ref{fig5} we stack the convolution layers successively. On the right, we stack the convolution layers as before, but we connect the original input to the output of the convolution block. This connection, known as a skip connection, plays a crucial role in retaining important information from earlier layers. It enables the network to effectively train deep networks by mitigating the vanishing gradient problem.The skip connection in ResNet facilitates the flow of gradients throughout the network during training. This means that the gradients can propagate back to earlier layers more easily, allowing for smoother and more efficient learning. By incorporating these skip connections, ResNet has achieved remarkable performance in various computer vision tasks and has become a state-of-the-art model for deep neural network training.

\begin{figure}
\centering
\includegraphics[scale=0.8]{resnet50_residualbloc.png}
\caption{Residual learning: a building block.}
\label{fig5}
\end{figure}

The Visual Geometry Group \textbf{(VGG-16)}~\cite{simonyan2014very} is a widely recognized deep Convolutional Neural Network (CNN) architecture developed by the Visual Geometry Group. The key contribution of VGG-16 lies in its design, which utilizes a stack of 3x3 filters. This approach demonstrated that using multiple small kernel size filters can achieve the same effect as using larger filters, while also providing the flexibility to increase the depth of the network. VGG models introduced the notion of depth as a crucial factor in the success of convolutional neural networks. By employing smaller filter sizes, VGG models are able of learning a larger number of filters, leading to a more complex representation of the input data. This approach marked a significant departure from earlier models that heavily relied on larger filters to capture low-level features. One of the notable achievements of VGG models is their ability to effectively capture both high-level and low-level features using smaller filters. This demonstrated that the depth of the network combined with smaller filters can produce powerful representations. This insight paved the way for subsequent advancements in deep learning architectures. In the case of VGG-16, the network comprises 16 trainable layers, as depicted in Figure~\ref{fig6}. The VGG-16 model has been widely adopted in various computer vision tasks and has achieved state-of-the-art performance on benchmark datasets. Its success has inspired further research and the development of more advanced CNN architectures.

\begin{figure}
\centering
\includegraphics[scale=0.58]{vgg16-1.png}
\caption{VGG16 architecture with building block of last layers.}
\label{fig6}
\end{figure}

The pre-trained models VGG16 and Wide ResNet-50 are used in this study. Normalized input images in mini-batches, with RGB images having three channel dimensions~($3\times \rmH\times \rmW$), where H and W are assumed to be 224, are required by these models. In our approach, the final classification layers are substituted with fully connected layers. Specifically, three separate layers replace the last layer. Diameter classification, pipe type classification, and differentiation of propagation mediums are handled by the first, second, and third layers, respectively. ReLU and softmax activation functions are employed. The initial training layers are frozen, and the modified layer is trained using the B-Scans dataset. A training duration of 30 epochs with a batch size of 32 is used. Adam~\cite{zhang2018improved,tato2018improving} serves as the model optimizer, and categorical cross-entropy is employed for loss calculation. To enhance computational efficiency and performance, techniques such as early stopping, data augmentation, and adaptive learning rates are employed. These techniques aim to improve both the speed of computation and the overall performance of the model.

\begin{figure}
\centering
\includegraphics[scale=0.7]{transfert learning.png}
\caption{Transfer learning strategy.}
\label{fig7}
\end{figure}

\subsection{Custom Model}

In this work, a deep convolutional neural network is developed for multi-label classification to classify 9 different diameters, 3 different pipe types, and 4 different propagation media. The custom CNN model is inspired by the inception network~\cite{cao2018recent}, it is a model developed by google. The main difference between the inception model and the ordinary CNNs~\cite{jmour2018convolutional} are the inception blocks. These consist in convoluting the same input with several filters and concatenating their results. The aim of this model is to introduce the concept of multipathing, which allows different characteristics to be simultaneously captured and extracted. The building block is illustrated in Figure~8, this block contains different convolution layers of different filter sizes $(1x1)$, $(5x5)$, $(7x7)$. The diversity of the kernel size will increase the capacity of the network to extract the most complex features. In the input layer, 32 filters of size $3 \times 3$ are used, followed by a MaxPool layer and then a batch normalization layer to increase the speed and stability of learning. After that, a convolution layer with 64 filters of size $3x3$ is applied, followed by a max pooling layer. Two inception building blocks are then defined, each containing convolution layers with 128 and 16 filters of size $(1x1)$, $(5x5)$, $(7x7)$, respectively. At the end of each inception block, all feature maps are concatenated. The activation function employed in all convolutional layers is the rectified linear unit (ReLU) defined as: the output of ReLU is equal to 0 if the input value is less than 0 and equal to the input value if the input is positive. Then, all the feature maps are transformed into a 1D vector using a flattened layer. A set of 3 dense layers has been defined, containing 256, 512, and 256 neurons respectively. Each layer is followed by an activation function ReLU. Finally, 3 branches were defined for each output, each branch includes upstream a set of three dense layers respectively consisting of 96, 48 and 48 neurons, each output layer has 9 neurons to recognize the diameter size, 4 neurons to specify the propagation medium, and 3 neurons to distinguish the pipe type. The output layers are followed by a softmax layer mainly used for multi-label classification~\cite{read2014deep} problems to predict the probability of each label. As in most CNN, the data split between the training and validation data is 80 per cent and 20 per cent respectively. The ADAM optimizer has been used in the present case.

\begin{figure}
\centering
\includegraphics[scale=0.6]{builduing block_customcnn.png}
\caption{Custom CNN Model - building block.}\label{custom}
\end{figure}

\section{Results}

The simulation dataset consists of nine different diameter values, three types of pipes, and four different propagation mediums. The dataset includes 50 different depths for each simulation scenario. Each simulation scenario generates a B-scan image, resulting in a total of 5,400 B-scan images. Each image is constituted by $224x224$ pixels. To further enhance the dataset, data augmentation techniques were applied, resulting in a total of 10,800 B-scan images. The data augmentation process involved key transformations, including no rotation of the images, zooming up to 10~\% of their original size, horizontal movement up to 10~\% of their width, and vertical movement up to 10~\% of their height.

\subsection{Data augmentation}

Enhancing model performance is a comprehensive process that involves employing various techniques to improve the accuracy, efficiency, and robustness of the model. This includes the use of advanced algorithms, preprocessing data, selecting relevant features, and tuning hyperparameters. By optimizing these factors, significant improvements can be achieved in the model's performance. The effectiveness and reliability of a deep learning model greatly depend on the quantity and diversity of the training data. To enhance data diversity and improve the model's interpretation of information, basic data augmentation techniques as described in reference~\cite{van2001art,shorten2019survey,
hu2019preliminary} are employed. These techniques introduce visual variability to the data, leading to enhanced accuracy. For the B-Scans GPR dataset, the applied data augmentation techniques involve the use of Horizontal and Vertical shifts, as well as Resizing.

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{accuracyresume.jpg}
\caption{Accuracy of the three developed models with Canny's operators.}
\label{fig9}
\end{figure}


\subsection{Early stopping techniques}

Early stopping~\cite{bai2021understanding,
mahsereci2017early} is a regularization technique commonly used in deep neural networks. It involves monitoring the model's performance on a validation dataset during training and stopping the training process after a certain number of epochs if the model's performance does not improve. The weights of the model are saved and updated throughout training. When further updates no longer result in performance improvement, training is halted, and the most recent optimal parameters are kept.

\subsection{Evaluation metrics}

Three indicators were used to evaluate the three Deep Learning models: Precision, Recall, and F1- score, the accuracy is the simplest indicator, it measures the percentage of correct predictions. It is defined as follows:
\begin{equation}
\text{accuracy} = \frac{TP + TN}{TN + TP + FP + FN}
\vspace*{5pt}
\end{equation}

\noindent Based on the results presented in Figure~\ref{fig9}, a higher accuracy of 96~\% is observed for the diameter identification using the VGG16 mode, and an efficiency of 98~\% for the pipe type identification achieved by the Resnet-50 model. We also reach a higher level of sensitivity for the identification of the propagation medium, which achieves 98~\% for both the VGG16 model and the customized CNN model.


TP (True positive) stands for the number of accurate diameter, pipe type and medium identifications, FP (False Positives) represents the number of incorrect identifications and FN (False negative) represents the number of failed recognitions. In GPR, accurate identifications mean that the target is well located and accurately classified. Incorrect identifications are indicative of a well-located target, but its geometry, as well as its type and medium, are not properly classified. The missed identifications reflect the target not being located. The Table~\ref{table3} shows the performance of the model Resnet-50 based on the 9 classes for the identification of diameters.\linebreak The overall accuracy is 96~\%. Table~\ref{table3} shows the classification report for the 9 diameter values, we find the precision that indicates the performance or the positive prediction given by the model, recall is a statistical metric that shows how many positive cases actually correspond to the predicted class. The F1 score provides the information about the incorrect predictions of the model meaning that 1 is the best and 0 is the worst. The F1 score is obtained by calculating the precision and recall of the model. From Figure~\ref{fig10}, out of 86 images it is seen that 80 are correctly predicted as having a diameter of 80mm and remaining 6 incorrectly assigned to other classes. So the precision of the diameter 80 mm class shown in Table~\ref{table3}, can be calculated as:
\begin{equation}
\text{Precision} = \frac{80}{86} = 0.93
\end{equation}

In Figure~\ref{fig10}, out of a total of 86 images labeled as 80 mm diameter, 80 images were correctly predicted, resulting in a recall of approximately 93.00~\% for the class ``diam80''.
\begin{equation}
\text{Precision} = \frac{80}{82} = 0.98
\end{equation}
The F1-score can be calculated as follows:
\begin{equation}
\text{F1-score} = \frac{2 \cdot \text{Precision} \cdot \text{recall}}{\text{Precision} + \text{recall}} = 0.95
\end{equation}

\begin{table}[htbp]
\centering
%\captionsetup{justification=centering}
\caption{Results of the ResNet-50 Classification Report: Diameter Identification on the Test Set.}
\begin{tabular}{||c c c c||}
\hline Labels & Precision & Recall & F1-Score \\
\hline\hline Diameter 16mm & 0.96 & 0.98 & 0.99 \\
Diameter 24mm & 1.00 & 0.98 & 0.99 \\
Diameter 32mm & 1.00 & 0.98 & 1.00 \\
Diameter 40mm & 1.00 & 0.95 & 0.98 \\
Diameter 48mm & 0.95 & 0.97 & 0.96 \\
Diameter 64mm & 0.85 & 0.98 & 0.96 \\
Diameter 72mm & 0.95 & 0.84 & 0.89 \\
Diameter 80mm & 0.93 & 0.98 & 0.99 \\
Diameter 100mm & 1.00 & 0.98 & 0.99 \\
\hline
\end{tabular}
\label{table3}
\end{table}

In Figure~\ref{fig10}, the confusion matrix provides insights into the model's accuracy in predicting diameters. It excelled in predicting diam16, diam32, and diam80, but misclassified diam72 as diam80 or diam64. The matrix depicts actual class labels on the vertical axis and predicted class labels on the horizontal axis. It showcases precise classification for all four categories (clay, concrete, fractal, sand) with no misclassifications. However, some instances of ``Water-filled pipe'' were misclassified as ``Vaccum'' while the model performed well in classifying ``Metallic'' and ``Vaccum'' categories.

\begin{figure}
\centering
\begin{subfigure}[]
{\includegraphics[scale=1.62]{matrixconfusion_resnet50_diameter.jpg}}
\end{subfigure}
\\
\begin{subfigure}[]
{\includegraphics[scale=0.125]{Resnet50mediuem_resnet50.jpg}}
\end{subfigure}
\\
{\hspace*{-12pt}
\begin{subfigure}[]
{\includegraphics[scale=0.13]{typepipe_resent50.jpg}}
\end{subfigure}}
\caption{Matrix-confusion of diameter identification - Medium of propagation and type of pipe classification with Resnet-50 Model.}
\label{fig10}
\end{figure}

\section{Discussion and conclusions}

In this study, the focal point was on developing a multi-label classification (MLC) model using transfer learning and data augmentation techniques. Three deep learning models, namely VGG-16, ResNet-50, and an adapted CNN, were utilized as pre-trained models for transfer learning. These models were trained on a synthetic dataset created using the TEMSI-FD software, which simulates the propagation of electromagnetic waves in complex media. To enhance the model's performance, various techniques were employed, including advanced algorithms, data preprocessing, feature selection, and hyperparameter tuning. Data augmentation techniques, such as Horizontal and Vertical shift, as well as Resizing, were used to improve data diversity and the model's interpretation of information. The regularization technique of early stopping was applied to the training process. This technique involves monitoring the model's performance on a validation dataset and stopping the training after a certain number of epochs if no improvement is observed. The weights of the model are saved and updated throughout training, and the most recent optimal parameters are used when further updates no longer improve performance. Based on the results presented the VGG16 model achieved a better accuracy of 96~\% in diameter identification, while the Resnet-50 model achieved an efficiency of 98~\% in pipe type identification. Both the VGG16 model and the customized CNN model achieved a higher level of sensitivity (98~\%) in the identification of the propagation medium. To evaluate the performance of the network in the presence of various underground objects, B-Scan images were created with up to 2~objects of different diameters, depths, and pipe types. In conclusion, this study successfully developed a multi-label classification model using deep learning techniques, transfer learning, and data augmentation. The models demonstrated high accuracy and sensitivity in identifying diameters, pipe types, and propagation mediums. The use of synthetic datasets, advanced algorithms, and regularization techniques contributed to the model's improved performance and reliability. Future work will focus on generating noisy data to assess the robustness of our models. As well as the development of a new algorithm based on transformers for extracting characteristics from Radar signals while combining A-Scan and B-Scan.


\section*{Declaration of interests}
The authors do not work for, advise, own shares in, or receive funds from any organization that could benefit from this article, and have declared no affiliations other than their research organizations.

\printbibliography
\end{document}
