Novel application of machine learning for estimation of pullout coefficient of geogrid

In the present study, the use of the RF regressiontechnique for estimation of pullout coefficient of geogrid embedded in different structural fills and atvariable normal stress based on

Novel application of machine learning for estimation

of pullout coefficient of geogrid

A Pant1and G.V Ramana2

1

Research Associate, Department of Civil Engineering, Indian Institute of Technology Delhi, India,

E-mail: aali.pant@gmail.com (corresponding author)

2

Professor, Department of Civil Engineering, Indian Institute of Technology Delhi, India,

E-mail: ramana@civil.iitd.ac.in

Received 02 June 2021, accepted 15 September 2021, published 09 March 2022

ABSTRACT: Pullout behaviour of geogrids is critical to understand for the design of mechanically

stabilized earth walls The pullout coefficients are determined through laboratory testing on geogrids

embedded in structural fill Random forest (RF) is a data-driven ensemble learning method that uses

decision trees for classification and regression tasks In the present study, the use of the RF regression

technique for estimation of pullout coefficient of geogrid embedded in different structural fills and at

variable normal stress based on 198 test results has been investigated using five-fold cross-validation.

80% of the data has been trained on the model algorithm and the accuracy of the model is then tested on

20% of the remaining dataset The performance of the model has been checked using statistical indices,

namely R2, mean square error, as well as external validation methods The validity of the model has also

been checked against laboratory tests conducted on geogrid embedded in four different fills The results

of the RF model have been compared to results obtained with three other regression models, namely,

Multivariate Adaptive Regression Splines, Multilayer Perceptron, and Decision Tree Regressor The

results demonstrate the superiority of the RF-based regression model in predicting pullout coefficient

values of geogrid.

KEYWORDS: Geosynthetics, random forest, machine learning, pullout test

REFERENCE: Pant, A and Ramana, G.V (2022) Novel application of machine learning for

estimation of pullout coefficient of geogrid Geosynthetics International, 29, No 4, 342 –355.

[https://doi.org/10.1680/jgein.21.00021a]

1 INTRODUCTION

Mechanically stabilized earth walls are subject to external

and internal modes of failure According to FHWA

(2001), the primary modes of failure against internal

stability of a mechanically stabilized earth wall are pullout

and tension failure of reinforcement Thus, in its design,

the knowledge of soil-geosynthetic interaction behaviour

is particularly important The reinforcement length

depends mainly on the assumed pullout coefficient

(FHWA, 2001) The pullout coefficient depends on

several factors such as type of soil (particle size

distri-bution, placement condition, and moisture), type of

reinforcement (geometry and mechanical properties),

boundary conditions and loading conditions (Moraci

et al 2014)

Pullout coefficient is determined through pullout

testing done on reinforcement (geosynthetic or metallic

grid) while being embedded in a fill material at different

normal stresses The specifications of a typical pullout

testing machine regarding box dimensions, loading

system, strain rate and clamps, and other parameters are already specified ASTM D6706-01 A number of researchers have studied the pullout behaviour of geogrids embedded in different types of fills (Moraci and Recalcati 2006; Teixeira et al 2007; Cardile et al 2016; Prasad and Ramana 2016a)

Pullout test is a large-scale and time-consuming experiment Thus analytical methods have been developed

to estimate pullout coefficient geogrid using properties of soil and geogrid (Jewell 1990; Cardile et al 2017) These methods depend on the engineering properties of soil (angle of shearing resistance) as well as the skin friction angle between soil and geogrid corresponding to each normal stress, calculating which can be tedious Huang and Bathurst (2009) developed statistical bi-linear and non-linear models for prediction of pullout capacity of geosynthetics based on a large database of published test results The non-linear models demonstrated superiority

to the approaches described by FHWA (2001) The models however were purely empirical and were trained and tested on the same dataset This limitation can be

overcome by regression model based on machine learning

(ML)

ML techniques are gaining momentum to make

accurate estimates in the field of geotechnical engineering

The utility of ML has been explored by researchers to

estimate basic geotechnical properties as well as those

pertinent to site characterization (Kirts et al 2018;

Tsiaousi et al 2018; Hu and Solanki 2021; Jalal et al

2021; Mittal et al 2021; Zhang et al 2021) However,

relatively fewer studies have been reported on predicting

the behaviour of geosynthetic reinforced soil using ML

Chou et al (2015) estimated the tensile loads generated in

geogrids used as reinforcement in geosynthetic-reinforced

soil structures using an evolutionary metaheuristic

intelli-gence model that was an improvement of the firefly

algorithm

Prasad and Ramana (2016b) analyzed experimental

pullout test data to model and capture the influence of

several geogrid parameters and structural fill properties

on pullout coefficient using an artificial neural network

(ANN) The authors stressed the need for a large database

for better prediction and analysis of results Debnath

and Dey (2018) used simplified vector regression for

prediction of the bearing capacity of unreinforced

sand bed and geogrid-reinforced sand bed resting over a

group of stone columns floating in soft clay Sharma et al

(2019) studied the application of ANN and genetic

programming methods in predicting the dynamic

response of geogrid reinforced machine foundation beds

Ghani et al (2021) studied the response of strip footing

resting on prestressed geotextile–reinforced industrial

waste using ANN and extreme learning machine Raja

and Shukla (2021) studied the settlement of shallow

reinforced soil foundations using five different types of

ML algorithms

ML applicability studies in geotechnical engineering

have primarily used a single sophisticated algorithm for

prediction (Sharma et al 2019; Mittal et al 2021; Raja

and Shukla 2021) Due to the complex behaviour of soil,

geosynthetics, and their interaction, utilization of

ensem-ble methods must be explored to model such behaviour

Ensemble learning combines multiple algorithms that

process several learner hypotheses in order to generate a

generalized hypothesis that gives better predictions

(Zhang et al 2021)

In this paper an ensemble learning based method,

namely random forest (RF), has been used to develop a

regression model to predict the pullout coefficient of

geogrid embedded in soil A robust dataset comprising

soil properties, geogrid parameters and pullout coefficient

from 198 published pullout test results has been compiled

ML models have then been trained using five-fold

cross-validation on randomly selected 80% of this data and then

tested on the remaining 20% of the set Laboratory

pullout tests have then been conducted on four different

fill materials, the results of which have been compared

with predictions from ML models In order to

demon-strate the superiority of the ensemble learning method,

ML models have also been developed using three other

widely used regression models

2 METHODOLOGICAL BACKGROUND

2.1 RF

RF is a bagging technique-based statistical learning theory that uses the bootstrap resampling method The RF method was developed by Breiman (2001) It extracts multiple random samples from the original dataset and models the decision tree for each of the bootstrap samples Each tree, trained using different bootstrap samples of the training data, acts as a regression function on its own Subsequently the predictions of multiple decision trees are combined and averaged to build the result The RF method creates many decision trees and provides the opportunity to evaluate over the combination of these trees

In the RF method, the structure where decision trees are formed is called the forest In the forest, each decision tree

is created by selecting samples from the data set by row-sampling with replacement technique and determin-ing the number of random variables determined from all variables at each node

RF regression involves construction of k number of trees {Tk(X ), m = 1, 2,…, k} The p-dimensional vector

X = {x1, x2,…, xp}, where p is the number of features in the dataset, that forms the forest is considered as the input vector of the algorithm This ensemble generates k outputs corresponding to k number of trees and yˆm, m = 1, 2,…, k

is denoted as the output of each tree (note: yˆm= Tk(X )) The average of the output of each tree is considered as the result of the algorithm

In the RF algorithm, a new training dataset (bootstrap samples) is selected by replacing the original training dataset for each regression tree structure This leads to several training data sets being omitted from the sample, which can

be reused These omitted data are known as out-of-bag (OOB) samples and constitute one-third of new training samples The other two-thirds of the data is used to derive the regression function Thus, a randomly drawn training sample from the original training set is selected for creating a decision tree each time, and one of the samples outside the bag is used for accuracy testing developing a generalized RF model The total learning error is denoted by yˆeand is given

in Equations 1 and 2, respectively as:

ˆyið Þ ¼Xi 1

k

Xk m¼1

ˆye¼1 n

Xn

i ¼1

ðˆyi yiÞ2 ð2Þ

where yˆi, yi and n represent the prediction of each tree created by using OOB samples, the true output, and the total number of OOB samples, respectively This error shows the prediction performance of the RF algorithm Figure 1 illustrates the schematic view of flowchart of the

RF algorithm

2.2 Multivariate adaptive regression splines (MARS) model

The MARS algorithm is a nonparametric regression method utilized for solving non-linear functions It

estimates the relationships between the input dimensions

and the output variable by dividing the training dataset

into a series of piecewise segments (splines) of varying

slope The splines are polynomials which can be linear or

non-linear (Raja and Shukla 2021) The MARS

model-ling method thus analyses the multivariate data and finds

the influence of features in the form of basic functions

(BF) for simulation of the target variable The terminal

point of each spline is known as the knot at which data

splits into two domains The splitting produces BFs,

which provide MARS with the freedom to adapt for

bends, thresholds, and departure from linear regression

Figure 2 illustrates the schematic view of the MARS

algorithm flowchart

BFs are selected using a stepwise search together with a

selection of knot locations through adaptive recursive

regression methods Thus, the MARS model consists of

forward and backward passes wherein in the forward pass,

potential knots and functions are added until residual

error minimizes In the backward pass, the large overfit model is pruned by removing the least effective terms from the model If X = X, …, XR is the input vector of R number of training points and y is the output variable, then y can be defined mathematically as in Equation 3:

y¼ f Xð Þ þ e ð3Þ where e is the distribution of error The function f is estimated by using basic functions, BF(X ) Thus, complex relations from high dimensional datasets can be extracted Equation 4 demonstrates the approximation functioning

of MARS, considering the linear piecewise segments which take the form of max(0, x− c) with a knot at point c:

max 0ð ; x  cÞ ¼ x c; if x  c

0; otherwise

(

ð4Þ

By applying a linear combination of several BFs and their interactions, the function f (X ) is defined as in Equation 5:

f Xð Þ ¼ βoþ XN

n¼1

βnBF Xð Þ ð5Þ

whereβois a constant andβnis the BF, which can either

be an individual spline function or the product of two spline functions in the model The backward algorithm that is used to prevent overfitting removes insignificant BFs A generalized cross-validation (GCV) is employed to eliminate the unimportant BFs based upon Equation 6:

GCV¼ RMSE

1 ððNF þ dNFÞ=RÞ

where RMSE is the root mean square error for the training dataset, NF is the number of BFs, d is the penalty factor, and R is the number of data points In-depth details and mathematical derivation of the MARS algorithm have been discussed by Friedman and Roosen (1995)

2.3 Multilayer perceptron (MLP) MLP is a supervised learning algorithm that is a feed-forward ANN, and generates results using back-propagation function on the training dataset (Singh et al 2012) Back-propagation reduces the error between pre-dicted and true output value MLP has a three-layered architecture, comprising an input layer consisting of n input nodes, and an output layer consisting of m output nodes The single or multi hidden layer converts the input command to output using optimizer functions as shown

in Figure 3 The training process of the MLP regression comprises two stages: (i) calculate the output value using forward propagation of the input values through the hidden layers, and estimate the error between the predicted and true value, and (ii) minimize the error by adjusting the connection weights and estimation of output with revised weights

Training data

Sample 1

Average of all predicted values

Prediction k

InBag 1 (2/3)

InBag 2 (2/3)

(2/3) OOB 1

(1/3)

OOB 2 (1/3)

(1/3)

n observations, m predictors

Tree 2 Tree 1

Tree k

Figure 1 Architecture of RF model

Construction phase (input data and number of BFs)

Pruning phase (determination

of GCV and deleting redundant BFs)

Final Mars Model and check for performance

Use MARS

model

Predict pullout

coefficient

Change number

of BFs

R2 ~ 1

Figure 2 Architecture of MARS model

Geosynthetics International, 2022, 29, No 4

2.4 Decision tree regressor (DTR)

Decision tree regression is a non-parametric supervised

ML method that uses binary rules to calculate a target

value The DTR algorithm is based on recursive

parti-tioning of input parameters to classify the data where it

splits the dataset by data entropy and performs a local

linear regression on the new subsets of data (Wei et al

2019) Multiple iterations are run on the dataset to give

different decisions and the best fit is determined by the set

of decisions that lead to least entropy The algorithm has a

recursive tree like structure of decision calls each of which

contributes to a change in entropy On every decision call,

the set of input parameters is split into two or more

subsets The recursive splitting continues over each subset

until the terminal nodes are reached The number of

subsets doubles every time the data splits which leads to

increasing depth of the decision tree The accuracy of

DTR is calculated from the mean square error value

between predicted and actual values

3 EXPERIMENTAL DATABASE

COLLECTION

3.1 Model inputs and outputs

Input parameters play the most significant role in training

a data-driven model Thus, it is critical to select

parameters that impact the output variable The factors

that influence pullout test results can be classified into

three categories: influence of design of pullout apparatus,

influence of geosynthetic reinforcement, and influence of

fill material Several researchers have assessed the

influ-ence of these factors on pullout behaviour of

geosyn-thetics While using ML for development of a data driven

predictive model, it is important to incorporate critical

input parameters for the development of a robust and

generalized model In order to develop a comprehensive

model and reduce its complexity it is also important to

limit the number of input parameters without

compromis-ing the model’s accuracy Thus, in this study, published

pullout test results that complied with the apparatus

guidelines of ASTM D6706-01 (2013) have been used

which allowed the authors to remove the influence of

design of pullout apparatus as an input parameter for model development

Pullout force is directly proportional to normal stress (Farrag et al 1993; Lee and Bobet 2005) Increase in normal stress reduces soil dilatancy, thus leading to an increase in passive soil resistance The rate of increase of pullout force decreases with increasing normal stress (Alfaro et al 1995) Thus, normal stress is a key input parameter to be considered for prediction of the pullout interaction factor Amongst the properties of soil, fines content and average particle size of fill (D50) of soil have been considered as input parameters Pullout resistance depends on the ratio between the geogrid opening size and

D50(Jewell et al 1984; Lopes and Lopes 1999) Pullout resistance decreases with an increase in percentage of fines

in fill (Pant et al 2019e)

Length of geogrid, spacing between longitudinal and transverse members, and ultimate tensile strength of geogrid have been considered as input parameters for development of prediction models Teixeira et al (2007) observed that an increase in geosynthetic specimen length leads to an increase of pullout resistance, stiffness and displacement at peak pullout resistance Pullout resistance increases linearly with geogrid specimen length Moraci and Recalcati (2006) investigated the influence of geogrid specimen length on the pullout behaviour of geogrids and reported that for longer geogrids (L = 1.5 m) under higher normal stresses pullout resistance increases progressively with increase in displacement On the other hand, under low normal stress, both short (L = 0.4 m) and long geogrid specimens (L = 1.5 m) show a progressive decrease of pullout resistance after the peak with further displacement

Palmeira and Milligan (1989) indicated that a decrease

in spacing between transverse members causes a reduction

in peak pullout resistance due to increasing interference between transverse members Teixeira et al (2007) reported that there is an optimum spacing (43 mm) between transverse ribs that maximizes pullout resistance Similarly, Calvarano et al (2012) reported that a peak pullout resistance is obtained at a spacing of 86.7 mm between transverse members A spacing less than or more than optimum spacing affects pullout resistance value Thus, spacing between transverse members has been shown to be a critical factor that influences pullout test results and has been considered as an input parameter for development of the data-driven predictive models Aperture size of geogrids has also shown a considerable effect on pullout interaction coefficient (Bernal et al 1997) An increase in pullout interaction coefficient is observed with decreasing aperture size Chen et al (2013) concluded that geogrid aperture size plays a more significant role than tensile strength or thickness of geogrid ribs Therefore, to consider the influence of aperture size of the geogrid, spacing between longitudinal members has been used as an input parameter in this study

Nayeri and Fakharian (2009) indicated that structural stiffness of the geogrid has a direct influence on pullout resistance at different normal stresses Abdel-Rahman

Input layer Input 1

Input 2

Input 3

Input 4

Input 5

Input 6

Hidden layer Output layer

Output

Figure 3 MLP architecture employed in this study

et al (2007), Prasad and Ramana (2016b) and

Wilson-Fahmy et al (1994) indicated that pullout

resist-ance increases non-linearly with an increase in the

ultimate tensile strength of the geogrid Hence, ultimate

tensile strength has also been considered as an input

parameter to develop a robust model for prediction of

pullout interaction coefficient

It has been demonstrated in the literature that other

factors such as fill material properties (for instance, angle

of shearing resistance of soil, relative density of soil), and

mechanical and physical properties of geogrid (like width

of geogrid, shape of aperture) also influence the pullout

capacity of geogrids The factors were however not

considered as input variables for model development,

primarily due to the lack of data of these parameters

in much of the published literature used as a database in

this study Moreover, it has been reported in the literature

that soil properties such as angle of shearing resistance

depend significantly on the normal stress, fines content

and D50 of soil that have been included as input

parameters for model development (Wang et al 2009;

Phan et al 2016)

Normal stress, D50, fines content (per unit weight),

length of geogrid specimen (L), spacing between the

longitudinal members (sL), spacing between the transverse

members of geogrid (sT) and ultimate tensile strength of

the geogrid (Tult) have thus been used as the input

parameters which affect the pullout coefficient of the

geogrid In the models developed, these parameters have

been used as input variables to predict pullout coefficient

of geogrid embedded in granular fill

3.2 Database description

The size of the dataset plays the most critical role in

development of a reliable prediction model in ML In

order to develop the models, published laboratory pullout

test results on geogrids embedded in different types of fill

materials were retrieved and compiled to develop a

robust database of 198 samples (Goodhue et al 2001;

Duszyń ska and Bolt 2004; Moraci and Recalcati 2006;

Abdel-Rahman et al 2007; Teixeira et al 2007; Vieira

et al 2016; Prasad and Ramana 2016a, 2016b; Abdi and

Mirzaeifar 2017; Wang et al 2018; Mirzaalimohammadi

et al 2019; Pant et al 2019a, 2019b 2019c, 2019e) Table 1

lists the statistical properties of the dataset summarizing

the central tendency, dispersion, and shape of the dataset

distribution Count refers to the number of individual

samples available, mean is the average value of any feature under consideration, std is the standard deviation of the feature, min is the minimum value of the feature in the dataset, 25%, and 75% refers to the percentile value of the feature, and max refers to its maximum value The developed ensemble learning models have also been verified on new data obtained from conducting laboratory pullout tests on geogrids embedded in four types of granular materials It must be pointed that these test results were not a part of the database used to develop the ML models Pullout tests were carried out on an apparatus with inner dimensions of 900 mm × 600 mm ×

600 mm (length × width × height) Air-dried material was filled in the test box in four layers of 150 mm in height and thereafter compacted to achieve a relative density of 80% Geogrid was fastened between clamps using countersunk bolts and positioned within the compacted fill material at

300 mm in height An airbag was used to apply uniformly distributed pressure over the fill surface The applied load and the displacements were monitored for a pullout displacement of 100 mm The pullout tests were per-formed at three different normal stresses (20, 40, and

80 kPa)

4 MODEL CONSTRUCTION AND IMPLEMENTATION

For prediction of pullout coefficient values, four data-driven regression-based models, – that is, RF, MARS, MLP, and DTR, were constructed and implemented The analysis and coding of algorithms in this study has been conducted on Python 3.8.5 The dataset of 198 samples was randomly divided into training and test datasets in

80 : 20 ratio While each algorithm was trained on the training dataset (158 randomly picked pullout test results), the predictability of the model was checked on test set (remaining 40 pullout test results) Five-fold cross validation was used on the training dataset for optimiz-ation of regression models Each dataset was randomly divided into five folds, four of which were utilized for training purposes while the model was tested on the remaining fifth fold The procedure was repeated over five runs Due to the unavailability of a large dataset, the number of folds was restricted to five only

The optimized RF model was created using the RF Regressor algorithm within the Scikit-learn library of

Table 1 Statistics of the dataset used in the study

Normal stress (MPa) D 50 (mm) Fines L (m) s L (m) s T (m) T ult (kN/mm) Pullout coeff

Geosynthetics International, 2022, 29, No 4

Python 3.8.5 The critical hyperparameters to tune when

using the algorithm are number of trees in the forest and

the size of the random subsets of features to consider

when splitting a node In this study a RF model with

600 trees was constructed that considered all the

seven input features for node split The influence

of limiting the maximum tree depth (total number of

splitting nodes) was assessed over a range of 1 to

the maximum possible number of nodes The tree

depth was not constrained because the model accuracy

did not decrease with increasing tree depth The

importance of the predictor was calculated as the

percentage increase in model mean squared error when

the predictor was permuted However, irrespective of the

predictor importance the model was fit using all the

seven predictors

The MARS model was constructed using pyearth

library in Python 3.8.5 A maximum of three degrees

were generated by forward pass in the optimized MARS

model developed in this study The maximum degree of

the terms was restricted to three as increasing it increased

the non-linearity of the model which led to overfitting as

well as increase in computation time The value of the

penalty parameter used in the model was three The

penalty parameter is used to calculate generalized cross

validation It is used during the pruning pass and aids in

determination of addition of a hinge or linear basis

function during the forward pass The number of extreme

data values of each feature not eligible as knot locations

was determined to be five Default values of other

hyperparameters were used in the development of the

optimized MARS model

Based on the iterations on number of hidden layers to

be used for the development of the MLP model, a model

with three hidden layers and 50 to 100 hidden units was

created using the MLP Regressor algorithm within the

Scikit-learn library of Python 3.8.5 The rectified linear

unit function was used to activate the hidden layers A

default stochastic gradient-based optimizer was used as

weight optimization function for the layers The weight

optimization function was iterated until it converged (i.e

tolerance for optimization reached within 1e−4), or 5000

iterations, after which training of data was stopped An

initial learning rate of 0.01 was used as a step-size

controlling parameter for updating weights Standard

values of other hyperparameters were used in the

devel-opment of the MLP model

The DTR model was constructed using the DTR

algorithm within the Scikit-learn library of Python

3.8.5 A decision tree with large depth leads to the

construction of a complex model that has more splits and

thereby can gather more information on the training data

leading to overfitting Thus, the maximum depth of

decision trees was restricted to ten in the present study

Friedman mean square error was used as the criterion

to measure the quality of split The decision trees were

made five-deep A minimum of two samples were specified

to be at each leaf node for a split point at any depth of

decision tree to be considered for splitting an internal

node

4.1 Correlation analysis The correlation coefficient between any two variables helps in the preliminary investigation of the strength of interdependency between two parameters The Spearman’s rank correlation coefficient (rs) exhibiting the correlations between each pairwise feature is shown in the heatmap in Figure 4 The maximum value of rs is limited to 1 and runs diagonally along Figure 4 The higher the rsvalue, the higher the correlation between the two features The sign of correlation signifies proportion-ality, where positive value indicates the pair being directly proportional and negative refers to an inverse relation between the features Table 2 summarizes the correlation

of all parameters according to the absolute value of rs Normal stress plays the most significant role in estimation

of pullout coefficient

4.2 Check for outliers Outliers are the values of target variable that are either very small or very large compared to the average values and may bring bias to the model developed Figure 5 shows a box plot of pullout coefficient values It contains the upper limit, lower limit, median, as well as the upper quartile (Q3) and lower quartile (Q1) of the available pullout coefficient data The Q1and Q3values are the 25 percentile and 75 percentile values of pullout coefficient as mentioned in Table 1 The upper limit and the lower limit

Normal stress

Normal stress Fines content

Fines content

L

L sL

sL sT

sT Tult

Tult Pull coeff

Pull coeff D50

D50

1.0

0.8

0.6

0.4

0.2

0

–0.2

–0.4

–0.6

Figure 4 Correlation coefficient matrix heatmap of feature variables and label

Table 2 r svalue interpretation

0 –0.10 Very weak Pullout coeff vs D 50 ,

Pullout coeff vs s T , Pullout coeff vs T ult

0.1 –0.2 Weak Pullout coeff vs L,

Pullout coeff vs s L

0.2 –0.5 Moderate Pullout coeff vs Fines 0.5 –0.7 Strong Pullout coeff vs Normal Stress

have been calculated using Equations 7 and 8:

Upper Lt: ¼ Q3 þ 1:5 Q3  Q1ð Þ ð7Þ

Lower Lt: ¼ Q3  1:5 Q3  Q1ð Þ ð8Þ

The maximum and minimum values of the upper and

lower limit values are limited to the maximum and

minimum values of pullout coefficient used in this study

as mentioned in Table 1 As can be observed from

Figure 5, no data point lies outside the lower and upper

limit of pullout coefficient values, indicating that the

entire dataset is representative of true pullout coefficient

values with no outliers

5 PERFORMANCE EVALUATION

METRICS

5.1 Training and testing performance

The statistical analysis between the observed and

pre-dicted values of the pullout coefficient was conducted to

assess the accuracy of all the models Three widely used

statistical performance criteria have been used as

per-formance indicators of the data-driven models (Raja and

Shukla 2021), namely: (i) coefficient of correlation (R2);

(ii) mean square error (MSE); and (iii) mean absolute percentage deviation (MAPD) The mathematical rep-resentation of the criteria has been summarized in Table 3

Fp,i and Fo,i are the predicted and observed values of pullout coefficient values, respectively

The values of the statistical parameters for the training and testing datasets have been tabulated in Tables 4 and 5, respectively The statistical metrics of each model is calculated for both training and testing dataset individu-ally A model that gives better values of performance evaluation indicators in the training dataset compared to the testing dataset indicates that the model exhibits low bias and high variance In other words, the model gives less error in predictions made on training dataset than on testing dataset Such a model is referred to as an overfit model and is not considered as a robust model for making predictions (difference between R2 values of DTR in training and testing dataset)

Ranks were given to the models for each criterion, where larger rank corresponds to better performance against the criterion under study The total score of ranks

is the sum of ranking score of a model for each statistical parameter Based on the total score, the final rankings of all the models were determined

From the results reported in Tables 4 and 5 of training and testing datasets, it can be observed that the perform-ance of the four models varied greatly, but the RF model significantly outperformed MARS, MLP, and DTR for both training and testing data The R2value of the RF model was found to be 0.97 for training dataset and 0.83 for testing dataset, which demonstrated that it had satisfactory estimative capabilities for predicting the pullout coefficient of geogrids In addition, the lower error indices (MSE and MAPD) for the RF model indicate unbiased estimations and less difference between observed and predicted response of geogrids Thus, amongst the four models used in this study, the highest R2and lowest error indices were obtained in the training dataset of the RF model, indicating that the RF model has excellent training ability Irrespective of the fact that the R2and error indices for the testing dataset of the

RF model showed a decrease of estimation accuracy compared with the training dataset, they are still higher than other data-driven models used in his study These observations signify that the RF model outperforms MARS, DTR and MLP models in predicting pullout coefficient of geogrids

2.00

1.75

1.55

1.25

Upper Lt.

Q3

Q1 Median

Lower Lt.

1.00

0.75

0.55

0.25

0

1 Pullout coeff

Figure 5 Pullout coefficient values boxplot for identification of

outliers

Table 3 Performance evaluation indicators

i¼1Fp;iFo;i Pn

i¼1Fp;i P n i¼1Fo;i

n P n i¼1 Fp;i  2

 Pn i¼1Fp;i

n P n i¼1 Fo;i  2

 Pn i¼1Fo;i

n

X n i¼1

Fo;i F p;i

= 0

n

X n i¼1

Fo;i F p;i

Fo;i



Geosynthetics International, 2022, 29, No 4

The RF model understands the complex nonlinear and

hierarchical soil-geosynthetic interaction behaviour It

also resists overfitting by demonstrating insensitivity to

noise in input data and has an unbiased error rate

measurement compared with other estimation methods,

resulting in higher estimation accuracy by the model

(Breiman 2001)

Tables 6 shows the combined performance of all

the models applied in the prediction of pullout coefficient

value In this table, considering the individual ranks

obtained by each model according to the statistical indices

(R2, MSE and MAPD in Tables 4 and 5), a total rank was

provided to each model based on either the train or test

total score, whichever was lower The ranking showed that

the RF model achieved the highest predictive accuracy

(total score = 12) The DTR model obtained the

second-best accuracy (total score = 5) Furthermore,

MLP (total score = 4) and MARS (total score = 3)

models showed lower accuracy in predicting the pullout

coefficient value of the geogrid in comparison to RF

The scatter plots of real and predicted values of pullout

coefficient values of test dataset obtained through the ML

models can be seen in Figures 6a–6d According to the

comparisons, it can be observed that the RF-based model

maintained a high prediction accuracy in the testing sets

Similar observations have been made on the training

datasets, the results of which have not been plotted as the reliability of a model depends on its accurate prediction of test datasets It can be noted that the values of pullout coefficient greater than 1 are normally under- or over-predicted by the models This may be attributed to the restrained dilatancy effect that is exhibited by structural fill during shearing at low normal stresses, thereby increasing the actual normal stress at the soil-geogrid interface which remains unaccounted for in the models developed in this study (Pant et al 2019a)

5.2 External model validation External validation is the process of comparing observed and predicted results using a certain set of statistical criteria Golbraikh and Tropsha (2002) developed an external model validation method that evaluates the reliability of model predictions based on model perform-ance on test dataset The method is designed to ensure model reliability even for a small dataset through rigorous statistical penalties For a model to be considered acceptable, it is mandatory for it to meet certain criteria that have been discussed below

For a model to be called 100% accurate, its ideal value

of correlation coefficient – that is, R2

must be 1 This means that one of the regression line gradients– that is, predicted versus observed values, or vice versa, passing

Table 4 Statistical parameters for the training dataset

Proposed model Network results in training dataset Ranking the predicted models Total ranking score Rank

Table 5 Statistical parameters for the testing dataset

Proposed model Network results in testing dataset Ranking the predicted models Total ranking score Rank

Table 6 Ranking of ML models based on training and testing scores

through the origin should approximate to 1 Thus,

k ¼

Pn i¼1Fp;iFo;i

Pn

k′ ¼

Pn

i ¼1Fp;iFo;i

Pn

where k and k′ are the slopes of regression lines through

the origin for fits to experimental and predicted data,

respectively The correlation coefficients passing through

the origin can be defined as Equations 11 and 12:

R2o¼ 1 

Pn i¼1F2p;ið1 kÞ2

Pn

R′2o ¼ 1 

Pn i¼1Fo;i2 ð1 k′Þ2

Pn

where Fp,mean and Fo,mean are the mean values of the

predicted and real pullout coefficient value of geogrid,

respectively R0 and R0′2

are the determination coefficients

of the predicted versus the observed values and of the observed versus the predicted values, respectively Rs,

referred to as a stabilization criterion, can be calculated using Equation 13:

R2s ¼ R21 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiR2 R2 ð13Þ According to the external validation method, any model is reliable if it meets at least two of the following conditions: (1) R2≥ 0.6; (2) 0.85 ≤ k ≤ 1.15 or 0.85 ≤

k′ ≤ 1.15; (3) R0 or R0′2

close to R2– that is, the ratio of absolute difference of R2from R0′2

(or R0′2

) to R2must be less than 0.1; and (4) Rs2≥ 0.5 A model is considered fully acceptable if it meets all four conditions discussed The results of the external model validation criteria for the test dataset have been summarized in Table 7 It may

be noted that for conditions 2 and 3, only one sub-criteria needs to be satisfied It is evident from the results that none of the four models met all the four conditions of the validation method including the stabilization criteria RF exhibited the best performance amongst the four models satisfying three of the four conditions, while the MARS,

2.0

Lab results RF 1.6

1.2

0.8

0.4

0

(a)

2.0

Lab results MARS 1.6

1.2

0.8

0.4

0

(b)

2.0

Lab results MLP 1.6

1.2

0.8

0.4

0

Count (c)

Count (d)

2.0

Lab results DTR 1.6

1.2

0.8

0.4

0

Figure 6 Scatter plot between the observed and the predicted values for testing datasets using (a) RF; (b) MARS; (c) MLP; and (d) DTR models

Geosynthetics International, 2022, 29, No 4

DTR and MLP models met only two conditions Meeting

a minimum of two criteria shows that the developed

models could predict the response value with reasonable

accuracy but amongst all modelling techniques, the

ensemble learning method – that is, the RF model, was

most accurate

6 COMPARISON OF MODEL

PREDICTIONS WITH LABORATORY

EXPERIMENTS

In order to verify the generalization ability of the

established pullout coefficient value prediction model,

laboratory pullout tests were conducted on geogrid

embedded in four different fill materials, the results of

which were compared with the predictions of ML models

developed in the previous section These experimental

data were not present in the model development phase

6.1 Material used

Bottom ash and fly ash were separately collected from

Dadri thermal power plant (TPP) and Jhajjar TPP The

materials were air dried, and a detailed geotechnical

characterization of the materials was then conducted

following standard procedures The geotechnical

proper-ties of the four materials have been summarized in Table 8

BA stands for bottom ash and FA stands for fly ash J_BA

refers to bottom ash from Jhajjar TPP while D_BA refers

to Dadri TPP Fly ash was finer than bottom ash and

contained a higher percentage of fines (particles less than

75μm) and lower D50

A uniaxial polyester (PET) geogrid was used as a

reinforcement in this study It was a polyvinyl coated PET

geogrid that consisted of knitted yarn fibers The

manufacturer provided ultimate tensile strength of the

geogrid was 80 kN/m in machine direction and 30 kN/m

in cross-machine direction The aperture size of the

geogrid was 27 × 29 mm

6.2 Laboratory test results The pullout resistance versus displacement results of the geogrid embedded in the two different well compacted bottom ash samples, J_BA and D_BA under 20 kPa,

40 kPa and 80 kPa normal stress have been presented in Figures 7a and 7b The geogrid exhibited strain softening behaviour, – that is, a gradual decrease of the pullout resistance after peak load, for both bottom ashes Similar observations were made for geogrid embedded in fly ash samples (Figures 8a and 8b The pullout resistance offered

by geogrid embedded in fly ash was almost 35–40% less than that offered in J_BA and D_BA This is due to a higher percentage of fines in fly ash than in bottom ash samples which leads to lower shear resistance mobilization

in fly ash Also, unlike bottom ash, fly ash samples exhibited stick-slip oscillations in its pullout resistance curves (Pant et al 2019d) at each normal stress

6.3 Determination of pullout coefficient

In order to interpret the pullout test, Moraci and Recalcati (2006) proposed Equation 14 to calculate the pullout coefficient (F ):

F ¼ PR

where PR, peak pullout resistance per unit of width (kN/m); L, Embedment length of the reinforcement (m);

σn, effective normal stress at the soil-reinforcement inter-face (kN/m2)

The value of F ranged from 1.04 to 0.41 in BA samples, and 0.7 to 0.28 in FA at the three normal stresses For a particular ash type, the value of F was observed to be higher at lower normal stress of 20 kPa which decreased with an increase in normal stress This behaviour in granular soil is attributed to suppression of the soil dilatancy (Fannin and Raju 1993; Teixeira et al 2007)

6.4 Comparison of predicted and actual pullout coefficient value

From pullout coefficient predictions based upon training and testing datasets, it has been shown that the RF model yields higher prediction performance than other models

in terms of all performance indicators The values of R2 and MAPD of the validation dataset have been summar-ized in Table 9 It can be observed that with an MAPD value of 7.7%, RF shows good predictive performance for the dataset with reasonable accuracy A similar obser-vation can be made for the value of R2where the slope of

Table 7 External validation for all the data-driven models according to the criteria suggested by Golbraikh and Tropsha (2002)

Table 8 Geotechnical characterization of materials used