In multi-objective optimization problems, good quality indicators are very important to the performance evaluation of algorithms. There are a number of performance indexes that can be utilized for measuring the performances of the different algorithms, Helbig and Engelbrecht(2013),Zitzler et al.(2003). In general, there are three aspects that need to be considered when evaluating the performance of algorithms: accuracy aspect, which relates to the convergence of the approximation set or how close the approximation set is to the Pareto optimal front; diversity aspect, which defines the relative distance between solutions in the approximation set; and cardinality aspect, relating to the number of solutions achieved in the approximation set. In this work, the following indexes are used to assess the performance of the algorithms, that consider all the above stated aspects.
5.4.1 Hypervolume
The result of a survey in Riquelme et al. (2015) shows that Hypervolume (HV) is the most used metric. HV considers all three aspects: accuracy, diversity, and cardinality.
HV calculates the largeness of the objective space enclosed by an approximation set and a specific reference point. HV can evaluate an approximation set obtained by an algorithm without knowledge of Pareto optimal front, as a result, it is suitable for optimization problems whose Pareto front is unknown. The HV calculation includes a reference point R(R1, R2, ..., RM), that is dominated by all solutions in an approximation set. Therefore, the accuracy of HV depends only on the choice of the reference point. HV is calculated using the following formula:
HV(A) =Leb( ∪
x∈A[f1(x), R1]×...×[fM(x), RM]) (5.3) whereM and Aare the number of objectives and the approximation set obtained by an algorithm, respectively. Leb(S) is the Lebesgue measure of a set S.
The algorithms are compared using statistical measurements of the obtained HV. Each algorithm is run 20 independent times and mean of HV on these 20 runs are calculated and this value is used as a main metric to compare the performance of the algorithms.
The number of solution evaluations using SUMO in each generation of NS-LS and SA-LS
are different. The execution of the algorithms is terminated when the maximum number of evaluation using SUMO is reached. Therefore, the number of generations in different runs are different. The output of k(th) run includes I pairs (nik, HVki) where I is the number of generations, nik is the number of evaluations using SUMO and HVki is the corresponding HV in i(th) generation. The average number of evaluations is determined by:
¯ n= 1
K
K
X
k=1
nik (5.4)
whereKis the total runs. Average HV on 20 runs corresponding to the average number of evaluations nave is calculated using the the following formula:
HV¯ = 1 K ×(
K
X
k=1
nik)× 1 K ×(
K
X
k=1
HVki
nik ) (5.5)
Furthermore, standard deviation is also used to measure how the the obtained HV spread out from the mean using the following equation:
SHV = s
PK
k=1(HVk−HV¯ )2 K−1
where SHV is the standard deviation, HVk and HV¯ are hypervolume of k(th) run and mean of HV on K runs, respectively. Best, worst, and median are also computed to provide further comparison of the algorithms. Best and worst are the maximum and minimum hypervolume values obtained in 20 runs, respectively. Arrange all HV obtained in 20 runs in ascending order and median is the average of the two middlemost numbers in this ordered set.
5.4.2 C-metric
The set coverage (C-metric): Suppose that A and B are two approximation sets of the optimization problem. The C-metric is defined as follows:
C(A, B) = |u∈B| 3v∈A:vu|
|B| (5.6)
The C-metric is used to compare the convergence of two approximation sets. C(A, B) refers to the percentage of the solutions in B which are dominated by at least one solution in A.C(A, B)> C(B, A) suggests that setA has better convergence than setB.
5.4.3 Diversity Indicators
Furthermore, to evaluate the diversity performance of the algorithms, two diversity indicators are used. The first diversity performance measure is the spacing metric of Schott (S), which measures how evenly the points of the approximated Pareto front are distributed in the objective space. Spacing is calculated as:
S= v u u t
1 N−1
N
X
m=1
(davg−dm)2 (5.7)
withdm=minj=1,...,NPM
k=1|fkm(x)−fkj(x)|whereN is the number of the solutions in the found Pareto front and M is the number of objective functions. davg is the average value of all dm values. The smallerS is, the more evenly distributed the solutions.
However,Sdoes not provide any information with regards to the spread of the solutions.
Therefore, maximum spread measurement is utilized as the second diversity indicator.
M S= v u u t
N
X
k=1
(maxk−mink)2 (5.8)
wheremaxk and mink are maximum and minimum values of the kth objective, respec- tively. M S measures the length of the diagonal of the hyperbox that is created by the extreme values of the non-dominated set. The bigger M S is, the more widely spread the solutions.
Table 5.1: Experimental parameters settings for NS-LS, SA-LS, and NSGA-II in Experiment 1.
No. Symbol Description Value
1 N Population size 120
2 maxEval Number of evaluations using the real fitness function 24000
3 pc Crossover probability 0.9
4 pm Mutation probability 0.1
5 Pmv Mutation probability of a variable in an individual 1.0/n
6 n Number of variables in an individual 30