Description of the MLP network models
The idea in using the MLP network in this chapter is to identify the assumed dependency of the daily peak-, valley-, and average load on earlier load and temperature data. The network is trained on the past data and is hoped to learn to approximate this unknown dependency.
The data over a period of one whole year is used for the network training. The peak-, valley- and average loads over the training period, from May 24, 1996 to May 23, 1997 are shown in figure 4.1. The temperature data on the same time period is also
used. The seasonal trend on the load can be easily seen. Also, the weekly load structure can be seen in the form of lower load values on weekends than on working days.
Figure 4.1: Peak-, valley- and average loads over the period May 24, 1996-May 23, 1997.
There are numerous ways to choose the architecture of the models. Here, a decision was made to use one network for all day types, and to include the type of the day as an input to the network. Another possibility would have been to use separate networks for each day type, but this was considered unnecessary on the basis of some preliminary testing. It was also decided that each network has one hidden layer between input- and output layers.
It was concluded that using networks with only one output node gives better results than forecasting peak-, valley- and average loads with one multi-output network. This means that separate networks are used in all cases.
Another features to be decided about the architecture of the network are the input variables and the number of hidden layer neurons. For the input variables, the following symbols are used:
= )
max(i
L maximum (peak) load of day i
= )
min(i
L minimum (valley) load of day i
0 50 100 1 5 0 2 0 0 2 5 0 300 3 5 0 400
10 20 30 40 50 60 70 80 90
D a y s
MW
= ) (i
Lave average load of day i
= )
max(i
T maximum temperature of day i
= )
min(i
T minimum temperature of day i
= ) (i
Tave average temperature of day i
To forecast the maximum load of a certain day, at least the maximum loads of the previous day and the corresponding day from the previous week, can be considered potential input variables. Also the temperature data of those days may be useful if temperature forecasts for the target day are available. Maximum, minimum and average temperatures are considered for this purpose.
Eight different input structures are tested separately for peak, valley and average loads. These are numbered from 1 to 8, and are listed in the following for maximum load forecasting. If the target is the minimum or average load, all maximum load values are replaced by corresponding minimum or average values respectively.
Output: Lmax(i) Input structures:
1. Lmax(i− 1),Lmax(i− 7),Lmax(i− 8),Tave(i),Tave(i− 1),Tave(i− 7),Tave(i− 8) 2. Lmax(i− 1),Lmax(i− 7),Tave(i),Tave(i− 1),Tave(i− 7)
3. Lmax(i− 1),Tmin(i),Tave(i),Tmax(i),Tmin(i− 1),Tave(i− 1),Tmax(i− 1) 4. Lmax(i− 1)
5. Lmax(i− 1),Tave(i)− Tave(i− 1) 6. Lmax(i− 1),Tmax(i)− Tmax(i− 1) 7. Lmax(i− 1),Tmin(i)− Tmin(i− 1)
8. Lmax(i− 1),Lmax(i− 7),Lmax(i− 8),Tmax(i),Tmax(i− 1),Tmax(i− 7),Tmax(i− 8) In addition to inputs listed above, each input structure contains four extra nodes.
These get binary values and inform the network of the day type of the target day. The day type classes are: 1) Mondays, 2) Tuesdays-Fridays, 3) Saturdays, and 4) Sundays.
Informing the network about the day type is important, because Saturdays and Sundays have much lower peak loads than working days.
A hyperbolic tangent function is used as the activation function by all feed-forward neural networks of this work. This function is a mapping into the interval [-1,1]. To enable the converging of the network training within a reasonable time, the desired output values should be scaled onto this interval (see, e.g., Haykin 1991). All temperature and load values are therefore scaled linearly between –1 and 1.
Predicting the shape of the load curve
Classifying the days
The load shape is predicted here by averaging some load curves of similar days in the load history. Therefore, days have to be grouped into classes of different day types.
Hsu and Yang (1991) used self-organizing feature maps to classify the days into groups, with each group comprising the days with similar load patterns. Their results are however not surprising and similar classifications can also be made without neural networks. They present a classification for the load data of Taiwan Power Company for the period of May 1986 classes being: 1) Sundays and holidays, 2) Mondays and days after holidays, 3) Saturdays, 4) weekdays except holidays.
It should be noticed that classification with a self-organizing feature map does not solve the obvious problem: the class of the target day has to be known in advance.
The feature map can not classify the day before the load values of the day are known.
Here, simple reasoning is used instead of more sophisticated methods. Two different classifications are used:
Classification 1: Classification 2:
a) weekdays a) Mondays
b) Saturdays b) Tuesdays
c) Sundays :
g) Sundays
The model is tested in this work only on normal days; special holidays are excluded from the test data.
The load shape based on peak- and valley loads
The shape of the load curve for a certain day contains 24 normalized load values.
When the load shape is combined with the forecast peak- and valley load values, the normalized load values are:
) ( )
(
) ( )
, ) ( , (
min max
min
i L i L
i L j i j L i Lnor
−
= − , (4.1)
where
) , (i j
Lnor = normalized load at hour j on day i )
, (i j
L = load at hour j on day i
= )
min(i
L valley load of day i
= )
max(i
L peak load of day i
When the load shape has been predicted, the hourly load forecast can be calculated:
) ( )
, ( ) ( )
( )
,
(i j Lmax i Lmin i L i j Lmin i
L nor
∧
∧
∧
∧
∧ +
−
= , (4.2)
where the hats indicate that the load values are forecasts.
The load shape based on average load
Wen the load shape is combined with the average load, the normalized load values are:
) ( ) , ( ) ,
(i j L i j L i
Lnor = − ave , (4.3)
where
= ) (i
Lave average load of day i The forecast for the hourly load is then:
).
( )
, ( )
,
(i j L i j L i
L nor ave
∧
∧
∧ = + (4.4)
Different averaging models
The forecast for the load shape is obtained by averaging some load shapes of the corresponding day type class in the load history. The number of these days has to be decided. Four different models are considered for day type classification 1, and three different models are considered for classification 2. These are:
Classification 1:
Model 1: For all day types two most recent example days are averaged.
Model 2: For all day types three most recent example days are averaged.
Model 3: For weekdays five most recent example days, for Saturdays and Sundays two most recent examples are averaged.
Model 4: For weekdays five most recent example days, for Saturdays and Sundays three most recent examples are averaged.
Classification 2:
Model 1: For all day types two most recent example days are averaged.
Model 2: For all day types three most recent example days are averaged.
Model 3: For all day types five most recent example days are averaged.
Test results
Error measure
In comparing different models, the average percentage forecasting error is used as a measure of the performance. This is defined:
% 1 100
1
×
−
= ∑
=
∧ N
i i
i i
ave L
L L
E N , (4.5)
where
=
N the number of cases to be forecast (number of days in the cases of peak-, valley- and average load forecasting, and number of hours in the case of hourly load forecasting).
i =
L the i:th load value
∧ =
Li the i:th load forecast
The reason for using the average percentage error is the fact, that its meaning can be easily understood. It is also the most often used error measure in the load forecasting literature used as reference of this work, and therefore allows for some comparative considerations (although the results are not directly comparable in different situations).
Another possibility as a performance measure would be the root-mean-square (RMS) percent error. This penalizes the square of the error as opposed to the average forecasting error. Therefore, the RMS takes the deviation of the errors into account more effectively. However, when both measures were calculated on some test models with relatively small errors, the orders of preference were in practice the same with both measures. Therefore, the average forecasting error will be used throughout this work.
Peak, valley, and average load forecasts
The results of forecasting the peak-, valley-, and average loads with MLP networks are given in this section. The eight different input structures (see page 39) were tested in all cases.
The training set for the networks consists of the data over one year, from May 24, 1996 to May 23, 1997. The idea is that all load and weather conditions are representative in the training in order to enable the model to adapt to all conditions.
The test data consists of the remaining data, from May 24, to August 18, 1997. Peak-, valley and average loads over the test period are shown in figure 4.2. In the beginning of the period, the weekly rhythm is clearly visible. At the end of the period, it becomes less distinct, especially for the peak- and valley loads. The reason is the summer holidays, which naturally weaken the weekly rhythm and also make the
Figure 4.2: Peak-, valley- and average loads over the period May 24, 1997- August 18, 1997
A more thorough testing would require the use of test data at all seasons of the year, but as the testing must not be carried out with the same data as used in the network training, all testing is limited to the only season where data in two successive years is available. If the testing were carried out on another season, and the data of the test season were simply excluded from the training set, then the data of the prevailing conditions would be under-representative in the training set.
The test season is in the summertime, when the holidays as an external factor have an influence on the load. This makes the forecasting difficult. Also the temperature has a much fainter effect in the summer than in other seasons.
The average percentage forecasting errors for the test data are shown in figures 4.3 (peak load), 4.4 (valley load), and 4.5 (average load). As the neural network forecasters can give slightly different results with different training sessions, the training and forecasting was performed three times with each network. The error values are averages from these three test runs.
The tests were carried out with 3, 5, 7 and 10 hidden layer neurons. These numbers were chosen on the basis of rough calculations for the reasonable size of the hidden layer, as discussed in chapter 3 (equation 3.3).
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0
5 1 0 1 5 2 0 2 5 3 0 3 5 4 0 4 5 5 0 5 5
D a y s
MW
Figure 4.3: The average percentage errors for peak load forecasts on the test period May 24 – August 18, 1997.
Figure 4.4: The average percentage errors for valley load forecasts on the test period May 24 – August 18, 1997.
Figure 4.5: The average percentage errors for average load forecasts on the test period May 24 – August 18, 1997.
0 1 2 3 4 5 6 7 8 9 1 0
1 2 3 4 5 6 7 8
I n p u t s t r u c t u r e s
Average percentage error
3 n e u r o n s 5 n e u r o n s 7 n e u r o n s 1 0 n e u r o n s 0
1 2 3 4 5 6 7 8 9 1 0
1 2 3 4 5 6 7 8
I n p u t s t r u c t u r e s
Average percentage error
3 n e u r o n s 5 n e u r o n s 7 n e u r o n s 1 0 n e u r o n s 0
1 2 3 4 5 6 7 8 9 1 0
1 2 3 4 5 6 7 8
I n p u t s t r u c t u r e s
Average percentage error
3 n e u r o n s 5 n e u r o n s 7 n e u r o n s 1 0 n e u r o n s
The input structure 4 gives the best results in all the cases. This contains only the peak load of the previous day as input. No temperature data is utilized. It can be concluded that in this test case the neural networks fail to make use of any other data than the peak, valley, or average load of the previous day.
The real values and forecasts with input structure 4 are shown in figures 4.6 (peak load), 4.7 (valley load) and 4.8 (average load).
Figure 4.6: Actual and forecast peak loads with 7 hidden layer neurons and input structure 4
Figure 4.7: Actual and forecast valley loads with 3 hidden layer neurons and input structure 4
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0
3 0 3 5 4 0 4 5 5 0 5 5
D a y s
MW
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0
1 4 1 6 1 8 2 0 2 2 2 4 2 6 2 8 3 0 3 2 3 4
D a y s
MW
Figure 4.8: Actual and forecast average loads with 7 hidden layer neurons and input structure 4.
The forecasts for the average load are more accurate than those for the peak and valley loads. The errors are on average less than 3.5% for the input structure 4.
On the other hand, for the valley load, the average error is only slightly less than 5%.
There are a few occasions, where the average forecasting error is unacceptably large (figure 4.4). This is a phenomenon, which has been observed from time to time during this work when forecasting with feed-forward networks: sometimes the network totally fails in the training phase. The forecasts can then be almost anything.
It is interesting to notice that the errors grow larger as the hidden layer size gets larger, except in the case of input structure four, which has only 1+4 input variables.
This suggests over-parameterization.
The results don't suggest the use of the data from the previous week. If, however, the peak load forecasts are made further than one day ahead, the situation may be different. If the only input to the network were the data from the day just before the target day, then in forecasting two days ahead, the input would consist only of forecast data. If, on the other hand, the input also contains data from the previous week, the accuracy could be better.
The data in July is quite much affected by the summer holidays, and the load seems to be difficult to forecast. If the tests are run only on the five first weeks of the data, the results are much better. In figure 4.9, the average errors for peak, valley and average
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0
1 5 2 0 2 5 3 0 3 5 4 0 4 5
D a y s
MW
load forecasts are given with the input structures 4, 5 and 6, using five hidden layer neurons.
Figure 4.9: The average percentage forecasting errors for peak-, valley-, and average load forecasts on the test period May 24 – June 28, 1997.
Load shape forecasts
To test the performance of the load shape forecasting model, the prediction was carried out with the unrealistic assumption that the peak-, valley- and average load values are known in advance.
To get a good opinion on the performances of different models, testing was carried out on four test periods, all at different seasons of the year. The average hourly forecasting errors are given in figures 4.10 – 4.13.
Figures 4.10 and 4.11 show the average errors for the model based on daily peak and valley loads. Figure 4.10 is for day type classification 1, and figure 4.11 for day type classification 2. The corresponding with daily average loads are shown in figures 4.12 and 4.13. Each bar represents the average error with a certain averaging model (see page 42).
0 1 2 3 4 5 6
P e a k lo a d V a lle y l o a d A v e r a g e l o a d
Average percentage error
I n p u t s t r u c t u r e 4 I n p u t s t r u c t u r e 5 I n p u t s t r u c t u r e 6
Figure 4.10: The load shape forecasting with peak- and valley loads: the average errors for day type classification 1.
Figure 4.11: The load shape forecasting using peak- and valley loads: the hourly average errors for day type classification 2.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
S e p t . 3 0 - N o v . 3 , 9 6 J a n . 1 6 - F e b . 1 3 , 9 7 A p r . 2 - A p r . 2 9 , 9 7 Jul. 2 - Jul. 31, 97
Average percentage error
M o d e l 1 M o d e l 2 M o d e l 3 M o d e l 4
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
S e p t . 3 0 - N o v . 3 , 9 6 J a n . 1 6 - F e b . 1 3 , 9 7 A p r . 2 - A p r . 2 9 , 9 7 Jul. 2 - Jul. 31, 97
Average percentage error
M o d e l 1 M o d e l 2 M o d e l 3
Figure 4.12: The load shape forecasting using the daily average load: the hourly average errors for day type classification 1.
Figure 4.13: The load shape forecasting using daily average load: the hourly average errors for day type classification 2.
The classification 1 appears superior to classification 2 in both cases. The averaging models 3 and 4 seem to give the best results. The average error is less than 3 % for all test periods, except for the last one.
As an illustration, the forecast and the real load over the one-week long period between April 8 – April 14, 1997 are shown in figures 4.14 (based on the peak- and
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
S e p t . 3 0 - N o v . 3 , 9 6 J a n . 1 6 - F e b . 1 3 , 9 7 A p r . 2 - A p r . 2 9 , 9 7 Jul. 2 - Jul. 31, 97
Average percentage error
M o d e l 1 M o d e l 2 M o d e l 3 M o d e l 4
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
S e p t . 3 0 - N o v . 3 , 9 6 J a n . 1 6 - F e b . 1 3 , 9 7 A p r . 2 - A p r . 2 9 , 9 7 Jul. 2 - Jul. 31, 97
Average percentage error
M o d e l 1 M o d e l 2 M o d e l 3
valley loads) and 4.15 (based on the average load). The classification 1 and averaging model 3 are used in both cases.
Figure 4.14: The load shape forecast with peak- and valley loads on the period April 8 – April 14, 1997. Classification 1 and averaging model 3 are used. The average error is 3.17 %.
Figure 4.15: The load shape forecast with average load on the period April 8 – April 14, 1997. Classification 1 and averaging model 3 are used. The average error is 3.27 %.
0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0
4 0 4 5 5 0 5 5 6 0 6 5 7 0
H o u r s
MW
0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0
3 5 4 0 4 5 5 0 5 5 6 0 6 5 7 0
MW
H o u r s
Combining peak, valley, and average load forecasts with load shape predictions
Following decisions were made on the models forecasting peak-, valley-, and average loads.
- input structure 4 is used - hidden layer has 7 neurons
Correspondingly, following decisions were made on the load shape prediction model:
- days are classified into three classes (Mondays-Fridays, Saturdays, Sundays) - in the class Mondays-Fridays, 5 example days are averaged
- in the classes for Saturdays and Sundays, 2 example days are averaged
Forecasting was performed twice on the test period May 24 – August 18, 1997: first peak- and valley forecasting, then average forecasting, was combined with load shape prediction. Average forecasting error was 5.03 % for the model using peak- and valley loads, and 4.48 % for the model using average load.
In figure 4.16, the actual load and forecast based on peak- and valley loads is shown for the two weeks long time period May 24 – June 6. The forecasting error for this time period is 4.76 %. The corresponding for the model using average load is given in figure 4.17. The forecasting error is in this case 4.35 %.
Figure 4.16: Actual load and forecast obtained by combining load shape prediction and peak- and valley load forecasts on the period May 24 – June 6, 1997. Average forecasting error is 4.76 %.
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0
2 5 3 0 3 5 4 0 4 5 5 0 5 5
H o u r s
MW
Figure 4.17: Actual load and forecast obtained by combining load shape prediction and average load forecasts on the period May 24 – June 6, 1997.
Average forecasting error is 4.35 %.
In both cases, the forecasting goes fine for the first week, the average error being around 3 %. However, in the second week, the level of the load fluctuates much more and large errors in forecasting the peak-, valley- and average loads cause notable systematic errors thus spoiling the hourly forecasts.
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0
2 5 3 0 3 5 4 0 4 5 5 0 5 5
MW
H o u r s