msc thesis

University of Nottingham Department of Mineral Resources Engineering MSc in Mineral Resources Engineering and Management ASSESSMENT OF NEURAL NETWORK PREDICTION TECHNIQUES FOR GRADE ESTIMATION Ioannis K. Kapageridis Thesis submitted to the University of Nottingham for the Degree of Master of Science September 1996 Acknowledgements 2 This thesis is submitted for the degree of MSc in Mineral Resources Engineering and Management. I would like to thank the supervisor of this project, Dr. Bryan Denby for his help, guidance and trust. I also thank him for his interest in my further research in the field of Neural Networks and Reserve Estimation. I must thank also Professor D. Potts for his consideration and help throughout the year of my studies in the University of Nottingham. Finally, I must thank Professor A. Triantafyllou of the Technological Education Institute of Kozani, Greece for allowing me to use the computer facilities in his laboratory during research and the writing of this thesis. Ioannis K. Kapageridis Nottingham 1996. Contents Assessment of Neural Network Prediction Techniques for Grade Estimation 3 Contents Contents 3 1. Introduction 5 1.1 The Problem of Grade Estimation 6 1.2 Conventional and Geostatistical Approach 7 1.2.1 The Variogram 8 1.2.2 Kriging 9 1.3 Neural Network Approach 13 2. Background to Artificial Neural Networks 14 2.1 General 14 2.2 Neural Network Learning 15 2.2.1 Supervised Learning Techniques 16 2.2.2 Steepest Descent 17 2.2.3 Conjugate Gradient 18 2.2.4 Simulated Annealing 19 2.2.5 Normalisation 19 2.3 Neural Network Architectures 20 2.3.1 General 20 2.3.2 Multi-Layer Perceptron (MLP) 20 2.3.3 Radial Basis Function Network (RBFN) 22 2.4 Statistical Mechanics and Neural Networks 25 2.5 Conclusions 28 3. Overview of the Application of Neural Networks for Grade Estimation 30 3.1 General 30 3.2 Neural Network Methods Applied to Grade Estimation 31 3.2.1 Single Network Approach 31 3.2.2 Modular Network Approach 34 3.3 Conclusions 35 4. Grade Estimation Using a Neural Network 37 Approach 37 4.1 Neural Network Development Environment 37 4.1.1 Spreadsheet Input Tool 39 4.1.2 Multi-Layer Perceptron Tool 41 4.1.3 Radial Basis Function Tool 42 4.1.4 Text Output Tool 43 4.2 Development of Example Deposits 44 4.2.1 General 44 4.2.2 Function Test One (FT1) 45 4.2.3 Copper A Deposit (CPA) 46 4.2.4 Copper B Deposit (CPB) 48 4.2.5 Chromium Deposit (CR) 50 4.2.6 Sampling Scheme and I/O Configuration 52 4.3 Training, Validation, and Testing 56 4.3.1 General 56 4.3.2 Training and Validation 59 4.3.3 Testing 61 4.4 Assessing the Results 61 4.4.1 Function Test One Results 63 4.4.2 Copper A Results 66 4.4.3 Copper B Results 69 4.4.4 Chromium Results 74 4.5 Conclusions 76 5. Summary 78 Contents Assessment of Neural Network Prediction Techniques for Grade Estimation 4 Appendix I - Deposits & Training Data Sets 79 Function Test One Example Deposit 79 Copper A Example Deposit 81 Copper B Example Deposit 83 Chromium Example Deposit 86 Appendix II - Neural Connection Reports 89 Topology Reports 89 Neural Connection Text Output Reports 101 Appendix III - List of Abbreviations 110 Appendix IV - Notations 111 References 113 Introduction Chapter 1 Assessment of Neural Network Prediction Techniques for Grade Estimation 5 1. Introduction The problem of grade estimation has been the subject of extended research carried out by people from different fields. As a result of this, a number of different methods were developed. These methods are based on geometrical, statistical and geostatistical techniques and theory. The driving force behind all this development was, and still is, the fact that most of the world’s rich and easily accessible deposits are close to complete exploitation while new deposits become difficult to find and even more difficult to implement as an economically viable exploitation programme. The ever increasing world-wide industrial activity creates extra pressure on the metals and minerals industry in the form of growing demand for metals and minerals, while the fluctuating prices of the latter alter rapidly the limit between what is profitable to exploit and what is not. The uncertainty created by all these facts leaves little space for assumptions during grade estimation. As all the methods developed and applied so far are based on large assumptions about the distribution of ore grades within a deposit, there is still a need for new methods which will require less assumptions and thus be closer to reality. Recently, a new prediction technique, coming from the field of Artificial Intelligence, has been employed for the problem of grade estimation. This is the application of Artificial Neural Networks (ANNs). It is the aim of this project to assess their performance in grade estimation. However, a brief overview of the already existing techniques is given in the next sections of this chapter together with a more detailed definition of the grade estimation problem and an introduction to the neural network approach to its solution. In Chapter 2, some background information is given on Neural Network theory and application. This is, by no means, a thorough and complete analysis of Neural Networks. Only the basic mechanics of their operation are given and the discussion is concentrated on the features and techniques to be used later on when NNs will be applied to the given problem. In Chapter 3, the pass from theory to application of Neural Networks is made. A first effort is made to assess their applicability for grade estimation from their own special features and from information given by other researchers in the area. Introduction Chapter 1 Assessment of Neural Network Prediction Techniques for Grade Estimation 6 Chapter 4 is the main part of this project where the actual assessment of ANNs prediction techniques takes place through training, validation and testing of NN systems using data from simulated and real deposits. For the purposes of this project the data were limited to two dimensions. Finally, in Chapter 5, a summary of the results and conclusions of the study described above is given. 1.1 The Problem of Grade Estimation In the geostatistical language, the problem of grade estimation would be that of finding the spatial distribution of grade in the three-dimensional space. According to G. Matheron: ‘The distribution of ore grades within a deposit is of mixed character, being partly structured and partly random. On one hand, the mineralising process has an overall structure and follows certain laws, either geological or metallogenic: in particular, zones of rich and poor grades always exist, and this is possible only if the variability of grades possesses a certain degree of continuity. However, even though mineralization is never so chaotic as to preclude all forms of forecasting, it is never regular enough to allow the use of a deterministic forecasting technique.’ (foreword from Mining Geostatistics, A.G. Journel and Ch.J. Huijbregts, 1978.) It is quite clear that grade estimation is a difficult problem due to the irregularities in mineralisation processes. The problem gets worse when one considers the fact that exploration is very expensive leading to very small quantities of information available on the grade distribution. This information, normally, contains a large number of errors and inconsistencies making the estimation of grade an even more difficult task. Usually the process of grade estimation involves the collection and assaying of samples from certain locations in a deposit and the interpolation, and sometimes, extrapolation to unknown points within the area of the deposit. Good sampling is essential to the whole process as it will have a great effect on the quality and accuracy of the results. Also the larger the number of samples the better results from the estimation. Introduction Chapter 1 Assessment of Neural Network Prediction Techniques for Grade Estimation 7 This is true only when the sampling points are chosen to be representative of the smaller areas of the deposit where they are taken from. Interpolating and extrapolating between known points is done, so far, using one of the methods given in the following section. 1.2 Conventional and Geostatistical Approach There are three main categories of estimation methods: geometrical, statistical, and geostatistical. In the geometrical methods are included the triangular, polygonal and the method of sections. These are illustrated in Figures 1.1 to 1.2. They are very simple to calculate, very quick but also very sensitive to errors. Their use is mainly in the first stages of estimation or when the deposit has a low spatial variability. The statistical methods involve fitting of surfaces to the extrapolation data, e.g. trend surface analysis and spline interpolation. There are also very simple methods such as the arithmetic mean method. Figure 1.3 illustrates some examples of statistical methods. Figure 1.1: Triangular method (or method of triangles) of estimation (Burnett, 1995). Geostatistics evolved in the 60s from the work of G.Matheron and other researchers. It includes basic statistical analysis and more specific tools like the Introduction Chapter 1 Assessment of Neural Network Prediction Techniques for Grade Estimation 8 variogram. The main interpolating method based on geostatistics is kriging which is one of the most common applications of the theory of regionalised variables. There are many books on the area such as from David (David, M. 1977), Journel and Huijbregts (1978) and Krige (1978). Before going in some detail in kriging it is necessary to discuss the theory and application of the variogram. 1.2.1 The Variogram The basic tool of every geostatistical analysis is the variogram or the semi-variogram. It is a way of measuring the spatial continuity of data, e.g. the grade. In other words, it is a graph of the average variability between samples vs. the distance between Figure 1.2: Polygonal method (or method of polygons) of estimation. them. A variogram is computed by averaging the squared differences between pairs of samples that are a given distance apart, as follows: 2 1 2 γ () ( )h n gg iih i N =− + ∑ (1.1) Introduction Chapter 1 Assessment of Neural Network Prediction Techniques for Grade Estimation 9 where N is the number of pairs at distance h, and h is the distance between the samples. The semi-variogram function γ (h) is computed for a number of different sample distances, to provide an experimental variogram that typically looks like the graph in Figure 1.4. The most important features of the variogram are the nugget effect, range, and sill. The nugget effect value is identified as the y-intercept of the variogram curve and represents random and short-distance variability factors such as sampling error, assaying error, and erratic mineralization. Most variograms increase in value from the nugget for some distance and then level off to a constant value. This distance is called the range of the variogram, and the variogram value is called the sill. The sill value is usually equal to the sample variance. A model, or equation, is fitted to the experimental variogram for further geostatistical evaluations such as kriging. The basic variogram models and their equations are illustrated in Figure 1.5 to 1.8. 1.2.2 Kriging Kriging and its derivatives are based on the original weighted averaging techniques developed by D.G. Krige. Kriging is a local estimation technique which provides the best linear unbiased estimator (abbreviated to BLUE) of the unknown characteristic studied (Journel and Huijbregts, 1978). Figure 1.3: Method of sections for reserve estimation (David, 1977). Introduction Chapter 1 Assessment of Neural Network Prediction Techniques for Grade Estimation 10 Figure 1.4: Typical graph of experimental variograms (David, 1988). Figure 1.5: Power and linear (1.0) model. Figure 1.6: Spherical model of variogram. Important factors in kriging calculations are: 1. The mean of the estimators should not be systematically smaller or bigger than the real value. This is set mathematically as follows: w i = ∑ 0 (1.2) 2. The error in calculating σ K 2 , which is expressed as variance (G-G*), or [...]... 1989), Carpenter and Grossberg (1986), Ripley (Ripley, 1996), Arbib (Arbib, 1995) and others Also for an overview of NN applications to mining related problems the reader is directed to C.C.H.Burnett PhD thesis (Burnett, 1995) 2.3.2 Multi-Layer Perceptron (MLP) Multi-layer perceptron is another name for a feed-forward neural network It is the most common type of network using the supervised learning techniques . Thesis submitted to the University of Nottingham for the Degree of Master of Science September 1996 Acknowledgements 2 This thesis is submitted for the degree of MSc in Mineral. University of Nottingham Department of Mineral Resources Engineering MSc in Mineral Resources Engineering and Management ASSESSMENT OF NEURAL NETWORK PREDICTION. allowing me to use the computer facilities in his laboratory during research and the writing of this thesis. Ioannis K. Kapageridis Nottingham 1996. Contents Assessment of Neural Network Prediction