Bài giảng Khai phá dữ liệu (Data mining) Genetic algorithm

Trịnh Tấn Đạt Khoa CNTT – Đại Học Sài Gòn Email: trinhtandat@sgu.edu.vn Website: https://sites.google.com/site/ttdat88/ Contents  Introduction: Genetic Algorithm (GA)  GA Operators and Parameters  Example  Homework Introduction History Of Genetic Algorithms  “Evolutionary Computing” was introduced in the 1960s by I Rechenberg  John Holland wrote the first book on Genetic Algorithms ‘Adaptation in Natural and Artificial Systems’ in 1975  In 1992 John Koza used genetic algorithm to evolve programs to perform certain tasks He called his method “Genetic Programming” What Are Genetic Algorithms (GAs)?  GAs are search and optimization techniques based on Darwin’s Principle of Natural Selection and Genetic Inheritance  A class of probabilistic optimization algorithms  Widely-used in business, science and engineering Basic Idea Of Principle Of Natural Selection “Select The Best, Discard The Rest” An Example of Natural Selection:  Rabbits are fast and smart  Some of them are faster and smarter than other rabbits Thus, they are less likely to be eaten by foxes   They have a better chance of survival and start breeding The resulting baby rabbits (on average) will be faster and smarter  Now, evolved species are more faster and smarter Genetic Algorithms Implement Optimization Strategies By Simulating Evolution Of Species Through Natural Selection Classes of Search Techniques Search Techniques Calculus Base Techniques Fibonacci Enumerative Techniques Guided random search techniques Sort Tabu Search DFS Hill Climbing Simulated Anealing Genetic Programming Dynamic Programming Evolutionary Algorithms Genetic Algorithms BFS Nature to Computer Mapping  GAs use a vocabulary borrowed from nature genetics Nature Computer (GAs) Population Set of solutions Individuals in environment Individual’s degree of adaptation to its surrounding environment Chromosome Solutions to a problem Gene Part of the encoding of a solution Selection, crossover and mutation in nature’s evolutionary process Stochastic operators Solutions quality (fitness function) Encoding for a Solution Working Mechanism Of GAs Simple Genetic Algorithm Simple_Genetic_Algorithm() { Initialize the Population; Calculate Fitness Function; While(Fitness Value != Optimal Value) { Selection;//Natural Selection, Survival Of Fittest Crossover;//Reproduction, Propagate favorable characteristics Mutation;//Mutation Calculate Fitness Function; } } Designing GAs ⚫ ⚫ ⚫ ⚫ ⚫ ⚫ How to represent chromosomes? How to create an initial population? How to define fitness function? How to define genetic operators? How to generate next generation? How to define stopping criteria? GA Examples Swap Mutation The following mutation operator is adapted to the path representation: GA Examples Why we cannot use single-point crossover Fail! GA Examples PMX-Crossover Also Partially Matched Crossover (PMX) avoids building illegal chromosomes: How Do GAs Work?  Maximize a function of k variable, f(x1,…,xk), f(x1,…,xk)>0 for all xi  Representation (binary string) We divide [ai,bi] into (bi-ai)*104 equal size ranges For each xi → binary string of length mi satisfies where xi [ai,bi], and (bi-ai)*104  2mi - To represent real value of a binary string Thus, each chromosome is represent by a binary string of length v=(string1 string2…stringk) GA Examples  Selection (Roulette wheel selection) GA Examples  Crossover(pc is probability of crossover)  Mutation (pm is probability of mutation)  For example, maximize the function [-3, 12.1] → 15.1*104 equal size ranges x1 → binary string of length 18 [4.1, 5.8] → 1.7 *104 equal size ranges x2 → binary string of length 15 GA Examples  The total length of a chromosome is m=18+15=33 bits GA Examples  Assume population of size (pop_size) equals 20 All 33 bits in all chromosome are initialized randomly New population Choose q11 and q4 ,etc  Crossover with pc=2.5% If r < 0.25, we select a given chromosome of crossover The chromosome v2’, v11’, v13’ and v18’ were selected for crossover  Mutation with pm=1% We have 33*20=660 bits, we expect (on average) 6.6 mutation per generation Generate 660 random numbers r , if r < 0.01, we mutate the bit After applying crossover and mutation, we have a new population New population Old population Conclusion  Genetic Algorithms (GAs) implement optimization strategies based on simulation of the natural law of evolution of a species by natural selection  The basic GA Operators are: Encoding Selection Crossover Mutation  GAs have been applied to a variety of function optimization problems  GA s have been shown to be highly effective in searching a large, poorly defined search space even in the presence of difficulties such as high-dimensionality, discontinuity and noise Homework 1) We're going to optimize a very simple problem: trying to create a list of N numbers that equal X when summed together  EX1: N = ; X = ; one solution is [2, 0, ,4, 2]  EX2: N = and X = 200, then these would all be appropriate solutions lst = [40,40,40,40,40] lst = [50,50,50,25,25] lst = [200,0,0,0,0] Ref : https://lethain.com/genetic-algorithms-cool-name-damn-simple/ Homework 2) Uses a genetic algorithm to maximize a function of many variables Ex : Consider the function: z = f(x,y) -x^2+2x-y^2+4y Find (x*,y*) to z is maximum Ref: https://github.com/philipkiely/floydhub_genetic_algorithm_tutorial/blob/mast er/geneticmax.py Ex 2: The equation is shown below: Y = w1x1 + w2x2 + w3x3 + w4x4 + w5x5 + w6x6 Given inputs values are (x1,x2,x3,x4,x5,x6)=(4,-2,7,5,11,1) Find the parameters (weights) that maximize such equation Ref: https://towardsdatascience.com/genetic-algorithm-implementation-inpython-5ab67bb124a6 Homework 3) Evolution of a salesman Ref : https://towardsdatascience.com/evolution-of-a-salesman-a-completegenetic-algorithm-tutorial-for-python-6fe5d2b3ca35