INCREASING THE SERVICE CAPABILITY FOR MULTIUSERS FOR VNU WEB PROXY WITH HAPROXY

ACKNOWLEDGEMENT First of all, I would like to express my sincere gratitude to my supervisor Ass.Prof. Ho Si Dam, and M.S Doan Minh Phuong, University of Engineering and Technology, Viet Nam National University (VNU), Hanoi for his enthusiastic guidance, warm encouragement and helpful research experiences. I would like to also thank the Center Application Internet Technology – CAIT for all their careful guidance and preparing facility to my application. I am grateful to thank all the teachers in University of Engineering and Technology, VNU who provide invaluable knowledge and life skills for me during the four academic years. I would like to also thank my friends in K53CA class whom help me during the four academic years. Last, but not least, my family is really the biggest motivation for me. My parents always encourage me when I have stress and difficulty. I would like to send them great love and gratefulness. Ha Noi, May …, 2012 Le Tri Thai ABSTRACT Our thesis has been completed objective and satisfied the requirement of subjective: Increasing the service capability for multiusers for VNU web proxy with Haproxy. We have successful integrated haproxy server load balancer into present system. We also modified old server to compatible with new scenario successfully. Lastly, all system was tuned up with higher capability and higher availability. Our work is to solve the problem of management servers. haproxy is stable, and does not any problem with all old system. The maintenance, upgrade backend servers, squid servers is easier than ever. Nowadays, the more servers is installed, the more works of administrators. But haproxy can save their time and working enforce. Squid proxy is stronger than before. The error request is reduce by 40%. The session rate is increate more than three times without deceasing of quality. The advantage is outweighing the disadvantage of difficult in installing and boosting system. Keyword: haproxy, squid proxy, capability List of Figures Figure 1.1. The University Campus Network. 5 Figure 2.1. Testing model of haproxy in support website. 8 Figure 2.2. Deployment model of haproxy load balancer. 10 Figure 4.1. System status generated by top command. 24 Figure 4.2. Filedescriptors in haproxy, server, and maxfile. 25 Figure 4.3. Haproxy server webpage statistics. 26 Figure 4.4. Filedescriptor shortage in squid proxy shown in cache.log. 29 List of Tables Table 3.2 Basic haproxy configuration 14 Table 3.3 Detail haproxy configuration 15 Table 3.4 Configuration of rsyslog for haproxy log 17 Table 3.5 Disable selinux in etcselinuxconfig 18 Table 3.6 Disable iptables service 18 Table 3.7 Restart haproxy. 18 Table 3.8 FD configuration. 20 Table 3.9. Statistic network by crontab. 20 Table 3.10. Calamaris analysis squid access log. 21 Table 4.1. Statistic of resource log in crond service. 23 Table 4.2. Incoming TCP request by status untuned 27 Table 4.4. Distribution histogram of request untuned 28 Table 4.5. Incoming TCPrequest by status tuned 29 Table 4.6. Requestdestination by 2ndleveldomain tuned. 30 Table 4.7. Distribution histogram of request tuned. 32 Table 4.8. Frequency load time website. 34 List of Acronyms CentOS : The Community ENTerprise Operating System VNU: Vietnam National University VNUnet: Vietnam National University network Table of Contents ..SemiFinalThai_thesis_v0.4.1.docx _Toc325917965 ABSTRACT iv List of Tables v List of Acronyms vi Chapter 1 1 INTRODUCTION 1 1.1 Motivation 1 1.2 Contributions and thesis overview 2 Chapter 2 3 INCREASING THE SERVICE CAPABILITY FOR MULTIUSERS FOR VNU WEB PROXY WITH HAPROXY 3 2.1 Squid web cache proxy 3 2.2 Server load balance 7 2.2.1 haproxy server load balancer 7 2.2.2 Haproxy specification 8 2.3 Operating system CentOS 10 2.3.1 File descriptors and maxfile 11 2.3.2 Sysctl.conf 11 2.3.3 SElinux 11 2.3.4 Service rsyslog 11 2.3.5 Service iptables 12 Chapter 3 13 DEPLOYMENT 13 3.1 Haproxy server configuration 13 3.2 Squid proxy server configuration 19 3.3 Logging tool. 20 3.3.1 Crond Service 20 3.3.2 Analysis squid log with calamaris 21 Chapter 4 22 EXPERIENCES AND RESULTS 22 4.1 Haproxy server 22 4.1.1 Uptime 22 4.1.2 Resource consuming 23 4.1.3 Web statistics and administration: 25 4.1.4 Haproxy conclusion 26 4.2 Squid proxy servers 27 4.2.1 Untuned proxy static 27 4.2.2 Tuned up proxy static 29 4.2.3 Squid conclusion 32 4.3 Users experience 33 Chapter 5 36 CONCLUSION AND FUTURE WORK 36 Reference 37

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY Le Tri Thai INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY Major: Computer Science HA NOI - 2012 VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY Le Tri Thai INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY Major: Computer Science Supervisor: Prof. Ho Si Dam M.S Doan Minh Phuong AUTHORSHIP “I hereby declare that the work contained in this thesis is of my own and has not been previously submitted for a degree or diploma at this or any other higher education institution. To the best of my knowledge and belief, the thesis contains no materials previously published or written by another person except where due reference or acknowledgement is made.” Signature:……………………………………………… i SUPERVISOR’S APPROVAL “I hereby approve that the thesis in its current form is ready for committee examination as a requirement for the Bachelor of Computer Science degree at the University of Engineering and Technology.” Signature:……………………………………………… ii ACKNOWLEDGEMENT First of all, I would like to express my sincere gratitude to my supervisor Ass.Prof. Ho Si Dam, and M.S Doan Minh Phuong, University of Engineering and Technology, Viet Nam National University (VNU), Hanoi for his enthusiastic guidance, warm encouragement and helpful research experiences. I would like to also thank the Center Application Internet Technology – CAIT for all their careful guidance and preparing facility to my application. I am grateful to thank all the teachers in University of Engineering and Technology, VNU who provide invaluable knowledge and life skills for me during the four academic years. I would like to also thank my friends in K53CA class whom help me during the four academic years. Last, but not least, my family is really the biggest motivation for me. My parents always encourage me when I have stress and difficulty. I would like to send them great love and gratefulness. Ha Noi, May …, 2012 Le Tri Thai iii  Our thesis has been completed objective and satisfied the requirement of subjective: Increasing the service capability for multi-users for VNU web proxy with Haproxy. We have successful integrated haproxy server load balancer into present system. We also modified old server to compatible with new scenario successfully. Lastly, all system was tuned up with higher capability and higher availability. Our work is to solve the problem of management servers. haproxy is stable, and does not any problem with all old system. The maintenance, upgrade backend servers, squid servers is easier than ever. Nowadays, the more servers is installed, the more works of administrators. But haproxy can save their time and working enforce. Squid proxy is stronger than before. The error request is reduce by 40%. The session rate is increate more than three times without deceasing of quality. The advantage is outweighing the disadvantage of difficult in installing and boosting system. Keyword: haproxy, squid proxy, capability iv List of Figures Figure 1.1. The University Campus Network 5 Figure 2.1. Testing model of haproxy in support website 8 Figure 2.2. Deployment model of haproxy load balancer 10 Table 3.2 Basic haproxy configuration 14 Table 3.3 Detail haproxy configuration 16 Table 3.4 Configuration of rsyslog for haproxy log 18 Table 3.5 Disable selinux in /etc/selinux/config 19 Table 3.6 Disable iptables service 19 Table 3.7 Restart haproxy script 19 Table 3.8 FD configuration 21 Table 3.9. Statistic network by crontab 21 Table 3.10. Calamaris analysis squid access log 22 Table 4.1. Statistic of resource log in crond service 24 Figure 4.1. System status generated by top command 25 Figure 4.2. File-descriptors in haproxy, server, and max-file 26 Figure 4.3. Haproxy server webpage statistics 27 Table 4.2. Incoming TCP request by status un-tuned 28 Table 4.4. Distribution histogram of request un-tuned 29 Figure 4.4. File-descriptor shortage in squid proxy shown in cache.log 30 Table 4.5. Incoming TCP-request by status tuned 30 Table 4.6. Request-destination by 2nd-level-domain tuned 31 Table 4.7. Distribution histogram of request tuned 33 Table 4.8. Frequency load time website 35 v  Figure 1.1. The University Campus Network 5 Figure 2.1. Testing model of haproxy in support website 8 Figure 2.2. Deployment model of haproxy load balancer 10 Table 3.2 Basic haproxy configuration 14 Table 3.3 Detail haproxy configuration 16 Table 3.4 Configuration of rsyslog for haproxy log 18 Table 3.5 Disable selinux in /etc/selinux/config 19 Table 3.6 Disable iptables service 19 Table 3.7 Restart haproxy script 19 Table 3.8 FD configuration 21 Table 3.9. Statistic network by crontab 21 Table 3.10. Calamaris analysis squid access log 22 Table 4.1. Statistic of resource log in crond service 24 Figure 4.1. System status generated by top command 25 Figure 4.2. File-descriptors in haproxy, server, and max-file 26 Figure 4.3. Haproxy server webpage statistics 27 Table 4.2. Incoming TCP request by status un-tuned 28 Table 4.4. Distribution histogram of request un-tuned 29 Figure 4.4. File-descriptor shortage in squid proxy shown in cache.log 30 Table 4.5. Incoming TCP-request by status tuned 30 Table 4.6. Request-destination by 2nd-level-domain tuned 31 Table 4.7. Distribution histogram of request tuned 33 Table 4.8. Frequency load time website 35 vi  CentOS : The Community ENTerprise Operating System VNU: Vietnam National University VNUnet: Vietnam National University network vii [...]... PID=`pidof haproxy` ; #stop all haproxy process with their ID for i in $PID; do kill $i; done; #make bash shell sleep 1 second sleep 1; #run haproxy again haproxy -f /etc /haproxy/ haproxy.conf; 19 #get processes ID of haproxy processes PID=`pidof haproxy` ; #renice haproxy process to -5 for i in $PID; do renice -5 -p $i; done; 3.2 Squid proxy server configuration Two squid proxy servers have IP address... restart #haproxy –f /etc /haproxy/ haproxy.conf We have not configured haproxy for run as service in CentOS, in temporary, using a bash shell to restart haproxy every time we want to change configuration This bash shell also re-nice the priority of haproxy process in server to -5 Full bash-shell is in Table 3.7 Table 3.7 Restart haproxy script #get processes ID of haproxy processes PID=`pidof haproxy` ;... authentication, decryption or caching [1] Squid is a caching proxy for the Web With Squid, we can use less bandwidth on our Internet connection when surfing the Web, reduce the client latency for a web pages, protect the hosts by proxying their web traffic, block unwanted website to users, control users by authenticate to access Internet, decrease the load to web server [10] Squid job is to be a proxy. .. .1 1.1 Motivation 1 1.2 Contributions and thesis overview .2 Chapter 2 3 INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY 3 2.1 Squid web cache proxy 3 2.2 Server load balance 7 2.2.1 haproxy - server load balancer 7 2.2.2 Haproxy specification 8 2.3 Operating system - CentOS... Contributions and thesis overview The purpose of this project is to increase capacity to serve users for web proxies system in Hanoi National University with Haproxy load balancer With load balancing, the jobs of upgrade, maintain squid proxy do not interrupt of Internet service, and the upgrade is also easier In addition, the deployment of Haproxy also achieves greater or equal performance compared with that... device into 4 squid proxy in VNUnet, we have to prepare 150 GB – 300 GB for each cache proxy Objective of Squid proxy in VNUnet The VNUnet is complexity network include up to hundreds of servers, providing all internet activity for student, staff office, and others There are webservers, portal webs, blackboard course, authentication server, resources, and so on They are together supporting for all people... to user the cached copy of that request This makes reduce bandwidth and latency for a web pages Squid – web caching do the jobs of storing the web resources for future reuse For example, when a user visits a site, the Squid cache stores the HTML page and other resources in that site Latter, others visits that site again, the validate cache which stored in squid memory should be response for them It... before In chapter 2, we present related work and background information needed for a better understanding of this paper We describe our approach and outline our implementation in chapter 3 We present our experiments and results in chapter 4 Finally, we conclude and offer suggestions for future works in chapter 5 2 Chapter 2 INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY. .. directory, then, compile and install haproxy Table 3.1 show more detail in action of installation #yum install gcc –y #wget http:/ /haproxy. 1wt.eu/download/1.4/src /haproxy- 1.4.20.tar.gz # tar xvfs haproxy- 1.4.20.tar.gz #cd haproxy- 1.4.20 # make install TARGET=linux2.6 ARCH=i686 Basic configuration for run intermediately is shown in Table 3.2 This configuration define the maximum connection of haproxy server... Table 3.3 Detail haproxy configuration global chroot /loadbalance /haproxy maxconn 20000 daemon user haproxy group haproxy # nbproc 2 defaults log 127.0.0.1 local0 info option httplog option http-server-close option forwardfor mode http clitimeout 5000 contimeout 4000 srvtimeout 6000 retries 2 # acl ads url_dom vcmedia # block if ads # Define the user authenticate to view the status of haproxy # stats . Contributions and thesis overview 2 Chapter 2 3 INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY 3 2.1 Squid web cache proxy 3 2.2 Server load balance 7 2.2.1 haproxy -. offer suggestions for future works in chapter 5. 2 Chapter 2 INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY In this chapter we introduce about squid web cache proxy. Then. Thai iii  Our thesis has been completed objective and satisfied the requirement of subjective: Increasing the service capability for multi-users for VNU web proxy with Haproxy. We have successful