Luận án tiến sĩ các phương pháp tiết kiệm năng lượng sử dụng công nghệ mạng điều khiển bằng phần mềm trong môi trường điện toán đám mây

MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY TRAN MANH NAM CÁC PHƯƠNG PHÁP TIẾT KIỆM NĂNG LƯỢNG SỬ DỤNG CÔNG NGHỆ MẠNG ĐIỀU KHIỂN BẰNG PHẦN MỀM TRONG MƠI TRƯỜNG ĐIỆN TỐN ĐÁM MÂY SDN-BASED ENERGY-EFFICIENT NETWORKING IN CLOUD COMPUTING ENVIRONMENTS DOCTORAL THESIS OF TELECOMMUNICATIONS ENGINEERING HANOI MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY TRAN MANH NAM CÁC PHƯƠNG PHÁP TIẾT KIỆM NĂNG LƯỢNG SỬ DỤNG CÔNG NGHỆ MẠNG ĐIỀU KHIỂN BẰNG PHẦN MỀM TRONG MƠI TRƯỜNG ĐIỆN TỐN ĐÁM MÂY SDN-BASED ENERGY-EFFICIENT NETWORKING IN CLOUD COMPUTING ENVIRONMENTS Specialization: Telecommunications Engineering Code No: 62520208 DOCTORAL THESIS OF TELECOMMUNICATIONS ENGINEERING Supervisor: Assoc.Prof Nguyen Huu Thanh HANOI PREFACE I hereby assure that the results presented in this dissertation are my work under the guidance of my supervisor The data and results presented in the dissertation are completely honest and have not been disclosed in any previous works The references have been fully cited and in accordance with the regulations Tôi xin cam đoan kết trình bày luận án cơng trình nghiên cứu hướng dẫn giáo viên hướng dẫn Các số liệu, kết trình bày luận án hoàn toàn trung thực chưa cơng bố cơng trình trước Các kết sử dụng tham khảo trích dẫn đầy đủ theo quy định Tác giả Trần Mạnh Nam ii ACKNOWLEDGEMENTS First and foremost, I would like to thank my advisor, Associate Prof Dr Nguyen Huu Thanh, for providing an excellent researching atmosphere, for his valuable comments, constant support and motivation His guidance helped me in all the time and also in writing this dissertation I could not have thought of having a better advisor and mentor for my PhD Moreover, I would like to thank Associate Prof Dr Pham Ngoc Nam, Dr Truong Thu Huong for their advices and feedbacks, also for many educational and inspiring discussions My sincere gratitude goes to the members (present and former) of the Future Internet Lab, School of `Electronics and Telecommunications, Hanoi University of Science and Technology Without their support and friendship it would have been difficult for me to complete my PhD studies Finally, I would like to express my deepest gratitude to my family They are always supporting me and encouraging me with their best wishes, standing by me throughout my life Hanoi, 19th Jan 2018 iii CONTENTS LIST OF FIGURES .viii LIST OF TABLES x INTRODUCTION CHAPTER AN OVERVIEW OF ENERGY-EFFICIENT APPROACHES IN CLOUD COMPUTING ENVIRONMENTS 1.1 Today's Internet 1.1.1 Cloud Computing Services and Infrastructures 1.1.2 Energy consumption problems .6 1.2 An Overview of Energy-Efficient Approaches .8 1.2.1 Energy consumption characteristics 1.2.2 Energy-Efficient Approaches' Classification 1.3 Software-defined Networking (SDN) technology 10 1.3.1 SDN Architecture 10 1.3.2 SDN Southbound API - OpenFlow Protocol 11 1.3.3 SDN Controllers 12 1.4 Difficulties on Network Energy Efficiency and Motivations .13 1.5 Dissertation’s Contributions 14 1.5.1 Proposing an energy-aware and flexible data center network that is based on the SDN technology 14 1.5.2 Proposing energy-efficient approaches in a network virtualization for cloud environments 14 1.5.3 Proposing an energy-aware data center virtualization for cloud environments 15 CHAPTER NETWORK SDN-BASED ENERGY-AWARE DATA CENTER 16 2.1 Background Technologies .16 2.1.1 DCN technique and architecture 16 2.1.2 Existing system 22 2.2 Power-Control System of a DC Network 22 2.2.1 Energy modeling of a network 23 2.2.2 The Diagram of the Power-Control System .25 2.3 Energy-Aware Routing based on Power Profile of Devices in Data Center Networks using SDN 29 2.3.1 Energy-Aware Routing and Topology Optimization Algorithm 30 2.3.2 Performance evaluation .36 2.4 Green Data Center using centralized Power-control of the Network and servers 39 2.4.1 Extended Power-Control System .40 2.4.2 Use case .41 2.4.3 Topology-aware VM migration algorithm 43 2.4.4 VM Migration cost and Power modeling of a Server 45 2.4.5 Experimental Results 45 2.5 Conclusion .48 iv CHAPTER ENERGY-EFFICIENT NETWORK VIRTUALIZATION FOR CLOUD ENVIRONMENTS 49 3.1 Network Virtualization and Virtual Network Embedding 51 3.2 Constructing Energy-Aware SDN-based Network Virtualization System 51 3.2.1 System’s Diagram 52 3.2.2 System’s workflow 53 3.3 Modeling and Problem Formulation .54 3.3.1 VNE Modeling .54 3.3.2 Objective and Constraints 55 3.3.3 Time-based Embedding Strategies 57 3.4 Energy-efficient VNE algorithms 58 3.4.1 Energy-cost Coefficient of Capacity 58 3.4.2 Virtual Node Mapping algorithms .59 3.4.3 Virtual Link Mapping (VLiM) Algorithm .62 3.5 Performance Evaluation 63 3.6 Conclusion .67 CHAPTER AN ENERGY-AWARE DATA CENTER VIRTUALIZATION FOR CLOUD ENVIRONMENTS 68 4.1 Virtual DC Technologies 69 4.1.1 Virtual data center embedding 69 4.1.2 Virtual machine migration and server consolidation 71 4.1.3 Discussion 71 4.2 Design Objectives 73 4.3 Problem Formulation 74 4.3.1 Data Center Modeling 74 4.3.2 Energy Modeling of DC Components 75 4.3.3 Energy-Efficient Problem Formulation .76 4.4 A New Concept for VDC Embedding 77 4.4.1 Energy-aware VDC architecture 77 4.4.2 Energy-aware VDC embedding algorithm 78 4.4.3 Joint VDC Embedding and VM Migration Algorithms 81 4.5 Performance Evaluation 84 4.5.1 Performance criteria 84 4.5.2 Numerical results 85 4.6 Conclusion .91 CHAPTER CONCLUSION AND FUTURE WORK 92 5.1 5.2 Major contributions 92 Future research directions .93 LIST OF PUBLICATIONS 94 REFERENCES 96 v ABBREVIATIONS APCI APEX ASIC BAU BFS CAPEX DC DCN D-ITG EA-NV EA-VDC ECO FM FPGA GH HEA-E HEE IaaS ICT ISP MoA MST NaaS NFV NV OLD OPEX PaaS PCS PM POD PSnEP RMD-EE SaaS SDSN SN SNMP Advanced Configuration & Power Interface Capital expenditure Application specific integrated circuits Business-as-usual Breadth-first Search Capital Expenditure Data center Data center network Distributed internet traffic generator Energy-aware network virtualization Energy-aware Virtual Data Center Eco sustainable Full migration Field programmable gate arrays GreenHead Heuristic Energy-aware VDC Embedding Heuristic energy-efficient Infrastructure-as-a-service Information and communication technologies Internet service provider Migrate on arrival Minimum spanning tree Network-as-a-service Network function virtualization Network virtualization OpenDayLight Operating expenses Platform-as-a-service Power-Control System Partial migration Optimized data centers Power scaling and energy-profile-aware Reducing middle node energy efficiency Software-as-a-service Software-Defined Substrate Network SecondNet Simple network management protocol vi TCAM VDC VDCE VLiM VM VmM VNE VNoM VNR Ternary content-addressable memory Virtual data center Virtual data center embedding Virtual link mapping Virtual Machine Virtual machine mapping Virtual network embedding Virtual node mapping Virtual network requests vii LIST OF FIGURES Figure 1.1: Estimate of the global carbon footprint of ICT (including PCs, telcos’ networks and devices, printers and datacenters) [15] Figure 1.2: Energy consumption estimation for the European telcos’ network infrastructures in the”Business-As-Usual” (BAU) and in the Eco-sustainable (ECO) scenarios, and cumulative energy savings between the two scenarios [16] .7 Figure 1.3: Operating Expenses (OPEX) estimation related to energy costs for the European telcos’ network infrastructures in the ”Business-As-Usual” (BAU) and in the Eco-sustainable (ECO) scenarios, and cumulative savings between the two scenarios [17] Figure 1.4: SDN Architecture 11 Figure 1.5: OpenFlow controller and switches 12 Figure 2.1: DCN Architecture [43] 18 Figure 2.2: Three-tier DCN Architecture [45] 18 Figure 2.3: Fat-tree DCN Topology 19 Figure 2.4: Dcell DCN Architecture [53] 19 Figure 2.5: BCube DCN Architecture [54] .20 Figure 2.6: Fat-tree architecture with k = 21 Figure 2.7: Diagram of the ElasticTree system [57] 22 Figure 2.8: Energy – Utilization relation of a network [58] .23 Figure 2.9: Power-control System of a Network .26 Figure 2.10: Fat-tree topology with Minimum Spanning Tree 28 Figure 2.11: Power Scaling Algorithm 32 Figure 2.12: Power Scaling and Energy-Profile-Aware - PSnEP algorithm (Proposed Algorithm 1) The flowchart describes the process between Edge and Aggregation switches 34 Figure 2.13: use-case with PSnEP algorithm in a DCN 35 Figure 2.14: PSnEP vs Power scaling (PS) with k=6 Fat-tree, mix scenario 38 Figure 2.15: Energy-saving level ratio of the PSnEP algorithm to the PS algorithm in different sizes 39 Figure 2.16: Extended Power-Control system (Ext-PCS) 40 Figure 2.17: Example 42 viii Figure 2.18: First-fit Migration [67] Algorithm 42 Figure 2.19: Topology-Aware Placement Algorithm 43 Figure 2.20: K=8, comparison with full mesh scenario 46 Figure 2.21: K=16, comparison with full mesh scenario 47 Figure 2.22: K=8, comparison with Honeyguide 47 Figure 2.23: K=16, comparison with Honeyguide 48 Figure 3.1: FlowVisor – Hypervisor-like Network Layer [71] 50 Figure 3.2: Example of a virtual network on top of a physical network 51 Figure 3.3: Energy-Aware Network Virtualization system’s Diagram .52 Figure 3.4: Online VNE mapping method 57 Figure 3.5: Online using Time Window method 58 Figure 3.6: The GUI of an Energy-aware network virtualization platform 64 Figure 3.7 AR– Online .65 Figure 3.8: AR – Online using Time Windows 65 Figure 3.9: Percentage of Power Consumption to Full State in Online Strategy .65 Figure 3.10 Percentage of Power Consumption to Full State in OuTW Strategy 65 Figure 3.11: Comparison of comsumed energy between Online and OuTW strategies 66 Figure 3.12: Comparison of acceptance ratio between Online and OuTW strategies 66 Figure 4.1: Traditional cloud service provider vs NaaS 68 Figure 4.2: Embedding virtual data center requests on a physical data center 70 Figure 4.3: Virtual data center embedding - Static mapping; 72 Figure 4.4: Virtual data center embedding - Dynamic mapping .72 Figure 4.5: Energy proportional property of energy-aware data centers 73 Figure 4.6: Energy-Aware VDC Architecture 78 Figure 4.7: VDC Embedding Flowchart 79 Figure 4.8: Flowchart of Partial Migration (PM) .83 Figure 4.9: Migration on Arrival .84 Figure 4.10: Fluctuation of system utilization (SecondNet) 86 Figure 4.11: DC Utilization per Load .87 4.12: Acceptance Ratio per VM 87 Figure 4.13: Acceptance Ratio per VDC 88 ix Next, the average power consumption of a VDC is evaluated, which is calculated by dividing the total power consumption of the physical DC by the number of served VDCs As can be seen in the Figure 55, when the system load increases: - - (1) the consumed power per each VDC is decreasing; (2) the consumed power of the proposed algorithms is much the same and is less than both SN and GN; (3) power consumption of the proposed algorithms decreases very slowly, which implies that power consumption of the physical DC stays nearly linear to the number of embedded VDCs, following the energy proportional property (see Sec 4.1.3); and (4) although the total power consumption of GH is the lowest (Figure 54), its average power consumption per VDC is the highest one due to the fact that GH can host only a very limited number of VDCs (Figure 55) Figure 4.55: Average consumed power per serving VDC 4.5.2.4 Complexity The average number of migrations for each strategy under different load situation is used as the metric to evaluate complexity As the migration of a virtual machine to a physical server takes time, a high number of migrations reduces the system performance significantly As expected, the number of migrations in FM under highly loaded situation (90%) can be as high as 9000 times (Figure 56) while the number of migrations in PM is under 100 times, independent of the load as only underutilized servers are required to consolidate 92 Figure 4.56: Number of migrations for different strategies 4.5.2.5 Discussions Finally, general comparison for the aforementioned strategies can be drawn As shown in the above numerical results, SecondNet performs well in terms of resource efficiency but it does not satisfy the requirement on energy-efficiency In contrast, GreenHead is energy-efficient in some sense, however, its utilization is very low in comparison to other methods Figure 4.57: Comparison of embedding - migration strategies On the other hand, the three new approaches PM, MoA and FM perform well in terms of both resource and energy efficiency Moreover, PM is simpler as the number of required migrations is much less, independent of system load when compared with MoA and FM The radar graphs in Figure 57 and Figure 58 illustrates this comparative analysis In 93 general, PM can be the most suitable strategy as it performance is nearly as good as FM and MoA and is much simpler 4.58: Different embedding-magrition strategies: (a) GreenHead, (b) SecondNet, (c) Partial Migration, (d) Migration on Arrival, (e) Full Migration 4.6 Conclusion This work firstly analyzes the resource fragmentation problem occurred when virtual data centers continuously join and leave the physical data center and its impact on the efficiency of VDC embedding algorithms Different joint VDC mapping and VM migration strategies are proposed to tackle that dynamic problem In comparison to some previous resource and energy-aware VDC embedding algorithms, the new strategies can remarkably improve both resource utilization and energy efficiency of the data center, while the complexity is kept at an acceptable level 94 CHAPTER CONCLUSION AND FUTURE WORK Many Telcos, Internet Service Providers (ISPs) and enterprises, have significantly employed large network infrastructures for the Internet services A large system consumes a huge energy volume, so that the network energy efficiency problem is very important recently Resolving energy-saving problems brings many advantages including: - Environmentally, it reduces the large amount of carbon emission from ICT sector; Economically, reducing energy consumption of the ICT data centers leads to reducing the costs of maintaining system Consequently, the Internet services’ cost will be reduced 5.1 Major contributions By using Software-defined Networking, energy-efficient approaches are studied in the network in several cloud DC environments such as: (1) in data center network that uses the promising DC topology, namely Fat-tree; (2) in the network virtualization concept; and (3) in the data center virtualization technology The contributions of this research are summarized as follows In the second chapter, the proposed SDN-based power-control system is presented This PCS platform allows administrators to monitor, control, and apply several energy-efficient algorithms This second chapter also presents two main energy-efficient approaches including: (1) energy-aware routing algorithm, namely power scaling and energy-profile-aware (PSnEP) algorithm, which is based on the power scaling algorithm and the power profile of a network device; and (2) topology-aware VM migration algorithm which migrates servers with two objectives: (a) minimizing the number of physical servers; and (b) reducing the number of switches for interconnecting these physical servers in order to turn-off more devices for energy efficiency The main advantage of this algorithm is that the migration process performs energy saving of servers as other common migration strategy, known as first-fit, while reducing the energy consumption of the network devices The experimental results show that the consumed power of the network devices can be saved up to 46% while remaining the energy-saving level of the servers An energy-aware network virtualization concept is described in the next chapter with its power monitoring and controlling abilities for cloud environments The proposed concept is based on the SDN technology and allows researchers to develop several energy-efficient virtual network embedding algorithms Two proposed energy-efficient embedding algorithms are proposed, namely heuristic energy-efficient node mapping and reducing middle node energy efficiency, with their experimental results of performance The SDN-based Energy-aware VDC approaches for cloud environments is presented in the fourth chapter The VDC technology is described in detail with its main problem, namely VDC embedding By integrating with VM consolidation technique, the joint VDC 95 embedding and VM migration algorithms is successfully deployed These algorithms with their experimental results are described in this chapter in details 5.2 Future research directions Although network energy efficiency has been attracted much attention from the research community, there are many difficulties to realize these technologies and transfer to the industrial market So that in the future work, we are going to establish the following tasks: - Realizing the energy efficiency of a data center network by using new cloud platform – OpenStack OpenStack software controls large pools of compute, storage, and networking resources throughout a datacenter, managed through a dashboard or via the OpenStack API OpenStack works with popular enterprise and open source technologies making it ideal for heterogeneous infrastructure [90] The OpenStack Platform contains the SDN controller - OpenDayLight and the compute management, which manage the VM provisioning and migration process - Developing the network virtualization and data center virtualization for detailed estimating the delay and packet loss In the near future, the Internet services with their characteristics will be embedded into the system Based on the services’ demand as well as their parameters such as downtime, latency and reliability, the system allocates its resources reasonably 96 LIST OF PUBLICATIONS Journals Thanh Nguyen Huu, Anh-Vu Vu, Duc-Lam Nguyen, Van-Huynh Nguyen, ManhNam Tran, Quynh-Thu Ngo, Thu-Huong Truong, Tai-Hung Nguyen, Thomas Magedanz (2015) “A Generalized Resource Allocation Framework in Support of MultiLayer Virtual Network Embedding based on SDN”, Elsevier - Computer Networks, 2015 - https://doi.org/10.1016/j.comnet.2015.09.042 Tran Manh Nam, Nguyen Huu Thanh, Hoang Trung Hieu, Nguyen Tien Manh, Nguyen Van Huynh, Tuan Hoang (2017) “Joint Network Embedding and Server Consolidation for Energy-Efficient Dynamic Data Center Virtualization”, Elsevier Computer Networks, 2017 - https://doi.org/10.1016/j.comnet.2017.06.007 Book Chapter Nam T.M., Huynh N.V., Thanh N.H (2016) “Reducing Middle Nodes Mapping Algorithm for Energy Efficiency in Network Virtualization” In: Advances in Information and Communication Technology ICTA 2016 Advances in Intelligent Systems and Computing, vol 538 Springer, Cham https://doi.org/10.1007/978-3319-49073-1_54 Conferences Nguyen Huu Thanh, Bui Dinh Cuong, To Duc Thien, Pham Ngoc Nam, Ngo Quynh Thu, Truong Thu Huong, and Tran Manh Nam (2013) “ECODANE: A Customizable Hybrid Testbed for Green Data Center Networks”, The International Conference on Advanced Technologies for Communications 2013 (ATC'13) IEEE, Hanoi, Vietnam Tran Manh Nam, Tran Hoang Vu, Vu Quang Trong, Nguyen Huu Thanh, Pham Ngoc Nam (2013) “Implementing Rate Adaptive Algorithm in Energy-Aware Data Center Network”, National Conference on Electronics and Communications (REV2013-KC01)., Hanoi, Vietnam Tran Manh Nam, Truong Thu Huong, Nguyen Huu Thanh, Pham Van Cong, Ngo Quynh Thu, Pham Ngoc Nam (2014) “A Reliable Analyzer for Energy-Saving Approaches in Large Data Center Networks”, IEEE ICCE - The International Conference on Communications and Electronics - 2014, Da Nang, Vietnam Tran Manh Nam, Nguyen Huu Thanh, Ngo Quynh Thu and Hoang Trung Hieu, Stefan Covaci (2015) “Energy-Aware Routing based on Power Profile of Devices 97 in Data Center Networks using SDN”, 12th Electrical Engineering/Electronics, Computer, Telecommunications And Information Technology Conference (ECTICON) - 2015, Hua Hin, Thailand Tran Manh Nam, Nguyen Huu Thanh, Nguyen Hong Van, Kim Bao Long, Nguyen Van Huynh, Nguyen Duc Lam, Nguyen Van Ca (2015) “Constructing EnergyAware Software-Defined Network Virtualization”, Proceedings of Asia-Pacific Advanced Network Research Workshop (APAN-NRW), August 10th - 14th 2015, Kuala Lumpur, Malaysia - (best student paper award) Tran Manh Nam, Nguyen Huu Thanh, Doan Anh Tuan (2016) “Green Data Center Using Centralized Power-Management Of Network And Servers”, The 15th international Conference on Electronics, Information, and Communication (IEEE ICEIC), Jan 2016, Da Nang, Vietnam Tran Manh Nam, Nguyen Van Huynh, Le Quang Dai, Nguyen Huu Thanh (2016) “An Energy-Aware Embedding Algorithm for Virtual Data Centers”, ITC28 International Teletraffic Congress, Sep - 2016, Wurzburg, Germany Tran Manh Nam, Nguyen Tien Manh, Truong Thu Huong, Nguyen Huu Thanh (2018) “Online Using Time Window Embedding Strategy in Green Network Virtualization”, International Conference on Information and Communication Technology and Digital Convergence Business (ICIDB-2018), Hanoi, Vietnam (presented) 98 REFERENCES [1] http://www.internetworldstats.com/stats.htm, "Usage and Population Statistics," [Online] [2] "Global Action Plan, An Inefficient Truth, Global Action Plan Report," october 2007 [Online] Available: http://globalactionplan.org.uk [Accessed 2016] [3] "Gartner Says Data Centers Account for 23 per cent of Global ICT CO2 Emissions,," Gartner, 2007 [Online] Available: http://www.gartner.com/it/page.jsp?id=530912 [Accessed 2016] [4] Bolla, R., Bruschi, R., Davoli, F., Member, S., & Cucchietti, F., "Energy Efficiency in the Future Internet : A Survey of Existing Approaches and Trends in EnergyAware Fixed Network Infrastructures," Ieee Communications Surveys & Tutorials, vol 13, no 2, 2011 [5] Boutaba, N M K Chowdhury and R., "A survey of network virtualization," Computer Networks, vol 54, no 5, p 862 – 876, 2010 [6] Chowdhury, N M M K., & Boutaba, R., "Network virtualization: state of the art and research challenges," IEEE Communications Magazine, vol 47, no 7, pp 2026, 2009 [7] Khan, A., Zugenmaier, A., Jurca, D., & Kellerer, W., "Network virtualization: a hypervisor for the Internet?," IEEE Communications Magazine, vol 50, no 1, p 136–143, 2012 [8] Zhang, Q., Zhani, M F., Jabri, M., & Boutaba, R., "Venice: Reliable virtual data center embedding in clouds.," in In Proceedings - IEEE INFOCOM, Toronto, ON, Canada, 2014 [9] Zhani, M F M., Zhang, Q., Simon, G., & Boutaba, R., "VDC Planner : Dynamic Migration-Aware Virtual Data Center Embedding for Clouds," in IFIP/IEEE International Symposium on Integrated Network Management, Ghent, Belgium, 2013 [10] S D M K L J a U B M P Gilesh, "Towards a Complete Virtual Data Center Embedding Algorithm," in IEEE 37th International Conference on Distributed Computing Systems, Atlanta, GA, USA, June - 2017 [11] Kreutz, D., Ramos, F M V., Esteves Verissimo, P., Esteve Rothenberg, C., Azodolmolky, S., Uhlig, S.,Uhlig, S., "Software-Defined Networking: A Comprehensive Survey," Proceedings of the IEEE, 103, vol 103, no 1, pp 14-76, 2015 [12] Feamster, H Kim and N., "Improving network management with software defined networking," IEEE Commun Mag, vol 51, no 2, p 114–119, 2013 [13] S Schenker, "The future of networking, the past of protocols," 2011 [Online] 99 Available: http://www.youtube.com/ watch?v=YHeyuD89n1Y [Accessed Jan 2014] [14] Luis M Camarinha-Matos, "Scientific Research Methodologies And Techniques," www.uninova.pt/cam/teaching/SRMT/SRMTunit1.pdf [15] "SMART 2020: Enabling the Low Carbon Economy in the Informa- tion Age," Global e-Sustainibility Initiative (GeSI), http://www.theclimategroup.org/assets/resources/publications/Smart2020Report.pdf [16] "U.S Energy Information Administration (EIA)," Official Energy Statistics from the U.S Governement, [Online] Available: http://www.eia.doe.gov [Accessed Dec 2016] [17] "Impacts of Information and Communication Technologies on Energy Efficiency," European Commission DG INFSO, http://ec.europa.eu/information society/newsroom/cf/itemdetail.cfm ?item id=4441, 2018 [18] "The Advanced Configuration & Power Interface (ACPI) Specification," HewlettPackard, Intel, Microsoft, Phoenix, and Toshiba., [Online] Available: http://www.acpi.info/ [19] R S Tucker, R Parthiban, J Baliga, K Hinton, R W A Ayre, W.V Sorin, "Evolution of WDM Optical IP Networks: A Cost and Energy Perspective," IEEE J Lightwave Technol, vol vol 27, no 3, pp pp 243-252, Feb 2009 [20] D Neilson, "Photonics for Switching and Routing," IEEE Journal of Selected Topics in Quantum Electronics (JSTQE), vol 12, no 4, pp 669-678, July-Aug 2006 [21] Bianzino, A P., Chaudet, C., Rossi, D., & Rougier, J., "A Survey of Green Networking Research," IEEE Communications Surveys Tutorials, vol 14, no 1, 2012 [22] Barr, Keith, ASIC Design in the Silicon Sandbox: A Complete Guide to Building Mixed-Signal Integrated Circuits, McGraw Hill Professional, 2007 [23] Józef KORBICZ et al., Synthesis of Compositional Microprogram Control Units for Programmable Devices, Poland: University of Zielona Góra Press, Poland, 2009 [24] J Baliga, R Ayre, K Hinton, and R S Tucker, "Photonic Switching and the Energy Bottleneck," in Internat Conf Photonics in Switching 2007, San Francisco, CA, USA, 2007 [25] L G Roberts, "A Radical New Router," IEEE Spectrum, vol 46, no 7, pp 34-39, 2009 [26] M Baldi, Y Ofek, "Time for a ”Greener” Internet,”," in Proc Green Communications Workshop in conjunction with IEEE ICC’09 (GreenComm09), Dresden, Germany, 2009 [27] J Noguera, I.O Kennedy, "Power Reduction in Network Equipment Through Adaptive Partial Reconfiguration," in Internat Conf on Field Programmable Logic 100 and Applications (FPL 2007), Amsterdam, Netherlands, 2007 [28] M B Srivastava, A.P Chandrakasan, R.W Brodersen, "Predictive System Shutdown and Other Architectural Techniques for Energy Efficient Programmable Computation," IEEE Trans Very Large Scale Integr (VLSI), vol 4, no 1, pp 4255, 1996 [29] M Allman, K Christensen, B Nordman, V Paxson, "Enabling an Energy-Efficient Future Internet Through Selectively Connected End Systems," in Proc ACM SIGCOMM HotNets Workshop (HotNets 07), Atlanta, GA, USA, 2007 [30] M Jimeno, K Christensen, B Nordman, "A Network Connection Proxy to Enable Hosts to Sleep and Save Energy," in Proc IEEE Internat Performance Computing and Communications Conf, Austin, Texas, USA, 2008 [31] Han, B., Gopalakrishnan, V., Ji, L., & Lee, S., "Network function virtualization: Challenges and opportunities for innovations," IEEE Communications Magazine, vol 53, no 2, p 90–97, 2015 [32] Correa, E S., Fletscher, L A., & Botero, J F., "Virtual Data Center Embedding: A Survey.," IEEE Latin America Transactions, vol 13, no 5, p 1661–1670, 2015 [33] B Raghavan et al, "Software-defined internet architecture: Decoupling architecture from infrastructure," in Proc 11th ACM Workshop Hot Topics Network, Redmond, Washington, USA, 2012 [34] N McKeown et al, "OpenFlow: Enabling innovation in campus networks," SIGCOMM Comput Commun Rev, vol 38, no 2, p 69–74, 2008 [35] "Open Networking Foundation (ONF)," 2014 [Online] Available: https://www [Accessed Jan 2015] [36] Gude, N., Koponen, T., Pettit, J., Pfaff, B., Casado, M., McKeown, N., & Shenker, S., "NOX: Towards an Operating System for Networks," ACM SIGCOMM Computer Communication Review, vol 38, no 3, pp 105-111, 2008 [37] "POX," [Online] Available: http://www.noxrepo.org/pox/ about-pox/ [Accessed Jan 2014] [38] D Erickson, "Beacon Home," [Online] Available: https://openflow.stanford.edu/display/Beacon/Home/ [Accessed May 2015] [39] Floodlight [Online] Available: Available: http://floodlight.openflowhub.org [Accessed May 2015] [40] "The OpenDaylight Platform," OpenDaylight, https://www.opendaylight.org/ [Accessed Dec 2015] [Online] Available: [41] G V N T i O.- ‐ B N Roteiro, "Efficient Resource Provisioning Using Virtualization Technology in Cloud Environment," International Journal of Innovative Research in Science, Engineering and Technology, vol 3, no 3, pp 2200-2205, 2014, Oct [42] Theophilus Benson, Aditya Akella and David A Maltz, "Network Traffic 101 Characteristics of Data Centers in the Wild," in IMC 10, Melbourne, Australia., 2010 [43] Z Q R M H G A V V Bin Wang, "A survey on data center networking for cloud computing," Computer Networks, vol 91, p 528–547, 2015 [44] A M N T Brian Lebiednik, "A Survey and Evaluation of Data Center Network Topologies," Distributed, Parallel, and Cluster Computing, 2016 [45] D Kliazovich, P Bouvry, S.U Khan, "GreenCloud: a packet-level simulator for energy-aware cloud computing data centers," Journal of Supercomputing, vol 62, no 3, p 1263–1283, 2012 [46] "Cisco Data Center Infrastructure 2.5," Design Guide, Cisco Validated Design, March 9, 2010 [47] S K L Z H L K H S M N M.-A L W a D C K Bilal, "Quantitative Comparisons of the State of the Art Data Center Architectures," Concurrency and Computation: Practice and Experience, vol 25, no 12, pp 1771-1783, 2013 [48] C E Leiserson, "Fat-trees: universal networks for hardware-efficient supercomputing," IEEE Transactions on Computers, vol 100, no 10, pp 892-901, 1985 [49] M Al-Fares, A Loukissas, and A Vahdat, "A scalable, commodity data center network architecture," ACM SIGCOMM Computer Communication Review, vol 38, no 4, p 63–74, 2008 [50] Greenberg, A., Hamilton, J R., Jain, N., Kandula, S., Kim, C., Lahiri, P., Sengupta, S., "VL2: A Scalable and Flexible Data Center Network.," ACM SIGCOMM Computer Communication Review, vol 39, no 4, 2009 [51] Andreyev., A., "Introducing data center fabric, the next-generation facebook data center network.," Facebook, Nov 2014 [Online] Available: https://code.facebook.com/posts/360346274145943/ [Accessed Jan 2017] [52] A Singh, J Ong, A Agarwal, G Anderson, A Armistead, R Bannon, S Boving, G Desai, B Felderman, P Germano et al.,, "Jupiter rising: A decade of clos topologies and centralized control in google’s datacenter network," in ACM Conference on Special Interest Group on Data Communication, 2015 [53] C Guo, H Wu, K Tan, L Shi, Y Zhang, and S Lu, "Dcell: a scalable and faulttolerant network structure for data centers," ACM SIGCOMM Computer Communication Review, vol 38, no 4, p 75–86, 2008 [54] C Guo, G Lu, D Li, H Wu, X Zhang, Y Shi, C Tian, Y Zhang, and S Lu, "Bcube: a high performance, server-centric network architecture for modular data centers," ACM SIGCOMM Computer Communication Review, vol 39, no 4, p 63– 74, 2009 [55] A Singla, C.-Y Hong, L Popa, P B Godfrey, "Jellyfish: Networking data centers randomly," in The 9th USENIX Conference on Networked Systems Design and Implementation, CA, USA, 2012 102 [56] Trinh, L Gyarmati and T A., "Scafida: A scale-free network inspired data center architecture," ACM SIGCOMM Computer Communication Review, vol 40, no 5, pp 4-12, 2010 [57] Heller, B., Seetharaman, S., Mahadevan, P., Yiakoumis, Y., Sharma, P., Banerjee, S., & McKeown, N., "ElasticTree : Saving Energy in Data Center Networks," in NSDI'10 Proceedings of the 7th USENIX conference, CA, USA, 2010 [58] Pham Ngoc, N., Nguyen Huu, T., Vu Quang, T., Tran Hoang, V., Truong Thu, H., Tran-Gia, P., & Schwartz, C., "A new power profiling method and power scaling mechanism for energy-aware NetFPGA gigabit router.," Computer Networks, vol 78, p 4–25, 2015 [59] Mahadevan, P., Sharma, P., Banerjee, S., & Ranganathan, P., "Energy Aware Network Operations.," in IEEE INFOCOM Workshops, 2009 [60] "NetFPGA Gigabit Card," netFPGA, 2015 [Online] Available: http://netfpga.org/ [Accessed Dec 2016] [61] Wang, X., Yao, Y., Wang, X., Lu, K., & Cao, Q., "CARPO: Correlation-aware power optimization in data center networks," in In 2012 Proceedings IEEE INFOCOM, 2012 [62] J Case, M Fedor, M Schoffstall and D J., "RFC 1157 - A Simple Network Management Protocol (SNMP)," 1990 [Online] Available: https://tools.ietf.org/html/rfc1157 [63] K W a Y.-H H Yi-Chih Lei, "Multipath Routing in SDN-based Data Center Networks," in 2015 European Conference on Networks and Communications (EuCNC) , Paris, France, 2015 [64] D P Omair Fatmi, "Distributed multipath routing for data center networks based on stochastic traffic modeling," in Networking, Sensing and Control (ICNSC), Miami, FL, USA, 2014 [65] "D-ITG, Distributed Internet Traffic Generator," 2017 [Online] Available: http://www.grid.unina.it/software/ITG/ [66] Fabien Hermenier, Xavier Lorca, Jean-Marc Menaud, Gilles Muller, Julia Lawall, "Entropy: a Consolidation Manager for Clusters," ACM SIGPLAN/SIGOPS, 2009 [67] Hiroki Shirayanagi, Hiroshi Yamada, and Kenji Kono, "Honeyguide: A VM Migration-Aware Network Topology for Saving Energy Consumption in Data Center Networks," ISCC 2012 IEEE, Cappadocia, Turkey., July 2012 [68] V De Maio, R Prodan, S Benedict, G Kecskemeti, "Modelling energy consumption of network transfers and virtual machine migration, Modelling energy consumption of network transfers and virtual machine migration," Future Generation Computer Systems, vol 56, p 388–406, 2016 [69] Chowdhury, N.K., & Boutaba, R et al., "Network virtualization: state of the art and research challenges," Communications Magazine, IEEE, vol 47, no 7, p 20–26, 2009 103 [70] Kleinberg, J., "Approximation algorithms for disjoint paths problems," MIT, Massachusetts Institute of Technology, 1996 [71] R Sherwood, G Gibb, K Yap, G Appenzeller, N McKeown, and G Parulkar, "FlowVisor: A network virtualization layer," in OpenFlow Switch Consortium OPENFLOW-TR-2009, 2009 [72] Rob Sherwood, Michael Chan,Glen Gibb, Nikhil Handigol,Te-Yuan Huang,Peyman Kazemian, Masayoshi Kobayashi, David Underhill, Kok-Kiong Yap, Guido Appenzeller, and Nick McKeown, "Carving Research Slices Out of Your Production Networks with OpenFlow," Newsletter ACM SIGCOMM Computer Communication, vol 40, no 1, 2010 [73] "Exploring networks of the future," http://www.geni.net/ [Accessed Dec 2015] GENI, [Online] Available: [74] "Applications, Linking Infrastructure and Ofelia,," OpenFlow in Europe, [Online] Available: http://www.fp7-ofelia.eu [Accessed dec 2016] [75] "OF@TEIN - OpenFlow@Trans-Eurasian Information Network," [Online] Available: http://oftein.net [Accessed Dec 2016] [76] Botero, J F., Hesselbach, X., Fischer, A., & de Meer, H., "Optimal mapping of virtual networks with hidden hops," Telecommunication Systems, vol 51, no 4, p 273–282, 2013 [77] Andreas Fischer, Juan Felipe Botero, Michael Till Beck, Hermann de Meer, and Xavier Hesselbach, "Virtual Network Embedding: A Survey," IEEE COMMUNICATIONS SURVEYS & TUTORIALS, vol 15, no 4, 2013 [78] M Y Y R J & C M Yu, "Rethinking virtual network embedding," ACM SIGCOMM Computer Communication Review, vol 38, no 2, p 17, 2008 [79] B M Waxman, "Routing of multipoint connections," IEEE J Sel Areas Communication, vol 6, no 9, pp 1617-1622, , 1988 [80] Skiena, S S., "The Algorithm Design Manual, 2nd Edition," Springer Pub- lishing Company, 2008 [81] A B J D M S D H X D M H Fischer, "ALEVIN - A framework to develop, compare, and analyze virtual network embedding algorithms," Electronic Communications of the EASST, vol 37, pp 1-12, 2011 [82] P Costa, M Migliavacca, P Pietzuch, A L Wolf, "NaaS: Network-as-aService in the cloud," in The 2nd USENIX Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services, USENIX, San Jose, CA, USA, 2012 [83] Fischer, A., Botero, J F., Beck, M T., de Meer, H., Hesselbach, X., Meer, H De, … Hesselbach, X., "Virtual network embedding: A survey," IEEE Communications Surveys and Tutorials, vol 15, no 4, p 1888–1906, 2013 [84] Yoonseon Han, Jian Li, Jae-Yoon Chung, Jae-Hyoung Yoo, & Hong, J W.-K., "SAVE: Energy-aware Virtual Data Center embedding and Traffic Engineering using SDN," in In Proceedings of the 2015 1st IEEE Conference on Network 104 Softwarization (NetSoft), London, UK, 2015 [85] Guo, C., Lu, G., Wang, H J H J., Yang, S., Kong, C., Sun, P., … Zhang, Y., "SecondNet: a data center network virtualization architecture with bandwidth guarantees," in Proceedings of the 6th International COnference, (VDC, Pennsylvania, USA, 2010 [86] A Amokrane, M F Zhani, R Langar, R Boutaba, G Pujolle, "Greenhead: Virtual data center embedding across distributed infrastructures," IEEE Transactions on Cloud Computing, vol 1, no 1, p 36–49, 2013 [87] H Goudarzi, M PedramH Goudarzi, M Pedram, "Energy-efficient virtual machine replication and placement in a cloud computing system," in IEEE Fifth International Conference on Cloud Computing, Honolulu, HI, USA, 2012 [88] F Farahnakian, P Liljeberg, J Plosila, "Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning," in 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, Torino, Italy, 2014 [89] A Beloglazov, R Buyya, Y C Lee, A Y Zomaya, "A taxonomy and survey of energy-efficient data centers and cloud computing systems," Advances in Computers 82 (2011) 47 – 111, pp 47-111, 2011 [90] "OpenStack," [Online] Available: https://www.openstack.org [Accessed Dec 2016] [91] J H D A M P P A Greenberg, "The cost of a cloud: Research problems in data center networks," ACM SIGCOMM Computer Communication Review, vol 39, no 1, p 68–73, 2009 [92] A Desai, R Oza, P Sharma, and B Patel, "Hypervisor : A Survey on Concepts and Taxonomy," no 3, vol 3, pp 222 - 225, 2013 [93] "CT503-MIX High-Speed LANforge-FIRE Traffic Generator," Candela Technology, 2017 [Online] Available: http://www.candelatech.com/ct503MIX_product.php [Accessed 2017] [94] A Al-Shabibi, M De Leenheer, M Gerola, A Koshibe, G Parulkar, E Sal- vadori, B Snow, "OpenVirteX: Make your virtual SDNs programmable," in Proceedings of the Third Workshop on Hot Topics in Software Defined Networking, HotSDN ’14, ACM, New York, NY, USA, 2014 [95] D Schwerdel, D Gunther, R Henjes, B Reuther, and P Muller, "German-lab experimental facility in Future Internet," FIS 2010, ser.Lecture Notes in Computer Science - Springer-Verlag Belin Heidelberg, vol 6369, p 1, 2010 [96] H T Nguyen, A V Vu, D L Nguyen, V H Nguyen, M N Tran, Q T Ngo, T H Truong, T H Nguyen, T Magedanz, "A generalized resource allocation framework in support of multi-layer virtual network embedding based on SDN, Computer Networks," Computer Networks, vol 92, no 2, pp 251 - 269, 2015 105 [97] L Ceuppens, A Sardella, D Kharitonov, "Power Saving Strategies and Technologies in Network Equipment Opportunities and Challenges, Risk and Rewards," in Proc Internat Symp on Applications and the Internet (SAINT 2008), Turku, Finland, 2008 [98] Singh, M Gupta and S., "Greening of the Internet," in ACM SIGCOMM Conf (SIGCOMM 03, Karlshue, Germany, Aug-2003 [99] R D Corin, M Gerola, R Riggio, F D Pellegrini, E Salvadori, "VeRTIGO: Network virtualization and beyond," in European Workshop on Software Defined Networking, Darmstadt, Germany, 2012 [100] Mohsen, Ehab, "Reducing System Power and Cost," XILINX White Paper: Artix-7 FPGAs, Sep-2016 106