Today'sInternet
CloudComputingServicesandInfrastructures
TheadvancesinInformationandCommunicationTechnologies(ICT)inthelastdecadeshavemassi velyinfluencedeconomyandsocietiesaroundtheworld.TheInternetservicesaswellas cloudcomputingservices aregrowing daybydayandplayaconsiderableroleinallaspects including education, business and entertainment As we can see in theTable 1.1á inthe last four years, the percentage of people using Internet witnesses an annual growth of3.5%,from39% worldpopulation’s percentageinDec-2013to51.7% inJune-2017 [1].
Date Number of users Worldpopulation’spercentage
To meet this booming of cloud services such as IaaS, NaaS, SaaS, cloud computingenvironments with their large network infrastructures have been deployed in a very largescale, even geographically distributed data centers with a huge number of devices.
Energyconsumptionproblems
Although the benefits of having that infrastructure are considerable, such a large systemconsumesthehigh volumeof energyand leads to consequent issues:
Figure 1.1: Estimate of the global carbon footprint of ICT (including
PCs,telcos’networksanddevices,printers anddatacenters)[15]
- Environmentally, the amount of energy consumption and carbon footprint of theITC-sector is remarkable (Figure 1.1) Gartner Company, the ICT research andadvisory company, estimates that the manufacture of ICT equipment, its use anddisposalaccountfor2%ofglobalCO2emissions,whichisequivalenttothecontributionsfr omtheaviationindustry[2].Thenetworkingdevicesandcomponentseliminate around37%ofthetotalICT carbon emission[3];
- Economically,thehugeconsumedpowerleadstothecostssustainedbytheproviders/ operators to keep the network up and running at the desired service levelandleadstotheirneedofcounterbalancingever- increasingcostofenergy(Figure
Figure 1.2: Energy consumption estimation for the European telcos’ network infrastructures inthe”Business-As-Usual”(BAU)and inthe Eco-sustainable(ECO)scenarios,and cumulativeenergysavingsbetween thetwoscenarios [16]
Because of these issues, the requirement of designing a high performance and energy- efficient network has become a crucial matter for Telcos and ISPs towards greener cloudenvironments.
Figure 1.3: Operating Expenses (OPEX) estimation related to energy costs for the
Europeantelcos’ network infrastructures in the ”Business-As-Usual” (BAU) and in the Eco- sustainable(ECO)scenarios,and cumulative savingsbetweenthetwo scenarios[17]
AnOverviewofEnergy-EfficientApproaches
Energyconsumptioncharacteristics
Efficient energy use, sometimes simply called energy efficiency concept, is far frombeing new in a computing system To the best of our knowledge, the first support of powermanagement system was published in 1999, namely “Advanced Configuration &
PowerInterface”(ACPI)standard[18].Thenceforth,moreenergy- savingmechanismsweredeveloped and introduced, especially in hardware enhancement with the new CPUs, whichcould be more efficient and consumed less energy Tucker [19] and Neilson [20] estimatedon IP routers that the control plane weighs 11%, data plane for 54% and power and heatmanagement for 35% Tucker and Neilson also broke out the energy consumption of dataplaneinmoredetailasdescribedinTable1.2.From54%energyconsumptionofdataplane,the buffer management weighs 5%, the packet processing weighs about 32%; the networkinterfaces weigh about 7%; and the switching fabric for about 10% This estimation workprovides a clear indication for developers in order to increase the energy-saving level ofnetworksin thefurther researches.
Energy-EfficientApproaches'Classification
From the general point of view, existing approaches are founded on few basic concepts.As shown in surveys of Raffaele Bolla et al [4] and Aruna Banzino et al [21], the largestpart of undertaken energy-efficient concepts is founded on few energy-saving mechanismsand power management criteria that are already partially available in computing systems.Theseapproaches,whichare depicted intheTable1.3,areclassifiedas(1)re-engineering;
There-engineeringapproachesfocusonintroducinganddesigningmoreenergy- efficientelementsinsidenetworkequipmentarchitectures.Noveltechnologiesmainlyconsistofnewsilicon (ex: for Application Specific Integrated Circuits (ASICs) [22], Field ProgrammableGateArrays(FPGAs) [23],etc.)andmemorytechnologies(ex:TernaryContent-Addressable Memory (TCAM), etc.) for packet processing engines, and novel networkmedia technologies (energy- efficient lasers for fiber channel, etc.) The approaches can bedivided into two sub- approaches as follows: (1)energy-efficient siliconwhich focuses ondeveloping new silicon technologies [24]; and (2)complexity reductionwhich focuses onreducing equipment complexity in terms of header processing, buffer size, switching fabricspeedupand memoryaccess bandwidth speedup[25][26].
The dynamicadaptationapproachesof networkresourcesareaimedatmodulatingcapacities of devices (working speeds, computational capabilities of packet processing…)accordingtothecurrenttrafficdemand[4].Theseapproachesarefoundedontwomainkin dsof power management capabilities provided by the hardware level, namelypower scalingandidlelogic.
Power scalingcapabilities allow dynamically reducing the working rate of processingengines or of link interfaces [27] [28] This is usually accomplished by tuning the clockfrequency and/or the voltage of processors, or by throttling the CPU clock (i.e., the clocksignalisgatedordisabledforsomenumberofcyclesatregularintervals).Ontheotherhand,idle logicallows reducing power consumption by rapidly turning off sub-components whenno activities are performed, and by re-waking them up when the system receives newactivities.Indetail,wake- upinstantsmaybetriggeredbyexternaleventsinapre-emptive mode (e.g., “wake-on-packet”), and/or by a system internal scheduling process (e.g., thesystem wakes itself up every certain periods, and controls if there are new activities toprocess).
Sleeping and standby approaches are founded on power management primitives,whichallow devices or part of them to turn themselves almost completely off, and enter very lowenergystates,whilealltheirfunctionalitiesarefrozen[4].Thus,sleeping/ standbystatescanbe thought as deeper idle states, characterized by higher energy savings and much largerwake-up times In more detail, the applications and services of a device (or its part) stopworkingandlosetheirnetworkconnectivity[29][30]whenitgoessleeping.Asaresult,thesleeping device loses its network ”presence” since it cannot maintain network connectivity,and answer to application/service-specific messages Moreover, when the device wakes up,it has to re-initialize its applications and services by sending a non-negligible amount ofsignalingtraffic.
Software-definedNetworking(SDN)technology
SDNArchitecture
[11]isanemergingnetworkingparadigmthatgiveshopetochangethelimitationsofcurrentnetworkinfrast ructures.First,itbreakstheverticalintegrationbyseparatingthenetwork’scontrollogic(thecontrolplane)fr omtheunderlyingroutersandswitchesthatforwardthetraffic(thedataplane)
[33].Second,withtheseparationofthecontrolanddataplanes,networkswitchesbecomesimpleforwar dingdevicesandthecontrol logic is implemented in a logically centralized controller (or network operatingsystem1),simplifyingpolicyenforcement and networkre-configuration and evolution.
AsimplifiedviewofthisarchitectureisshowninFigure1.4.Itisimportanttoemphasizethat a logically centralized programmatic model does not postulate a physically centralizedsystem In fact, the need to guarantee adequate levels of performance, scalability, andreliability would preclude such a solution.Instead, production-level SDN network designsresort to physically distributed control planes The separation of the control plane and thedataplanecanbedonebyawell- definedprogramminginterfacebetweentheswitchesand
Northbound API (ex: RestAPI,…) SDN controller (ex: POX, Floodlight, ODL)
SDN Data Forwarding element (ex:
Network applications the SDN controller The controller exercises direct control over the state in the data planeelements via this well-defined application programming interface (API), as depicted inFigure1.4.Themost notableexampleof such anAPIis OpenFlow[34], [35].
SDNSouthboundAPI-OpenFlowProtocol
OpenFlow [34] [35] is the first and also the most widely known SDN protocol forsouthboundAPI,itprovidesthecommunicationprotocolbetweenthecontrolplaneonSDNcontrollera ndtheforwardingplanesonOpenFlowswitches.OpenFlowspecifieshow theseplanescommunicateandinteractwitheachothersincetheconnectionissetupuntiltheend.The OpenFlow protocol is layered above the Transmission Control Protocol, leveraging theuse of Transport Layer Security (TLS) The default port number for controllers to listen onis6653 forswitches thatwant to connect.
An OpenFlow switch has one or more tables of packet (Figure 1.5) handling rules (flowtable) Each rule matches a subset of the traffic and performs certain actions (dropping,forwarding, modifying, etc.) on the traffic Depending on the rules installed by a controllerapplication, an OpenFlow switch can be instructed by the controller behave like a router,switch, firewall, or perform other roles (e.g., load balancer, traffic shaper, and in generalthose of a middlebox) A flow-table contains several flow entries, each flow entry consistsof threemain parts:
- Match rule: this includes various fields to match on a packet: IP source address,
IPdestination address, MAC source address, MAC destination address, TCP sourceport address, etc A field can be left empty, which means any packets can matchwith this field.
- Action:thisactionisappliedtothematchpacket.Actionsincludeforwardingpackettoanother port, drop packet, etc.
- Stats: this part records the number of packet and byte that has matched with thisflowentry.Italsorecordsthedurationfromthestartingtimeuntilcurrent.Thisstatscomponenti susuallyused formonitoringand inmanagementfunctions.
When a packet arrives, it will be paired with the first matching flow entry in the flowtable If the packet is not matched with any entries, the switch will send anOpenFlowPacketInmessagetothecontrollerwhichwilltakeappropriateactionsafterwards.Afterthat,thec ontrollerwillsendanOpenFlowFlowModmessagebacktotheswitchinordertocreateanewentrymatchingt hispackettogetherwithsomeaction.Thatway,iflatersimilarpacketsgetinto theswitch, theswitch doesnot needto askthe controllerforfurtheraction.
SDNControllers
In Software-defined Networking, SDN Controller does exactly what its name suggests,controlling the network as the “brain” of network It has the global view of a network, withall information about the network topology, flow tables of the OpenFlow switches, etc.Using this information, the SDN Controller manages OpenFlow switches via southboundAPIs(e.g.OpenFlow)andleadstothedeploymentofapplicationsandbusinesslogic’above’vi anorthbound APIs.
The first developed SDN Controller is NOX which was introduced by Natasha Gude etal.in[36].Subsequently,otheropensourcecontrollerswerealsodeveloped,e.g.POX[37],Beacon [38],and Floodlight (forked from Beacon) [39] Later, multiple vendors such asCisco,IBM,HPE,VMwareandJuniperjoinedtheSDNControllermarketandeachofthempossessedthe irownproducts.FromBeacon,HPE,Cisco,andIBMControllershavemovedtowardsOpenDayligh t(ODL)[40].Despitebeingoneoftheearlycontrollers,andbeinglesspopularthanits counterparts,thePOXcontroller,written in Python,is stillfullyfunctional, easy to be grasped, installed and configured, that makes it ideal for academic researchers intheir experiment That also explains why this POX controller is selected in this dissertationfortheSDNarchitecture.
DifficultiesonNetworkEnergyEfficiencyandMotivations
Although the concept of network energy efficiency is not new, there are still issues inrealization of the energy-efficient network due to the inflexibility of a network and the lackofan energy-awarenetwork.Thesedifficulties aredepictedas follows:
- Inflexiblenetwork:First,cloudservicesup-to- datefrequentlyandleadtothechangeofnetworkinfrastructure.Onthecontrary,oneimporta ntpointofnetworksnowadays is the inflexibility issue Administrators should plan and prepare well forany changes in the network, which might require re-designing, re-configuring andmigrating In many cases, there is a big challenge for any developers to apply anynew approaches and evaluate them Consequently, the network flexibility is vitallynecessary Secondly, there are difficulties in evaluating the energy-saving levels ofnew energy-efficient approaches in a network due to the lack of the power-controlsystem of a network. Developers struggle when they propose and evaluate a newenergy- savingapproach.
- Cloud computing has been blooming in the last few years as a promising paradigmthatfacilitatesnewservicemodelssuchasInfrastructure-as-a-
Service(IaaS),Platform-as-a-Service(PaaS),Network-as-a-
Service(NaaS).Onthecloudcomputingenvironments,virtualization technique ssuch asnetworkvirtualization
[5] [6] [7] anddata center virtualization[8] [9] [10] have rapidly been developedand attracted much attention from industrial communities.Currently, virtualizationworksmainlyfocusontheresourceoptimizationandresourceprovisioning approaches[7]
[41],whilethereareonlyfewworksfocusingontheenergyefficiency.Oneofthemaindifficulties ofnetworkenergyefficiencyinvirtualizationtechnologiesisthelackofenergymeasurementmeth odofthenetworkinfrastructureincloudenvironments.Consequently,theimplementationof energy- awareplatforms,whichworkwellfornetworkvirtualizationanddatacentervirtualization,isani mportantandpromisingapproachintheenergyefficiencyareaofthenetworking.
Above difficulties as well as the potentials of SDN technology are great motivation fortheconstructionofSDN-basedenergy- efficientnetworkingincloudcomputingenvironments.Inthisdissertation,severalenergy- efficientnetworkingapproachesareproposedwithspecificalgorithmsand,equallyimportant,experi mentalresults.Thedetailedcontributionsaredescribed in thenext section.
Dissertation’sContributions
Proposinganenergy- awareandflexibledatacenternetworkthatisbasedontheSDNtechnology 14
In the ideal case for energy-efficiency, devices should consume energy proportional totheirtrafficdemand
(load).Thatis,energyconsumptioninalowutilizationscenarioshouldbemuchlowerthaninacaseofhig htrafficutilization.Theenergyconsumptionofthewholenetwork depends on the number of active network devices and their current working states.Consequently,understandingpowerprofileofnetworkdevicesisanimportantissueinordertocontributetoth eenergyefficientapproachandbuildapower- controlsystemofanetwork.Toachievethetarget,thefollowingworksareimplemented:
(1)profilinganenergyconsumption of a single network device as well as of the whole network; (2) constructing apower-control system for a network that allow administrator to monitor and control theenergystatesofeachnetworkdeviceaswellasthewholenetwork; (3)proposingtheenergy-aware routing algorithm that is based on the power profile of network devices; and (4)integratingthepower- controlsystemofnetworkdeviceswiththepowercontrolsystemofaphysicalmachine,andthenproposing aVMmigrationtechniquesfortheoptimizationoftheenergy consumption The detailed information of above contributions is described in thechapter2.
Proposingenergy-
e f f i c i e n t approaches in a n e t w o r k v i r t u a l i z a t i o n f or cloudenviron ments.
Recently, cloud computing has emerged in recent years as a promising paradigm thatfacilitates such new service models as Infrastructure-as-a-Service (IaaS), Platform-as- a-Service(PaaS),Network-as-a-
Service(NaaS).Forsuchkindsofcloudservices,virtualizationtechniquesincludingnetworkvirtualiza tion[5][6][7]anddatacentervirtualization [8] [9] [10]have been rapidly developed and attracted much attention fromthe research communities as well as the industrial market.
As for network virtualization, animportantquestionishowtorealizeandevaluateanenergy- savinglevelofnetworkvirtualization mechanisms in cloud environments The current lack of an energy-awarenetwork virtualization constitutes significant difficulties in deploying and evaluating theenergy- efficientn e t w o r k W i t h t h e s e a b o v e m o t i v a t i o n s , a n e n e r g y - a w a r e n e t w o r k
Proposinga n e n e r g y -
Beside the network virtualization, data center virtualization in cloud environments is anew trend of thecloudservices which aim to create several virtual data centers on top of aphysicaldatacenter.Amajorchallengeofnetworkvirtualizationindatacentersisthevirtualdata center embedding (VDCE) problem as solving VDCE is NP-hard For that reason,currentresearchmostlyfollowsheuristicandmeta- heuristicapproaches.Inthisresearch,theenergy- efficientdatacentervirtualizationisemphasizedwiththefollowingcontributions:
- Proposinganenergy-awaredatacentervirtualizationplatformandaddressingchallenges in providing energy-efficient VDCE These platform works under theconditionofdynamicVDCrequests,inwhichvirtualdatacenterrequestsarriveandleave the physical data center dynamically The evaluation results show that theperformance of conventional static VDCE algorithms is unstable and degradedunderdynamicconditions.
- Proposing a novel VDC embedding algorithm, namely HEA-E algorithm, with thefollowing objectives: (1) resource efficiency that deals with efficient mapping ofvirtual resources on substrate resources in terms of CPU, memory and networkbandwidth;and(2)energyefficiencythatdealswithminimizingtheenergycons umption of the virtual data center while satisfying the mapping demands. Theproposed VDC embedding algorithm is also integrated with new remapping andserverconsolidationstrategies,whicharedevelopedtoovercomethedynamicVDCmapping problem and to mitigate the complexity of the joint embedding migrationapproach Evaluation results show that our approach performs better than someexistingonesintermsofacceptanceratio,resourceutilizationandenergyconsumpt ion
For network energy efficiency, most efforts have focused onre- engineeringapproachesthat are applying on single network device [24] [25] [26] Although these approaches havegained good power-saving results, they only focused on saving energy of single device Infact, a cloud data center network (DCN) recently consists of thousand of devices and isdesigned with different topologies The traffic demands of a DCN continuously changeminute by minute and the DCN is typically provisioned for peak workload while runningwellbelowcapacitymostofthetime[42].Consequently,theperformanceofaDCNstronglydepends on the topology optimizing and traffic routing This property also helps improvingenergy efficiency in low traffic demand scenario by optimization a DC network topology,turning on the only necessary part and re-routing the traffic on this The remaining part ofnetworkcomponentsthenisputintothesleepingmodeinordertoreducepowerconsumption.
Fromthispointofview,acentralizedpower-controlsystemthathasmonitoring,topologyoptimizations and traffic routing abilities for a DCN is necessary Based on this system,several energy-efficient algorithms can be proposed and deployed with worthwhile powersavings and optimal performance effects. Consequently, the power-control system (PCS) ofaDCN is proposed withfollowingcontributions:
- Propose a power-control system that has following capabilities: (1) monitor theenergyconsumptionstatusaswellasitsefficiency;(2)controltheworkingstatesofthe devices due to the energy consumption of the system; and (3) implementingseveralenergy- efficienttopologyoptimization andtrafficroutingapproaches.
- Proposeanovelenergy-awareroutingalgorithmthatefficientlyworkswithdifferenttypes of network devices in term of power saving The algorithm routes a trafficdemandbasedonthepowerprofileofanetworkdeviceandalsobasedonthepower- scalingapproach.
- Integrate with server management for constructing the centralized power- controlsystemforboth servers and data center network.
This Section describes the related work to this chapter including a DCN technique andarchitecture; the energy model of a network device; and current existing energy- awarenetworkarchitectures.
DCN in a Data center creates the links among elements inside this network and providesconnectivity among them DCN architecture,which lays out network components andinstallsn e t w o r k t e c h n i q u e s w i t h i n a d a t a c e n t e r , i s u s u a l l y i m p l e m e n t e d f r o m t w o s u b - technical points First, selecting networking techniques inside a DCN, which satisfy thebandwidthdemandsandservicerequirements.Secondly,designinganetworktopologythatsatisfi estherequirementandbuildsacost-effectiveDCNtoscaleupthedatacenter.Sothatin this Section, the existing
DCN architectures models are described The suitable
DCNnetworkingtechniqueandtopology,thataresatisfiedtherequirementofbuildinganenergy- awarenetworkplatformin this dissertation, arealso presented.
For building power-control system of a data center, DCN should have the flexibility tomanage and upgrade its resources For example, DCN should quickly detect the novelnecessary topology that satisfies the traffic requirement and re-route the traffic onto thistopology, or DCN could quickly detect starved VMs and schedule residual resources, e.g.,migrating these VMs to an idle server with low overhead Both above examples require acentralizedand flexiblecontrol planetocoordinatethe DCN devices.
The traditional model for networking, despite being effective to the certain extent whereithastouseantiquatedmethodsofpassingdata,couldnotmeettheflexibilitylevelrequiredto deliver today's massive amounts of data Moreover, when the hardware and software arecoupled,thenetworkbecomesexpensivetomaintain,scale,andharderforuserstoinnovateandadmi nistrators to tuneapplications.
To address these issues, we are turning to Software-defined Networking technology(SDN) SDN services, typically controlled and monitored from centrally located sources,havetheglobalviewoftheentirenetwork.WithSDN,trafficflowismanagedwithsoftwareapplicatio ns, which are significantly more dynamic, being a solution for optimization andtuning which are not available in local management of switches and routers On the otherhand, scalability is easier to be achieved in SDN, since the software scales to as manyswitches or routers as there are in the network Adding hardware simply creates newpathways for the software to manage, monitor, and uses to create the most efficient trafficflow With a central SDN solution, the network routing also could be customized easier,shapingittothespecificinterestsandneedsofthatdatacenter.Byusingalgorithmstocreatea solution, SDN relies on OpenFlow, Puppet, and other protocols to remain agile, flexible,andcost-efficient.
Recently,researchworkshaveshownthattheDCNtopologyiscategorizedasahierarchicalmo del, recursivemodel, or rack-to-rackmodel (Figure2.1)[43][44].
Rack to rack Model Safida, Jellyfish Recursive Model Dcell, Bcube
Hierarchical Model Fat-tree, VL2
Hierarchicalmodelnetworkswiththeirelements(devices)arearrangedinmultiplelayersand characterize network traffic differently One of the most advantages of this model isreducing congestion within a network because an upper layer switch prevents an overloadof traffic that would otherwise all go through the same switch in a lower layer Three-tierarchitecture [45] [46] is the most widely deployed DCN architecture that follows a layeredapproach to arrange the network switches in three layers The network elements (switchesandrouters)arearrangedinthreelayersnamely: (a)accesslayer,(b)aggregationlayer,and
The legacy three-tier DCN architecture does not have the capability to meet the currentdata center bandwidth and growth trend The major shortcomings of the legacy DCNarchitecture can be expressed in terms of: scalability, cost, energy consumption, cross-section bandwidth, and agility [47] To accommodate the shortcomings of the legacy DCNarchitecture,new architectures areproposed bytheresearchcommunity.
Fat-tree [48] [49], one of the new architectures that was proposed by Al-Fares et al,consists ofcore,aggregation, andedgelayers(Figure 2.3).This architecture aims tomaximizetheend-to-endbisectionbandwidth.InaDCNwithFat- treetopology,switchesattheaggregationandedgelayersarearrangedinblocks,namelyPerformanceOpti mizedDataCenters(PODs),whichareresponsibleforroutingend-to- endcommunications.Coreswitches in Fat-tree topology simply maintain the connectivity among these PODs, so thatFat-treetopologyreducesthe traffic load over thecorelayer.
Another example of the hierarchical model is the VL2 based on DCN architecture [50].The architecture uses a flat automated addressing scheme that facilitates the placement ofservers anywhere in the network without configuring the address manually.
(a) automated addressing, (b) potential for transparent service migration, (c) load- balancingtraffic flowforhighcross-sectionbandwidth,and (d)end devicesbasedaddressresolution.
Thehierarchicalmodelismostly usedinDCNnetworkandshowsmanyrealisticevaluation results. Nowadays, since most of the commodity network devices support thesearchitectures that are used to connect the massive number of servers with each other.Facebook[51]andGoogle[52]aretypicalexamplesofusingthehierarchicalmodelintheirDCN whereboth of thesetechnologygroups use theFat-treetopology.
A recursive DCN consists of individual cells, each of which contains a single switch andnumber of servers, and each server bridges different cells In a recursive model, DCell [53]andBCube[54]aretypicalexamples of this model andimplemented in manyDCN.
BackgroundTechnologies
This Section describes the related work to this chapter including a DCN technique andarchitecture; the energy model of a network device; and current existing energy- awarenetworkarchitectures.
DCN in a Data center creates the links among elements inside this network and providesconnectivity among them DCN architecture,which lays out network components andinstallsn e t w o r k t e c h n i q u e s w i t h i n a d a t a c e n t e r , i s u s u a l l y i m p l e m e n t e d f r o m t w o s u b - technical points First, selecting networking techniques inside a DCN, which satisfy thebandwidthdemandsandservicerequirements.Secondly,designinganetworktopologythatsatisfi estherequirementandbuildsacost-effectiveDCNtoscaleupthedatacenter.Sothatin this Section, the existing
DCN architectures models are described The suitable
DCNnetworkingtechniqueandtopology,thataresatisfiedtherequirementofbuildinganenergy- awarenetworkplatformin this dissertation, arealso presented.
For building power-control system of a data center, DCN should have the flexibility tomanage and upgrade its resources For example, DCN should quickly detect the novelnecessary topology that satisfies the traffic requirement and re-route the traffic onto thistopology, or DCN could quickly detect starved VMs and schedule residual resources, e.g.,migrating these VMs to an idle server with low overhead Both above examples require acentralizedand flexiblecontrol planetocoordinatethe DCN devices.
The traditional model for networking, despite being effective to the certain extent whereithastouseantiquatedmethodsofpassingdata,couldnotmeettheflexibilitylevelrequiredto deliver today's massive amounts of data Moreover, when the hardware and software arecoupled,thenetworkbecomesexpensivetomaintain,scale,andharderforuserstoinnovateandadmi nistrators to tuneapplications.
To address these issues, we are turning to Software-defined Networking technology(SDN) SDN services, typically controlled and monitored from centrally located sources,havetheglobalviewoftheentirenetwork.WithSDN,trafficflowismanagedwithsoftwareapplicatio ns, which are significantly more dynamic, being a solution for optimization andtuning which are not available in local management of switches and routers On the otherhand, scalability is easier to be achieved in SDN, since the software scales to as manyswitches or routers as there are in the network Adding hardware simply creates newpathways for the software to manage, monitor, and uses to create the most efficient trafficflow With a central SDN solution, the network routing also could be customized easier,shapingittothespecificinterestsandneedsofthatdatacenter.Byusingalgorithmstocreatea solution, SDN relies on OpenFlow, Puppet, and other protocols to remain agile, flexible,andcost-efficient.
Recently,researchworkshaveshownthattheDCNtopologyiscategorizedasahierarchicalmo del, recursivemodel, or rack-to-rackmodel (Figure2.1)[43][44].
Rack to rack Model Safida, Jellyfish Recursive Model Dcell, Bcube
Hierarchical Model Fat-tree, VL2
Hierarchicalmodelnetworkswiththeirelements(devices)arearrangedinmultiplelayersand characterize network traffic differently One of the most advantages of this model isreducing congestion within a network because an upper layer switch prevents an overloadof traffic that would otherwise all go through the same switch in a lower layer Three-tierarchitecture [45] [46] is the most widely deployed DCN architecture that follows a layeredapproach to arrange the network switches in three layers The network elements (switchesandrouters)arearrangedinthreelayersnamely: (a)accesslayer,(b)aggregationlayer,and
The legacy three-tier DCN architecture does not have the capability to meet the currentdata center bandwidth and growth trend The major shortcomings of the legacy DCNarchitecture can be expressed in terms of: scalability, cost, energy consumption, cross-section bandwidth, and agility [47] To accommodate the shortcomings of the legacy DCNarchitecture,new architectures areproposed bytheresearchcommunity.
Fat-tree [48] [49], one of the new architectures that was proposed by Al-Fares et al,consists ofcore,aggregation, andedgelayers(Figure 2.3).This architecture aims tomaximizetheend-to-endbisectionbandwidth.InaDCNwithFat- treetopology,switchesattheaggregationandedgelayersarearrangedinblocks,namelyPerformanceOpti mizedDataCenters(PODs),whichareresponsibleforroutingend-to- endcommunications.Coreswitches in Fat-tree topology simply maintain the connectivity among these PODs, so thatFat-treetopologyreducesthe traffic load over thecorelayer.
Another example of the hierarchical model is the VL2 based on DCN architecture [50].The architecture uses a flat automated addressing scheme that facilitates the placement ofservers anywhere in the network without configuring the address manually.
(a) automated addressing, (b) potential for transparent service migration, (c) load- balancingtraffic flowforhighcross-sectionbandwidth,and (d)end devicesbasedaddressresolution.
Thehierarchicalmodelismostly usedinDCNnetworkandshowsmanyrealisticevaluation results. Nowadays, since most of the commodity network devices support thesearchitectures that are used to connect the massive number of servers with each other.Facebook[51]andGoogle[52]aretypicalexamplesofusingthehierarchicalmodelintheirDCN whereboth of thesetechnologygroups use theFat-treetopology.
A recursive DCN consists of individual cells, each of which contains a single switch andnumber of servers, and each server bridges different cells In a recursive model, DCell [53]andBCube[54]aretypicalexamples of this model andimplemented in manyDCN.
Dcell [53] is a server-centric hybrid DCN architecture where one server is directlyconnected to other servers (Figure 2.4) The Dcell follows a recursively build hierarchy ofcellsandeachserverinthisnetworkconsistsmanynetworkinterfacecards(NICs).InDCell,a server is connected to a number of other servers and switches via communication links,whichareaassumed to bebidirectional.
TheBCubenetworkarchitecture[54]isserver-centricapproachandcontainstwotypeofdevices: (1) server with multiple ports; and (2) switches that connect a constant number ofserver.BCubeisarecursivelydefinedstructure.ABCube 0is simplynserversconnectingtoan n-port switch ABCube 1is constructed from n BCube 0 sandnn-port switches. Moregenerically,aBCube k(k ≥1))isconstructedfrom nBCube k−1 sandnkn-portswitches.Eachserver in aBCube k hask + 1ports, which are numbered from level-0 to level-k It is easy toseethataBCube kha s N=nk+1serversandk+1levelofswitches,witheachlevelhavingnkn- portswitches.
Figure 2.5shows aBCube 1 withn = 4with 2 levels Source-based routing is performedusing intermediate nodes as packet forwarder which is ensuring, decreasing the hammingdistance between each consecutive intermediate host to the destination Periodic searchingfor the optimal path is performed in order to cope with any failures in the network One-to-all,all-to-oneandall-to- alltrafficcanalsoberoutedbyusingredundant(k+1)portsoneachhost.
Although most recursive DCNs architectures have the high scalability to allow DCexpansion and there are also cost-effective because of using cheap switches, they are notcommonlyseen in aDC.In particular,most oftherecursiveDCNsemployacomputationalserver as anetwork deviceandlackadequate fieldtestingof theirdesigns.
Due to the current occurrences of the bottleneck in the backbone links of two previousmodels, recent researchon DCN architecturesfocuses on making a direct connectionbetweendifferentracks,asopposedtosettingtrunklayers.In[55],aDCNtopology,namelyJe llyfish, uses a random graph to build end-to-end communication Instead of a fixedtopology,J e l l y f i s h n e t w o r k s i m p l y g u a r a n t e e s t h a t e a c h s w i t c h h a s p o r t s c o n n e c t i n g t o anotherswitchwhileremainingportsareusedtoconnecttoservers.In[56],L.GyarmatiandT.A.Trinhpropo sedarack-to- rackarchitecture,namelyScafida,whichaddresseslargenodedegreewithinaDCNtopology.Scafidaarchi tectureisscale-freetopologywherethelongestpath has fixed upper bound Scafida provides methodologies to construct such a topologyfor datacenterswhilemaking reasonablemodificationstooriginalscale- freenetworkparadigm Scafida consists of a heterogeneous set of switches and hosts in terms number ofports/links/interfaces.Thetopology isbuiltincrementally by adding anodeandthen,randomly connecting all the available ports to existing empty ports The number of ports islimitedbytheavailableportsonanode,unlikeoriginalscale- freenetworks.Suchanetworkprovideshighfault tolerance.
Power-ControlSystemofaDCNetwork
In this section, a power-control system (PCS) is proposed, which is extended fromElasticTree system [57] by adding a new module and a new function This system allowsadministratortomonitorandcontroltheworkingstateaswellastheenergyconsumptionofa network In the next sections, the energy modeling of whole DC network is depicted first,andthenthedetaileddiagramand componentsofthissystemaredescribedin details.
Intheidealcaseofenergyefficiency[58],devicesshouldconsumeenergyproportionallytotheirutiliza tion(Figure2.8).Thatmeans,energyconsumptioninalowutilizationscenarioshouldbemuchlowerthanint hecaseofhightrafficutilization.IntheFigure2.8,U(%)andP(%) are the utilization in percentage of a device and the power consumption in percentageof a device, respectively U = 100% means that a device is working in full resource state,andP0%means thatadeviceis consumingamaximum energy.
Recently, research communities [58] focus on answering an important question,how tooptimize the consumed energy volume by a device proportionally to its actual load. Theenergyconsumptionofwholenetworkdependsonthenumberofactivenetworkdevicesandtheir current working states Consequently, understanding power profile of switch is animportant research point, which leads to an energy efficient approach and the establishmentof power-management system of a network As an initial step to understanding the energyconsumption patterns of a variety of networking devices, a detailed power instrumentationstudyis conducted.
In [59] Priya et al developed a power model to estimate the power consumed by anyswitches.Thelinear power model of aswitch,P sw , is defined:
Where:𝑃𝑐ℎ𝑎𝑠𝑠𝑖𝑠is the power consumed by the switch’s chassis;𝑃𝑙𝑖𝑛𝑒𝑐𝑎𝑟𝑑is the consumedpower oflinecardwith no active ports; and𝑛𝑢𝑚𝑙𝑖𝑛𝑒𝑐𝑎𝑟𝑑i s a n a c t u a l n u m b e r o f c a r d s t h a tare plugged into the switch Variableconfigin the summation represents the possibleconfigurations for working speeds of ports,𝑛𝑢𝑚𝑝𝑜𝑟𝑡𝑐𝑜𝑛𝑓𝑖𝑔(𝑖)and𝑃𝑐𝑜𝑛𝑓𝑖𝑔(𝑖)are number ofportsand powerconsumed byaport runningat workingspeedsi,respectively. d,
In another reference [58], Pham et al proposed an energy model of a NetFPGA- basedOpenFlowSwitch[60].ThismodeliscontributedforpowerscalingmethodintheNetFPGAcardwhic hmakesthisswitchbecomeenergy-aware.TheNetFPGAisthelow-costreconfigurable hardware platform optimized for high-speed networking The NetFPGAincludes all of the logic resources, memory, and Gigabit Ethernet interfaces which arerequired to build a complete programmable switch, router, and/or security device. Becausetheentiredatapathisimplementedinhardware,thesystemcansupportback-to-backpacketsat full Gigabit line rates and has a processing latency measured in only a few clock cycles.Thelinear modelcan beformallyexpressed as follow:
N100,N1000:numberofportsinlinkstateIdle,10Mbps,100Mbps,1Gbps,respectively
- P0,P10,P100,P1000:powerconsumptionofportsinlinkIdle,10Mbps,100Mbp s,1Gbps,respectively
In the other work, reference [61] provided an energy model that also divided the energyconsumption of port into two part including static consumed power dynamic consumedpower These static and dynamic sub-powers are denoted asPport sand Pportrespectively.Thisenergymodel is defined as follows:
- P sw :thepowerofoneswitch,includingpowerconsumedbyboththechassis,the line-cards and the ports;
- din j :the data rateof theincomingflowat portj;
Aswecansee,therearefewmodelingmethodsfortheenergyconsumptionsofnetworking devices These methods have many similarities in components, so that in thisdissertation,theabovemethodsaresummarizedandageneralenergymodelisproposedfora network switch that is used in the completely analytical model The general model isdescribedas follows:
Value𝑃 𝑠𝑤 denotes the power consumption of the whole switch;𝑃 𝑠𝑡 is a static (baseline)power consumption of a switch (same as𝑃 𝑐ℎ𝑎𝑠𝑠𝑖𝑠 𝑜𝑟 𝑃 𝑠𝑡𝑎𝑡𝑖𝑐 in the eq 2.1, 2.2 and 2.3),excludingenergyconsumedinitslinecards;𝑛 𝑝 denotesthenumberofportsworkingatstatepwhile
𝑃 𝑝 isthepowerconsumptionofaportworkingatstatep;𝑃̇d e n o t e sasetofworking speeds of a port Currently, there are many working speeds of a switchport such as 40Gbps,10Gbps,1Gbps,100Mbps,andidle.𝑃 𝑒𝑥𝑡 denotesanextensionconsumedpower.Forexamp le𝑃𝑒𝑥𝑡isPFPGA-Corei n c a s e o fGigabit NetFPGA-basedswitch.
1Gbps, 100Mbps,andidle.The consumed energy of each port’s state isdifferent Consequently, the energy consumption of a network is calculated as the totalconsumed energy of all switches with their states including ports’ forwarding speeds. Fromthegeneralmodelofoneswitchintheequation(2.5),theenergymodelofwholenetworkisdescribed moredetail inbelow equations:
Inthisdissertation,thepower-controlsystem (PCS)isproposed,whichisextendedfromElasticTree system [57] In ElasticTree, Heller et al proposed to use monitoring protocolsimple network management protocol(SNMP) [62] as an exchanged protocol for switchcontrollingandmonitoring.AlthoughSNMPis a worldwideprotocolthatisusedintermofnetwork monitoring, it is still inflexible protocol and has many limitations interm ofcontrolling network in real time To due with these limitations, theSDN-based PCS systemis proposed with several extensions These extensions of PCS from ElasticTree system aredescribedas:(1)extendingthepowercontrolmoduleforsupportingtheOpenflowprotocol,thecoreprotocolofSDNt echnology.Implementingt h e OpenFlowprotocolonboth
SOFTWARE-DEFINED NETWORKING CONTROLLER OptimizerRouting
Optimize topology based on current traffic and energy condition
Route the current flows based on optimal topology
Traffic state, Full-mesh and MST topology
(SDN switches, DCN topology, Device Power Profile)
Openflow protocol SSL/TCP Openflow protocol SSL/TCP controller and switchesmakes a seamless protocol for controlling andmonitoring thenetwork; (2) adding themonitoringmodule for real-time monitoring the network state andtraffic byusingOpenFlowprotocol.
The PCS architecture’s diagram is depicted inFigure 2.9.The data center networkconsistsofallSDNswitcheswiththeirconnections,networktopologyandthepowerprofileof switches The SDN controller of this PCS consists of four main modules, namely:monitoring,optimizer,routingandpowercontrol.Themonitoringmodulecollectsinforma tionfromaDCNandvisualizesitstrafficstate,topologyandcurrentstatusofenergyconsumption.
Theoptimizermodule finds the most energy-efficient subnet that satisfies thecurrent offered traffic based on the traffic flows, the topology and the energy-profile of aswitchoftheDCN.Afterthiscalculation,theoptimizermoduleoutputstheactivetopology,contains active devices and connections among them, to theroutingmodule andpowercontrolmodule.Afterwards,thepowercontrolmodulechangesthepowerstatesofswitches,linec ards,and interfaces,whereas theroutingmodulechooses thepaths for allflows.
Figure2.9:Power-controlSystem of aNetwork
TheSDNcontroller,whichcontainsalloptimizer,routing,monitoringandpowercontrolmodules, communicates with a DCN via the secure channel, known as OpenFlow protocol.Thedetailed descriptions ofDCN and
Aswementionedabove,therearemanyadvantagesofFat- treenetworktopologyinaDCnetwork.Accordingly,inthisdissertation,theDCNnetworktopologyoftheP CSistheFat-tree topology AkFat-tree is a DC network architecture with three layers which areedge,aggregationandcoreusingk-portswitches There arekPODs and each POD containsk/2edge switches andk/2aggregation switches Each k-port switch in the edge layer uses k/2portswhichdirectlyconnecttok/2serverswhileremainingk/
𝑓𝑢𝑙𝑙 aggregation layer of the hierarchy There are(k/2) 2 k-port core switches Each core switchhas one port connected to each of k PODs Thei th port of any core-switches is connected toPODiso that consecutive ports in the aggregation layer of each POD switch are connectedtocoreswitches on(k/2)strides.
EachEdgeswitchconnecttok/2servers,sothatnumberofserversthatkFat-treetopology supports is:𝑛
Consequently, the power consumption of the whole fat-tree network,𝑃 𝑁𝑊 , when allswitchesareturned-onwiththeirportsrunningatmaximumspeeds(fullstate)isdefinedas:
Where𝑃 𝑠𝑡 isastatic(baseline)powerconsumptionofaswitch,𝑃 𝑒𝑥𝑡 denotesanextensionconsumed power, and𝑃 𝑃𝑚𝑎𝑥 is a power consumption of port in a maximum working state(speed).
In order to minimize the energy consumption of a network, a minimum spanning tree(MST) topology is used which puts a part of the data center network in sleep mode inunderutilizedloadsituation.Inacaseofnotrafficdemand,theDCnetworkmaintainsaMST forminimumconnectivitybetweenservers.AsdepictedinFigure2.10,atinitialphasewhenthereis no trafficdemand amongservers, then:
- onlytheleftmostcoreswitchSw core a n dtheleftmostaggregationswitchSw agg a r eturnedon with theirnetwork interfaces runningatthe lowestoperatingspeed;
- all accessswitchesSw acc areturnedon andrunat thelowest operatingspeed.
Inkfat-treetopology,theremainingofMSTworkingtopologyare:onecoreswitch;k aggregations w i t c h ( o n e f o r e a c h P O D ) ; a n d 𝑘 2edges w i t c h e s w h i c h w o r k i n t h e l o w e s t
Thenumberoflowest port(the10Mbps is usedasthe lowestoperatingspeedof aport):
(2.17) Then,an energyconsumption ofthe networkinMST workingtopologyis described as:
In the diagram of the power-control system, the SDN controller is built as the centralcomponent which monitors and controls all DCN devices The SDN controller performsdifferent functionalities such as: collecting DCN information; network monitoring; routinganddefiningflowtablesforOpenFlowswitches;executesoptimizationalgorithmforenergyefficienc y As we mentioned above, the SDN controller is extended from the POX, whichsupportsenergy- awarefunctionalities.TheFigure2.9illustratestheSDNcontrollerwithitscomponents:Optimizer,Moni toring, Power Control, and Routing.
GreenDataCenterusingcentralizedPower- controloftheNetworkandservers 39
Although the algorithm and the system that are presented in Section2.2and Section2.3areenergy- awareandworkwellwithseveralnetworkdevices,theyonlyfocusontheenergyconsumptionofnetworkinord ertosatisfytherequirementoftrafficdemandamongservers.Fromthetechnicalpointofview,iftheVMswithi naDCcanbeallocatedtophysicalservers
Algorithm (Idle logic + Topology-aware live migration)
State of servers (server’s state, VMs) NETWORK ROUTING MIGRATION
State of Networking devices State of Networking devices
DATA CENTER NETWORK DEVICES AND SERVERS
NETWORK Control state of switches and links (turn on/ off)
POWER CONTROL+ based on the states and topology of the network, the traffic demand among these VMs willbeeasier-to-be- feasible.Consequently,integratingtopologyoptimizationwithVMsallocationandmigrationpro cesswillbepromisinglyapproachfor theenergyefficiencyinadatacenter network.
This section presents the proposed integration idea which will be explained in detailsshortly In the fat-tree topology of a data center, these are near, middle, far,andmixtrafficscenarios(seethesection2.1.1.2.4).Theneartrafficflowhasitssourceanddestinat ionthatconnect to the same edge switch so that the exchanged traffic traverses over only edgeswitches Besides, in the far traffic scenario, the source and destination of a flow resides indifferent PODs, so in this situation the flow traverse over edge, aggregation and coreswitches As a result, the system in near traffic scenario only needs to turn-on only edgeswitchandhencethetrafficdemandiseasier-to-be-feasiblethanitcouldbeinthefartraffic.From this point, if VMs placement and migration processes are also aware of the networktopologyand state, theenergy-savinglevelof thenetworkcan be also increasing.
Inordertorealizetheaboveidea,thePower-controlsystem(PCS),which ispresentedinSection2.2,isextendedbyaddingtheserverandVMmonitoring+moduletothearchite cture.Thismonitoring+moduleprovidesthemonitoringabilityoftheworkingstatesandtheenergycons umptionofphysicalserversaswellasthecurrentstateofallVMsintheDCN The novel simple diagram, is described inFigure 2.16,is addedserverandmigrationsub-modules In this architecture, the extended power control system (Ext-PCS) managesandcontrols both serversand network entities ofadata center.
Figure2.16:ExtendedPower-Controlsystem(Ext-PCS)
Ext-PCS consists of four logical modules, known asoptimizer+, monitoring+, powercontrol+andconfiguring+,which are shown inFigure 2.16.The role ofoptimizer+is tocalculatethemigrationplanandfindtheenergy-efficientnetworksubset,whichsatisfiesthe conditions of servers Its inputs are a topology, the state and the power models of switchesandservers.Theoutcomesoftheoptimizer+aretheservermigrationplanandasetofactivecompo nents (servers and network devices) which will be sent to bothpower control+andconfiguring+modules.Power control+changes the power states of switches and servers,whileconfiguring+module implements the server migration and chooses the paths for thewholenetworkin datacenters. Thefollowing workflow shows theoperation of this systemin details.
- Step1:Monitoring+identifiesthenetworkstates(on/offswitchesandlinks),trafficflows, servers’ states (active/inactive) and VMs states After that,monitoring+modulesends this information tooptimizer+module.
- Step 2:Optimizer+runs the proposed topology-aware VM placement algorithm,which will be presented in the section2.4.3, to identify the migration plan andnetwork configuration.Optimizer+sends the results of its algorithm to bothpowercontrol+andconfiguring+modules.
- Step3:A f t e r r e c e i v i n g r e s u l t s f r o m t h eo p t i m i z e r + m o d u l e ,t h ec o n f i g u r i n g + implements routing algorithm to route the traffic flows Then this module migratesVMsfollowingtheVMmigrationplan.Theconfiguration+sendstheconfirmationsig naltopower control+moduleit finishes routingand migratingprocesses.
- Step 4: Thepower control+module changes the network and servers state when itreceivestheconfiguration signal from theconfiguration+.
ThisSectiondescribesanexampleoftheproposedplatform.Inputsofthissystemarethepower model of devices, traffic demand and VMs distribution Resources of each VM areCPU and memory (RAM) According to [66], the state of VMs is divided into two states,namelyactiveVMandinactiveVM.
- ActiveVM(handlingtasks): bothCPU andRAM arebeingused.
Figure2.17isanexampleofapartofthedatacenter.Inthisexample,thenetworkiskOat- treetopologywhichsupports16physicalservers.EachphysicalserverhoststhreeVMswith their corresponding requirements of CPU and RAM (The requirement of the system isdescribedinTable2.4).ThesystemmaintainstheMST(MinimumSpanningTree)ofthe
VM01VM03VM05 VM02VM04VM06
VM07VM09VM11VM13VM15 VM08VM10VM12VM14VM16
VM01VM03 VM02VM04 VM09
Fat-tree topology that was described in the section2.2.2.2 Also in this example, the red- color switches are active switches which are under the exchange of traffic flows while thegreen-colorswitchesare idleswitches.
On the one hand,Figure 2.18shows the migration strategy of the popular VM migrationalgorithm,alsoknownasfirst-fitalgorithm[67].ThisFirst- fitalgorithmfocusesonminimizing number of active physical servers The VMs allocation plan is described infollowingsteps:VM07to S8(server8),VM08 toS6,VM09 toS2,VM10toS7,VM01toS3.After the process, two physical servers S4 and S5 can be turned-off for energy saving Alltheswitches keep turnedon to maintainthetrafficbetween servers.
On the other hand,Figure 2.19shows proposed idea and algorithm The topology- awareVMmigrationalgorithmisdeployedwhichallocatetheserversbasedontrafficdemandandnetwork topology The allocation plan is:VM14 to S3, VM09 to S1, VM10 to S2, VM11 toS4,VM12toS8.Aswecansee,afterthere-allocation,theserversS5,S6andtheedgeswitch
VM01VM03 VM02VM04 VM09VM10
In the Fat-tree topology with MST working topology, which is presented in the section2.2.2.2, the number of switches that could be turned off depends on the required number ofservers connecting to switches In other words, the fewer switches and PODs, which areconnectingtotheworkingservers,themoreswitcheswhichcouldbeturned-off.Figure2.19shows that switch E3 can be also turned-off by using the proposed topology-aware VMmigrationalgorithm.
The constraints of this algorithm are quite similar and focusing on the resources of thedevices Each physical server hosts many virtual machines (VMs) So that the resources’requirement of hosted VMs must be less than the physical capacity, RAM and CPU, of theserver.Theseconstrainsaredepicted in the Eq.(2.25and Eq.(2.26
The bandwidth demand off all VMs that are hosted on a server must be less than thenetworkcapabilityof theserver.This constrainis presented in Eq.(2.27.
In this dissertation, the proposed topology-aware VM migration algorithm migratesserverswithtwoobjectives:
(1)minimizethenumberofphysicalservers;and(2)reducethenumber of switches being used for interconnecting of these physical servers The inputs ofthis system are traffic matrix, VMs and servers states The idea of this algorithm is toaccumulate physical servers that are hosting VMs to fewer PODs and edge switches, andthen either turn off more switches or put them to their low working state for the energyefficiency.
Each migration process contains a source server, the server hosts current VM, and thedestination server, where the VM must be migrated to Firstly, the algorithm builds a list ofsource servers,𝐿𝑠𝑟𝑐 These source servers are sorted in order of increasing number of activeVMs hosting in servers (line 4 -Pseudocode 1:) Then the servers, which are hosting thesame active VMs of𝐿𝑠𝑟𝑐list, are re-sorted again by prioritizing the position to neighboringserver:near→middle→far(line4-
Pseudocode1:).Thelistofdestinationservers,𝐿 𝑝 ,issortedinorderofdecreasingnumberofactiveVMsth atstayonservers(line8-Pseudocode1:).
After that, all possible active VMs, that can be migrated from𝐿𝑠𝑟𝑐𝑡 𝑜 𝐿 𝑝 b y u s i n gbubble sortalgorithm,are found The𝑚𝑖𝑔: 𝑣𝑚𝑖→ 𝑆 𝑝 condition is checked by constraintswhich are described in the equations(2.25),(2.26)and(2.27) The migrating process forinactiveVMs is similar to thesame process in active VMs.
5.//Alltheserver withthe same active VMs is re-sortedbynear →middle→far
7.//Createalist ofdestination server bydecreasingnumberof activeservers
𝑠𝑡𝑎𝑡𝑒(𝑆 𝑝 ,𝑡)ofaserverwhetheri th serverturnedon(1)orturnedoff(0).TheUtilisdefinedasthepercent ageofaserverutilizationintermofCPUorMemory.Theenergyconsumptionof a server at timetis defined in below equation, where γ and δ are energy coefficient andbaselinepower, respectively.
The power cost of migration comprises of the power used by the source physical server,which starts the migration; and the power used by the destination server All the cost iscaused by the increase of server resources (CPU, memory) and I/O resource
(network) Themigration energy consumption is calculated as the sum of an energy consumption of thesourceserverand the destination server.
C# based Ext-PCS simulator is deployed which investigates the energy performance ofdata centers under different traffic conditions, VM requirements, network topologies andenergy models of devices Moreover, the tool allows us to implement and analyze differentenergy- awareoptimizationalgorithmsandVMmigrationstrategies.Forperformanceevaluation, thescenario generatormodule is built, which randomly creates a scenariofollowingtwosteps:
- VMsdistribution:with therequirements ofCPU &RAM.
- Traffic flows between the active VMs: The traffic matrix is generated according torealistictrafficdistributions[42].Thebenefitofthistraffic generatoristhatithelpsus providing an appropriate energy-saving approach in favor of a specific trafficpattern.
Conclusion
Inthissection,theextendedpower-controlsystem(Ext-PCS)ofaDCNisproposed.Thesystem is awareness of energy consumption and has the ability to support administrators inmonitoring,controllingandapplying severalenergy-efficientstrategiessuchaspowerscaling, power scaling with energy-profile-aware algorithms The chapter also presents twomainenergy-efficient approaches including:
- (1) the energy-aware routing algorithm, namelypower scaling and energy-profile- aware(PSnEP),whichisbasedonpowerscalingandbasedonthepowerprofileofnetworkdevi ces.TheexperimentalresultsshowthattheproposedPSnEPalgorithmeffectively reduces the energy consumption of the network and works well withseveral networking devices The energy-saving level increases up to 41%, which ismoreefficient than thecommon power scalingalgorithm;
- and (2) topology-aware VM migration algorithm which could migrate the serversfor two goals: (a) minimizing the number of physical servers; and (b) reducing thenumber of switches used for interconnections of these physical servers in order toturn-off more devices for energy efficiency The most significant advantage of thisalgorithm is that the migration process saves the energy consumption of servers asmuch as other migration (first-fit) while reducing the energy consumption of thenetwork devices The experimental results show that the consumed power of thenetwork devices can be saved up to 46%,which is significant compared to the full-meshscenario,while the energy- savinglevel oftheserversremainsunchanged.
Asdescribedinchapter2,theproposedpower-controlsystemworkswellinadatacenternetwork and satisfies a rapid growth in the number of DC servers as well as the number oftheInternetservices.Incloudcomputingenvironments,manyservicesmodelssuchasIaaS,NaaS, PaaS have emerged in the last few years as a promising paradigm For such kinds ofthese services, virtualization technologies includingnetwork virtualizationanddata centervirtualizationhave quickly developed In this chapter, the energy efficiency in the networkvirtualizationtechnologyistakenintoaccountwhiletheenergyefficiencyinthedatacentervir tualizationwillbepresentedinthenextchapter.Firstly,networkvirtualizationispresented as a technology with huge potential [5] [6] [7] in term of green networking.Network Virtualization (NV) allows multiple separate Virtual Networks to be run on thesame physical substrate network From a theoretical perspective, network virtualizationshoulddealwithis,howtomapavirtualnetworkontopofthephysicalinfrastructurebase donspecificrequirementsandconstraints.
- One major challenge of network virtualization is, thevirtual network embeddingproblem that deals with efficient mapping of virtual resources on substrate networkresources[69].Tobespecific,anefficientutilizationofphysicalnetworkresourcesstron glydependsonthevirtualnetworkembeddingalgorithmsundersuchconstraintslikenode, linkresources,admissioncontrolrequestandsoforth.Solvingthe virtual network embedding (VNE) problem is NP-hard, as it is related to themultiway separator problem [70] For that reason, current research mostly followstheheuristicand meta-heuristicapproaches.
- Furthermore, virtual network embedding research often focuses on a virtual nodeand virtual link embedding in combination as well as the optimization approach ofthe NV resource allocation To the best of our knowledge, there are only fewresearchstudieswhichaddresstheenergy- efficientVNEproblem.Themainreasonis the lack of energy-aware NV platform, which allows researchers to develop newNVpowerefficiencyapproaches,evaluatetheirperformanceas wellas efficiency.
Thecombinationofmoreadvancedtechnologies,suchasSDNandnetworkvirtualization, enables the realization of a programmable and flexible network.
Moreover,optimizedvirtualizationtechnologywillalsoconstitutelessdependenceonenergyconsu mption.TheimplementationofnetworkvirtualizationwithSDNtechnologywilloffernotonlyanintegr atedorchestrationexperience,butalsoaunificationofsubstrateandvirtualinnovations.FlowVisor[71] [72]isoneofthemostsuccessfulSDN-basednetworkvirtualization layers, widely used in Network Virtualization testbeds such as GENI [73],Ofelia [74] and OF@TEIN [75] In Future Internet Lab - HUST, we also deployed aReServNetPlatform which is based onFlowVisorplatform.
Architecturally, FlowVisor acts as a transparent proxy and it is mentioned as one of thefirst hypervisor-like virtualization architectures for network infrastructure, resembling thehypervisor model that is common for computing and storage (Figure 3.1) Network devicesgenerateOpenFlowprotocolmessages,whichgototheFlowVisorandarethenroutedtotheappropriate OpenFlow controller bynetworkslice.
From the implementation perspective, although there is a common consensus in thenetwork research communities that SDN is likely to be the technology to introduce newvirtualization concepts, there is still a gap between theory and practice in the SDN- basednetwork virtualization An important question is how to realize and evaluate the energy-saving level of network virtualization mechanisms by using SDN in real cloud computingenvironments.Thecurrentlackofanenergy- awarenetworkvirtualizationplatformconstitutessignificantdifficultiesindeployingandevaluatingth enetworkenergyefficiency.With these above motivations, the concept of energy-aware network virtualization withpower of energy monitoring and controlling is proposed in this section The contributionsaredescribedas follows:
- Constructing an Energy-Aware Network Virtualization (EA-NV) platform in cloudenvironments.WithinputsareVirtualNetworkRequests(VNRs),thesystemperforms separate VNE algorithms and evaluates their performance as well aspower-savinglevel.
Efficient(HEE)VNEalgorithmandReducingMiddlenodeEnergy efficiency (RMN- EE)VNE algorithm.These proposed algorithms increaseenergy-saving level while maintaining a reasonable resource optimization, knownasacceptanceratio.
The rest of this chapter is organized as follows.Section3.1provides the backgroundknowledgeofnetworkvirtualizationandtheconceptofvirtualnetworkembedding.Sect ion
3.2presents the construction of energy-aware SDN-based network virtualization platform.The modeling and problem formulation is described in Section3.3. Sections3.4and3.5provide the proposedenergy-efficientvirtualnetwork embedding algorithmsandtheirperformance evaluation,respectively.ThelastSectionconcludesthechapter’swork.
Network VirtualizationandVirtualNetworkEmbedding
Networkvirtualizationisahighlyflexibleandcost- effectivetechnologythatsatisfiesthecontinuouslyrisingdemandfortheInternetservicesofthecurrentnet work.NVprovidesanabstraction of coexistence of multiple virtual networks on the same physical substratenetwork The most significant advantage of this technology is being able to share resourcesamongheterogeneous logical virtual networks.
Figure3.2:Exampleof a virtual networkontop of aphysicalnetwork
In network virtualization, a virtual node might map to several substrate nodes, and thevirtual links connected among these nodes are spanned to several substrate links. Thismapping process is known as a VNE, which aims to reach the maximum exploitation ofsystem resources This optimality can be computed in regard to different objectives such asrevenue, energy efficiency, QoS, economical profit…etc Usually, the VNE problem isdividedintotwodifferentsub- problems,namelyVirtualNodeMapping(VNoM)andVirtualLinkMapping(VLiM)[5]
[6].VNoMmapsvirtualnodesontosuitablesubstratenodes.Onevirtualnodemayhavevariousmappingsol utionsandvirtualnodesunderdifferentVNRcanmapontothesamesubstratenode.WhentheVNoMprocess isfinished,VLiMprocessmapsvirtualrequestedlinksontosubstratelinks.Onevirtuallinkbetweentwovirtu alnodescouldspanseveral substrate links.
System’sDiagram
NetworkRequests(VNRs)Ex:VNR1&VNR2
Slicing, port mapping, Switches, Controllers coordinating and configuring
Extended FlowVisor Power Power monitoring: monitor NW power consumption Power controlling: Control NW devices' state
Substrate network (Openflow-enable Switches)
VNR1(nodes&linksdemand) VNR2(nodes&linksdemand)
Figure 3.3shows the proposed diagram of the Energy-aware network virtualizationsystem There are four blocks, namelyManagement; OpenFlow Controllers;
ExtendedFlowVisor;andSubstrateNetwork.InputsofthissystemareseveralVNRsincludingvirtual nodeswiththeircapacitydemandandvirtuallinkswiththeirbandwidthdemand.Withtheseinputs,Manage mentblock uses thevirtual network embedding(VNE) module to solve theembedding problems and then, sends the embedding results to theExtended FlowVisorforvirtual network slicing and power controlling For each VNR,Managementalso creates avirtualcontrollerbyusingcontrollermanagementsub- block.TheSubstratenetworkconsistsofOpenFlow- enabledswitches,whicharecontrolledandmonitoredbytheextendedFlowVisorblock.Eachvirtu alnetworkthatisontopofthesubstratenetworkcanconnecttoits SDNcontroller bySlicermodulevia theOpenFlowSSLsecure channel.
System’sworkflow
1) Step 1 - Identifying inputs:Managementblock identifies the set of VNRs and asubstratenetworkwhichcontain followinginformation:
(1) asetofvirtualnodeswiththeirparameterswhichareCPUandmemory;(2)linksbetween the nodes with their bandwidth capacity demands; and (3) its controllerinformation if necessary (controller type: POX or Floodlight) The VNR can beimplemented either as a virtual SDN request with the SDN controller ability or asimplenetworkrequestwith theconnectionamongits nodes.
- The substrate network contains the physical network information including allphysical nodes with their configurations and capacity (the CPU, memory of thenodesand bandwidthof thelinks).
2) Step 2 - Virtual Network Embedding: After receiving a set of arriving VNRs, theVNE sub-block maps the requests based on the available capacity of the system andbased on the selected VNE algorithm In this phase, any VNE algorithms can bedeveloped and evaluated flexibly The mapping results of embedding processes,virtual node mapping (VNoM result) and virtual link mapping (VLiM result), aresent to theOpenFlow controllermodule viacontroller managementsub-block andtohypervisor- likeextended FlowVisorlayer.
3) Step 3- Slicer andPower control:TheextendedFlowVisorconfiguresvirtualnetworks by creating flowspaces and allocates resources to them by usingSlicersub-block.
Besides, based on the VNE results, thepowersub-block controls thesubstrate network by sending extended OpenFlow messages to change the states ofsubstrate nodes and links Thepowersub-block keeps on monitoring the energystatesofthenetworkdevicesandsynchronizesthisinformationwiththeManagem entblock.
4) Step4-ConfiguringOpenFlowControllers:Basedonthesuccessfulmappingresultsof each VNR, theOpenFlow Controllerblock configures corresponding controller.This controller is created and connected to its virtual networks via the extendedFlowVisorlayer.
5) Step 5: - Finishing and monitoring: Power consumptions and states of the substratenetwork are monitored byPowerandGUIsub-blocks The stats could containfollowinginformation:
ModelingandProblemFormulation
This Section presents the modeling of the virtual network embedding process with allrelatedinformationandaspects.ThisVNEisformulatedwithitstheobjectiveandconstraints.
A weighted graphG S (N S ,L S )denotes a substrate network whereL S is a set of substratelinks andN S is a set of substrate nodes Each substrate node𝑛 𝑆 ∈ 𝑁 𝑆 has available (i.e.,leftover)
CPU capacity which is denoted as𝐶𝑐𝑎𝑝(𝑛 𝑆 ),and each substrate link𝑙 𝑆 ∈ 𝐿 𝑆 has itsavailable bandwidthcapacity𝐵𝑐𝑎𝑝(𝑙 𝑆 ).
Thevirtualnetworkrequesti th isdefinedas𝑉𝑁𝑅 𝑖 (𝑁 𝑖 ,𝐿 𝑖 )∈𝑉𝑁𝑅 (𝑁 𝑅 ,𝐿 𝑅 ).Where𝑁 𝑖 ,𝐿 𝑖 aresetsofvirtualnodesandvirtuallinks,respectively.𝐵𝑐𝑎𝑝(𝑙 𝑆 𝑆
𝑛 ℎ ,𝑛 𝑘 ℎ 𝑘 defined as a set of virtual nodes that are successfully mapped onto substrate node𝑛 𝑆 ∈𝑁 𝑆 And𝑉𝐿(𝑙 𝑆 )isdenotedasasetofvirtuallinksthataresuccessfullymappedontothesubstrateli nk𝑙 𝑆
N i Setof virtual Nodes in VNR i
V N (nS) Setof virtual nodes which aremapped on substratenoden S
V L (lS) Setof virtual linkswhich aremapped ontosubstrate linkl S
Eachp ϵ P(a S ,b S )is a set of substrate links Then the maximum bandwidth of eachpath,𝑅 𝑝 ,is defined as the available bandwidth of a substrate link that has the lowestbandwidthofavailablecapacity.
The two functions𝑓𝑖, 𝑦𝑖in the below equations are the mapping functions of virtual nodemappingand virtual link mapping, respectively.
Thep o w e r s t a t e𝑃 𝑠𝑤 o fs u b s t r a t e n o d e𝑛 isg i v e n bytheequation Wh er et he bi n ar y n i 𝑖
(1)allsubstratenetworknodesuchasswitches,routers;and(2)allthelinksthatinterconnectthese switchesandrouters.Theobjectiveisdefinedasinthe(3.7. min∑𝑠 𝑡 𝑎 𝑡 𝑒 n(𝑡)×(𝑃 𝑠𝑡 + ∑
𝑁𝑛𝑒𝑖(𝑛 𝑖 )isd e n o t e d as a s e t o f ne ig hb or in gn od es o f vi rt ua l n o d e
𝑛 𝑖 ∈ 𝑁 𝑖 ← 𝑉𝑁𝑅 𝑖 A VNR is composed of a set of virtual segments that normally include acouple of virtual nodes and the virtual link between these nodes Then the virtual nodeconstraintisthatthevirtualnodesofthesamesegmentmustbelocatedondifferentsubstratenodes(Equ ations(3.8)and(3.9)).
Forthel i n k ma pp in g function𝑦𝑖:𝑙 𝑖 → 𝐿 𝑁 ,a virtual l i n k𝑒 𝑖 𝑖𝑖 c a nma po nt os ub st ra te
(leftover)bandwidthofsubstratelink.Consequently,thelinkmappingconstraintisd escribedas inEquation(3.10.
Each virtual segment of a virtual request has the corresponding load for the physicalinfrastructure.Anexpressionfortheloadofthei th virtualnetworkrequest𝑉𝑁𝑅𝑖isgivenbyth eequation:
- 𝜔,𝜓arethefactorsfortheCPUcapacity ofvirtualnodesandbandwidthofvirtuallinksrequirement,respectively Thedefault valuesare𝜔=𝜓=1.
- 𝐶𝑑(𝑛 𝑖 )𝑎𝑛𝑑 𝐶𝑑(m i )areCPUdemandsofvirtualnode𝑛 𝑖 a n dvirtualnode𝑚 𝑖 inthevirt ual segments𝑠𝑒𝑔𝑖of𝑉𝑁𝑅𝑖,respectively.
There are two main forms of time-based methods for VNE solving process: (1)offlinemethods and (2)onlinemethods.Onlinemethods take VNRs on a first-in-first-out (FIFO)basis,allocatingvirtualresourceswhentherequestsarrive.Whiletheseapproachesaremoresuitablei ndealingwithhighdynamicity,itcomestothecostofbeinglessoptimalsolutionssuch as decreasing acceptance rate Things might be further complicated by looking atheterogeneous resourcesin the substrate network (i.e. resources that are described bydifferent sets of parameters or that have widely varying attributes) Adapting algorithms tosupport such an assumption, therefore, remains an open issue for now On the other hand,offlinemethods take a given set of VNRs together with the description of the requirement.Based on the request, system executes an algorithm to find out the mapping results Whilethis approach may achieve good mapping results, it does not consider a dynamic arrivalprocess of the VNRs as real service providers (i.e. each VNR arrives at a different time andmust be mapped in real time) In this dissertation, the Online using Time Window (OuTW)is also proposed In this OuTW time-based method, all the requests, which are arriving attime window TW, will be collected and embedded at the end of this time window In thetime window, the arrived requests can be flexibly re-arranged to optimize themappingresource.Theoperationsoftheir algorithms aredescribedas follows:
1) Online: takes the VNRs on FIFO basis when the requests arrive If any VNR isunsuccessful,theframeworkrejectsthisrequestandcontinueswiththenextrequest.
2) Online using Time Window(OuTW): arriving requests are processed within a timewindow as a request queue Inside the time window, VNRs can be arranged in anypriorities such as energy consumption, the number of virtual nodes If there are anyunsuccessful embedded requests, the request will be deferred to the next windowtimeand will be tried again.
3) Offline: takes a given set of all VNRs together In fact, the offline method is onlineusinginfinity-sizetimewindow.
Energy-efficientVNEalgorithms
MappingvirtualnodesandlinksontosubstratenetworkisalsoknownasVirtualNetworkEmbedding (VNE)problem which consists of two sub-problems, namely virtual nodemapping(VNoM) and virtual link mapping(VLiM).
- VNoM:Virtualnodes shouldbe mappedto substratenodes.
- VLiM: Virtual links are allocated into paths connecting the corresponding nodes inthesubstrate network.
Two-stagesmappingapproachesareusedinthissectionforembeddinganarrivingVNR.The first stage solves VNoM problem and send its result to the second stage The VLiMproblem is solved in this second stage Each stage uses its energy-aware metric that isdescribedas follows:
- VNoM metric: anenergy-cost coefficientof capacity𝐸𝐶𝑐for VNoM embeddingalgorithm is proposed and described in the Eq.(3.12 For each VNR with its CPUrequirement,𝐸𝐶𝑐is defined as the consumed power for each capability unit of asubstratenode(Eq.
(3.12).The𝑃𝑐𝑢𝑟a n d 𝑃𝑎𝑓𝑡𝑒𝑟a rethecurrentconsumedpowerand theestimatedconsumedpowerafterthemapping.The𝐶 𝑃 (𝑡)istheleftovercapacityofadevice. Sothat,thseenergy-costcoefficient𝐸𝐶𝑐meanstheenergy-costforeachcapacityunit.
- VLiM metric: For the VLiM algorithm, the link mapping stage uses energy- costcoefficientofbandwidth,𝐸𝐶𝐵.FromtheVNoMresult,VLiMfindsallthepossible paths and theoretically evaluates the power consumption that should be paid morefor this path This energy consumption consists of additional power link𝑃𝑙andpower of any necessary middle hops [76] which are currently OFF and must beturned-on for this path.𝐸𝐶𝐵is obtained with the equation(3.13)with the binaryindicator𝑠𝑡𝑎𝑡𝑒𝑗is state of theneed-to-be-turned-onnode The𝑠𝑗= 1when thismiddle node should be turn on, and otherwise𝑠𝑗= 0 The power consumption of theneed-to- be-turned-onnode,middlenode𝑃𝑚𝑖𝑑,isdescribedasinequation(3.14).
Two energy-efficient algorithms are proposed for the VNoM mapping process, namelyheuristic energy-efficient (HEE)node mapping andreducing middle node energy efficiency(RMN-EE)mapping The capacity greedy node mapping, the popular mapping strategy, isalsoimplementedforperformanceevaluationandcomparisonwiththeproposedalgorithms.
SolvingVNEproblemisknownastheNP-Hardproblem[70].Therefore,manyresearchers focus on contributing greedy-based and heuristic-based VNE algorithms [77].Recent articles mainly focus on improving the revenue of the Internet service providers aswell as acceptance rate However, there are few works consider the power-saving target ofnetworksin cloud computingenvironments.
Inthisdissertation,firstly,thecapacitygreedy-basedVNoMalgorithm[78],isknownas
CapGreedy, is re-implemented for comparison with our proposed algorithm ThisCapGreedychooses thenodewiththelargestCPUcapacity(Cap)valueformappingselectionandturn-offallunusednodes after the mapping This Capacity greedy-based algorithm, which is described inPseudocode2 ,i s s c a l a b i l i t y a n d e a s y t o i m p l e m e n t T h e l i n e 7 c o n s t r a i n t , 𝑉𝑛(𝑛 𝑆 )∩
𝑁𝑛𝑒𝑖(𝑛 𝑅 )≠ ∅, ensures that the separate neighboring virtual nodes cannot be located on thesame substrate node.𝑁𝑛𝑒𝑖(𝑛 𝑅 )is a set of all neighbor of node n, where𝑉𝑛(𝑛 𝑆 )is a set of allvirtual node thatmappedonto the substrate node𝑛 𝑆
In this section, a Heuristic Energy-efficient VNoM (HEE) algorithm is proposed.
ThisHEEalgorithmfocusesonsatisfyingthetrafficdemandandminimizingthenumberofactivesubstrate nodes All inactive nodes will be turned off when no activities are performed Inthissection,aHeuristicEnergy- efficientVNoM(HEE)algorithmisproposedwhichrealizesnodemappingin order of priorityasfollows:
- Turn offthe unused nodes forenergysaving.
The strategy of this mapping priority is focusing on energy efficiency by choosing thesubstratenodsewiththebest𝐸𝐶𝑐value.ThisalgorithmisdescribedmoredetailinPseudocode
In network virtualization, one virtual node can be mapped to only one substrate node,while a virtual link can be represented by a path that stays in the group of consecutivephysical links in substrate network A middle node is considered as an intermediate node inthe substrate path mapped to a virtual link Although the middle nodes are transparent tocustomers, they constantly consume energy So that if the number of middle nodes can bereduced while satisfying the VNRs, the energy consumption of the whole network can beefficiently saved RMN-EE is a heuristic-based algorithm that focuses on minimizing thenumberof activenodesand reducingthe numberof middlenodes.
Algorithm:ReducingMiddle Node EnergyEfficiencyMapping(RMN-EE)
Pseudocode 4: Reducing Middle Node Energy Efficiency
The algorithm is described in thePseudocode 4: The unused nodes are turned-off forreducing the energy consumption To achieve this target, the mapping algorithm shouldminimize the number of active nodes at the VNoM step as well as avoid purely forwardingmiddlenodes.Inordertodothat,allvirtuallinksareaimedtomapdirectlyontothesubstratelink(a physicallinkthatisdirectlyconnectedbetweentwosubstratenodes).
Intuitively,thenumberofsubstrateneighborsfornodemappingpartiallyguaranteesthedirectlinkm apping So that, the RMN-EE algorithm ranks the substrate nodes in non-increasing orderof number of neighbor nodes first (line 3-4) If substrate nodes have the same number ofneighbornodes,thesenodesare also ranked bythe node’s capacity(line5-6).
AfterwhentheVNoMalgorithmisexecuted,anenergy-awareVLiMmappingalgorithm,which is proposed in this work, is used to detect the substrate path for all virtual linksrequests.Therearetwoprocessingstepsofthisalgorithm.First,theproposedalgorithmusestheBreadt h First Searchto detects all possible paths for virtual links After that, thealgorithm chooses the best energy-efficient path based on the𝐸𝐶𝐵metric The operation ofVLiMisgiven bythe pseudocodebelow.
6 Findall paths for(src,dst)
8 //Sort paths by𝐸𝐶𝐵in non-decreasingorder
PerformanceEvaluation
The experimental results of several algorithms under different time-based strategies areshown in this section The VNoM algorithms areCapGreedy,Heuristic Energy-
EfficientMapping(HEE) andReducing Middle Node Energy Efficiency(RMN-EE) algorithms FortheVLiM process,onlyEnergy-AwareVirtualLink Mappingalgorithmisused.
In this work, the energy-aware network virtualization platform is developed which isbasedontheFlowVisorproject[71]
[72]andtheMininet[12]emulator.Inthisplatform,theGraphicUserInterface(Figure3.6)isconstructed whichallowsuserstomonitorandmanagetheresourcesofsubstratenetwork,aswellasthedemandsofVN Rs,moreeasily.ThisJava- basedGUIstaysintheManagementblockandcommunicateswithFlowVisorandOpenFlowcontro llers.Via this UI administrators caneasily establish several tasksasfollows:
- Configure suitable OpenFlow controllers In proposed system, users can create twotypes of controllers that are POX and Floodlight In the near future, the types ofsupportedcontrollerswill be extended.
Figure3.6:The GUIof anEnergy-awarenetworkvirtualizationplatform
For creating a substrate network, in this test, random substrate topologies from 7 to 11nodes are generated by using Waxman algorithm [79] The below equation determines theprobability that there exists a link connecting two arbitrary nodesuandvin the Waxmanalgorithm:
Parametersαandβare in the range of (0, 1];dis the distance in Cartesian coordinatesbetween nodeuandv,Lis the maximum distance between any two nodes in the graph Arise in the parameterαincreases the probability of existing links between any nodes in thegraph, and an increase inβyields a larger ratio of long links to short links Similar to somepreviouswork[80][81],inthisdissertation,theαandβaresetas0.5foraverageconnectivity and link distance All physical links have the same capacity of 100Mbps.Besides, several VNE requests are generated randomly, each may include several trafficdemands The investigated network traffic utilization (or traffic demand ratio) varies from10%to90%.Foreachtrafficutilizationvalue,1000testsareperformed,eachwitharandomphysicaltop ology.
Theacceptanceratio(AR)andenergyconsumption(EC)levelofthesystemareconsidered in this work The acceptance ratio is known as the ratio of the number ofsuccessfully embedded VNRs to the arriving VNRs Energy consumption ratio is the ratioof the energy consumption of the system to the maximum energy consumption when allsubstratenodes and linksareworkingatmaximum speeds.
Figure3.7AR–Online Figure3.8:AR–Onlineusing Time Windows
TheFigure3.7andFigure3.8showtheacceptanceratiosofthreealgorithmsintwotime-based methods,OnlineandOnline using Time Windows In fact, the acceptance ratio of aVN technology represents the resource optimization of the system As we can see in thesefigures, the acceptance ratios of three algorithms are different in both time-based methods.From the low load point, these acceptance ratios are quite good The acceptance ratios aregoing down when the Load value is increased Besides, for the load from 10 percents to 90percents,thedifferenceamongalgorithms areremarkable.
Figure 3.9: Percentage of Power Consumption to
Figure 3.10 Percentage of Power Consumption toFull Statein OuTWS t r a t e g y
TheFigure 3.9andFigure 3.10show the energy consumption of the system underdifferent load value from 10% up to 90% The results show that, by turning-off all unuseddevices,theconsumedenergyofthesystemisproportionaltotheVNRsload,evenwiththeCap Greedy algorithm The energy consumption’s trends of both online and online usingwindows time (OuTW) are quite similar in term of energy proportionality When the HEEand RMN-EE algorithms map virtual nodes, these methods use the energy-cost coefficientvalue as a metric, so that the energy- consumption of the system is being optimized andreduced Thanks to the concerning to the number of neighbors when doing node mapping,theRMN-EEcanreducemiddlenodes.Consequently,theenergyconsumptionofthewholenetworkis decreased.
Beside the comparison of the acceptance ratio and the energy-saving level of thesealgorithms, the effect of the two time-based methods are also compared in this section TheFigure 3.11andFigure 3.12show the comparison of power consumption and acceptanceratio of the system in two time-based strategies,onlinevsOuTW.In this comparison, onlytheRMN-EE algorithm, the bestalgorithm in this section,isused.
As we can see in theFigure 3.11andFigure 3.12, the acceptance ratio and the energysaving level of the OuTW method are better than the Online method Actually, the OuTWmethoddoesnotmapaVNRimmediatelywhenthisVNRarrives.Anyrequestscometothesystem will be stored in the time windows In the end of the time window, all the VNRs ofthis time window will be re-ordered first, and then mapped to the system Consequently,thanks to the re-ordering process, the energy-saving level as well as the acceptance ratio ofOuTWarebetterthantheOnlinemethod.Incontrary,whenweusetheOuTW,someVNRshaveto wait in thetime window instead ofbeingserved in real-time (online).
In OuTW, the time window size also affects the performance as well as the efficiency ofthesystem.Actually,ifthewindowsizeiszero,wewillhaveanonlinestrategy.Incontrary,if the time window is infinity, it actually is the offline strategy So that, the influence ofwindow size onto the acceptance ratio and energy- saving level is evaluated As we can seeinTable3.2,whenthewindowsizeischangedfrom𝑡𝑑t o 3𝑡𝑑,theacceptancerateamong them are a slight difference Where𝑡𝑑is the average duration of the living time of VNRs.The large size of time window provides better results then the small one in both aspectsincludingpowerconsumptionandacceptanceratio.Thepenaltyofthelargewindowsize,inourp oint of view,will bethe notreal-time serving therequest when itarrives.
Table 3.2: Acceptance ratio and power consumption of the system underdifferent windowsizein OuTW
Conclusion
Anenergy-awarenetworkvirtualizationplatformisproposedanddeployedinthischapter The platform performs several VNE algorithms with its power monitoring andcontrolling abilities This platform also provides the evaluation ability of performance andpower-saving level of the network with several VNE algorithms In this chapter, two novelHeuristicEnergy-efficientVNE(HEE- VNE)algorithmsarealsoproposed,namelyHeuristic Energy-Efficient (HEE)virtual network embedding andReducing Middle nodeEnergy efficiency (RMN-EE)algorithm.The proposed algorithms increase energy-savinglevel while maintains a reasonable acceptance ratio As we can see the results of this work,the HEE and RMN-EE algorithms save more energy consumption of the whole network aswellas acceptanceratio of thenetwork.
Cloudcomputing isbecoming increasingly importantnowadaysasitsupportsnewbusiness models such as Infrastructure-as-a-Service (IaaS), Platforms-a-Service (PaaS) andSoftware-as- a-Service(SaaS).Oneimportantcomponentofcloudcomputingisdatacenters,which are used by cloud service providers to house cloud-based resources and services.
AsshowninFigure4.1,intraditionalcloudcomputingparadigms,acloudserviceprovidercantypically build their own data centers as the infrastructure to implement and offer cloudservices.Alternatively,intheNetwork-as-a- Service[82]model,theserviceprovidercanbedecoupled into two new roles, namely the Infrastructure Provider (InP) that deploys andmaintains network infrastructure and the Service Provider (SP), which is in charge ofdeployingend-to-endservices.IntheNaaSmodel,thecloudserviceproviderscanmakeuseof data centers belonging to third-party infrastructure providers In either former and latercase,datacentervirtualizationcomesintoplay,whichisaconceptofnetworkvirtualization(NV)tha tallowscreatingmultiple,separatevirtualdatacenters(VDC)ontopofthephysicaldatacenter.
- Costsaving:datacentervirtualizationallowsreducingcapitalexpenditure(CAPEX)andoperationa lcosts(OPEX)asseveralcloudserviceproviderscansharethesamephysicaldata centerofathird-partyinfrastructure provider.
- Flexibility,resourceefficiencyanddynamicity:aVDCcanbeprovisioneddynamically on demand based on service requirements, scalability, time durationandrequiredresources.Itcanalsobeexpandedandshrunkandusedasapay-as- you-usepricingmodelforuserstorenttheirpersonalizedtopologynetworks.
- Energy saving: Recent surveys have shown that the energy consumption in a datacenter considerably contributes to its operation costs [4] A remarkable part of thelargeenergyvolumeconsumedindatacenterstodayisduetotheover-provisioningof such network resources as switches, links, and servers to meet the stringentrequirements on reliability By dynamically scale up and down the VDC instead ofmaintaining a fixed number of physical servers,under-utilization can be avoidedthatleads to moreenergy-efficient usageof thedatacenter.
A major challenge of data center virtualization is the virtual data center embedding(VDCE) problem as solving VDCE is NP-hard For that reason, current research mostlyfollows heuristic and meta-heuristic approaches In this chapter, an energy-efficient datacentervirtualizationis focused on with thefollowingcontributions:
- Addressing some challenges in providing energy and resource efficient virtual datacenter embedding under the condition of dynamic VDC requests, in which virtualdata centers arrive and leave the physical data center dynamically The evaluationresultsshowthattheperformanceofconventionalstaticVNEalgorithmsisunstable andis degraded under dynamicconditions.
- A novel VDC embedding algorithm is proposed with the following objectives:(1)resourceefficiencythatdealswithefficientmappingofvirtualresourcesonsubstrate resources in terms of CPU, memory and network bandwidth; and(2)energyefficiencythatdealswithminimizingenergyconsumptionofthevirtualdatacenterw hilemeetingmappingdemands.Inthiswork,theproposedVDCembeddingalgorithm is integrated with new remapping and server consolidation strategies,which are developed to overcome the dynamic VDC mapping problem and tomitigate the complexity of the joint embedding migration approach Evaluationresults show that our approach performs better than some existing ones in terms ofacceptanceratio, resourceutilization andenergyconsumption.
VirtualDCTechnologies
Asalreadyaddressedabove,datacentervirtualizationisaconceptofnetworkvirtualization, which allows creating multiple virtual and separated data centers on top of asingle physical data center infrastructure based on specific requirements and constraints Amajor challenge in network virtualization is the virtual network embedding problem thatdeals with efficient mapping of virtual resources on the substrate resources The virtualnetwork embedding problem is typically modeled by: (1) virtual network requests (VNR)withnodeandlinkdemands;and(2)thephysicalsubstratenetwork(SN)withnodeandlinkresourc es [83] Demands and resources then have to be matched in order to complete theembedding.Virtualnetworkembeddingcanbedecomposedintotwosub- problems,namelythevirtual nodemapping (VNoM)and thevirtuallinkmapping(VLiM) problems.
In the context of data center virtualization, VDC embedding requires mapping VDCcomponents, such as virtual machines (VM) and network devices (switches and links) ontophysicalservers, physicalnodesandlinks.Figure 4.2illustratesthe VDCembeddingprocess Although sharing generalembedding concepts,VDC embedding differs fromvirtualnetwork embeddingin several aspects:
- Virtual node mapping:beside mapping virtual nodes on physical nodes, which aresuchn e t w o r k d e v i c e s as switchesa n d routers,virtualn o d e m a p p i n g i n V D C embedding also deals with mapping virtual machines (VM) on physical servers.Thus the virtual node mapping algorithm in VDC embedding is more complex as itshouldconsidernotonlynetworknoderesourcesbutalsoserverresources.Moreover, as a virtual machine in a data center can often be migrated from onephysical machine to another, the topology of the VDC can be dynamically changedduringitslifetime,whichposesagreaterchallengeintheefficiencyanddynamicityofvi rtual networkembeddingalgorithms.
- Energy efficiency: in cloud computing paradigms, nowhere in the cloud providerinfrastructure,includinginaccessandcorenetworks,ismoredenselydeployedwithserv ers and network devices than in data centers Thus energy efficiency in datacentersisbecomingimportantasitcanreduce operationcostsandmake theprovider’s network more environmental friendly For that reason VDC embeddingalgorithmsshouldmeetmultipleobjectives,knownasresourceandenergyef ficiency.
Althoughthegeneralvirtualnetworkembeddingproblemhasbeenrelativelywellstudiedin the literature, there is only few research work that has addressed the VDC embeddingproblem with consideration of energy efficiency. VDC Planner [9] and Venice [8] wereproposed as VDC embedding methods based on migration aware model to maximize therevenue of InPs For energy efficiency, Han et al.
[84] proposed SAVE - an SDN assistedVDCembeddingsystem.However,thosemethodsdidnotconsiderthelifetimeofVDCsaswell as arrivingand leavingtime oftheirrequests.
Gue etal.[85]proposeda data centernetworkarchitecturecalledSecondNetthatincorporateaheuristicalgorithmforresourcealloc ationtoVDC.AlthoughSecondNetisnotan energy-aware approach, it focuses on resource-efficient VDC embedding and providingbandwidth guarantees among multiple VMs in a multitenant virtualized data center.WhenembeddingaVDCrequest,SecondNetchoosesaserverclusterthatcanaccommoda tethe number of VMs in the request with the VM locations as near to each other as possible, thustryingtoavoidvirtuallinkstraversingmorephysicalnodesandincreasetheacceptancerate.However,the virtuallinkembeddingalgorithmperformedafterwardsusestheBreadthFirstSearch (BFS) algorithm [80] to find the shortest path for link mapping In some cases, theshortest path may require to turn on more immediate physical switches instead of usingON_Stateswitches that leadto morepower consumption.
Amokraneetal.[86]introducedGreenHead-aholisticresourcemanagementframeworkfor embedding VDCs across geographically distributed data centers connected through abackbone network GreenHead is an energy- aware VDC embedding approach First, it triesto concentrate VMs in a part of the physical substrate, so that a part of the data center ishighly utilized and the rest of the DC can be put in sleep mode For virtual link mapping,GreenHeadmakesuseofBFStofindtheshortestpathsbetweenVMs.However,itdoesnotta ke the available bandwidth of links on shortest paths into account If the bandwidthavailability does not satisfy bandwidth demands, the VDC request is rejected immediatelywithout any trying the second or third shortest mapping As a result, this mapping strategydirectlyaffects theacceptanceratio of theVDC embeddingprocess
Data center virtualization is further empowered by virtual machine migration and serverconsolidation that relocate VMs within cloud data centers VM migration is performed toadaptdynamicworkloadtothephysicalserversandtoachievevariousresourcemanagementobjectivessucha sloadbalancing,powermanagement,faulttolerance,andsystemmaintenance Recently there is some work focusing on the energy efficiency of servers byusing servers consolidation and placement algorithms for VMs that can be mapped ontophysicalservers[87][88] [89].Nevertheless,theseapproachesonlyfocusonasingle groupof VMs requests and not on the embedding of virtual data centers that include multiplegroupsofVMrequestsatatime.Additionally,approachesthatcombinedatacentervirtualization with VM migration are hard to find in the literature From the technicalperspective,OpenStack[90]isanemergingopensourceplatformthatfacilitatescloudificat ion OpenStack is a cloud operating system that controls large pools of compute,storage, and networking resources throughout a data center In OpenStack, live migrationcan be performedeasily.When combining with OpenFlow/SDN, theOpenStack-
SDNenvironmentcanprovidefullflexibledatacentervirtualization,inwhichadditionalfunction scan be extended to supportmigration and virtualembeddingoptimization.
In this Section, some issues in providing virtual data centers are discussed In most ofreal-world situations, a virtual data center request (VDCR) has to be tackled as a dynamicproblem.T h a t i s , a V D C R o f t e n a r r i v e s i n r e a l - t i m e a n d i s n o t k n o w n i n a d v a n c e I f successfullyembeddedinthephysicalsubstrate,thenewlycreatedvirtualdatacenterstaysinthes ystem in arandom amount oftime beforeleaving.
In contradiction to static virtual data center embedding algorithms (Figure 4.3) that donot take the aforementioned dynamic problem into account, dynamic virtual data centerembeddingalgorithms(Figure4.4)havetohandleVDCRsinasequenceastheyarriveratherthan handle them at once In order to tackle this issue, dynamic virtual data embeddingalgorithms should have the ability to relocate one or more VDCRs There are several issuesrelatedto energy-awarevirtual datacenter embeddingas thefollows:
- Resourcefragment:sinceeachVDCRhasitsownvirtualtopologywiththecorresponding resourcedemands,suchasanumberofVMs,bandwidthandsoforth,asthevirtualdatacentersdyn amicallyjoinandleavethesystem,resourcesintermsof server capacity, available link bandwidth become fragmented The consequenceis the degradation of data center’s utilization The impact of the dynamic problemonthe system performancewill bediscussed inmoredetails inthenext Section.
- EfficientVDCremapping:asmentionedabove,inorderfordynamicvirtualnetworkembedding algorithms to reach optimal objectives (e.g., optimal resource or energyefficiency), relocation of a part or even all existing virtual data centers is necessaryupon the arrival or departure of a virtual data center As illustrated inFigure 4.4, atanyt i m e s w h e n a r e q u e s t j o i n s o r l e a v e s t h e s y s t e m , t h e v i r t u a l d a t a c e n t e r embeddingalgorithmshouldreconsiderallexistingrequests.Thisrelocationprocess consists of two steps: (1) re-mapping virtual machines onto new physicalservers, that is, migrating VMs from existing physical machines to new ones; (2)remappingvirtualnetworknodesandvirtuallinksofrelocatedvirtualdatanetworkson the physical DC network Thus it is worthwhile to note that a dynamic virtualdata center embedding algorithm is the combination of embedding andserverconsolidation Since the aforementioned relocation process might principally takeplace every time when a virtual data center joins or leaves, the complexity of there-mappingalgorithms should be taken into account.
Figure4.5:Energyproportional propertyof energy-awaredatacenters
- Energy proportionality:from the energy perspective, one important issue in thedesignofenergy- awaredatacenterishowtooptimizetheenergyvolumeconsumedbythedatacenterproportiona llytoitsactualload.Thatis,energyconsumptioninalow utilization scenario should be much lower than in case of high utilization, asshown inFigure 4.5.In other words, the average energy consumption per virtualdata center should be kept ideally constant, independent of the number of acceptedvirtual data centers in the system However, in today DCN, energy consumptionbetween low and high utilization is not much different It is due to the fact thatservers and network devices in conventional data centers should be operational allthe time to meet the stringent requirements on reliability, regardless of their actualload.
DesignObjectives
By using SDN technique, joint virtual data center embedding and server consolidationarefocused on, whichshould meet thefollowing objectives:
- Resourceefficiencyofthephysicaldata centershouldbeimprovedin the sensethatthe overall utilization can be increased Thus more virtual data centers can beacceptedwith limited physicalresources.
- Energy efficiency: besides resource efficiency, the overallenergy consumptionshouldbereducedandasproportionaltothedata center’sutilizationaspossible.
- Trade-offbetweendynamicityandcomplexity:asdiscussedabove,toreachoptimality the VNE algorithm might be performed every time when a virtual datacenter joins or leaves the system This poses a challenge in the implementationcomplexity.In ourapproach, thecomplexityshould be keptacceptable.
ProblemFormulation
𝑆 𝑝 denotesasetofphysicalservers,𝑁 𝑝 denotesasetofnetworkdevicesand𝐿 𝑝 d e n o t e saseto fphysicallinksamongthedevices.Theservers’attributesgenerally includememory andC P U c a p a c i t y.𝑀 𝑐𝑎𝑝(𝑆 𝑝 )a n d 𝐶 𝑐𝑎𝑝(𝑆 𝑝 )d e n o t ea v a i l a b l e ( i e , l e f t o v e r )m e m o r y a n d
CPUofphysicalserver𝑆 𝑝 whichbelongstothesetofphysicalservers𝑆 𝑝 ,respectively.The attributeofaphysicallink𝐿 𝑝 isthebandwidthofthislinkandisdenotedby𝐵𝑐𝑎𝑝(𝐿 𝑝 ).
A sequence of virtual data center requests joins and leaves over time Similarly to𝐺 𝑝 ,i th VDC request is modeled as a weighted graph𝑅 𝑣 (𝑉𝑀𝑖, 𝐿 𝑣𝑖 , 𝑡𝑖, 𝑑𝑖)where𝑡𝑖and𝑑𝑖denotethe arrival time and duration of this VDC, respectively.𝑉𝑀𝑖denotes the set of virtualmachineswiththeircorrespondingCPU𝐶𝑑(𝑉𝑀𝑖)andmemory𝑀𝑑(𝑉𝑀𝑖)resources’dem and.𝐿𝑣 𝑖 d e n o t e sthematrixofvirtuallinks’demandwiththeirbandwidthattributeand each𝑙 𝑣𝑖 ∈ 𝐿 𝑣𝑖 d e n o t e savirtuallinkdemandwithitsbandwidthfromthe source𝑣𝑚 destination𝑣𝑚𝑑. tothe
The main challenge in a data center virtualization is a VDC embedding problem whichmaps VDC requests onto a physical DC Embedding VDC request𝑅 𝑣 onto physical datacenter𝐺 𝑝 meansfindingasubsetof(𝑆 𝑝 ,
𝐿 𝑝 )at𝑡𝑖whichsatisfiestheresources’requirementof𝑉𝑀𝑖and𝐿 𝑣𝑖 VDCembeddingproblemisdivid edintotwosub-problems:(1)virtual machine mapping (VmM) that maps𝑉𝑀𝑖onto the physical servers; and (2) virtuallink mapping(VLiM) that maps matrixof link request𝐿 𝑣 𝑖 o n t o s u b s t r a t e l i n k s
Let𝑐𝑎𝑝:𝑆 𝑝⋃𝑁 𝑝⋃𝐿𝑝→𝑅⃗→beafunctionthatassignsavailableresourcesofphysicalDCinclud ingservers,networkdevicesandlinkswhere𝑅⃗→isthevectorspaceofresources.
Similarly,function𝑑𝑒𝑚𝑖:𝑉𝑀𝑖⋃𝐿 𝑣𝑖 →𝑅⃗→assignsdemandstoanelementofaVDCrequest.Conseq uently, VDC embedding consists two functions𝑓𝑖ofVmMa n d𝑘𝑖ofVLiMwhicharedescribedas follows:
𝑠𝑡𝑎𝑡𝑒𝑡: 𝑆 𝑝 ⋃𝑁 𝑝 ⋃𝐿 𝑝 → 𝐺 𝑝 denotes the function assigns a state at timetof all elements of thephysicalDCbybinaryvalues,whichreturns1whenturningon(ON_State)and0otherwise(OFF
In a data center, the energy cost is mainly consists of four components: network devices,physical servers, building infrastructure, and energy draw (electrical utility costs) [91].
Theconsumedpowerofnetworkdevicesisapproximately15%andphysicalserversapproximately consume 45% of total energy consumption of whole DC.
Additionally,reducingthedissipatedenergyinaDCsuchasheatbynetworkdevicesandserverswillalso decrease the consumed power of other ancillary parts including power draw and coolingsystem.Sincebothtwocomponents,buildinginfrastructureandenergydraw,aresignificant lyproportionaltotheworkingstatesoftheserversandnetworkdevices.Consequently,the energysavinglevel of aDCcanbedonebysimplydecreasingthepowerconsumptionofits networkdevicesand physicalservers.
In section2.2.1.1, the energy consumption of a network device is modeled with itsworkings t a t e a t t i m et,t h e s t a t e o f a n e t w o r k e l e m e n tn i sd e f i n e d b y b i n a r y i n d i c a t o r
𝑠𝑡𝑎𝑡𝑒(𝑛, 𝑡)which is: 0 if the is turned off; or 1 (otherwise) The energy consumption of thenetwork part of DC (switches and links),𝐸𝑁(𝑡), at timetis denoted as the sum of allswitches, including their static power𝑃𝑠𝑡𝑎𝑡𝑖𝑐(i.e., baseline power) of the core of the device,andthe powerconsumptionofinterfaces under theiroperatingspeeds.
Theenergymodelingofaphysicalserveris alreadydescribed in the section2.4.4.1.
The power cost of migration comprises of the power used by the source physical server,which starts the migration; and the power used by the destination server All the cost iscaused by the increase of server resources (CPU, memory) and I/O resource
(network) Themigration energy consumption is calculated as the sum of an energy consumption of thesourceand thedestination serversat timet.
Basically, the main objective of energy-aware methods in data centers is to reduce thetotal energy consumption of a physical DC In order to do that, the energy-aware
VDCembeddingalgorithmsshouldfocusondecreasingthenumberofONstateservers,ONstatenetwor kdevicesaswellastheirlinks.Theobjective,asdefinedinEq.(4.6andEq.
(4.7,istominimizetheconsumedpowerofDCcomponentsincludingnetworkdevices𝐸𝑁(𝑡),physica l servers𝐸𝑆(𝑡)and migrationprocess𝐸 𝑡
- Virtual machine mapping (VmM) constraints: The requirement of computationalresources and memory of any𝑣𝑚𝑗must be lower than those of physical serverhostingit.
- Eachv i r t u a l m a c h i n e o f a n y V D C r e q u e s t s w h i c h c a n b e m a p p e d o n o n l y o n e physicalserver𝑆 𝑝 ,sothatthebinaryindicator𝑥 𝑖o f 𝑝 𝑣𝑚𝑗mappingtoserver𝑆 𝑝 is
- VDCconstraints:Allphysicalelementsthata𝑉𝐷𝐶𝑖isallocatedonmustbeturnedon.T heseconstrains aredescribed in thebelow equations.
ANew Conceptfor VDCEmbedding
An energy-aware virtual data center architecture EA-VDC is constructed which allowsdeployinganyenergy-awareembeddingalgorithms.AsshowninFigure4.6,thearchitectureconsists of two main blocks:Managementand DCHypervisor TheManagementblockconsistsofthreesub- blocks:
DCbasedonSDNtechnologybyinteractingwithboththeNetworkHypervisorandVMHypervisorsub- blocks.Therearealready several platforms for server hypervisor such as HyperV, OpenStack, KVM, XEN,and VMware [92],
[90] One of the most challenges of the VDC architecture is the networkhypervisor.BydevelopingtheFlowVisor- basedReServNetplatform,anewresourcemanagementa n d a l l o c a t i o n c o n c e p t i s i m p l e m e n t e d f o r n e t w o r k v i r t u a l i z a t i o n , w h i c h i s
Server ServerServerServer Server ServerServerServer Server ServerServerServer Server ServerServerServer presented in the last chapter By combining server and network hypervisor based on SDNtechnology, the proposed architecture could be a promising approach for energy efficiencyofDC virtualization.
The VDCembedding consists of two sub-embedding processes, whichare: VirtualMachineMapping(VmM)whichmapsvirtualmachinerequestsontothephysicalmachines;and
(2) virtual link mapping (VLiM) which creates virtual links interconnecting newlymappedVMs ontop of thephysical network substrate.
In this dissertation, a Heuristic Energy-Aware VDC Embedding (HEA-E) algorithm isproposedf o r v i r t u a l d a t a c e n t e r e m b e d d i n g H E A - E i s a n e n e r g y - e f f i c i e n t e m b e d d i n g algorithmthatisbasedonsmartsleeping/ standbythatisnotonlytoreducepowerconsumption but also to improve resource utilization of the physical data center When aVDC request arrives, VmM maps virtual machines onto physical servers Subsequently,based on VmM results, VLiM creates virtual links interconnecting newly mapped VMs ontop of the physical network substrate In order to improve the acceptance rate, HEA-Ere-performs VmM mapping if VLiM does not perform successfully The flowchart of VDCembeddingis described inFigure4.7.
Virtualmachinemapping(VmM):threegroupsofserversaredefined,namelyneargroup,middlegroup andfargroup These groups are correspond to: (1) near traffic that flowsbetween two servers connected to the same edge switch; (2) middle traffic that should flowthroughanaggregationswitch;and(3)fartrafficthattraversestoacoreswitch,respectivelyWhenaVDC requestarrives, the followingactionsaretaken:
- Step 1: Finding all possible mapping groups that have the number of availablephysical servers greater or equal to the number of VMs in the request To improvereliability, a source and destination VMs pair will not be mapped on the samephysicalserver.
- Step 2: For all selected server groups, choose candidate groups with the leastphysicalserverssothatthepowerconsumptionofphysicalserverscan besaved.
- Step3:ForallcandidategroupswiththeleastnumberofphysicalserversinStep2,chooseagro upthathasasmanyserversthatareneartoeachotheraspossible.Thatis, the candidate groups are prioritized in thenear→middle→farorder By doingthis, it is likely that the virtual links interconnecting these VMs should traversedoverlessimmediatehops,thusreducingthepowerconsumptionofnetworkdevices.Th eHEA-EVmM algorithmis described inthepseudocodebelow.
Pseudocode6: HEA-EVirtualMachineMappingAlgorithm (Proposed Algorithm6)
Virtual link mapping (VLiM): The virtual link mapping is inspired by Heller et al.
[57]approach Heller proposed theElastic Treeconcept in order to reduce consumed energy ofDC network by maintaining a minimal logical topology on top of the Fat-Tree based onactual traffic demands In HEA-E, VLiM, which is described inPseudocode 7, firstlyarranges the matrix of virtual link requests of a VDC in non-increasing order of bandwidthdemands.Thenwitheachrequestcorrespondingtoasourcevirtualmachinevm sand desti nationmachinevm d ,theVLiMalgorithmidentifiesthetypeoftrafficscenario(i.e.,near,middle or far) according to their relative positions that are already decided by the VmMalgorithm.It then performs thecorrelative actions:
- In near traffic scenario: If the VMs are in the physical machines that are connectedto the same edge switchsw edge , the request is mapped onto the physical linksconnectingthe VMs through that switch.
- In middle or far traffic scenarios: Otherwise, find all possible paths with necessaryset of active switches Select the path satisfied the traffic demand that has leastavailablebandwidth andconsumes the least additional energy
- Ifthevirtuallinkmappingdoesnotsucceed,HEA-Eperformsvirtualnodere- mappingprocess(Figure4.7).
Pseudocode7:HEA-EVirtual LinkMapping(ProposedAlgorithm7)
The aforementioned resource fragment problem occurs when VDCs continuously joinandleavethephysicaldatacenterovertime.WhenaVDCleavesthesystem,somephysicalservers that hosted that VDC may be left underutilized On the other hand, the new comingVDCs might be mapped to some new servers as the existing servers are not fitted to newrequests This situation leads to inefficient VDC embedding in term of energy as well asresources In these casesserver consolidationis necessary Server consolidation is done bymigratingVMs from some servers to others In this section, three VM-migration strategiesareproposed:
- (2) migration on arrival (MoA) that tries to re-optimize the whole system by re- mapping all existing requests every time when a new VDC request arrives in thesystem;
- and (3) full migration (FM) that executes remapping of all existing requests upon aVDCjoiningor leaving.
Intuitively, the full migration approach is the best one in terms of energy and resourceefficiency However, the pay-off is the increase in complexity as the number of migrationsmightbeveryhigh.Inthenextsection,differentperformancemetrics,includingenergyand resourceefficiencyas well as complexitywill beevaluated.
Thebasic principlesof PMarethe as follows:
- In PM,underutilized serversare servers that are not used up their 100% capacity.PMrearrangestheunderutilizedphysicalserversbymigratingsomeVMsfrommostund erutilized servers to less underutilized ones so that after the arrangement, someservers become idle and can be turned off The set of underutilized servers is listedinList-inefficient-serverin thenon-decreasingorder ofnumber of VMs.
- Considering underutilized servers inList-inefficient-serverin the increasing order.Foreachunderutilizedserver Pi, there is a set of source VMs(i.e,list-Src-VM(Pi)), that is, VMs need to be migrated to other servers There aredifferentpossibilitiestomigrateasourceVMtoadestinationserver.Withinthesetof candidate destination servers (i.e.,List-Des-Server) corresponding to the givenVM, choosing a server that has shortest paths (that is, lowest link metric) to otherVMs on the same VDC of the source VM The selected pathsshould satisfybandwidthdemands andcorrespond totheVDC topology.
MigrationonArrivalistheembeddingstrategythatintegratesHEA-Ewiththeremappingprocess (Figure 4.9) When a new VDC request arrives in the system, MoA firstly checksthe number of requested VMs, which is known asCheck VDC sizein the flowchart inFigure 4.9.If the physical DC can accommodate the new request, the following steps arecarriedon:
- AddthenewrequesttothelistofexistingrequestslistServingVDCandmakeuseofHEA-Eto re-embedall the requests on thelist.
- Iftheembeddingissuccessful,acceptthenewrequest.Otherwise,rejecttherequestandkeep theexistingmappings unchanged.
The Full Migration (FM) strategy is basically similar to Migration on Arrival The onlydifference is that, FM performs remapping every time there is a change in the system,suchas the arrival or departure of a VDC From the theoretical point-of-view, FM is the beststrategy in terms of resource and energy efficiency However, FM requires the system tocontinuouslyremapallrequests,includingtheexistingsuccessfulmappings,thusitincreasesthe complexityof thesystem
PerformanceEvaluation
The performance of the proposed joint HEA-E embedding and migration strategies arecomparedwiththreeVDCembeddingalgorithms,namelyHEA-
Ewithoutmigration(denoted as HEA-E), GreenHead [86] (GH) and SecondNet [85] (SN). Three criterias havebeen used for the performance evaluation, which are resource efficiency, energy efficiencyand complexity In this analysis, HEA-E without migration is used as the benchmark toinvestigatethe efficiency of new proposedmigrationstrategies,whileSecondNetandGreenHeadareusedforbenchmarkingthere sourceefficiencyandenergyefficiency,respectively.Whenthereisanincomingrequest,theseconventio nalalgorithmssimplymapthe new VDC request on the leftover resources of the physical substrate without anyremappings.
A Java-based simulator is developed in this dissertation All results shown in this workare derived from a Fat-Tree topology withk = 8, corresponding to a data center with 128servers.E a c h p h y s i c a l s e r v e r c a n h o s t a t m o s t 4 V M s , s o t h a t t h e t o t a l c a p a c i t y o f t h e simulated system is 512 VMs The energy profiles of servers and network devices in thesimulationsarebased on:
- Energy-aware commercial 24-port HPE Enterprise switch [59], which is able toadjusttheclockfrequenciesofitsnetworkinterfacesfordifferentpowerstates.Thebaseline power consumptionP static (see Sec.2.2.1.1) and the power consumption ofports under various operating speeds (10M bps, 100M bps and 1000M bps) aresummarizedinTable1.1.
- The Dell PowerEdge R710 server with Quad-core Intel 5520, 32GB RAM. Thedetailenergyprofile is shown in theTable2.5.
VDC requests are generated randomly following Poisson distribution with arrival rate ofλ = 8 VDCs per hour The duration of a VDC is exponentially distributed with 2hoursaveragelifetime.ThenumberofVMsperVDCrequestisuniformlydistributedfrom4 to
32.Inordertoinvestigatetheperformanceofembedding/ migrationstrategiesunderdifferentloadscenarios,alltrafficdemandsbetweenVMpairsarealsouniformlydi stributedbetween10Mbpsand90Mbps, so that the load of the physical DC varies from 10% to more than100% 50 simulation runs are performed for each test scenario, the results are then theaverageof thesemeasurements.
VMs within a VDC are interconnected with a random topology generated by Waxmanalgorithm The below equation represents the probability that there exists a link connectingtwoarbitrarynodes𝑢and𝑣in theWaxman algorithm:
Parametersα,βareintherangeof(0;1),disthedistanceinCartesiancoordinatesamongVM u and v, L is the maximum distance between any two nodes in the graph A rise in theparameter α increases the probability of existing links between two any nodes in the graph,andanincreaseinβyieldsalargerratiooflonglinkstoshortlinks.Inthesimulations,αandβareset as 0,5for averageconnectivityandlinkdistance.
As requests continuously join and leave the system, resource fragmentation may occur,which consequently leads to temporary degradation of resource efficiency, such as systemutilizationandacceptancerate.Figure4.10showstheutilizationofSecondNetinshorttimescale.
As can be seen, the system utilization fluctuates over time This can be explained as thefollows As some VDCs leave the system, some servers become underutilized, however, ifthese servers are spread out over the DC, they may not be able to host VMs of the newincoming VDC requests as the VLiM algorithm cannot successfully embed virtual linkssatisfying bandwidth demands This leads to the decline in utilization as these new requestsmight be rejected As more VDCs leave the system some time later, more servers becomeavailable that increase again the acceptance rate The more the utilization fluctuates, themore the performance of the
DC is unstable Server consolidation can be useful in this caseasithelpsrearrangeavailableserverstoincreasetheacceptanceratioandsystemutilization.This fluctuation phenomenon can be evaluated using the standard deviation of systemutilizationaroundthemeanutilization.Standarddeviationofutilizationisexpectedtobeassmal laspossible.AsshownintheTable4.1,thethreenewlyproposedstrategies(PM,MoA,FM)aremorestable thanthe conventional mappingalgorithms.
Resource efficiency can be evaluated by using the system utilization Utilization is thenumber of VMs that can be accepted by the physical DC per its capacity in terms ofVMs.As can be seen inFigure 4.11, as load increases, utilization increases up to a certain value,then it saturates Thanks to the migration as well as remapping strategies, the number ofserving VMs that the system can accept in PM, MoA, FM under the same incoming load issignificantlyhigher thanthatofGHandSN MoAperforms aswell as FM.
Furthermore, the acceptance ratio of the VMs and VDCs are shown in in the Figure4.12andFigure4.13,respectively.TheseacceptanceratiosofVMsandVDCsdescribetheratiosof the number of accepted VMs and VDCs to the number of arrived requested VMs andVDCs, respectively As we can see in the figures, as load increases, acceptance ratios of allalgorithms decreases However, the acceptance ratio of the proposed algorithms is betterthanGH and SN.
Total power consumption of the physical DC is shown inFigure 4.14 In low loadedconditions, the power consumption of GreenHead is the lowest, the following is the powerconsumption of FM, MoA The power consumption of PM is a bit higher than that of
FM,PMandGHbutlowerthanSN.Whensystemloadincreases,serversandnetworkcomponents of the physical data center are gradually turned on to accommodate incomingrequests,thuspowerconsumptionshouldincreaseaccordingly.Inthehighlyloadedsitu ation,sincethesystemworksatitsfullcapacity,thepowerconsumptionsofallalgorithms are the same It is while worthy to note that, while GH is very energy-efficient,its acceptance rate and utilization are very low so that
GH is only able to operate in the lowloadedsituation (less than 20%).
Next, the average power consumption of a VDC is evaluated, which is calculated bydividing the total power consumption of the physical DC by the number of served VDCs.Ascan beseen in theFigure4.15,when thesystem load increases:
- (2) the consumed power of the proposed algorithms is much the same and is lessthanboth SN and GN;
- (3) power consumption of the proposed algorithms decreases very slowly, whichimpliesthatpowerconsumptionofthephysicalDCstaysnearlylineartothenumberofembe ddedVDCs, followingtheenergyproportional property(seeSec.4.1.3);
- and (4) although the total power consumption of GH is the lowest (Figure 4.14), itsaveragepowerconsumptionper VDCisthehighestoneduetothe factthatGHcanhost onlyaverylimited number ofVDCs (Figure4.15).
Theaveragenumberofmigrationsforeachstrategyunderdifferentloadsituationisusedas the metric to evaluate complexity As the migration of a virtual machine to a physicalservertakestime,ahighnumberofmigrationsreducesthesystemperformancesignificantly.Asexpect ed,thenumberofmigrationsinFMunderhighlyloadedsituation(90%)canbeashigh as 9000 times (Figure4.16) while the number of migrations in PM is under 100 times,independentofthe loadas onlyunderutilized serversarerequired to consolidate.
Finally,generalcomparisonfortheaforementionedstrategiescanbedrawn.Asshowninthe above numerical results, SecondNet performs well in terms of resource efficiency but itdoesnotsatisfytherequirementonenergy- efficiency.Incontrast,GreenHeadisenergy-efficient in some sense, however, its utilization is very low in comparison to othermethods.
On the other hand, the three new approachesPM, MoAandFMperform well in terms ofboth resource and energy efficiency Moreover,PMis simpler as the number of requiredmigrationsismuchless,independentofsystemloadwhencomparedwithMoAandFM.Theradar graphsinFigure4.17andFigure4.18illustratesthiscomparativeanalysis.In general,
4.18: Different embedding-magrition strategies: (a) GreenHead, (b) SecondNet, (c)
Conclusion
Thisworkfirstlyanalyzestheresourcefragmentationproblemoccurredwhenvirtualdatacenters continuously join and leave the physical data center and its impact on the efficiencyof VDC embedding algorithms Different joint VDC mapping and
VM migration strategiesare proposed to tackle that dynamic problem In comparison to some previous resource andenergy- awareVDCembeddingalgorithms,thenewstrategiescanremarkablyimprovebothresource utilization and energy efficiency of the data center, while the complexity is kept atanacceptablelevel.
ManyTelcos,InternetServiceProviders(ISPs)andenterprises,havesignificantlyemployedl argenetworkinfrastructuresfortheInternetservices.Alargesystemconsumesahuge energy volume, so that the network energy efficiency problem is very importantrecently.Resolvingenergy- savingproblemsbringsmanyadvantages including:
- Economically,reducingenergyconsumptionoftheICTdatacentersleadstoreducing the costs of maintaining system Consequently, the Internet services’ costwillbereduced.
Majorcontributions
By using Software-defined Networking, energy-efficient approaches are studied in thenetwork in several cloud DC environments such as: (1) in data center network that uses thepromising DC topology, namely Fat-tree; (2) in the network virtualization concept; and (3)inthedatacentervirtualizationtechnology.Thecontributionsofthisresearcharesummarized as follows In thesecondchapter, the proposed SDN-based power- controlsystemispresented.ThisPCSplatformallowsadministratorstomonitor,control,andapplysev eral energy-efficient algorithms Thissecondchapter also presents two main energy-efficient approaches including: (1) energy-aware routing algorithm, namelypower scalingandenergy-profile- aware(PSnEP)algorithm,whichisbasedonthepowerscalingalgorithmandthepowerprofileofa networkdevice;and(2)topology- awareVMmigrationalgorithmwhichmigratesserverswithtwoobjectives:
(a)minimizingthenumberofphysicalservers;and(b)reducingthenumberofswitchesforinterconnectin gthesephysicalservers in order to turn-off more devices for energy efficiency The main advantage of thisalgorithm is that the migration process performs energy saving of servers as other commonmigrationstrategy,knownasfirst- fit,whilereducingtheenergyconsumptionofthenetworkdevices.Theexperimentalresultsshowthatthecons umedpowerofthenetworkdevicescanbesaved up to 46% whileremainingthe energy-savinglevel of theservers.
An energy-aware network virtualization concept is described in the next chapter with itspowermonitoringandcontrollingabilitiesforcloudenvironments.Theproposedconceptisbased on the SDN technology and allows researchers to develop several energy- efficientvirtualnetworkembeddingalgorithms.Twoproposedenergy-efficientembeddingalgorithms are proposed, namelyheuristic energy-efficientnode mapping andreducingmiddlenodeenergyefficiency,with theirexperimentalresults of performance.
The SDN-based Energy-aware VDC approaches for cloud environments is presented inthefourthchapter.TheVDCtechnologyisdescribedindetailwithitsmainproblem,namelyVDCembed ding.ByintegratingwithVMconsolidationtechnique,thejointVDCembedding and VM migration algorithmsis successfully deployed.These algorithms withtheirexperimental results aredescribed in this chapterin details.
AcceptanceRatio per VM
Total power consumption of the physical DC is shown inFigure 4.14 In low loadedconditions, the power consumption of GreenHead is the lowest, the following is the powerconsumption of FM, MoA The power consumption of PM is a bit higher than that of
FM,PMandGHbutlowerthanSN.Whensystemloadincreases,serversandnetworkcomponents of the physical data center are gradually turned on to accommodate incomingrequests,thuspowerconsumptionshouldincreaseaccordingly.Inthehighlyloadedsitu ation,sincethesystemworksatitsfullcapacity,thepowerconsumptionsofallalgorithms are the same It is while worthy to note that, while GH is very energy-efficient,its acceptance rate and utilization are very low so that
GH is only able to operate in the lowloadedsituation (less than 20%).
Next, the average power consumption of a VDC is evaluated, which is calculated bydividing the total power consumption of the physical DC by the number of served VDCs.Ascan beseen in theFigure4.15,when thesystem load increases:
- (2) the consumed power of the proposed algorithms is much the same and is lessthanboth SN and GN;
- (3) power consumption of the proposed algorithms decreases very slowly, whichimpliesthatpowerconsumptionofthephysicalDCstaysnearlylineartothenumberofembe ddedVDCs, followingtheenergyproportional property(seeSec.4.1.3);
- and (4) although the total power consumption of GH is the lowest (Figure 4.14), itsaveragepowerconsumptionper VDCisthehighestoneduetothe factthatGHcanhost onlyaverylimited number ofVDCs (Figure4.15).
Theaveragenumberofmigrationsforeachstrategyunderdifferentloadsituationisusedas the metric to evaluate complexity As the migration of a virtual machine to a physicalservertakestime,ahighnumberofmigrationsreducesthesystemperformancesignificantly.Asexpect ed,thenumberofmigrationsinFMunderhighlyloadedsituation(90%)canbeashigh as 9000 times (Figure4.16) while the number of migrations in PM is under 100 times,independentofthe loadas onlyunderutilized serversarerequired to consolidate.
Finally,generalcomparisonfortheaforementionedstrategiescanbedrawn.Asshowninthe above numerical results, SecondNet performs well in terms of resource efficiency but itdoesnotsatisfytherequirementonenergy- efficiency.Incontrast,GreenHeadisenergy-efficient in some sense, however, its utilization is very low in comparison to othermethods.
On the other hand, the three new approachesPM, MoAandFMperform well in terms ofboth resource and energy efficiency Moreover,PMis simpler as the number of requiredmigrationsismuchless,independentofsystemloadwhencomparedwithMoAandFM.Theradar graphsinFigure4.17andFigure4.18illustratesthiscomparativeanalysis.In general,
Different embedding-magrition strategies: (a) GreenHead, (b) SecondNet, (c) PartialMigration,(d)Migration onArrival, (e)FullMigration 91 LISTOFTABLES Table1.1:TheInternet’susersintheworld[1]
Thisworkfirstlyanalyzestheresourcefragmentationproblemoccurredwhenvirtualdatacenters continuously join and leave the physical data center and its impact on the efficiencyof VDC embedding algorithms Different joint VDC mapping and
VM migration strategiesare proposed to tackle that dynamic problem In comparison to some previous resource andenergy- awareVDCembeddingalgorithms,thenewstrategiescanremarkablyimprovebothresource utilization and energy efficiency of the data center, while the complexity is kept atanacceptablelevel.
ManyTelcos,InternetServiceProviders(ISPs)andenterprises,havesignificantlyemployedl argenetworkinfrastructuresfortheInternetservices.Alargesystemconsumesahuge energy volume, so that the network energy efficiency problem is very importantrecently.Resolvingenergy- savingproblemsbringsmanyadvantages including:
- Economically,reducingenergyconsumptionoftheICTdatacentersleadstoreducing the costs of maintaining system Consequently, the Internet services’ costwillbereduced.
By using Software-defined Networking, energy-efficient approaches are studied in thenetwork in several cloud DC environments such as: (1) in data center network that uses thepromising DC topology, namely Fat-tree; (2) in the network virtualization concept; and (3)inthedatacentervirtualizationtechnology.Thecontributionsofthisresearcharesummarized as follows In thesecondchapter, the proposed SDN-based power- controlsystemispresented.ThisPCSplatformallowsadministratorstomonitor,control,andapplysev eral energy-efficient algorithms Thissecondchapter also presents two main energy-efficient approaches including: (1) energy-aware routing algorithm, namelypower scalingandenergy-profile- aware(PSnEP)algorithm,whichisbasedonthepowerscalingalgorithmandthepowerprofileofa networkdevice;and(2)topology- awareVMmigrationalgorithmwhichmigratesserverswithtwoobjectives:
(a)minimizingthenumberofphysicalservers;and(b)reducingthenumberofswitchesforinterconnectin gthesephysicalservers in order to turn-off more devices for energy efficiency The main advantage of thisalgorithm is that the migration process performs energy saving of servers as other commonmigrationstrategy,knownasfirst- fit,whilereducingtheenergyconsumptionofthenetworkdevices.Theexperimentalresultsshowthatthecons umedpowerofthenetworkdevicescanbesaved up to 46% whileremainingthe energy-savinglevel of theservers.
An energy-aware network virtualization concept is described in the next chapter with itspowermonitoringandcontrollingabilitiesforcloudenvironments.Theproposedconceptisbased on the SDN technology and allows researchers to develop several energy- efficientvirtualnetworkembeddingalgorithms.Twoproposedenergy-efficientembeddingalgorithms are proposed, namelyheuristic energy-efficientnode mapping andreducingmiddlenodeenergyefficiency,with theirexperimentalresults of performance.
The SDN-based Energy-aware VDC approaches for cloud environments is presented inthefourthchapter.TheVDCtechnologyisdescribedindetailwithitsmainproblem,namelyVDCembed ding.ByintegratingwithVMconsolidationtechnique,thejointVDCembedding and VM migration algorithmsis successfully deployed.These algorithms withtheirexperimental results aredescribed in this chapterin details.
Although network energy efficiency has been attracted much attention from the researchcommunity, there are many difficulties to realize these technologies and transfer to theindustrialmarket Sothat in thefuturework,wearegoingtoestablish thefollowingtasks:
- Realizingtheenergy efficiency ofadatacenternetworkbyusing newcloudplatform– OpenStack.OpenStacksoftwarecontrolslargepoolsofcompute,storage, and networking resources throughout a datacenter, managed through adashboardorviatheOpenStackAPI.OpenStackworkswithpopularenterpriseandopensou rcetechnologiesmakingitidealforheterogeneousinfrastructure[90].TheOpenStack Platform contains the SDN controller - OpenDayLight and the computemanagement,which managethe VM provisioningand migrationprocess.
- Developing the network virtualization and data center virtualization for detailedestimating the delay and packet loss In the near future, the Internet services withtheir characteristics will be embedded into the system Based on the services’demand as well as their parameters such as downtime, latency and reliability, thesystemallocates its resources reasonably.
1 Thanh Nguyen Huu, Anh-Vu Vu, Duc-Lam Nguyen, Van-Huynh Nguyen,Manh- NamTran ,Quynh-ThuNgo,Thu-HuongTruong,Tai-
HungNguyen,ThomasMagedanz (2015) “A Generalized Resource Allocation
Framework in Support ofMultiLayerVirtualNetworkEmbeddingbasedonSDN”,Elsevier-
ComputerNetworks,2015 -https://doi.org/10.1016/j.comnet.2015.09.042
2 Tran Manh Nam , Nguyen Huu Thanh, Hoang Trung Hieu, Nguyen Tien Manh,Nguyen Van Huynh, Tuan Hoang (2017) “Joint Network Embedding and
ServerConsolidation for Energy-Efficient Dynamic Data Center Virtualization”, Elsevier -ComputerNetworks, 2017 -https://doi.org/10.1016/j.comnet.2017.06.007
1.Nam T.M , Huynh N.V., Thanh N.H (2016).“Reducing Middle Nodes
MappingAlgorithmforEnergyEfficiencyinNetworkVirtualization”.In:AdvancesinInf ormation and Communication Technology ICTA 2016 Advances in IntelligentSystems and Computing, vol 538 Springer, Cham. https://doi.org/10.1007/978-3-319-49073-1_54
1 NguyenHuuThanh,BuiDinhCuong,ToDucThien,PhamNgocNam,NgoQuynhThu,Truo ngThuHuong,andTranManhNam (2013).“ECODANE:ACustomizable Hybrid
Testbed for Green DataCenterNetworks”, The
InternationalConferenceonAdvancedTechnologiesforCommunications2013(ATC'13)- IEEE,Hanoi,Vietnam
2 TranManh Nam ,TranHoangVu,VuQuangTrong,NguyenHuuThanh,PhamNgocNam.
AwareDataCenterNetwork”,NationalConferenceonElectronicsandCommunications (REV2013-KC01).,Hanoi, Vietnam
3 Tran Manh Nam , Truong Thu Huong, Nguyen Huu Thanh, Pham Van Cong, NgoQuynh Thu, Pham Ngoc Nam (2014) “A Reliable Analyzer for Energy- SavingApproachesinLargeDataCenterNetworks”,IEEEICCE-
TheInternationalConferenceon Communicationsand Electronics- 2014, DaNang,Vietnam
4 Tran Manh Nam , Nguyen Huu Thanh, Ngo Quynh Thu and Hoang Trung Hieu,StefanCovaci.(2015).“Energy-
AwareRoutingbasedonPowerProfileofDevices inDataCenterNetworksusingSDN”,12thElectricalEngineering/Electronics,Computer, Telecommunications And Information Technology Conference (ECTI-CON)-2015, Hua Hin, Thailand.
5 Tran Manh Nam , Nguyen Huu Thanh, Nguyen Hong Van, Kim Bao Long, NguyenVan Huynh, Nguyen Duc Lam, Nguyen Van Ca (2015) “Constructing Energy-AwareSoftware-DefinedNetworkVirtualization”,ProceedingsofAsia- PacificAdvanced Network Research Workshop (APAN-NRW), August 10th - 14th 2015,Kuala Lumpur, Malaysia-(best student paperaward)
6 Tran Manh Nam , Nguyen Huu Thanh, Doan Anh Tuan (2016) “Green Data CenterUsingCentralizedPower-
ManagementOfNetworkAndServers”,The15thinternational Conference on Electronics, Information, and Communication (IEEE -ICEIC),Jan 2016, Da Nang, Vietnam
7 Tran Manh Nam , Nguyen Van Huynh, Le Quang Dai, Nguyen Huu Thanh. (2016).“AnEnergy-AwareEmbeddingAlgorithmforVirtualDataCenters”,ITC28- InternationalTeletrafficCongress, Sep-2016,Wurzburg, Germany.
8 Tran Manh Nam , Nguyen Tien Manh, Truong Thu Huong, Nguyen Huu Thanh(2018).“OnlineUsingTimeWindowEmbeddingStrategyinGreenNetworkVirtu alization”,InternationalConferenceonInformationandCommunicationTechnology and DigitalConvergence Business(ICIDB-2018), Hanoi,Vietnam.(presented)
[1] http://www.internetworldstats.com/ stats.htm," U s a g e a n d P o p u l a t i o n S t a t i s t i c s , " [Online].
[Online].Available:http://globalactionplan.org.uk.[Accessed 2016].
[3] "GartnerSaysDataCentersAccountfor23percentofGlobalICTCO2Emissions,,"Gartner, 2007. [Online] Available:http://www.gartner.com/it/page.jsp?idS0912.[Accessed2016].
[4] Bolla, R., Bruschi, R., Davoli, F., Member, S., & Cucchietti, F., "Energy EfficiencyintheFutureInternet:ASurveyofExistingApproachesandTrendsinEnergy-
AwareFixed Network Infrastructures,"Ieee Communications Surveys & Tutorials,vol 13,no. 2,2011.
[6] Chowdhury,N.M.M.K.,&Boutaba,R.,"Networkvirtualization:stateoftheartandresearch challenges,"IEEE Communications Magazine,vol 47, no 7, pp 20-26,2009.
[7] Khan, A., Zugenmaier, A., Jurca, D., & Kellerer, W., "Network virtualization: ahypervisorfortheInternet?,"IEEECommunicationsMagazine,vol.50,no.1,p.136–143, 2012.
[8] Zhang, Q., Zhani, M F., Jabri, M., & Boutaba, R., "Venice: Reliable virtual datacenter embedding in clouds.," inIn Proceedings - IEEE INFOCOM, Toronto, ON,Canada,2014.
[9] Zhani, M F M., Zhang, Q., Simon, G., & Boutaba, R., "VDC Planner : DynamicMigration-AwareVirtualDataCenterEmbeddingforClouds,"inIFIP/
[10] S D M K L J a U B M P Gilesh, "Towards a Complete Virtual Data CenterEmbeddingAlgorithm,"inIEEE37thInternationalConferenceonDistributedCo mputingSystems, Atlanta, GA, USA, June-2017.
[11] Kreutz,D.,Ramos,F.M.V.,EstevesVerissimo,P.,EsteveRothenberg,C.,Azodolmolky, S.,Uhlig,S.,Uhlig,S.,"Software-
DefinedNetworking:AComprehensiveSurvey,"ProceedingsoftheIEEE,103,vol 103,no.1,pp.14-76,
[12] Feamster,H.KimandN.,"Improvingnetworkmanagementwithsoftwaredefin ednetworking,"IEEE Commun Mag,vol 51, no 2,p 114–119, 2013.
Available:http://www.youtube.com/watch?v=YHeyuD89n1Y
Matos,"Scientific ResearchMethodologi es And Techniques," www.uninova.pt/ cam/teaching/SRMT/SRMTunit1.pdf.
[15] "SMART 2020: Enabling the Low Carbon Economy in the Informa- tion
(GeSI),http://www.theclimategroup.org/assets/re-sources/publications/
U.S.Governement,[Online].Available:http://www.eia.doe.gov.[AccessedDec2016].
[17] "Impacts of Information and Communication Technologies on Energy Efficiency,"EuropeanCommissionDGINFSO,http://ec.europa.eu/informationsociety /newsroom/cf/itemdetail.cfm?item idD41, 2018.
[18] "The Advanced Configuration & Power Interface (ACPI) Specification," Hewlett- Packard,Intel,Microsoft,Phoenix,andToshiba.,[Online].Available:http:// www.acpi.info/.
[19] R S Tucker, R Parthiban, J Baliga, K Hinton, R W A Ayre, W.V. Sorin,"Evolution of WDM Optical IP Networks: A Cost and Energy Perspective,"IEEE J.LightwaveTechnol,vol vol 27, no.3, pp pp 243-252, Feb 2009.
[20] D.Neilson,"PhotonicsforSwitchingandRouting,"IEEEJournalofSelectedTopicsinQuantu mElectronics(JSTQE),vol 12,no.4, pp.669-678, July-Aug.2006
[21] Bianzino,A.P.,Chaudet,C.,Rossi,D.,&Rougier,J.,"ASurveyofGreenNetworking Research,"IEEE Communications Surveys Tutorials,vol 14, no 1,2012.
[22] Barr,Keith,ASICDesigninthe SiliconSandbox: ACompleteGuideto Build ingMixed-SignalIntegrated Circuits, McGrawHill Professional,2007.
[23] JózefKORBICZetal.,SynthesisofCompositionalMicroprogramControlUnitsforProgram mableDevices,Poland:Universityof ZielonaGóraPress,Poland, 2009.
[24] J.Baliga,R.Ayre,K.Hinton,andR.S.Tucker,"PhotonicSwitchingandtheEnergyBottleneck," inInternat Conf Photonics in Switching 2007, San Francisco, CA,USA, 2007.
[25] L.G.Roberts,"ARadicalNewRouter,"IEEESpectrum,vol.46,no.7,pp.34- 39,2009.
[26] M.Baldi,Y.Ofek,"Timefora”Greener”Internet,”,"in Proc.G r e e n CommunicationsWorkshopinconjunctionwithIEEEI C C ’ 0 9 (GreenComm09),Dresd en,Germany, 2009.
AdaptivePartialReconfiguration,"inInternat.Conf.onFieldProgrammableLogican dApplications (FPL 2007),Amsterdam, Netherlands, 2007.
[28] M.B.Srivastava,A.P.Chandrakasan,R.W.Brodersen,"PredictiveSystemShutdownandOtherAr chitecturalTechniquesforEnergyEfficientProgrammableComputation,"IEEETrans.V eryLargeScaleIntegr.(VLSI),vol.4,no.1,pp.42-55,1996.
[29] M Allman, K Christensen, B Nordman, V Paxson, "Enabling an Energy- EfficientFutureInternetThroughSelectivelyConnectedEndSystems,"inProc.ACMSI
GCOMMHotNetsWorkshop(HotNets 07),Atlanta, GA,USA, 2007.
[30] M Jimeno, K Christensen, B Nordman, "A Network Connection Proxy to EnableHosts to Sleep and Save Energy," inProc IEEE Internat Performance
ComputingandCommunications Conf, Austin, Texas, USA,2008.
[31] Han, B., Gopalakrishnan, V., Ji, L., & Lee, S., "Network function virtualization:Challengesandopportunitiesforinnovations,"IEEECommunicationsMagazine,v ol.53, no 2, p.90–97, 2015.
[32] Correa,E.S.,Fletscher,L.A.,&Botero,J.F.,"VirtualDataCenterEmbedding:ASurve y.,"IEEELatin America Transactions,vol 13, no.5, p 1661–1670, 2015.
[33] B Raghavan et al, "Software-defined internet architecture: Decoupling architecturefrom infrastructure," inProc 11th ACM Workshop Hot Topics Network, Redmond,Washington,USA, 2012.
SIGCOMMComput Commun.Rev,vol 38,no 2, p.69–74, 2008.
[36] Gude, N., Koponen, T., Pettit, J., Pfaff, B., Casado, M., McKeown, N., & Shenker,S.,"NOX:TowardsanOperatingSystemforNetworks,"ACMSIGCOMMComputerC ommunicationReview,vol 38, no 3, pp 105-111, 2008.
[37] "POX,"[Online].Available:http://www.noxrepo.org/pox/about-pox/.[AccessedJan2014].
Available:https://openflow.stanford.edu/display/Beacon/Home/.[Accessed May2015].
[39] Floodlight [Online] Available: Available: http://floodlight.openflowhub.org. [Accessed May2015].
[40] "The OpenDaylight Platform," OpenDaylight, [Online].
Available:https://www.opendaylight.org/.[Accessed Dec2015].
InnovativeResearchinScience,EngineeringandTechnology,vol.3,no.3,pp.2200-
[43] Z.Q.R.M.H.G.A.V.V.BinWang,"Asurveyondatacenternetworkingforcloudcomputing,"Com puter Networks,vol 91, p 528–547, 2015.
[44] A.M.N.T B ri an L e b i e d n i k , "A Survey andE va lu at i on ofDataCenter Net work Topologies,"Distributed, Parallel, and ClusterComputing,2016.
[45] D Kliazovich, P Bouvry, S.U Khan, "GreenCloud: a packet-level simulator forenergy-awarecloudcomputingdatacenters,"JournalofSupercomputing,vol.62,no.3, p 1263–
"QuantitativeComparisons of the State of the Art Data Center Architectures,"Concurrency andComputation:Practiceand Experience,vol 25, no.
[48] C.E.Leiserson,"Fat-trees:universalnetworksforhardware- efficientsupercomputing,"IEEE Transactions on Computers,vol 100, no 10, pp. 892-901,1985.
[49] M Al-Fares, A Loukissas, and A Vahdat, "A scalable, commodity data centernetwork architecture,"ACM SIGCOMM Computer Communication
[50] Greenberg, A., Hamilton, J R., Jain, N., Kandula, S., Kim, C., Lahiri, P., Sengupta,S.,"VL2:AScalableandFlexibleDataCenterNetwork.,"ACMSIGCOMMCo mputerCommunication Review,vol 39, no 4,2009.
[51] Andreyev., A., "Introducing data center fabric, the next-generation facebook datacenternetwork.,"Facebook,Nov2014.[Online].Available:https://code.facebook.c om/posts/360346274145943/.[AccessedJan 2017].
[52] A.Singh,J.Ong,A.Agarwal,G.Anderson,A.Armistead,R.Bannon,S.Boving,G.Desai, B. Felderman, P Germano et al.,, "Jupiter rising: A decade of clos topologiesand centralized control in google’s datacenter network," inACM
Conference onSpecialInterest Group on Data Communication, 2015.
[53] C Guo, H Wu, K Tan, L Shi, Y Zhang, and S Lu, "Dcell: a scalable and fault- tolerantnetworkstructurefordatacenters,"ACMSIGCOMMComputerCommunication
[54] C.Guo,G.Lu,D.Li,H.Wu,X.Zhang,Y.Shi,C.Tian,Y.Zhang,andS.Lu,"Bcube:a high performance, server-centric network architecture for modular data centers,"ACMSIGCOMMComputer
[55] A.Singla,C.-Y.Hong,L.Popa,P.B.Godfrey,"Jellyfish:Networkingdatacenters randomly,"i nT h e 9 t h U S E N I X C o n f e r e n c e o n N e t w o r k e d S y s t e m s D e s i g n a n d Implementation,CA, USA, 2012.
[56] Trinh, L Gyarmati and T A., "Scafida: A scale-free network inspired data centerarchitecture,"ACMSIGCOMMComputerCommunicationReview,vol.40,no.5,pp.4-12, 2010
[57] Heller,B.,Seetharaman,S.,Mahadevan,P.,Yiakoumis,Y.,Sharma,P.,Banerjee,S.,&McKeow n,N.,"ElasticTree:SavingEnergyinDataCenterNetworks,"inNSDI'10Proceedingsof the7th USENIX conference, CA,USA, 2010.
[58] Pham Ngoc, N., Nguyen Huu, T., Vu Quang, T., Tran Hoang, V., Truong Thu, H.,Tran-Gia, P., & Schwartz, C., "A new power profiling method and power scalingmechanismforenergy-awareNetFPGAgigabitrouter.,"ComputerNetworks,vol.78,p 4–
[59] Mahadevan,P.,Sharma,P.,Banerjee,S.,&Ranganathan,P.,"EnergyAwareNetworkOper ations.,"inIEEE INFOCOM Workshops, 2009.
[60] "NetFPGAGigabitCard,"netFPGA,2015.[Online].Available:http:// netfpga.org/.[AccessedDec2016].
[61] Wang,X.,Yao,Y.,Wang,X.,Lu,K.,&Cao,Q.,"CARPO:Correlation-awarepoweroptimization in data center networks," inIn 2012 Proceedings IEEE INFOCOM,2012.
[62] J Case,M.Fedor, M.SchoffstallandD J., "RFC1157- A Simple NetworkManagementProtocol(SNMP),"1990.[Online].Available:https://tools.ietf.o rg/html/rfc1157.
[63] K W a Y.-H H Yi-Chih Lei, "Multipath Routing in SDN-based Data CenterNetworks,"in2015EuropeanConferenceonNetworksandCommunications(Eu
[64] D P Omair Fatmi, "Distributed multipath routing for data center networks based onstochastic traffic modeling," inNetworking, Sensing and Control (ICNSC), Miami,FL,USA, 2014.
[65] "D-ITG,DistributedInternetTrafficG e n e r a t o r , " 2 0 1 7 [Online].Available:http:// www.grid.unina.it/software/ITG/.
MarcMenaud,GillesMuller, Julia Lawall, "Entropy:aConsolidationManager forClusters,"ACMSIGPLAN/SIGOPS,2009.
[67] HirokiShirayanagi,HiroshiYamada,andKenjiKono,"Honeyguide:AVMMigration- AwareNetworkTopologyforSavingEnergyConsumptioninDataCenterNetworks,"ISCC
[68] V.DeMaio,R.Prodan,S.Benedict,G.Kecskemeti,"Modellingenergyconsumptionofnetworktr ansfersandvirtualmachinemigration,Modellingenergyconsumption ofnetworktransfersandvirtualmachinemigration,"FutureGenerationComp uter
[69] Chowdhury, N.K., & Boutaba, R et al., "Network virtualization: state of the art andresearch challenges,"Communications Magazine, IEEE,vol 47, no 7, p 20–
[71] R Sherwood, G Gibb, K Yap, G Appenzeller, N McKeown, and G. Parulkar,"FlowVisor: A network virtualization layer," inOpenFlow Switch
[72] Rob Sherwood, Michael Chan,Glen Gibb, Nikhil Handigol,Te-Yuan Huang,PeymanKazemian,MasayoshiKobayashi,DavidUnderhill,Kok-
KiongYap,GuidoAppenzeller,andNickMcKeown,"CarvingResearchSlicesOutofYourProdu ctionNetworkswithOpenFlow,"NewsletterACMSIGCOMMComputerCommunication,vol 40, no 1, 2010.
[73] "Exploringnetworksofthefuture,"GENI,[Online].Available:http://www.geni.net/.
[74] "Applications,Linking InfrastructureandOfelia,,"OpenFlowinEurope, [Online] Available:http://www.fp7-ofelia.eu.[Accessed dec2016].
[75] "OF@TEIN - OpenFlow@Trans-Eurasian Information Network,"
[Online].Available:http://oftein.net.[Accessed Dec2016].
[76] Botero,J.F.,Hesselbach,X.,Fischer,A.,&deMeer,H.,"Optimalmappingofvirtualnetworkswithhid denhops,"TelecommunicationSystems,vol.51,no.4,p.273–282,2013.
[77] Andreas Fischer, Juan Felipe Botero, Michael Till Beck, Hermann de Meer, andXavierHesselbach,"VirtualNetworkEmbedding:ASurvey,"IEEECOMMUNICA
[78] M.Y.Y.R.J.&.C.M.Yu,"Rethinkingvirtualnetworkembedding,"ACMSIGCOMMCom puter Communication Review,vol 38, no 2, p.17, 2008.
[80] Skiena,S.S.,"TheAlgorithmDesignManual,2ndEdition,"SpringerPub- lishingCompany,2008.
[81] A.B.J.D.M.S.D.H.X.D.M.H.Fischer,"ALEVIN-Aframeworktodevelop,compare, and analyze virtual network embedding algorithms,"
Electronic Communicationsof theEASST,vol 37,pp 1-12, 2011.
[82] P.Costa,M.Migliavacca,P.Pietzuch,A.L.Wolf,"NaaS:Network-as-aServicein thecloud,"inThe2ndUSENIXWorkshoponHotTopicsinManagementofInternet,Cloud,and
EnterpriseNetworksandServices,USENIX,SanJose,CA,USA, 2012.
[83] Fischer,A.,Botero,J.F.,Beck,M.T.,deMeer,H.,Hesselbach,X.,Meer,H.De,…Hesselbach, X.,
"Virtual network embedding: A survey,"IEEE CommunicationsSurveysand Tutorials,vol 15, no 4, p 1888–1906, 2013.
[84] Yoonseon Han, Jian Li, Jae-Yoon Chung, Jae-Hyoung Yoo, & Hong, J W.-
[85] Guo, C., Lu, G., Wang, H J H J., Yang, S., Kong, C., Sun, P., … Zhang, Y.,"SecondNet:adatacenternetworkvirtualizationarchitecturewithbandwidthguarant ees,"inProceedingsofthe6thInternationalCOnference,(VDC,Pennsylvania,USA, 2010.
[86] A Amokrane, M F Zhani, R Langar, R Boutaba, G Pujolle, "Greenhead: Virtualdata center embedding across distributed infrastructures,"IEEE Transactions onCloud Computing,vol 1,no 1, p 36–49, 2013.
[87] H Goudarzi, M PedramH Goudarzi, M Pedram, "Energy-efficient virtual machinereplicationandplacementinacloudcomputingsystem,"inIEEEFifthInternationalConf erenceon Cloud Computing, Honolulu, HI,USA, 2012.
[88] F.Farahnakian,P.Liljeberg,J.Plosila,"Energy- efficientvirtualmachinesconsolidationinclouddatacentersusingreinforcementlearning,"in22nd
EuromicroInternational Conference on Parallel, Distributed, and Network-Based Processing,Torino,Italy, 2014.
[89] A Beloglazov, R Buyya, Y C Lee, A Y Zomaya, "A taxonomy and survey ofenergy- efficientdatacentersandcloudcomputingsystems,"AdvancesinComputers82(2011) 47 –
[90] "OpenStack,"[Online].Available:https://www.openstack.org.[AccessedDec2016].
[91] J H D A M P P A Greenberg, "The cost of a cloud: Research problems in datacenter networks,"ACM SIGCOMM Computer Communication Review,vol 39, no.1, p 68–73, 2009.
[92] A Desai,R.Oza,P.Sharma, andB.Patel,"Hypervisor:ASurveyonConceptsandTaxonomy,"no 3,vol 3, pp 222 -225, 2013.
[93] "CT503-MIXHigh-SpeedLANforge-FIRETrafficGenerator,"CandelaTechnology,2017. [Online].Available:http://www.candelatech.com/ct503-MIX_product.php.
[94] A Al-Shabibi,M.DeLeenheer,M.Gerola,A.Koshibe,G Parulkar,E.Sal- vadori,
B Snow,"OpenVirteX:Makeyour virtualSDNsprogrammable,"inProceedingsof