Simulation of Cloud Computing Eco-Efficient Data Centre

Ibrahim Alzamil MSc Computing and Management Session (2011/2012)

The candidate confirms that the work submitted is their own and the appropriate credit has been given where reference has been made to the work of others. I understand that failure to attribute material which is obtained from another source may be considered as plagiarism. Ibrahim Alzamil (Signature of student) _______________________________

Abstract
Cloud Computing has created a new era in the IT industry, offering businesses and individuals technical and economical benefits, such as utilising computing resources through virtualisation; reducing operational costs by moving the cost of ownership to the Cloud providers, and paying only for what they use. However, the data centres that underpin the Clouds consist of huge computing resources that consume large amounts of power. A data centre can even consume the same amount of electricity that is consumed by a city. Thus, maintaining data centres can incur huge operational costs for the providers. Moreover, this would increase the impact on the environment in terms of CO2 emissions. Therefore, this project will evaluate some techniques, namely DVFS and VM migration, which can be deployed to enhance the eco-efficiency of data centres. Through using the CloudSim simulator, a number of experiments will be conducted to investigate the efficiency of these techniques in terms of reducing energy consumption.

Acknowledgement
First and foremost, I thank Allah for granting me knowledge, health, and the patience to successfully complete this project. I would also like to show my gratitude and appreciation to the following people: Dr. Karim Djemame, my supervisor, for his generous and constructive feedback and continuous encouragement throughout the project. Dr. Raymond Kwan, my assessor, for his critical and constructive feedback for the interim report and valuable comments in the progress meetings. I also wish to take this opportunity to thank Dr. Mariam Kiran for her advice about where to start with learning to use CloudSim. Special thanks and gratitude go to all my colleagues and friends, especially Abdulaziz Albalti and Khalid Alhammad, who have been there cheering me on and supporting me. Finally, I would like to express my indebtedness to my parents, my brothers, and my sisters who have always been supportive and encouraged me morally.

II

List of Acronyms
DVFS: Dynamic Voltage and Frequency Scaling WBS: Work Breakdown Structure NS2: Network Simulator IT: Information Technology SOA: Service-Oriented Architecture SIP: Strategic Inflection Point PC: Personal Computer SLAs: Service Level Agreements SaaS: Software as a Service PaaS: Platform as a Service IaaS: Infrastructure as a Service OS: Operating System DDoS: Distributed Denial of Service TCO: Total Cost of Ownership CO2: Carbon Dioxide ICT: Information and Communication Technology QoS: Quality of Service PUE: Power Usage Effectiveness CCTV: Closed Circuit Television ERE: Energy Reusable Efficiency III

CPU: Central Processing Unit VM: Virtual Machine PTM: Power and Thermal Management WM: Workload Monitor DPM: Dynamic Power Management IDEs: Integrated Development Environments MIPS: Millions of Instructions per Second RAM: Random Access Memory

IV

List of Figures
Figure 1. Four-Layer Cloud Architecture Figure 2. Layered Cloud Computing Architecture Figure 3. Initial Layered CloudSim Architecture Figure 4. Current Layered CloudSim Architecture .................................................................... ....................................................... ....................................................... ....................................................... .......................................... 7 8 17 18 23

Figure 5. An iterative approach for experiments development

Figure 6. Comparison between energy consumption mean values for each experiments - Scenario 1 ..................................................................................................................................... 27 Figure 7. A screenshot sample of using Excel to calculate mean values, standard deviation, and confidence interval ...........................................................................................................

71

Figure 8. Comparison between energy consumption mean values for each experiments - Scenario 2 ..................................................................................................................................... 28 Figure 9. Comparison between energy consumption mean values for each experiments - Scenario 3 ..................................................................................................................................... 30 Figure 10. Comparison between energy consumption mean values for each experiments - Scenario 4 ..................................................................................................................................... 31 Figure 11. Comparison between energy consumption mean values for each experiments - Scenario 5 ..................................................................................................................................... 32 Figure 12. Comparison between energy consumption mean values for each experiments - Scenario 6 ..................................................................................................................................... 34 Figure 13. Comparison between energy consumption mean values for each experiments - Scenario 7 ..................................................................................................................................... 36 Figure 14. Comparison between energy consumption mean values for each experiments - Scenario 8 ..................................................................................................................................... 37

List of Tables
Table 1. Typical Characteristics of Host, VMs, and Cloudlets ........................................... ... 25 67 Table 2. Results of Scenario 1 experiments with the deployment of DVFS mechanism

Table 3. Results of Scenario 1 experiments without the deployment of power aware mechanism ...................................................................................................................................... 69 Table 4. Results of Scenario 2 Experiments with the deployment of DVFS mechanism ... 83

Table 5. Results of Scenario 2 experiments without the deployment of power aware mechanism ...................................................................................................................................... 85 Table 6. Results of Scenario 3 Experiments with the deployment of DVFS mechanism ... 87

Table 7. Results of Scenario 3 experiments without the deployment of power aware mechanism ...................................................................................................................................... 89 Table 8. Results of Scenario 4 Experiments with the deployment of DVFS mechanism ... 91

Table 9. Results of Scenario 4 experiments without the deployment of power aware mechanism ...................................................................................................................................... 93 Table 10. Results of Scenario 5 Experiments with the deployment of DVFS mechanism ... 95

Table 11. Results of Scenario 5 experiments without the deployment of power aware mechanism ...................................................................................................................................... 96 Table 12. Results of Scenario 6 Experiments with the deployment of DVFS mechanism ... 98

Table 13. Results of Scenario 6 experiments without the deployment of power aware mechanism ...................................................................................................................................... 100 Table 14. Results of Scenario 7 Experiments with the deployment of VM migration mechanism with DVFS enabled ......................................................................................................................... 102 Table 15. Comparison for energy consumption mean values between scenario 7 and 1 ... 35

Table 16. Results of Scenario 7 Experiments with the deployment of VM migration mechanism with DVFS enabled ......................................................................................................................... 103 Table 17. Benchmarking of energy consumption between the deployment of DVFS mechanism and non power aware (NPA) mechanism in a data centre ........................................................ 38

VI

Table of Contents
Abstract ............................................................................................................... Acknowledgement ...................................................................................... List of Acronyms ................................................................................................... List of Figures ................................................................................................... List of Tables ................................................................................................... Table of Contents ................................................................................................... 1. Introduction ................................................................................................... 1.1. Overview .......................................................................................................... 1.2. Aim and Objectives ............................................................................................. 1.3. Minimum Requirements and Further Enhancements ......................................... 1.4. Deliverables .......................................................................................................... 1.5. Motives and Relevance to the Degree Programme ......................................... 1.6. Methodology .......................................................................................................... 1.7. Initial Project Plan ............................................................................................. 1.7.1.Key Activities ............................................................................................. 1.7.2.Milestones ............................................................................................. 2. Review of Literature and Technologies .............................................................. 2.1. Introduction to Cloud Computing ................................................................... 2.1.1.What is Cloud Computing? ................................................................... 2.1.2.Architecture of Clouds ................................................................................ 2.1.3.Types of Clouds Services ................................................................................ 2.1.4.Types of Clouds Deployment ................................................................... 2.1.5.Virtualisation ............................................................................................. 2.2. Issues Associated with Clouds ................................................................................ 2.2.1.Cost of Clouds Maintenance ................................................................... 2.2.2.Eco-efficient Data Centre Management ...................................................... 2.2.2.1. A Scientific Model of Eco-efficient Data Centre ............................ 2.2.2.2. Management of Power Consumption ......................................... 2.2.2.2.1. VM Allocation ................................................................... 2.2.2.2.2. Power and Thermal Management ............................ 2.2.2.2.3. Dynamic Voltage and Frequency Scaling ............... 2.3. CloudSim Toolkit ............................................................................................. 2.3.1.Architecture of CloudSim ................................................................................ 2.3.2.Usability .......................................................................................................... 2.3.3.Capabilities ............................................................................................. 2.3.4.Limitations ............................................................................................. 3. Methodology ................................................................................................... 3.1. Project Management ............................................................................................. 3.1.1.Revised Project Plan ................................................................................ 3.1.1.1. Key Activities ................................................................................ 3.1.1.2. Milestones ................................................................................ VII I II III V VI VII 1 1 1 2 2 2 3 4 4 5 6 6 6 7 9 9 10 11 12 12 13 14 15 15 15 16 16 18 18 19 21 21 21 21 21

4.

5.

6.

7. 8.

3.1.2.Progress Meetings ................................................................................ 3.2. Problem Summary ............................................................................................. 3.3. Project Methodology ............................................................................................. 3.4. Analysis and Design of Simulated Experiments ...................................................... Implementation of Experiments .......................................................................... 4.1. DVFS Mechanism ............................................................................................. 4.1.1.Scenario 1: Increasing the No. Of Hosts, VMs and Cloudlets ............... 4.1.1.1. Objectives and Design ................................................................... 4.1.1.2. Discussion of the Results ...................................................... 4.1.2.Scenario 2: Increasing the No. Of Hosts ...................................................... 4.1.2.1. Objectives and Design ................................................................... 4.1.2.2. Discussion of the Results ...................................................... 4.1.3.Scenario 3: Increasing the No. Of VMs ...................................................... 4.1.3.1. Objectives and Design ................................................................... 4.1.3.2. Discussion of the Results ...................................................... 4.1.4.Scenario 4: Increasing the No. Of Cloudlets ......................................... 4.1.4.1. Objectives and Design ................................................................... 4.1.4.2. Discussion of the Results ...................................................... 4.1.5.Scenario 5: Increasing the No. Of Hosts and VMs ......................................... 4.1.5.1. Objectives and Design ................................................................... 4.1.5.2. Discussion of the Results ...................................................... 4.1.6.Scenario 6: Increasing the No. Of Hosts and Cloudlets ............................ 4.1.6.1. Objectives and Design ................................................................... 4.1.6.2. Discussion of the Results ...................................................... 4.2. VM Migration .......................................................................................................... 4.2.1.Scenario 7: Increasing the No. Of Hosts, VMs and Cloudlets ............... 4.2.1.1. Objectives and Design ................................................................... 4.2.1.2. Discussion of the Results ...................................................... 4.2.2.Scenario 8: Increasing the No. Of Hosts ...................................................... 4.2.2.1. Objectives and Design ................................................................... 4.2.2.2. Discussion of the Results ...................................................... 4.3. Overall Discussion and Results Comparison ...................................................... Evaluation ............................................................................................................... 5.1. Evaluation According to Minimum Requirements ......................................... 5.2. Evaluating the Methodology ................................................................... 5.3. Evaluating the Results ............................................................................................. 5.4. Limitations .......................................................................................................... 5.5. Future Work .......................................................................................................... Conclusion ............................................................................................................... 6.1. Summary .......................................................................................................... 6.2. Problems Encountered ................................................................................ Bibliography ................................................................................................... Appendices ................................................................................................... 8.1. Appendix A: Project Reflection ................................................................... 8.2. Appendix B: Interim Report ................................................................... VIII

22 22 23 24 26 26 26 26 26 28 28 28 29 29 29 30 30 31 32 32 32 33 33 33 34 34 34 35 36 36 37 38 40 40 41 42 43 43 45 45 45 46 50 50 52

8.3. 8.4. 8.5. 8.6. 8.7.

Appendix C: Initial Project Plan ................................................................... 54 Appendix D: Revised Project Plan ................................................................... 56 Appendix E: Weekly Progress Report ................................................................... 58 Appendix F: Work in Progress Presentation ...................................................... 64 Appendix G: Collected Results of Scenario 1: Increasing the No. Of Hosts, VMs and Cloudlet DVFS Mechanism ............................................................................................. 67 8.8. Appendix H: A Source Code Sample of the CloudSim Java Classes ............... 72 8.9. Appendix I: Collected Results of Scenario 2: Increasing the No. Of Hosts DVFS Mechanism ....................................................................................................................... 83 8.10.Appendix J: Collected Results of Scenario 3: Increasing the No. Of VMs DVFS Mechanism ....................................................................................................................... 87 8.11.Appendix K: Collected Results of Scenario 4: Increasing the No. Of Cloudlets DVFS Mechanism .......................................................................................................... 91 8.12.Appendix L: Collected Results of Scenario 5: Increasing the No. Of Hosts and VMs DVFS Mechanism .......................................................................................................... 95 8.13.Appendix M: Collected Results of Scenario 6: Increasing the No. Of Hosts and Cloudlets DVFS Mechanism ............................................................................................. 98 8.14.Appendix N: Collected Results of Scenario 7: Increasing the No. Of Hosts, VMs and Cloudlets VM Migration ............................................................................................. 102 8.15.Appendix O: Collected Results of Scenario 8: Increasing the No. Of Hosts VM Migration ....................................................................................................................... 103

IX

1. Introduction
1.1. Overview
Cloud Computing has become an extremely attractive topic of research and application these days. NIST has defined Cloud Computing as a model for enabling convenient, on-demand network access to a shared pool of configurable Computing resources [1, p. 1]. Cloud security and storage have grasped lots of attention in this field [2]. In terms of storage, it acts as the lower layer of cloud infrastructure that supports the other layers functionality. Also, since it is important to make data storage redundant in different locations as contingency planning to ensure availability of cloud services, it would incur huge operating costs for the providers [2]. Therefore, this raises an important issue of how to improve the eco-efficiency of data centres to add value to the operational and maintenance costs of the cloud storage.

1.2. Aim and Objectives

Cloud Computing has gained huge popularity for what it can offer for business and individuals in terms of cost effectiveness. However, maintaining the Clouds data centres can cost lots of money and energy. Therefore, the overall aim of this project is to identify and evaluate a technique, such as DVFS, that can be used to improve the eco-efficiency and performance of Clouds data centres. In order to achieve this aim, a number of objectives have been outlined: Understand the importance of cloud computing, architecture of clouds, types of clouds, models of clouds, and virtualization. Understand some issues associated with Cloud Computing in terms of data centre management. Understand Cloud simulation, in particular CloudSim toolkit in terms of its usability, capability and limitations to offer a feasible evaluation of the eco-efficiency techniques used in data centres.

1.3. Minimum Requirements and Further Enhancements

The minimum requirements have been identified as follows: Design a simulated scenario of a cloud data centre using a DVFS mechanism Implement the scenario Collect and evaluate the simulation results Evaluate the mechanism used to improve the eco-efficiency of data centre management in Cloud Computing Further enhancements are: Design and implement additional scenarios of a Cloud data centre using a DVFS mechanism, and others without using a DVFS mechanism Design and implement additional scenarios of a Cloud data centre using VM Allocation mechanism Collect and evaluate the simulation results

1.4. Deliverables
The main deliverable of this project is to provide an evaluation and recommend the ecoefficiency technique/s that can be used to improve the performance of cloud data centres. Also, in order to do that, there will be the implementation of a set of CloudSim java classes to simulate the experiments (see Appendix H for a source code example).

1.5. Motives and Relevance to the Degree Programme

The primary motive is to find a challenging project that will allow me to develop new skills and enhance my knowledge in the field of Cloud Computing. Cloud Computing is currently a popular topic and is believed to be the future of the IT industry. Consequently, identifying and evaluating the eco-efficiency of cloud data centres is essential to its development. This project builds upon the knowledge and skills gained from several modules taken during the course of studying the MSc in Computing and Management. One of the modules, Advanced Distributed Systems, has provided a fundamental understanding of Web Services and Cloud Computing, from which this project will primarily be built. The knowledge and skills gained from Problem Solving with Computers, in terms of programming with Java and understanding the logic of 2

attempting to solve some IT problems, are very useful. These skills can be further enhanced to add value to this project. Business Information Systems increased the awareness of some key topics, such as choosing the appropriate methodology for this project. Techniques for Knowledge Management introduced key knowledge management topics; these topics will be investigated to evaluate the experimental results of this project. Additionally, knowledge learned from the business modules, which are Understanding Organisation, Marketing for Management, and Strategic Management, will be useful in enhancing the researching, critiquing, documenting, and management skills needed to construct this project.

1.6. Methodology
To achieve the aim of this project, the chosen methodology should offer a way to design and build an experimental scenario of cloud computing, implement the scenario, and evaluate its performance, and more specifically, the eco-efficiency, of the data centres of the cloud. There are some methodologies that can be adopted for this project, but it is important to determine which one would be more applicable. Direct experiments, mathematical modelling, and simulation methodologies are going to be investigated next. Firstly, direct experiments can be used to implement the scenario of a Cloud data centre in reality and obtain accurate results. However, it may be time consuming to build and implement the Cloud. It also requires many resources. This project, however, is restricted with a limited time scale. As such, this method is not feasible for this project. Secondly, mathematical modelling, involving mathematical equations, can be used to evaluate the energy and performance trade-off of cloud computing environments [3]. Therefore, this method may be appropriate for this project as it can offer benchmarking of the results. Nonetheless, thirdly, it would be better to use a simulation that not only offers mathematical equations to measure the energy and performance, but also provides an entire environment to simulate a scenario, starting from designing the desired experiments, running experiments, and showing the results after the calculations. More justification of using a simulation as a methodology for this project will be discussed in Section 2.3. There are different simulation tools that can be used for the experimental development of Cloud Computing (e.g. GreenCloud, CloudSim, and CloudAnalyst). GreenCloud, built on top of NS2, can be used to determine the detail of the consumed energy in the components of a data centre and 3

in the packet-level communications between these components [4]. However, because its simulation takes a long time and requires high memory usage, its scalability is limited only to small data centres [5]. CloudSim is an extensible toolkit used to model and simulate cloud infrastructures, including data centres, users, user workloads, and application provision environments [6] [7]. CloudSim can be scalable and should have low simulation overheads [5] [6]. CloudAnalyst is built on top of the CloudSim toolkit; it provides visual modelling, easy to use graphical user interfaces, and large-scale application simulations deployed on the cloud infrastructure. Application developers can use CloudAnalyst to determine the best approaches for allocating resources among available data centres to serve specific requests and determine the related costs for these operations [7]. After reviewing the options, it was determined that the CloudSim toolkit will be used for this project, due to the support of the specifications that come with it, as well as the support of some of the eco-efficiency techniques which will be addressed throughout the project. More details about CloudSim will be presented in Section 2.3. Briefly, the development approach of this project will start with analysing and designing a scenario of the simulated experiment, running the scenario, verifying the results, and finally, evaluating the results. More details about the development approach will be explained in Section 3.3.

1.7. Initial Project Plan

In order to control and manage the project, a Work Breakdown Structure (WBS) and a Gantt chart have initially been created during the planning stage, as shown in Appendix C. Also, some changes may take place during the implementation phase in the case of encountering any challenges. This project plan will help to track the completion of each development stage within the time-scale to ensure the delivery of a successfully completed project.

1.7.1. Key Activities

A. Initiating: this stage covers the initial meeting with the supervisor to discuss the project idea and set up the aim and minimum requirements for the project. B. Planning: in this stage, a feasible planned project structure has been set and initially agreed on with the supervisor. C. Executing: this stage includes the main activities of the projects development; the literature review and background research about Cloud Computing and eco-efficiency issues in data 4

centres; analysing and gaining a deep understanding of the issues and how to provide an evaluated solution; designing a simulated scenario of cloud computing to evaluate the ecoefficiency technique as solution, and finally, implementing the scenario. D. Evaluation: this stage involves evaluating the results of using the eco-efficiency technique. E. Controlling: this stage covers the progression status of the project including weekly progress reports and a work in progress presentation. F. Closing: this stage is made up of the write-up of the dissertation, finalising, and the submission of the dissertation.

1.7.2. Milestones
In order to measure the progress of the project and ensure meeting the aim and minimum requirements, milestones, known as self-goals, have been included in the project plan to indicate the starting point and deadline for each stage [8]. These milestones are: 1. Decide on Project (10/02/2012) 2. Decide on Planning (09/03/2012) 3. Analysis and Implementation (01/03/2012) 4. Monitoring (28/05/2012) 5. Evaluation (09/07/2012) 6. Preparing for the Final Project and Dissertation (16/07/2012) 7. End of Project (30/08/2012)

2. Review of Literature and Technologies

2.1. Introduction to Cloud Computing
To begin with, as the way people live and work together has evolved over time, so Information Technology (IT) has come to play a significant role in adapting to and meeting changing business requirements. Software systems have been developed from being monolithic one-tier systems to more complex and decoupled n-tier systems, which emphasises, along with the Internet evolution, the importance and need to have suitable integration technologies to enable businesses to communicate over networks [9]. Therefore, a significant evolution of integration technology has been the development of Service-Oriented Architecture (SOA), which is a pattern used for understanding and maintaining business activities that bridge large distributed systems [10]. Additionally, this development has introduced middleware technologies that enable communication and the sharing of resources between heterogeneous systems. Thus, having all these above mentioned technologies available, along with Grid Computing which is already in place, has led to the development of a new era of IT, which is Cloud Computing. Cloud Computing is considered to be a Strategic Inflection Point (SIP is a time of change in the life of an economy, a company, a society, or a person) in IT, as it introduced software technology for virtualisation, metalevels, and virtual integration in 2010. The SIP of Cloud Computing has arrived after a 15 year gap since the last SIP, which was the wide adoption of the internet in 1994 [11]. Also, Buyya et al [12] argue that based on the increasingly common perceived vision of computing, it will become the fifth utility (after electricity, gas, water, and telephony), providing the general public with the basic level of computing services used in their daily routines. The following subsections will provide further background information about Cloud Computing.

2.1.1. What is Cloud Computing?

Instead of individuals and businesses using their own PCs and other IT resources to perform their daily work, they can use the services offered by Cloud Computing at a reasonable cost, and leave the maintenance and cost of ownership to the cloud providers. There are many definitions of Cloud Computing suggested by different IT experts; each definition focuses on certain aspects of the Cloud. Vaquero et al [13] argue that to offer a comprehensive definition of Cloud Computing is like a large pool of easily usable and accessible virtualised resources, such as hardware, development 6

platforms, and/or services. These resources can be dynamically re-configured to adjust to a variable load (scale), allowing also for an optimum resource utilisation. This pool of resources is typically exploited by a pay-per-use model in which guarantees are offered by the Infrastructure Providers by means of customised Service Level Agreements (SLAs) (p. 51). Therefore, the main Cloud Computing characteristics and features, which can be derived from the definition above, are virtualisation, dynamic scalability, resource utilisation, and a business model of on-demand services based on SLAs. Therefore, Cloud Computing can be seen as providing useful opportunities for some businesses. Basically, it can offer economical and technological benefits. In terms of economic benefits, organisations can reduce the huge costs of large capital investments in IT resources by simply paying only for offered on-demand services (during peak time) from a Cloud provider, as well as lowering the cost of labour and IT resources maintenance [14] [17]. In terms of technological benefits, Cloud Computing can offer easier group collaboration, such as sharing documents between users from anywhere, regardless of their operating systems. Also, it can offer on-demand scaling of resources, such as acquiring unlimited storage capacity [14]. However, Cloud Computing comes with some disadvantages as well. For example, it requires constant connection to the Internet in order to access the services. Also, it mostly requires high bandwidth for data transfer [14]. More issues with Cloud Computing will be discussed in Section 2.2.

2.1.2. Architecture of Clouds

Cloud Computing architecture consists of multiple different layers, as is the case with most distributed systems. Foster et al [15] categorise Cloud Computing architecture, based on a comparison with Grid architecture, into four layers, which are fabric, unified resource, platform and application, as shown in Figure 1 below: Application

Cloud Architecture

Platform

Unified Resource

Fabric Figure 1. Four-layer Cloud Architecture [15] 7

Firstly, the fabric layer is the lower level layer of the architecture and includes the raw hardware resources, like storage resources. Secondly, the unified resource layer contains abstract resources, through virtualisation, that form integrated resources exposed to the upper layer and end users. Thirdly, the platform layer depends on the unified resources layer and includes an additional set of dedicated tools, middleware, and services in order to provide an environment for the applications development and deployment. Fourthly, the applications that run in the clouds are contained in the application layer [15]. Moreover, another study by Buyya et al [16] shows that Cloud architecture mainly consists of user-level middleware, core-middleware, and system level, as shown in Figure 2 below:

Figure 2. Layered Cloud Computing Architecture [16]

Beginning from the top of the architecture, Cloud application layers contain applications which can be accessed by end-users directly. Alternatively, users own applications can be deployed in this layer [6]. In addition, the user-level middleware layer contains software frameworks that assist the developer to create an environment for applications to be developed, deployed and executed in Clouds. Moreover, the platform level services that set the run-time environment are performed in the core-middleware layer to host and control services at user-level application. Finally, the system level layer is where massive physical resources, like servers, exist, and these resources are managed by the virtualisation services set above this layer [16].

2.1.3. Types of Clouds Services

Organisations can access different levels of services from Clouds with a pay-per-use policy. These types of Cloud services can be categorised based on the layers of Cloud architecture mentioned earlier. So, there are mainly three Cloud services, namely SaaS (Software as a Service), PaaS (Platform as a Service), and IaaS (Infrastructure as a Service). SaaS service is about offering applications, like Google docs, to end users to use on the Cloud vendors servers. Cloud vendors offer PaaS services, like Google App Engine, to allow users to host their applications instead of buying new resources to house them. Furthermore, IaaS service is about using Cloud providers virtualised low level resources; and this service can be named Everything as a Service. For example, if the user wants to use the storage of the vendors data centres, it can be named Storage as a Service [17]. Based on the architecture of Cloud Computing, SaaS is supplied at the user level, and PaaS is provided in the core middleware, and IaaS is supplied at the system level [6]. Data centres form the low level foundation of Cloud Computing, and these centres consist of thousands of inter-connected servers, usually located in areas with low probability of natural disasters and cheaper energy rates [18]. SaaS, PaaS, and IaaS services are sometimes provided by different cloud vendors.

2.1.4. Types of Clouds Deployment

There are many different types of Cloud deployment, which are mainly public, private, hybrid, community, and federated Clouds. Users can have open access, at fairly cheap cost, to use public Clouds services through the Internet, while access to private Cloud services is restricted to only those who own the Clouds and their subsidiaries [19]. Public Clouds offer IT resources and services to individual and businesses to save the overheads of having in-house built resources. Private Clouds allow organisations to utilise their IT resources efficiently (through virtualisation) as well as increase security since it can be managed internally. Moreover, the concept of hybrid Clouds is about connecting internal Clouds (private) to external Clouds (public) [11]. This combination usually happens when businesses using private Clouds for their business-critical systems need to scale up their resources at low cost by outsourcing more IT services from public Cloud providers to fulfil their non-critical system needs. Furthermore, a specific community of organisations who have common concerns can share their Clouds infrastructure to collaborate by forming a community Cloud [1]. Thus, a community Cloud can be formed through a combination of public, private, or hybrid Clouds.

Additionally, federated Clouds usually occur when two private Clouds owned by different organisations have a form of contract to share some of their Clouds resources, as forming an alliance can better utilise their IT resources effectively and gain the advantage of using others capabilities.

2.1.5. Virtualisation
Virtualisation is a vital technology of Cloud Computing which offers two important featuresabstraction and encapsulation [15]. It is about creating an abstract layer between hardware and software. Usually, the virtualisation layer is set above the physical layer of the Clouds architecture. Virtualisation technology is used widely in Cloud Computing data centres owing to the benefits offered, such as utilising resources, lowering costs, easier management of servers, server consolidation, and live migration of virtual machines [20]. Virtualisation is mostly used in Cloud Computing platforms as means to optimise resource usage [21]. Through virtualisation, the number of hardware resources used in Clouds can be reduced to minimise the capital cost as well as the cost of power consumption and cooling systems [22]. For instance, through server consolidation, multiple (virtual) servers can be allowed to run simultaneously on a single physical server. Also, live migration of the virtual machine to the not fully utilised physical servers would allow more and more physical servers to be turned off, which would lead to better achievement of energy efficiency for data centres [20] [15]. Furthermore, visualisation in Cloud Computing can offer dynamic configurations for different applications resource requirements, and aggregate these resources for different needs. Additionally, it can improve responsiveness by monitoring, maintaining and provisioning resources automatically. Therefore, all these features offered by virtualisation are used in Clouds in order to meet the criterion of the business requirements of SLAs [15]. Moreover, there are mainly two ways of virtualisation deployment, as stated by Hardy et al [22]. Firstly, when the virtualisation layer is installed on an existing operating system as an application, it is called hosted system virtualisation or application virtualisation. Secondly, when the virtualisation layer is installed directly on clean x86-based hardware, instead of a conventional operating system, it is called hypervisor systems, and is usually used for server consolidation [22]. Furthermore, there are several types of virtualisation, some of which are full virtualisation, hardware assisted virtualisation, partial virtualisation, paravirtualisation, hybrid virtualisation and operation system-level virtualisation [23]. Full virtualisation allows a guest operating system to be run unmodified and in isolation by simulating, almost completely, actual hardware. Hardware assisted virtualisation is about improving the efficiency of a guest environment by improving the 10

computational performance of a virtual machine without interpreting and isolating the instructions passed directly to the host processor. Partial virtualisation is about simulating most but not the entire underlying host environment. Paravirtualisation, hybrid virtualisation, and operating systemlevel virtualisation can be deployed as a way of utilising partial virtualisation. Paravirtualisation provides a similar software interface or APIs of virtual machines to that of the underlying host hardware. Hybrid virtualisation is about combining the advantages of both hardware assisted virtualisation and paravirtualisation to obtain better performance than with software-only paravirtualisation [24]. Finally, operating system-level virtualisation is about creating multiple isolated virtual machines with the requirement that the OS of the virtual machine should be the same as the host [23].

2.2. Issues Associated with Clouds

As mentioned earlier, Cloud Computing, as the latest technology in the IT industry, has attracteds lots of businesses nowadays owing to its promising benefits, such as cost reduction for capital investment, scalability, on-demand services, and energy saving. However, there are some issues associated with clouds, such as security, availability of services, reliability of data, unpredictable performance, and the risk of data lock-in [25]. Firstly, since the data of consumers is stored somewhere in the Cloud and not on their site anymore, security issues have become one of the major concerns for consumers use of Cloud Computing technology. Also, private data leakage would raise a concern, as consumers data has to be moved from a single-tenant to a multi-tenant environment [26]. Moreover, businesses logically would not be willing to be in the position of putting all their eggs in one basket by moving all their business data to the Cloud where it could be vulnerable to hacking attacks. For example, Distributed Denial of Service (DDoS) attacks can be set by a hacker by hosting a malicious application in a cloud which his victim, another user, is subscribing to. Launching these attacks with a large number of zombie machines from multiple resources can pass heavy flood of packets that may affect the performance or availability of the service. However, Sabahi [26] argues that Clouds can be used as a solution against these DDoS attacks because the Cloud can offer the flexibility of providing more resources instantly, to tolerate such attacks and avoid site shutdown. Even so, more investigations need to be carried out to discover to what extent Cloud technology can handle this type of attack in order to ensure the availability and reliability of the services.

11

Moreover, the key strategy of Cloud providers offering services at cheap cost is to attract more customers till they gradually move their core business requirements to the Cloud, which would then increase the risk of data lock-in. After relying on the Cloud, it would be very difficult for the consumer to move back to their own system because it would require some additional costs. Therefore, the Cloud providers can take advantage of consumers being bound to their services and include more add-on services. Furthermore, there are some cost issues associated with the Clouds in terms of maintenance and management of the resources, especially the data centres, which will be discussed next.

2.2.1. Cost of Clouds Maintenance

To begin with, Total Cost of Ownership (TCO) is not only about the capital expenditure required to own an IT infrastructure, but also about the ongoing cost of managing and operating the IT infrastructure in a business [27]. Therefore, in addition to buying hardware and software assets to build the infrastructure of the Cloud and provide services, there are more ongoing costs, like having highly skilled IT experts employed to maintain and ensure its availability and performance all the time. Also, as a Cloud provides more and more services to a large consumer base, enhancement of resource utilisation is needed to meet the demand and gain maximum economies of scale by producing more output with less expensive resource consumption. Besides, since Cloud providers follow a strategy of offering Cloud services at cheap cost to attract more consumers to use their businesses, the added-value activities for maintaining the Clouds should be at the lowest cost possible in order enable the Cloud providers to achieve their strategy as well as gain a competitive advantage. Thus, such techniques should be implemented to reduce the cost of maintaining the data centres, which will be discussed more in the next Section.

2.2.2. Eco-efficient Data Centre Management

Eco-efficiency can be directly linked to being environmentally friendly. It is about how to manage the data centres of the Clouds in ways that have less impact on the energy consumed, as well as on carbon dioxide (CO2) emissions. Therefore, mechanisms and policies should be put in place to help understand how green the data centre of a Cloud is. The dramatic increase in greenhouse gas emissions is having a detrimental effect on the global climate, like increasing temperatures, dryness, and floods. Also, the ICT industry has been

12

contributing to this growth; as has been stated by Smarr, who explains that the carbon emission from this sector is expected to triple from 2002 to 2020 [28]. Cloud infrastructure is likely embodied by enormous and power-consuming data centres that generate large amounts of heat when processing. So, in order to effectively remove this heat, data centres are equipped with great cooling systems that consume huge amounts of energy in order to work [29]. Therefore, there is great pressure on Cloud providers to improve such mechanisms for the energy efficiency of Cloud data centres to both reduce costs and support the prices offered to customers, as well as to ensure meeting QoS to satisfy the customers expectations [29]. 2.2.2.1. A Scientific Model of Eco-efficient Data Centre There are some metrics that can be used in order to measure and understand how eco-efficient a Cloud data centre is. For example, Power Usage Effectiveness (PUE) is a well known metric that can be used to define how eco-efficient the infrastructure of a data centre is. PUE has been defined by The Green Grid organisation as a measurement for identifying how a data centre uses its power efficiently by actually calculating how much of the power is consumed by the servers in contrast to other overheads, like cooling [30] [31]. To illustrate, PUE is calculated by dividing the Total Data Centre Energy Usage by the IT Energy Usage (PUE = Total Energy/IT Energy). The total energy includes IT energy plus the energy consumed for lighting, security, CCTV, heating and cooling, and all other facilities in the data centre. As has been stated by Gillan [32], the average PUE efficiency in typical data centres these days is about 2.0. So, as the PUE increases, the more inefficient the data centre infrastructure is, and the closer the PUE is to 1.0, the more efficient the data centre infrastructure is considered to be. Thus, PUE optimisation aims to lower the total energy consumed by all facilities, to be almost equal to the energy consumed by IT equipment. In other words, it means minimising the energy used by the facilities, other than IT equipment, to almost zero consumption. However, using PUE metrics alone is not enough to consider data centre infrastructure to be very eco-efficient. Therefore, a new metric, Energy Reusable Efficiency (ERE), has been defined by the Green Grid. This metric measures how efficiently the wasted energy that leaves the data centre is reused in adjacent, non-data centre spaces [33]. For example, the heat generated from the data centre can be converted beneficially to electricity by using reverse heat engines and technologies. The energy reproduced should be for use somewhere else on the campus, but outside the control volume boundary of the data centre, so that it will not affect PUE measurement [33]. So, both PUE and ERE are useful in analysing the efficiency of energy usage in a data centre, but the former 13

assesses the efficiency of the energy used by the data centre infrastructure, and the latter assesses the efficiency of altering the wasted energy that leaves the data centre to be reused in other spaces outside the boundary of the data centre [34]. Moreover, it has been argued that PUE may be a poor operational metric because it does not consider IT efficiency or IT load distribution [35] [36]. In fact, more optimal energy savings would be achieved theoretically when there is utilisation of the IT resources to reduce the running machines and further reduce the energy consumed [36]. Virtualisation is a very important part of utilising the resources, as discussed before in Section 2.1.5, but more utilisation for resources like CPU utilisation can add more value to energy efficiency and cost reduction. Also, as the technology advances, producing new servers that run faster and consume the same energy as legacy servers, it can be beneficial to have a life cycle strategy for replacement. For instance, replacing the old servers with new powerful servers would help to increase the energy efficiency of data centres by reducing the number of servers required to deliver the same IT workload [36]. Thus, the definition of a scientific model of eco-efficient data centre depends on how efficiently the energy is consumed in a data centre with minimum costs and less, or most favourably no, impact on the environment in terms of CO2 emissions. There are a number of metrics that should be used to measure the efficiency of energy usage in the data centre from different aspects. As new metrics are being developed, it would help to drive the data centres towards being more eco-efficient. 2.2.2.2. Management of Power Consumption The energy supplied in data centres is consumed for computational operation, cooling systems, networks, and other overheads. In terms of computational operations, there are some energy-saving techniques that can be deployed to monitor and control energy consumption. These eco-efficient energy techniques have become one of the hot topics in the IT business these days because of the benefits gained, not only from an economic perspective, but also from an environmental perspective. Cloud service providers can save huge costs by efficiently utilising their data centre to its maximum capacity. Also, governments pressure companies to conduct their businesses with less impact on the environment, in terms of the emission of CO2. Hence, Clouds service providers can also improve their SLAs by being friendly to the environment to attract more customers. Furthermore, in terms of power efficiency, it can be said that data centre A is better than data centre B if A can consume less power and process the same workload as B, or A can consume the same power but with more workload compared with B [37]. Thus, lots of research has been

14

conducted to suggest some techniques for reducing power consumption without degrading the performance in the data centres, some of which will be discussed next. 2.2.2.2.1. VM Allocation Firstly, a study by Corradi et al states that Virtual Machine (VM) consolidations can be used as a means of reducing the power consumption of cloud data centres [37]. To illustrate, this technique tries to allocate more VMs on less physical machines as far as possible to allow maximum utilisation of the running of physical machines. For instance, when there are two VMs, instead of allocating each one to a physical server that has not been fully utilised, this technique tries to allocate both VMs on one physical server and switch the other server off to save energy. Therefore, using this technique in a data centre can reduce the operational costs and increase the efficiency of energy usage. However, it is important to note that the number of VMs in one physical machine should not be too high to the extent that it may degrade the performance of VMs [37]. 2.2.2.2.2. Power and Thermal Management Furthermore, another study conducted by Pakbaznia et al emphasises the importance of developing power and thermal management (PTM) to improve the energy efficiency of data centres [38]. Basically, this approach supports making the decision for the number and placement of ON servers, as well as maintaining the supplied cold air temperature, in order to reduce the power consumption for both the servers and cooling units. However, in order for this approach to achieve its goal, it requires a prediction of the incoming workload in terms of requests per second, which can be achieved by using Workload Monitoring (WM). Thus, as stated in this study, this approach endeavours to use a combination of chassis consolidation and efficient cooling to reduce the overall power consumption [38]. 2.2.2.2.3. Dynamic Voltage and Frequency Scaling Dynamic Voltage and Frequency Scaling (DVFS) can be used as one of the eco-efficiency techniques to reduce the energy consumed in computing servers. This technique observes the workload offered and then alters the CPU power consumption accordingly, which would then change the performance level as well [39]. Also, DVFS along with Dynamic Power Management (DPM) have been widely deployed and extensively studied [4]. Usually, the idle servers consume two-thirds of the peak load in order to keep the other components, like memory and disks, up and running; and the remaining one-third of the consumption is based on the CPU utilisation, which can be managed by DVFS. On the other hand, the DPM technique may be more efficient by powering

15

down the servers, including all the components. Nonetheless, DPM would consume a greater amount of energy compared to DVFS when there is a need to turn these servers back on to run [39]. It can be deduced from the simulated experiments conducted by Kim, Beloglazov, and Buyya [40] that data centre providers can increase their profits by reducing power consumption through the deployment of a DVFS technique. A DVFS technique is designed in a way that decreases the energy consumption of the CPU by controlling and reducing the supply of voltage and frequency.

2.3. CloudSim Toolkit

IT companies who are willing to offer some services in the Cloud can use a simulation-based approach to perform some benchmarking experiments with the services to be offered in dependable, scalable, repeatable, and controllable environments before real deployment in the Cloud [6]. Therefore, they can test their services in a controlled environment free of cost, and through a number of iterations, with less effort and time. Also, by using simulation, they can carry out different experiments and scenarios to identify the performance bottlenecks of resources and develop provisioning techniques before real deployment in commercial Clouds [6] [41]. Therefore, CloudSim has been developed to fulfil these requirements by simulating and modelling extensible Clouds; more details about CloudSim will be discussed next.

2.3.1 Architecture of CloudSim

CloudSim can be defined as a new, generalised, and extensible simulation framework that allows seamless modelling, simulation, and experimentation of emerging Cloud Computing infrastructure and application services (p.24) [6]. Initially, the framework of CloudSim consists of multiple layers starting from the lowest layer of SimJava up to the top layer of User Code, as shown in Figure 3.

16

Figure 3. Initial Layered CloudSim Architecture [16] At the lowest layer, SimJava provides the base engine of the simulation that supports the implementation of core functionalities essential for the higher-level frameworks of the simulation, like queuing and processing of events; formation of system components (services, hosts, brokers, VMs); interaction between these components, and administration of the simulation clock [16]. On top of that layer is the GridSim layer which supports modelling high-level and fundamental Grid components, such as networks, resources, data sets, and information services [16]. Then, the CloudSim layer forms the next level of the architecture that extends the core functionalities of the GridSim layer. This layer supports modelling virtualised Cloud-based data centre environments, including VMs, memory, storage and bandwidth. Also, this layer can manage instantiating and simultaneously implementing a large scale Cloud infrastructure composed of thousands of system components (VMs, hosts, data centres, and application) [16]. Finally, User Code is the top-most layer of the simulation toolkit, which reveals the configuration of functionality for the system components, such as the number and specification of hosts and the scheduling policies of the broker [16]. At this layer, a developer can model and perform robust experiments and scenarios of Cloud environments based on custom policies and

17

configurations already supported by the CloudSim, in order to evaluate and tackle some Cloud issues like the complexities of Cloud infrastructure and application. Moreover, the architecture of the CloudSim toolkit has been reviewed and changed with its new release, as can be seen in Figure 4.

Figure 4. Current Layered CloudSim Architecture [6]

Therefore, based on Figure 4, the current layers of the simulation architecture are the CloudSim core simulation engine, CloudSim, and User Code. The SimJava layer in the previous release has some limitations, like not allowing the simulation to programmatically reset at run-time, not supporting a new simulation entity to be created at run-time, and the SimJava nature of being multithreaded can cause performance overheads with the increasing size of the system. Thus, SimJava has been removed and replaced with CloudSim core simulation engine in the new release of CloudSim to overcome these limitations and enable more advanced operations that were not backed up by SimJava [6].

2.3.2 Usability
In order to use the CloudSim toolkit, users need to have a basic background in Java programming language because it is written in Java [6]. Also, it requires users to write some code to use the components from its library in order to simulate the desired scenarios. Therefore, it is not 18

just about setting the parameters, running the program, and collecting the results, but it also requires a deep understanding of how the program works. In addition, a little knowledge about Integrated Development Environments (IDEs), like NetBeans or Eclipse, will be useful to ease installing the toolkit and the development of scenarios. Furthermore, CloudSim provides a library that can be used to build a ready-to-use solution, such as CloudAnalyst which is built on top of CloudSim, to offer an easy to use graphical user interface [7].

2.3.3 Capabilities
CloudSim has some compelling features and capabilities that can be extended to model a custom Cloud Computing environment. According to Calheiros et al [6], CloudSim can offer flexibility and applicability and with less time and effort to support initial performance testing. It can support simulating, from small-scale up to large-scale cloud environments containing data centres, with little or almost no overheads in terms of initialisation and consumption of memory. Also, it has a virtualisation engine that allows the creation of multiple virtualised services that can be independently managed on a single node of the data centre. Moreover, it supports, in addition to other features, energy-awareness provisioning techniques at resource, VM, and application level, such as VM allocation and DVFS [6]. For managing the energyconscious techniques in a data centre, CloudSim architecture contains the key components CloudCoordinator, Sensor, and VMM. The Sensor component, which is attached to every host, is used by the CloudCoordinator to monitor particular performance parameters, like energy consumption and resource utilisation. Thus, through the attached Sensors, CloudCoordinator passes real-time information, like load conditions and processing share, of the active VMs to the VMM. Then, VMM uses this information to perform the appropriate application of DVFS and resizing of VMs. Also, according to VMs policy and current utilisation of resources, CloudCoordinator constantly issues VM migration commands and changes the power state of nodes to adapt the allocation of VMs [6].

2.3.4 Limitations
CloudSim is a powerful tool for modelling and simulating Cloud computing, but it has some limitations. Firstly, it is not a ready-to-use tool that that would just require setting parameters only. Actually, it does require writing some Java code to use its library, as discussed earlier. Also, the capabilities of CloudSim are sometimes limited and require some extensions. For instance, CloudAnalyst has been developed as an extension of CloudSim capabilities to offer a separation of

19

the simulation experimentation exercise from the technicalities of programming, using the library in order to ease modelling by simply focusing on the complexity of the simulated scenario, without spending much effort and time on the language in which the simulator is interpreted [7].

20

3. Methodology
3.1. Project Management
Managing the project appropriately by using the right tools can be a key factor to its success [42]. Project management should involve the activities of planning, follow-up and control, and ensure the project is being completed according to the specification and within the permitted budget and timescale.

3.1.1. Revised Project Plan

For the project undertaken, a project plan was initially created, as discussed earlier in Section 1.7. But, because of uncertainties about the hurdles encountered in the early stages, the project plan was revised during the Implementation stage because it took longer than initially expected. The reasons for revising the plan are the lack of a user manual for how to use the CloudSim toolkit and time spent on running the experiments. There was no user guide or online tutorial for how to use the toolkit; therefore, it took a while to perform reverse engineering to understand the simulation source code and how it works and behaves. Also, running the experiments took a very long time. More details of these issues and how they were solved will be explained in Section 6.2. In addition, more time was allocated for the Implementation stage to add more novelty to the project by implementing further scenarios instead of having only one. Thus, the project plan has been revised without exceeding the limited timescale for the project deadline (see the revised plan in Appendix D). 3.1.1.1. Key Activities The key activities for the project are still the same as initially discussed in Section 1.7.1. 3.1.1.2. Milestones The start dates for some of the milestones have been updated, as follows: 1. Decide on Project (10/02/2012) 2. Decide on Planning (09/03/2012) 3. Analysis and Implementation (01/03/2012) 21

4. Monitoring (28/05/2012) 5. Evaluation (09/08/2012) 6. Preparing for the Final Project and Dissertation (30/07/2012) 7. End of Project (30/08/2012)

3.1.2. Progress Meetings

In order to track the progression status of the project, weekly progress meetings with the supervisor have been conducted almost from the beginning of the project till the end. During these meetings, it was very useful to discuss the progress of the project, reveal any issues encountered, and set plans based on the discussion output. After the submission of the interim report, weekly progress reports were documented and brought to the meetings for tracking and discussing the tasks accomplished for that week, and setting the tasks to be completed for the following week (see these weekly progress reports in Appendix E). In addition, a work in progress presentation was conducted with both the supervisor and assessor to show and discuss the overall progress of the project and obtain feedback to enhance the project accordingly (see the presentation slides in Appendix F).

3.2. Problem Summary

Cloud infrastructure is equipped with large and power consuming data centres that bring challenges to managing the power consumption eco-efficiently [29] [40]. As has been stated by Scheihing [43], data centres consume about 10 to 100 times more energy per square foot than typical office buildings. They can even consume the same amount of electricity that is used by a city [44]. Also, high energy consumption in a data centre can lead to extreme heat dissipation which would raise the cooling costs and make the servers vulnerable to risks and failures. All these issues would also inflate the operational costs in order to maintain and keep the data centre functioning [45]. Additionally, governmental policies across the world push data centre providers to reduce energy waste and become greener with regard to the environment. Thus, using certain eco-efficient mechanisms, like DVFS and VM allocation, to reduce the energy consumption can help to enhance the sustainability and profitability of the data centres and make them greener.

22

3.3. Project Methodology

As discussed previously, the methodology will be based on using a simulation to evaluate the mechanisms that can be used to improve the eco-efficiency of data centres. Thus, the development approach of this project will start with analysing and designing a scenario of the simulated experiment, running the scenario, verifying the results, and finally, evaluating the results. The designing, running and verifying of the stages will follow an iterative approach, as shown in Figure 5, to ensure the appropriateness of the simulated scenario for this project.

Figure 5. An iterative approach for experiments development

The main reason for following an iterative approach is to give more flexibility to quickly altering the settings of the design of the experiments in case of any mistake or error that would terminate running and completing the experiments. Also, after completing each experiment successfully, the iterative approach allows going back and changing the objectives of the experiment in order to investigate more aspects of Cloud data centres. The analysis stage is more about understanding the background to the CloudSim toolkit, how it has been used by others to simulate some experiments, and setting the objectives for the experiments to be simulated. The design stage is more about forming the settings for the experiments according to the objectives. Running the experiments and collecting the results then takes place. After that, verifying and interpreting the results and making sure that the experiment is completed successfully is done. Finally, evaluating and contrasting the results with the other work already carried out in the literature takes place.

23

3.4. Analysis and Design of Simulated Experiments

As discussed previously, the methodology adopted for this project is based on simulating some experiments by using the CloudSim toolkit to evaluate eco-efficiency mechanisms in Cloud data centres. Therefore, as mentioned in Section 2.3, CloudSim has been analysed and reviewed in terms of its usability and capabilities of offering an environment for experimenting with some scenarios of Cloud data centres. The main objective of the experiments is to evaluate the energy consumption in data centres. In terms of the design, CloudSim version 2.1.1 has been used because it has more features than the earlier release, as discussed in Section 2.3.1., and it provides examples of power provisioning techniques that can be used for conducting the experiments in this project. CloudSim consists of significant classes that form the building blocks of the simulator. For example, Cloudlet is the class that forms Cloud-based applications services like business workflow, content delivery and social networking [6]. For each life cycle of the Cloudlet execution, it has to undertake the pre-assigned instruction length and data transfer overhead including its creation, allocation to VM, scheduling in VM, and termination [46]. Also, Host class is responsible for modelling the physical resources like storage or compute servers. This class encapsulates essential details about the host, such as the amount of memory and storage, and lists and types of the processing cores. Furthermore, Vm class models the VM component which is hosted and managed by a Cloud host component that stores the related characteristics to the VM, including accessible memory, storage size, and processor [6]. Therefore, as a design for the scenarios to be experimented, they will focus on altering the settings of three parameters, which are the number of hosts, VMs, and Cloudlets. The characteristics of these three parameters will be kept as the typical values already set in CloudSim as it has already been experimented and validated by Calheiros et al to quantify CloudSim efficiency in simulating Cloud Computing environments [6]. Also, the values of these characteristics represent the typical settings of the machines found in a data centre. These typical characteristics of hosts, VMs, and Cloudlets are shown in Table 1.

24

Table 1. Typical characteristics of hosts, VMs, and Cloudlets Hosts Max Power Static Power MIPS RAM Storage Bandwidth VMs MIPS Rating Number of CPUs RAM Bandwidth Image Size Cloudlets Length Number of CPUs File Size Output Size 150000 (equivalent to 10 mins execution on 250 MIPS) 1 300 300 { 250, 500, 750, 1000 } 1 128 MB 2500 Mbps 2500 MB 250 Watts 0.7 (70%) { 1000, 2000, 3000 } 10000 MB 1000000 MB 100000 Mbps

Thus, provisioning requests for each VM would demand 1 CPU core (with variation of 250 MIPS to 1000 MIPS), 128 MB of RAM, and 2.5 GB of storage. Moreover, the simulation has some randomness with the results of the experiments. Therefore, the experiments will be run N-times to increase the reliability of the results by having a sample size and then calculating the mean values, standard deviation, and confidence interval with a confidence level of 95%- a commonly used level that will give very conservative results, as stated by Payton et al [47].

25

4. Implementation of Experiments
This chapter will show some simulated scenarios of a Cloud data centre using CloudSim, with the overall objective of evaluating the techniques based on the variation of energy consumption metric. Firstly, it will include several scenarios conducted with the deployment of a DVFS mechanism. Furthermore, it will show two more simulated scenarios using VM migration with DVFS enabled. Each scenario will have a number of experiments that will vary based on the value settings of the key parameters, hosts, VMs and Cloudlets, as discussed in the previous Section 3.4. They will start with the default values of 10 hosts, 20 VMs, and 20 Cloudlets as the minimum values, and then they will be incremented to represent the higher scales of a data centre for each succeeding experiment. The maximum values for each scenario will vary from one to another based on the objectives, as will be stated later on.

4.1. DVFS Mechanism

4.1.1. Scenario 1: Increasing the No. of Hosts, VMs, and Cloudlets
4.1.1.1. Objectives and Design The objectives of the following experiments are to evaluate how increasing the number of resources and services would reflect on the energy consumption; how the deployment of a DVFS mechanism would contribute towards reducing energy consumption compared to an environment without the deployment of this mechanism, and what the maximum parameters values that CloudSim can handle are. The parameter settings for the number of hosts, VMs, and Cloudlets will initially be according to the default values already set in CloudSim, and then they will be doubled each time in order to observe the variation in energy consumption. 4.1.1.2 Discussion of the Results The experiments for the two scenarios, with and without the deployment of DVFS, have been run and the results have been collected and shown in Table 2 and Table 3 in Appendix G. Thus, in terms of the parameter settings of this scenario, experiment 1 has the minimum values of 10 hosts, 26

20 VMs, and 20 Cloudlets, and experiment 15 has the maximum values of 4603 hosts, 9206 VMs, and 9206 Cloudlets. From the results collected and calculated, Figure 6 below has been produced to explain the difference between the two scenarios based on the mean values of the energy consumption metric.
600 500 400 300 200 100 0 Experiment 10 Experiment 11 Experiment 12 Experiment 13 Experiment 14 -100 Experiment 15 Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7 Experiment 8 Experiment 9 Without Power Aware Mechanism

With DVFS Mechanism

Figure 6. Comparison between energy consumption mean values for each experiment

Based on the results shown in Figure 6, doubling the number of resources and services would also double the energy consumption for both scenarios, with and without the deployment of DVFS. However, the energy consumed with the deployment of a DVFS mechanism is about three times less than the energy consumption with the scenario with no deployment of DVFS. Therefore, the deployment of DVFS would save about 2/3 of the consumed energy because the DVFS mechanism seeks to decrease the dynamic energy consumption by reducing the supplied voltage and frequency [40]. Also, this scenario should have stopped at experiment 10 where it cannot accept doubling the parameters values any more, but it was carried out for the purpose of finding out the maximum parameter values that CloudSim can handle, which took approximately 31 hours to complete. As can be observed from experiment 10 to 15, the scenario has changed, since the values of the parameter settings are not doubling this time because of the limitation of CloudSims ability to handle them. As a result of this particular scenario, the purpose of finding the maximum values that CloudSim can handle has been achieved, and they are 4603 Hosts, 9206 VMs, and 9206 Cloudlets, as shown in experiment 15 in Appendix G.

27

4.1.2. Scenario 2: Increasing the No. Of Hosts

4.1.2.1. Objectives and Design The objectives of the following experiments are to evaluate how increasing the number of hosts would reflect on the energy consumption, and how the deployment of a DVFS mechanism would contribute towards reducing energy consumption compared to an environment without the deployment of this mechanism. The parameter settings for the number of hosts, VMs, and Cloudlets will initially be according to the default values already set in CloudSim, and then the number of hosts will be doubled each time to observe the variation in energy consumption. 4.1.2.2. Discussion of the Results Based on the results collected from the two scenarios, with and without the deployment of DVFS, shown in Table 4 and 5 in Appendix I, Figure 8 below has been produced to show the difference between the two scenarios based on the mean values of the energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 14 has the maximum values of 81920 hosts, 20 VMs, and 20 Cloudlets.
9000 8000 7000 6000 5000 4000 3000 2000 1000 0 -1000 With DVFS Mechanism Without Power Aware Mechanism

Figure 8. Comparison between energy consumption mean values for each experiment

28

Based on the results shown in Figure 8, doubling the number of machines would also double the energy consumption for the scenario without the deployment of DVFS, because more machines would be running and waste energy if there is no power control mechanism. But with the deployment of DVFS, the energy consumed is almost unchanged and stays the same for each experiment because only the number of machines required are running to provide the services, while the rest of the machines are not running, which would be reflected greatly in saving energy.

4.1.3. Scenario 3: Increasing the No. Of VMs

4.1.3.1. Objectives and Design The objectives of the following experiments are similar to scenario 2, but this time the number of VMs will be increased, while the Hosts and Cloudlets remain constant. Thus, the design of the experiments will involve doubling the value only of the VMs parameter each time to observe the variation in energy consumption. 4.1.3.2. Discussion of the Results Based on the results collected from the two scenarios, with and without the deployment of DVFS, shown in Table 6 and 7 in Appendix J, Figure 9 below has been produced to show the difference between the two scenarios based on the mean values of energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 14 has the maximum parameter values of 10 hosts, 163840 VMs, and 20 Cloudlets.

29

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

With DVFS Mechanism Without Power Aware Mechanism

Figure 9. Comparison between energy consumption mean values for each experiment

Based on the results shown above in Figure 9, doubling the number of VMs from experiment 1 to 2 increased the energy consumption by about three times for the scenario with the deployment of DVFS, and then the energy consumption stayed the same for the subsequent experiments. Interestingly, this scenario may indicate that the number of under-utilised VMs used has increased from experiment 1 to 2, since the number of services and physical machines has stayed the same. In terms of the scenario without the deployment of DVFS, it remained almost constant and unchanged for all the experiments. However, the deployment of DVFS in experiment 1 saved about 70% of energy consumption compared to the other scenario. Also, for the remaining experiments, it still saved about 16% of the consumed energy compared to the scenario without the deployment of DVFS.

4.1.4. Scenario 4: Increasing the No. Of Cloudlets

4.1.4.1. Objectives and Design The objectives of the following experiments are similar to the previous scenarios, but this time it will increase the number of services while the Hosts and VMs remain constant. Thus, only the value of the Cloudlets parameter will be doubled each time to observe the variation in energy consumption.

30

4.1.4.2. Discussion of the Results Based on the results collected from the two scenarios, with and without the deployment of DVFS, shown in Table 8 and 9 in Appendix K, Figure 10 below has been produced to show the difference between the two scenarios based on the mean values of the energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 13 has the maximum parameter values of 10 hosts, 20 VMs, and 61440 Cloudlets.
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 With DVFS Mechanism Without DVFS Mechanism

Figure 10. Comparison between energy consumption mean values for each experiment

Based on the results shown above in Figure 10, doubling the number of services has not, surprisingly, increased the energy consumption for both scenarios. They almost stayed the same for all the experiments. Also, the figure above clearly shows that the deployment of a DVFS mechanism saves about 70% of energy consumption compared to the other scenario without the deployment of this mechanism.

31

4.1.5. Scenario 5: Increasing the No. Of Hosts and VMs

4.1.5.1. Objectives and Design The objectives of the following experiments are similar to the previous scenarios, but this time it will involve increasing the number of Hosts and VMs only, while the number of Cloudlets remains constant. Thus, the value of Host and VM parameters only will be doubled each time to observe the variation in energy consumption. 4.1.5.2. Discussion of the Results Based on the results collected from the two scenarios, with and without the deployment of DVFS, shown in Table 10 and 11 in Appendix L, Figure 11 below has been produced to show the difference between the two scenarios based on the mean values of the energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 10 has the maximum parameter values of 3840 hosts, 7680 VMs, and 20 Cloudlets.
400 350 300 250 200 150 100 50 0 -50 -100 Without Power Aware Mechanism With DVFS Mechanism

Figure 11. Comparison between energy consumption mean values for each experiment

Based on the results shown in Figure 11 and tables 10 and 11 in Appendix L, doubling the number of Hosts and VMs would also double the energy consumption for both scenarios, with and

32

without the deployment of DVFS. But, interestingly, with the deployment of DVFS, the energy consumed has increased three times from experiment 1 to 2, and for the subsequent experiments it just doubled. Similar to scenario 3, this increase may indicate that the number of under-utilised VMs used has increased from experiment 1 to 2 since the number of services stayed the same. The energy consumed in experiment 1 with the deployment of a DVFS mechanism is almost three times less than for the same experiment without the deployment of this mechanism. Also, for the remaining experiments, the energy consumption for the DVFS scenarios was almost twice as less than the consumption with the other scenario.

4.1.6. Scenario 6: Increasing the No. Of Hosts and Cloudlets

4.1.6.1. Objectives and Design The objectives of the following experiments are similar to the previous scenarios, but this time it involves increasing the number of Hosts and Cloudlets only, while the number of VMs remains constant. Thus, the value of Hosts and Cloudlet parameters will be doubled each time to observe the variation in energy consumption. 4.1.6.2. Discussion of the Results Based on the results collected from the two scenarios, with and without the deployment of DVFS, shown in Table 12 and 13 in Appendix M, Figure 12 below has been produced to show the difference between the two scenarios based on the mean values of the energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 13 has the maximum parameter values of 30720 hosts, 20 VMs, and 61440 Cloudlets.

33

3500 3000 2500 2000 1500 1000 500 0 -500 Without Power Aware Mechanism With DVFS Mechanism

Figure 12. Comparison between energy consumption mean values for each experiment

Based on the results shown in Figure 12 and the tables in Appendix M, doubling the number of Hosts and Cloudlets would also double the energy consumption for the scenario without the deployment of DVFS. But, with the deployment of DVFS, the energy consumed has remained the same. These results are exactly the same as the results shown in the previous scenario 2. However, the only difference is that in this scenario, CloudSim accepts doubling the values till experiment 12 and in experiment 13 the values have been increased by just 50%, while in scenario 2 it continues by handling two more experiments till experiment 14 without crashing the simulation. This may suggest that increasing the number of Cloudlets would not affect the energy consumption, as this can also be observed in scenario 4 as well.

4.2 VM Migration
4.2.1. Scenario 7: Increasing the No. Of Hosts, VMs, and Cloudlets
4.2.1.1. Objectives and Design The objectives of the following experiments are to evaluate how increasing the number of resources and services would reflect on the energy consumption in an environment with the deployment of a VM migration mechanism with DVFS enabled, compared to those results shown in scenario 1 with DVFS mechanism deployment only. 34

The parameter settings for the number of hosts, VMs, and Cloudlets will initially be according to the default values already set in CloudSim, and then they will be doubled each time in order to observe the variation of energy consumption. Also, it is important to note that the utilisation threshold will be set as the typical value of 0.8 (80%) in order to prevent allocating too great a number of VMs on one physical host to the extent that it may degrade the performance of VM. 4.2.1.2. Discussion of the Results Based on the results collected from the two scenarios, with the deployment of VM migration and DVFS enabled and the deployment of DVFS only, shown in Table 14 in Appendix N and Table 2 in Appendix G, Table 15 and Figure 13 below have been produced to show the difference between the two scenarios based on the mean values of the energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 9 has the maximum values of 2560 hosts, 5120 VMs, and 5120 Cloudlets. Table 15. Comparison for energy consumption mean values between scenario 7 and 1 VM Migration and DVFS enabled (Scenario 7) Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7 Experiment 8 Experiment 9
0.221429 0.41 0.757143 1.435714 2.96 5.99 11.87 23.24857 46.80571

DVFS Mechanism Only (Scenario 1)

0.245714 0.50 1.005714 2.094286 4.187143 8.428571 16.91 33.87 67.69571

Percentage of Energy Reduction Achieved

9.88% 18% 24.71% 31.45% 29.30% 28.93% 29.80% 31.36% 30.86%

35

80 70 60 50 40 30 20 10 0 -10 -20
Figure 13. Comparison between energy consumption mean values for each experiment

With VM Migration and DVFS Enabled With DVFS Mechanism Only

Thus, based on the results shown in the previous Table 15 and Figure 13, doubling the number of resources and services would also double the energy consumption for both scenarios, with the deployment of VM migration and DVFS enabled and with the deployment of DVFS mechanism only. But the energy consumed with the deployment of VM migration and DVFS enabled is about 30% less than the energy consumption with the deployment of DVFS, except for only the first three experiments which have fewer reduction values, as can be seen in Table 15 above. Therefore, the deployment of VM migration along with DVFS being enabled would save more energy because the VM migration mechanism would further reduce energy by allocating more VMs to less physical machines to allow maximum utilisation of the running of physical machines [37].

4.2.2. Scenario 8: Increasing the No. Of Hosts

4.2.2.1. Objectives and Design The objectives of the following experiments are to evaluate how increasing the number of hosts would reflect on the energy consumption in an environment with the deployment of VM migration with DVFS enabled, compared to those results in scenario 2 with DVFS mechanism deployment only. The parameter settings for the number of hosts, VMs, and Cloudlets will initially be according to the default values already set in CloudSim, and then the number of hosts only will be doubled each time to observe the variation in energy consumption. Also, the utilisation threshold will remain at the typical value of 0.8 for all the experiments to prevent performance degradation. 36

4.2.2.2. Discussion of the Results Based on the results collected from the two scenarios, with the deployment of VM migration and DVFS enabled, and the deployment of DVFS only, shown in Table 16 in Appendix O and Table 4 in Appendix I, Figure 14 below has been produced to show the difference between the two scenarios based on the mean values of the energy consumption. For this scenario, experiment 1 has the minimum values of the parameter settings with 10 hosts, 20 VMs, and 20 Cloudlets, and experiment 14 has the maximum values of 81920 hosts, 20 VMs, and 20 Cloudlets.
0.3 0.25 0.2 0.15 0.1 0.05 0

With VM Migration and DVFS Enabled With DVFS Mechanism Only

Experiment 1

Experiment 4

Experiment 7

Experiment 10

Experiment 11

Experiment 12

Experiment 13

Figure 14. Comparison between energy consumption mean values for each experiment

Based on the results shown in Figure 14, doubling the number of machines would mostly not affect the energy consumption for both scenarios because DVFS is enabled for both to turn on only the required number of machines, and the rest are turned off. But with the deployment of VM migration and DVFS enabled in scenario 7, the energy consumed is less than the energy consumption in scenario 2 with DVFS deployment only, because the VM migration would further reduce the energy consumption by utilising the current running physical machines [37].

37

Experiment 14

Experiment 2

Experiment 3

Experiment 5

Experiment 6

Experiment 8

Experiment 9

4.3 Overall Discussion and Results Comparison

Overall, as can be seen from the results of the experiments, deploying DVFS and VM migration mechanisms can enhance the eco-efficiency of a data centre by lowering the performance overheads. Based on the scenarios from 1 to 6, a data centre with the deployment of DVFS mechanism would consume significantly less energy than a data centre with no deployment of a power aware mechanism, which reflects how effectively the DVFS mechanism has been designed to dynamically control the input voltage and clock frequency of a CPU to decrease the power consumption, as stated by Raj and Shriram [45]. A summarised comparison of the energy consumption between the deployment of the DVFS mechanism and no deployment of a power aware mechanism for these scenarios can be seen in Table 17 below. Table 17. Benchmarking of energy consumption between the deployment of DVFS mechanism and non power aware (NPA) mechanism in a data centre Variation of Energy Variation of Energy consumption for each consumption for each subsequent Average energy savings subsequent experiment with experiment with NPA DVFS Scenario 1: Doubling the number of hosts, Double Double About 70% VMs, and Cloudlets Varies (Dramatic increase in Scenario 2: Doubling Stable Double energy savings as the scale of the number of hosts the experiments goes higher) About 70% in experiment 1 Scenario 3: Doubling Almost stable Stable About 16% for the remaining the number of VMs experiments Scenario 4: Doubling the number of Stable Stable About 70% Cloudlets Scenario 5: Doubling About 70% in experiment 1 the number of hosts Almost double Double About 55% for the remaining and VMs experiments Scenario 6: Doubling Varies (Dramatic increase in the number of hosts Stable Double energy savings as the scale of and Cloudlets the experiments goes higher)

38

Based on the results shown in the table above, scenario 2 and 6 are almost the same, which suggests along with scenario 4 that increasing the number of cloudlets does not influence energy consumption. Moreover, the results of scenario 1 are similar to those of scenario 5. Although the energy saving with the deployment of DVFS compared to NPA in scenario 5 is about 70% only for the first experiment, and for the remaining it is about 55%; while in scenario 1 it is about 70% for all the experiments. Furthermore, scenario 3 suggests that increasing the number of VMs only would not influence the energy consumption while the number of hosts and cloudlets are constant. Similarly, scenario 4 suggests that increasing the number of cloudlets only, would not influence the energy consumption while the number of hosts and VMs remain constant. However, increasing the number of hosts only would influence the energy consumption, as can be seen in scenario 2 with NPA mechanism. Thus, it can be concluded that the key parameter that influences energy consumption most significantly is the number of hosts. Also, the number of VMs would also influence the energy consumption only if this is increased with the number of hosts. Additionally and most importantly, the deployment of the DVFS mechanism shows significant energy savings, as can be seen in the last column in Table 17 above. Therefore, the DVFS mechanism does a good job saving energy, and this can be justified by the way it has been designed to control the voltage supply and frequency scaling of a CPU. Moving on, for the deployment of VM migration with DVFS enabled, a further reduction of energy consumption has been shown in scenarios 7 and 8 compared to scenarios 1 and 2 respectively with the deployment of DVFS mechanism only. For scenario 7 with doubling the number of hosts, VMs and Cloudlets, it shows about a 30% further reduction in energy consumption, while in scenario 8 with doubling the number of hosts only it shows about 20% further reduction in energy consumption. This can be justified by the way that the VM migration mechanism is designed to allocate more VMs on less physical machines to enable maximum utilisation of the running of physical machines.

39

5. Evaluation
This chapter will evaluate the project from three different aspects. Firstly, it will evaluate the project in terms of meeting the aim and minimum requirements. Secondly, it will evaluate the appropriateness of the chosen methodology. Thirdly, it will evaluate the results of the experiments, which will include a comparison of the two techniques investigated and an overall comparison of the results of the experiments conducted in this project with those already contained in the literature.

5.1. Evaluation According to Minimum Requirements

The aim of this project is to evaluate DVFS mechanisms that can be used to improve the ecoefficiency of Clouds data centres. The minimum requirements for this project, as previously stated in Section 1.3, are: Design a simulated scenario of a cloud data centre using DVFS mechanism Implement the scenario Collect and evaluate the simulation results Evaluate the mechanism used to improve the eco-efficiency of data centre management in Cloud Computing In order to achieve the aim and minimum requirements, the literature review in Section 2.1 has thoroughly investigated some important aspects of Cloud Computing, including the architecture of Clouds, types of Cloud services, types of Cloud deployment, and virtualisation. Also, in Section 2.2, it has investigated some issues associated with Clouds, including the cost of Cloud maintenance and, most importantly, management of eco-efficiency in data centres. In Section 2.3, it has investigated the CloudSim toolkit in terms of its architecture, usability, capability, and limitations. Moreover, in Section 3.4 and throughout Chapter 4, this project has not only designed and implemented one simulated scenario of a Cloud data centre using a DVFS mechanism successfully, but also another five scenarios, along with comparisons with other scenarios in an environment without the deployment of a power aware mechanism, in order to compare and evaluate the energy reduction. In addition, two more scenarios with the deployment of VM migration and DVFS enabled have been designed and implemented to add more novelty to the project, by allowing further

40

energy reduction and improvement of the eco-efficiency of a data centre. Thus, the aim and minimum requirements have been achieved with further enhancements as well.

5.2. Evaluating the Methodology

The methodology used in this project is based on using a simulation for experimenting with scenarios of a Cloud data centre to evaluate the mechanisms that can be used to improve ecoefficiency. Using the simulation as a methodology for this project has been effective for the reasons that it offers an environment that is scalable, reliable, repeatable, and controllable for performing some benchmarking experiments, with low overheads in terms of cost and effort [6]. In terms of the simulator, CloudSim toolkit has been validated and investigated with regard to its efficiency in modelling some scenarios of Clouds by the people who developed it from the University of Melbourne. Therefore, CloudSim has been widely used by researchers and organisations, such as HP labs in the US, to model some scenarios for supporting their investigation into different aspects of Clouds, like the energy-efficient management of data centres [6]. For this project, CloudSim has been effective in terms of its flexibility and ability to support modelling a Cloud data centre, from small-scale to large-scale experiments. Additionally, it supports modelling some energy aware provisioning techniques, such as VM allocation and DVFS, as discussed in Section 2.3.3. Even so, it has some limitations, such as the findings of the experiments conducted in scenario 1 which reveal the maximum limit of the parameter values that CloudSim can handle. Also, running the experiments sometimes takes a long time to complete, especially when the scale of the experiment is high. Because of the randomness of CloudSim simulation, each experiment had to be run seven times in order to obtain more reliable results, which also increases the time required for running the experiments. In terms of the development approach mentioned in Section 3.3, the iterative approach has been found suitable and effective for the scenarios conducted in this project because it provides clear and easy to follow stages for the scenarios development. Also, it allows speedy adjustment of the design settings in the case of the experiments that have not been run successfully.

41

5.3. Evaluating the Results

In this section, an evaluation will be presented from two dimensions. First, it will evaluate and compare the results of the two mechanisms, DVFS and VM migration, in terms of how each one would contribute towards enhancing the eco-efficiency of data centres. Second, it will contrast the results of the experiments conducted in this project with those already included in the literature. Firstly, as has been stated by Raj and Shriram [45], a DVFS mechanism focuses on utilising the CPU by manipulating the voltage and clock frequency of a CPU to decrease the power consumed. Based on the results of the experiments simulated in Section 4.1, scenarios 1 to 6 show significant energy reduction with the deployment of a DVFS mechanism in a data centre, ranging from 16% reduction up to 70%, and an even greater reduction for scenarios 2 and 6. Furthermore, the VM allocation mechanism maximises the utilisation of the physical machines running by dynamically migrating more VMs to less physical machines as much as possible, but to the extent that it does not downgrade its performance [30]. Based on the results from the simulated experiments in Section 4.2, scenarios 7 and 8 show further energy reduction with the deployment of VM migration with DVFS enabled in a data centre, ranging from 20% up to 30%. Thus, based on the results from all of the scenarios, the deployment of both DVFS and VM migration in a data centre would reduce energy consumption significantly, and it can be deduced that it would also contribute greatly to enhancing the eco-efficiency of data centres. However, it would have added more novelty to the project if the experiments had investigated the efficiency of mechanisms based on variant metrics instead of only the energy consumption metric. Secondly, one study has concluded with the overall evaluation that VM migration with the goal of server consolidation is always suitable for reducing power consumption; however, it can lead to performance degradation when the number of consolidated VMs exceeds the specified limit of a service threshold [37]. Similarly, the experiments conducted with VM migration in this project have also shown the applicability of the mechanism to reduce power consumption without performance degradation, by setting the threshold to 0.8 (80% utilisation or less). Moreover, another study involved conducting an experiment to evaluate VM migration with the application of DVFS [6]. The results show that such an energy conscious technique can considerably decrease the power consumption of the hosts in a data centre by up to 50%. Whereas an evaluation of the VM migration with DVFS enabled conducted in this project, shows the results to be a reduction in power consumption of up to 30%, from which it can be deduced that the difference 42

may be because of the different settings of the parameter values for the experiments. Also, this project has included scenarios consisting of multiple experiments, starting from modelling small scale data centres up to the large scale; whereas the experiment conducted in the other study referred to, has only modelled one scenario representing one scale of data centre.

5.4. Limitations
The assessment of the two experimental mechanisms, DVFS and VM migration, in this project has been limited to measuring the metric of energy consumption only. If another metric, such as SLAs violation, was included, it would have added more novelty to the project in terms of measuring the efficiency of the mechanisms by representing some trade-offs. However, due to the lack of knowledge concerning the architecture of CloudSim before conducting this project, it was difficult to extend the CloudSim to include another metric for measuring the SLAs violation, especially due to the limited time-scale allocated to the project. Therefore the evaluation of these two mechanisms could be enhanced by measuring their efficiency based on variant metrics including energy consumption and SLAs violation. Some of the scenarios simulated may possibly have had similar results with repeated points, and this can be justified due to each scenario not being designed and implemented based on specific needs and real requirements; instead, they were all based on generic requirements and objectives using typical value settings. This could be avoided in the future by studying and analysing a specific real data centre, and then performing the simulation experiments to evaluate the efficiency of deploying the DVFS and VM migration techniques to enhance the eco-efficiency of that particular data centre.

5.5. Future Work

In addition to overcoming the limitations mentioned in the previous Section 5.4, direct experiments involving the scenarios could be set up in a laboratory to obtain more accurate results for measuring the performance of energy-conscious techniques, such as DVFS and VM allocation, in a real environment. However, the development approach would be different to that used in this project. In essence, it would start with planning and setting up the experiments in terms of the computing resources, including machines and networks. Then, there should be a device put in place for measuring the consumed energy. Finally, after the implementation comes analysing and evaluating the results. 43

This project has focused on evaluating the techniques that could be used in a data centre as a way of improving the eco-efficiency of Cloud Computing in terms of the infrastructure layer. However, the eco-efficiency of Cloud Computing can be improved for different layers as well. For example, for the top layer, designing a framework to develop applications that would consume less energy when running on the Cloud infrastructure would also contribute towards the eco-efficiency of Cloud Computing.

44

6. Conclusion
6.1. Summary
As the usage of Cloud Computing is continually increasing, the energy consumption of the computing resources, such as data centres, which underpin the Clouds, will increase as well. This would increase the burden of the operational costs and the environmental impact if there is no control over the energy consumption. Therefore, controlling and reducing the energy consumption in data centres would not only lower the operational costs, which in return would increase the profits, but also lower the environmental impact in terms of CO2 emissions. Thus, this project has investigated and evaluated two techniques, namely DVFS and VM migration, which can be deployed in data centres to control the energy consumption and improve their eco-efficiency. The experiments explained in Chapter 4 have shown that significant energy reduction in data centres could be achieved with the deployment of DVFS and VM migration.

6.2. Problems Encountered

A number of problems have been faced while conducting the project, particularly when conducting the experiments. Since CloudSim is open source software, there is no user manual for it, which made it difficult to understand how to use the toolkit. This meant it took longer than expected to carry out further research and reverse engineering in order to understand how the tool works for modelling some Cloud scenarios. The Google group for CloudSim community was very useful when attempting to tackle this issue. Also, the time for running the experiments was very long, especially when the scale of the scenario is high. Also, because of the randomness of the simulation, each experiment had to be run multiple times. For example, scenario one had 15 experiments and each experiment was run seven times, which means that 105 runs were conducted through eclipse in order to complete scenario 1 only. Thus, it was very difficult and time consuming to complete the rest of the scenarios in the same way; therefore, by using shell scripting, a number of experiments were run simultaneously to speed up conducting the remaining scenarios.

45

7. Bibliography
[1] Mell, P. and Grance, T. (2011) The NIST Definition of Cloud Computing. [Internet], National Institute of Standards and Technology, September, p.1-3. Available from: <http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf> [Accessed 8th June 2012] [2] Deng, J et al (2010) Research and Application of Cloud Storage. 2nd International Workshop on Intelligent Systems and Applications (ISA), 22-23 May, pp. 1-5. [3] Yeo, S. and Lee, H. (2011) Using Mathematical Modelling in Provisioning a Heterogeneous Cloud Computing Environment. Computer, 44(8), August, pp. 55-62. [4] Kliazovich, D. et al (2010) GreenCloud: A Packet-level Simulator of Energy-aware Cloud Computing Data Centers. IEEE Global Telecommunication Conference (GLOBECOM 2010), 6-10 December, pp.1-5. [5] Grag, S. and Buyya, R. (2011) NetworkCloudSim: Modelling Parallel Applications in Cloud Simulations. Fourth IEEE International Conference on Utility and Cloud Computing (UCC), 5-8 December, pp. 105-113. [6] Calheiros, R. et al (2010) CloudSim: A Toolkit for Modelling and Simulation of Cloud Computing Environments and Evaluation of Resources Provisioning Algorithms. Software Practice and Experience 41, 24 August, pp. 23-50, John Wiley & Sons, Ltd. [7] Wickremasinghe, B., Calheiros, R. and Buyaa, R. (2010) CloudAnalyst: a CloudSim-based Visual Modeller for Analysing Cloud Computing Environments and Applications. 24th IEEE International Conference on Advanced Information Networking and Applications, 20-23 April, pp. 446-452. [8] Doest, L. Et al (2006) Personal Goal Facilitation Through Work: Implications for Employee Satisfaction and Well-Being. Applied Psychology: An International Review, 55(2), pp. 192-219. [9] Krafzig, D. et al (2005) Enterprise SOA: Service-Oriented Architecture: Best Practice. Upper Saddle River, Pearson Education, Inc. [10] Josuttis, N. (2007) SOA in Practice: The Art of Distributed System Design. California, OReilly Media, Inc. [11] Chorafas, D. (2011) Cloud Computing Strategies. United States of America: CRS Press, Taylor and Francis Group. [12] Buyya, R. et al (2009) Cloud Computing and emerging IT Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility. Future Generation Computer Systems 25, pp. 599-616. [13] Vaquero, L. et al (2009) A Break in the Clouds: Towards a Cloud Definition. ACM SIGCOMM Computer Communication Review, 39(1), January, pp. 50-55.

46

[14] Miller, M. (2008) Cloud Computing: Web-Based Applications That Change the Way You Work and Collaborate Online. United States of America: Que Publishing. [15] Foster, I. et al (2008) Cloud Computing and Grid Computing 360-Degree Compared. Grid Computing Environment Workshop, GCE 0. 12-16 November, pp. 1-10. [16] Buyya, R., Ranjan, R., and Calheiros, R. (2009) Modelling and Simulation of Scalable Cloud Computing Environment and the CloudSim Toolkit: Challenges and Opportunities. International Conference on High Performance Computing and Simulation, HPCS 09. 21-24 June, pp. 1-11. [17] Velte, A., Velte, T., and Elsenpeter, R. (2010) Cloud Computing: A Practical Approach. United States of America: McGraw-Hill. [18] Tsai, W., Sun, X., and Balasooriya, J. (2010) Service-Oriented Cloud Computing Architecture. Seventh International Conference on Information Technology. 12-14 April, pp. 684-689. [19] Rehman, Z., Hussain, F., and Hussain, O. (2011) Towards Multi-Criteria Cloud Service Selection. Fifth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing. 30 June 2 July, pp. 44-48. [20] Ye, K. et al (2010) Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing: A Performance Perspective. IEEE/ACM International Conference on Green Computing and Communications & IEEE/ACM International Conference on Cyber, Physical and Social Computing. 18-20 Devember, pp. 171-178. [21] Duran-Limon, H. et al (2011) Using Lightweight Virtual Machines to Run High Performance Computing Applications: The Case of Weather Research and Forecasting Model. Fourth IEEE International Conference on Utility and Cloud Computing. 5-8 December, pp. 146-153. [22] Hardy, J. et al (2012) Assessment and Evaluation of Internet-based Virtual Computing Infrastructure. 15th IEEE International Symposium on Object/Component/Service-Oriented RealTime Distributed Computing. 11-13 April, pp. 39-46. [23] Armstrong, D. and Djemame, K. (2009) Towards Quality of Service in the Cloud. Proceedings of the 25th UK Performance Engineering Workshop, pp. 226-240. [24] Nakajima, J. and Mallick, A. (2007) Hybrid Virtualisation Enhanced Virtualisation for Linux. Proceedings of Linux Symposium, 2, 27th-30th June, pp. 87-96. [25] Tsaravas, C. and Themistocleous, M. (2011) Cloud Computing and EGovernment: A Literature Review. European, Mediterranean & Middle Eastern Conference on Information Systems. 30-31 May, pp. 154-164. [26] Sabahi, F. (2011) Cloud Computing Security Threats and Responses. IEEE 3rd International Conference on Communication Software and Networks. 27-29 May, pp. 245-249. [27] Li, X. et al (2009) The Method and Tool of Cost Analysis for Cloud Computing. IEEE International Conference on Cloud Computing, CLOUD 09. 21-25 September, pp. 93-100.

47

[28] Smarr, L. (2010) Project GreenLight: Optimizing Cyber-Infrastructure for a Carbon-Constrained World. Computer, 43(1), January, pp. 22-27. [29] Moreno, I., and Xu, J. (2011) Energy-Efficiency in Cloud Computing Environments: Towards Energy Savings without Performance Degradation. International Journal of Clouds Applications and Computing, 1, pp. 16-33. [30] Jaureguialzo, E. (2011) PUE: The Green Grid metric for evaluating the energy efficiency in DC (Data Center). Measurement method using the power demand. IEEE 33rd International Telecommunications Energy Conference (INTELEC). 9-13 October, pp.1-8. [31] Tellbach, D. (2012) What are Next Steps to Make PUE Statements more Helpful? Datacentre Transformation Conference, [Internet], at the University of Leeds, 26-27 June. Available from: <http://www.dtconference.com/presentations> [Accessed 30th July 2012] [32] Gillan, P. (2012) Fresh Air The Coolest Way to Save Energy: Direct Free Cooling Utilising Outside Air. Datacentre Transformation Conference, [Internet], at the University of Leeds, 26-27 June. Available from: <http://www.dtconference.com/presentations> [Accessed 30th July 2012] [33] Patterson, M. et al (2010) WP29-ERE: A Metric for Measuring the Benefits of Reuse Energy from a Data Center. [Internet], The Green Grid, 22 September. Available from: <http://www.thegreengrid.org/en/Global/Content/white-papers/ERE> [Accessed 30th July 2012] [34] VanGeet, O. (2010) FEMP Best Practice Guide for Energy-Efficient Data Center Design. [Internet], Federal Energy Management Program (FEMP), prepaid by the National Renewable Energy Laboratory (NREL), US. Available from: <http://www.nrel.gov/docs/fy11osti/47201.pdf> [Accessed 30th July 2012] [35] Brady, E. (2012) A Critical Evaluation of the PUE Metric. Datacentre Transformation Conference, [Internet], at the University of Leeds, 26-27 June. Available from: th <http://www.dtconference.com/presentations> [Accessed 30 July 2012] [36] Rudgyard, M. (2012) End-To-End Monitoring and Management in Data Centre; a Fresh Approach. Datacentre Transformation Conference, [Internet], at the University of Leeds, 26-27 June. Available from: <http://www.dtconference.com/presentations> [Accessed 30th July 2012] [37] Corradi, A., Fanelli, M., and Foschini, L. (2011) Increasing Cloud Power Efficiency through Consolidation Techniques. IEEE Symposium on Computers and Communications. 28 June 1 July, pp. 129-134. [38] Pakbaznia, E., Ghasemazar, M., and Pedram, M. (2010) Temperature-Aware Dynamic Resource Provisioning in a Power-Optimized Datacenter. Design, Automation & Test in Europe Conference & Exhibition. 8-12 March, pp. 124-129. [39] Kliazovich, D., Bouvry, P., and Khan, S. (2010) DENS: Data Center Energy-Efficient NetworkAware Scheduling. IEEE/ACM International Conference on Green Computing and Communications (GreenCom) & International Conference on Cyber, Physical and Social Computing (CPSCom). 18-20 December, pp. 69-75.

48

[40] Kim, K., Beloglazov, A., and Buyya, R. (20009) Power-aware Provisioning of Cloud Resources for Real-time Services. Proceedings of the 7th International Workshop on Middleware for Grids, Clouds and e-Science MGC 09, 30th November, pp. 1-6. [41] Quiroz, A. et al (2009) Towards autonomic workload provisioning for enterprise grids and clouds. 10th IEEE/ACM International Conference on Grid Computing, 13-15 October, pp. 50-57. [42] Jessen, S. (2011) The Impact on Project Success of Using Technology in Modern Project Planning and Control. IEEE International Technology Management Conference (ITMC), 27-30 June, pp. 801805. [43] Scheihing, P. (2007) Creating Energy-Efficient Data Centers. Data Center Facilities and Engineering Conference, Washington, DC, 18th May. [44] Markoff, J. and Lohr, S. (2002) Intels Huge Bet Turns Iffy. The New York Times, [Internet], 29th September. Available from: <http://www.nytimes.com/2002/09/29/business/intel-s-huge-bet-turnsth iffy.html?pagewanted=all&src=pm> [Accessed 6 August 2012] [45] Raj, V. and Shriram, R. (2011) Power Aware Provisioning in Cloud Computing Environment. International Conference on Computer, Communication and Electrical Technology ICCCET, 18th19th Marsh, pp. 6-11. [46] Shiraz, M. and Gani, A. (2012) Mobile Cloud Computing: Critical Analysis of Application Deployment in Virtual Machines. International Proceedings of Computer Science and Information Technology, vol.27, February, pp. 11-16. [47] Payton, M., Greenstone, M., and Schenker, N. (2003) Overlapping Confidence Intervals or Standard Error Intervals: What Do They Mean in Terms of Statistical Significance. Journal of Insect Science, 3(34), 30 October, pp. 1-6. [48] Beloglazov, A. and Buyya, R. (2011) Optimal Online Deterministic Algorithms and Adaptive Heuristics for Energy and Performance Efficient Dynamic Consolidation of Virtual Machines in Cloud Data Centres. Concurrency and Computation: Practice and Experience, [Internet] 7th October. Available from: <http://beloglazov.info/papers/2012-optimal-algorithms-ccpe.pdf> [Accessed 1st June 2012]

49

8. Appendices
8.1. Appendix A: Project Reflection
This project has been very interesting and challenging as it provided me with vital skills and knowledge about Cloud Computing and the techniques used to improve the eco-efficiency of data centres. Having completed the project, I would like to share some experiences and lessons learnt with others interested in doing a similar project. First of all, once you have chosen a topic that is interesting to you, you should set your objectives and create a thorough project plan as early on as possible. It might be difficult to foresee everything at the beginning, but planning will help you to think ahead about how to effectively manage your time in order to complete the project successfully. Moreover, it is important to complete the background reading and some of the writing before May, when exams start, so that you have enough time to write a solid interim report, which has to be submitted about three weeks after the exams. Indeed, if the topic chosen is new to you, it would be very difficult to complete both the initial reading and writing within three weeks. Choosing an appropriate methodology for the project is another key success factor to achieving a better outcome. As my project was aimed at investigating several techniques through experiments, I found that using simulations and applying an iterative approach was most suitable for conducting the experiments because it gave me more flexibility. Personally, I expected the CloudSim simulator to be easy to use, but once I started exploring it, I faced numerous challenges, especially in terms of how to use it. If you use CloudSim for your project, it is important to start learning about the tool by studying and running simple examples available in the program; then explore it more deeply until you understand how it works and how you can employ it for your project. Also, it is important to explore the tool while carrying out the background reading before May so that you do not need to waste time learning it once you start the implementation, which was the most important lesson I learnt. In addition, if you encounter any issues while using the tool, visit the online discussions for CloudSim, which are available on Google group; you may find the answers to your queries there, or you can post your own queries to be answered by the people who developed CloudSim, who manage the group.

50

In terms of project management, ideally, have one project plan and stick to it. This will help to ensure that everything goes smoothly and as planned. However, sometimes you may have to change the project plan due to unexpected issues or the underestimation of some tasks. Thus, revise the project plan regularly to ensure successful completion of the project within the limited timescale. Furthermore, weekly meetings with the supervisor gave me the opportunity to discuss the issues encountered as well as gain more advice on improving the project. To make the meetings effective, prepare an agenda for each meeting to discuss the tasks you have achieved during that week and to set tasks to be completed in the following week. In addition, detailed and constructive feedback from both the supervisor and assessor for the interim report and the work in progress presentation was extremely useful, as it directed me to where improvements should be made in the project. If I were to do the project again, I would have prepared and included more work in the interim report and the presentation in order to obtain more feedback. Finally, it has been my experience that discussing the project with friends can help identify some interesting points for improvements, although not necessarily from a technical perspective. This approach may, however, help with the overall organisation of the project. Overall, I have learnt a lot from completing this project and have truly enjoyed the experience.

51

8.2. Appendix B: Interim Report

52

53

8.3. Appendix C: Initial Project Plan

54

55

8.4. Appendix D: Revised Project Plan

56

57

8.5. Appendix E: Weekly Progress Report

Weekly Progress Report 1

Date: 22/June/2012
Tasks undertaken and progress accomplished for this week Read more about how to use CloudSim toolkit Understand the examples provided with CloudSim Learn more about using CloudSim Analyse and design a scenario to simulate DVFS mechanism Planned tasks for next week

Supervisor Name: Dr. Karim Djemame Student Name: Ibrahim Alzamil Student ID: 200627581

58

Weekly Progress Report 2

Date: 29/June/2012
Tasks undertaken and progress accomplished for this week Read and learn more about using CloudSim, and particularly understand the DVFS example provided in CloudSim-2.1.1. Analyse and design four scenarios to simulate DVFS mechanism. Understand how to calculate the mean, standard deviation, and confidence interval for the energy consumption statistics of each experiment. Run the experiments for each scenario and collect the results. Calculate the results and find out the mean, standard deviation, and confidence interval for each experiment. Analyse and discuss the results Find out the maximum values of the parameters setting for each scenario that CloudSim can handle. Do two more scenarios by only increasing: 1 - the no of hosts and VMs, 2 - the no of hosts and cloudlets, Look at VM allocation mechanism if there is enough time. Planned tasks for next week

Supervisor Name: Dr. Karim Djemame Student Name: Ibrahim Alzamil Student ID: 200627581

59

Weekly Progress Report 3

Date: 6/July/2012
Tasks undertaken and progress accomplished for this week Find out the maximum values of the parameters setting that CloudSim can handle in scenario 1, and run the experiments, collect the results, and then calculate the results to get the mean, standard deviation, confidence interval for each experiment. Analyse and discuss the results Try to complete scenario 2 and do the same as for scenario 1. But still uncompleted because the simulation takes a very long time! Learn how to use shell scripting to speed up the simulation time. Extend the scenarios 2, 3, and 4 Do two more scenarios by only increasing: 1 - the no of hosts and VMs, 2 - the no of hosts and cloudlets Planned tasks for next week

Supervisor Name: Dr. Karim Djemame Student Name: Ibrahim Alzamil Student ID: 200627581

60

Weekly Progress Report 4

Date: 12/July/2012
Tasks undertaken and progress accomplished for this week Learn how to use shell scripting to speed up the simulation time. Extend the scenarios (2, 3, and 4) up to experiment 14 or to a point where CloudSim cannot handle doubling the parameter values. Run and collect the results. Calculate the results to find out the mean, standard deviation, and confidence interval. Use the built-in function in Excel to calculate the confidence interval instead of the previous way, and adjust all the results accordingly. Do two more scenarios by only increasing: 1 - the no of hosts and VMs, 2 - the no of hosts and cloudlets. Analyse and discuss the results Prepare table of contents and a draft chapter of the report to submit on 20th of July. Design and implement two scenarios of VM migration mechanism Planned tasks for next week

Supervisor Name: Dr. Karim Djemame Student Name: Ibrahim Alzamil Student ID: 200627581

61

Weekly Progress Report 5

Date: 20/July/2012
Tasks undertaken and progress accomplished for this week Review Project Plan Review aim and minimum requirements Prepare table of content Analyse and designing two scenarios of VM migration mechanism Run experiments for each scenario and collect results, and fixing the bug found to continue running Calculate the results and find out the mean, standard deviation, and confidence interval for each experiment Analyse and discuss the results Complete scenario 2 Prepare a draft chapter of Implementation Plan for Evaluating the results Planned tasks for next week

Supervisor Name: Dr. Karim Djemame Student Name: Ibrahim Alzamil Student ID: 200627581

62

Weekly Progress Report 6

Date: 27/July/2012
Tasks undertaken and progress accomplished for this week Complete scenario two of VM migration Write a draft chapter of implementation Review table of contents Plan for evaluation Start writing up: Review and adjust the Interim Report (first two chapters) based on the feedback from supervisor and assessor Write up the Methodology chapter Review the Implementation chapter based on the supervisor feedback Evaluate the project Write up the Conclusion Finalise the project and write Personal Reflection Planned tasks for the remaining weeks

Supervisor Name: Dr. Karim Djemame Student Name: Ibrahim Alzamil Student ID: 200627581

63

8.6. Appendix F: Work in Progress Presentation

Problem summary
What is the problem?
Cloud infrastructure is likely embodied by enormous data centres that consume huge energy in order to work [29]. Managing cloud data centres eco-efficiently

Work in Progress

Why is it important?

Ibrahim Alzamil

Reduce energy consumption Reduce operational costs Increase profit Reduce CO2 emissions and be more friendly with the environment

Applying some power aware mechanisms to control and reduce power consumption
Dynamic Voltage and Frequency Scaling (DVFS)
Alters the CPU power consumption based on the offered workload.

Aim:
Evaluate a technique that can be used to improve the eco-efficiency and operational costs.

Objectives:
Understand the importance of cloud computing, architecture of clouds, types of clouds, models of clouds, and virtualization. Understand some issues associated with cloud computing. Understand CloudSim toolkit in terms of its usability, capability and limitations to offer a feasible evaluation of the eco-efficiency techniques used in data centres.

Virtual Machine (VM) Allocation

Migrates more VMs on less physical machines to allow maximum utilisation of running physical machines.

64

Minimum Requirements:

Further enhancements:

Designing and implementing five more scenarios of a cloud data centre using DVFS mechanism and others without using DVFS mechanism Designing and implementing two more scenarios of a cloud data centre using VM Allocation mechanism Collecting and evaluating the results

Run

Verify Results

Six simulated scenarios with DVFS mechanism compared to an environment with no deployment of power aware mechanism

Objectives:

Two simulated scenarios with the deployment of VM migration and DVFS-enabled compared to an environment with DVFS deployment only

Design:

How increasing the No. of resources and services would affect energy consumption How the deployment of DVFS would contribute to reduce energy compared to an environment without this mechanism
Based on the parameter settings of hosts, VMs, and cloudlets Doubling the number of hosts, VMs and cloudlets for each experiment Now, will go through demonstration of CloudSim

Run:

Results:
600 500

Energy Consumption kWh

400 300 200 100 0 -100

With DVFS Mechanism Without Power Aware Mechanism

Figure 1: Comparison between energy consumption mean values for each experiment

65

Evaluation

Analysis

Designing and implementing a simulated scenario of a cloud data centre using DVFS mechanism Collecting and evaluating the results Providing eco-efficiency evaluation of the mechanism used to handle some of the issues associated with data centre management of cloud computing

Design

Writing up the dissertation

Reviewing Introduction and LR chapters based on the interim report feedback Writing up the methodology chapter Reviewing Implementation chapter based on the feedback Evaluating the project Writing up the Conclusion Finalising and writing personal reflection

Thank you

66

8.7. Appendix G: Collected Results of Scenario 1: Increasing the No. Of Hosts, VMs, and Cloudlets DVFS Mechanism
Table 2. Shows 15 experiments with the deployment of DVFS mechanism. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.50 kWh 0.49 kWh 0.50 kWh 0.50 kWh 0.49 kWh 0.49 kWh 0.50 kWh 1.00 kWh 1.02 kWh 1.01 kWh 1.03 kWh 0.99 kWh 0.98 kWh 1.01 kWh 2.12 kWh 2.10 kWh 2.09 kWh 2.08 kWh 2.11 kWh 2.07 kWh 2.09 kWh 4.20 kWh 4.18 kWh 4.17 kWh 4.21 kWh 4.14 kWh 4.19 kWh 4.22 kWh 8.44 kWh 8.41 kWh 8.43 kWh 8.39 kWh 8.44 kWh 8.45 kWh 8.44 kWh 16.94 kWh 16.93 kWh 16.92 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.249674 Lower Bound = 0.241755

Exp.1

0.245714

0.005345

Exp.2

No of Hosts = 20 No of VMs = 40 No of Cloudlets = 40

0.50

0.005345

Upper Bound = 0.499674 Lower Bound = 0.491755

Exp.3

No of Hosts = 40 No of VMs = 80 No of Cloudlets = 80 No of Hosts = 80 No of VMs = 160 No of Cloudlets = 160 No of Hosts = 160 No of VMs = 320 No of Cloudlets = 320 No of Hosts = 320 No of VMs = 640 No of Cloudlets = 640 No of Hosts = 640 No of VMs =

1.005714

0.017182

Upper Bound = 1.018443 Lower Bound = 0.992986

Exp.4

2.094286

0.017182

Upper Bound = 2.107014 Lower Bound = 2.081557

Exp.5

4.187143

0.026904

Upper Bound = 4.207073 Lower Bound = 4.167213

Upper Bound = 8.444244 8.428571 0.021157 Lower Bound = 8.412898 Upper Bound = 16.94361

Exp.6

Exp.7

16.91

0.041519

67

1280 No of Cloudlets = 1280 No of Hosts = 1280 No of VMs = 2560 No of Cloudlets = 2560 No of Hosts = 2560 No of VMs = 5120 No of Cloudlets = 5120 No of Hosts = 3840 No of VMs = 7680 No of Cloudlets = 7680 No of Hosts = 4320 No of VMs = 8640 No of Cloudlets = 8640 No of Hosts = 4590 No of VMs = 9180 No of Cloudlets = 9180 No of Hosts = 4598 No of VMs = 9196 No of Cloudlets = 9196 No of Hosts = 4602 No of VMs = 9204 No of Cloudlets = 9204 No of Hosts =

Exp.8

Exp.9

Exp.10 +%50

Exp.11 +%12.5

Exp.12 +%6.25

Exp. 13 +%0.2

Exp.14 +%0.1

Exp.15

Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1

16.86 kWh 16.85 kWh 16.96 kWh 16.93 kWh 33.86 kWh 33.83 kWh 33.91 kWh 33.91 kWh 33.90 kWh 33.83 kWh 33.85 kWh 67.68 kWh 67.66 kWh 67.75 kWh 67.69 kWh 67.64 kWh 67.69 kWh 67.76 kWh 101.46 kWh 101.60 kWh 101.69 kWh 101.51 kWh 101.53 kWh 101.77 kWh 101.71 kWh 114.28 kWh 114.39 kWh 114.31 kWh 114.21 kWh 114.26 kWh 114.46 kWh 114.09 kWh 121.54 kWh 121.47 kWh 121.41 kWh 121.44 kWh 121.42 kWh 121.32 kWh 121.53 kWh 121.77 kWh 121.65 kWh 121.75 kWh 121.80 kWh 121.58 kWh 121.81 kWh 121.66 kWh 121.90 kWh 121.91 kWh 121.74 kWh 121.77 kWh 121.56 kWh 121.75 kWh 121.75 kWh 121.81 kWh

Lower Bound = 16.8821

Upper Bound = 33.89671 33.87 0.036056 Lower Bound = 33.84329

Upper Bound = 67.72853 67.69571 0.044293 Lower Bound = 67.6629

Upper Bound = 101.6961 101.61 0.11619 Lower Bound = 101.5239

Upper Bound = 114.3747 114.2857 0.120119 Lower Bound = 114.1967

Upper Bound = 121.5032 121.45 0.075656 Lower Bound = 121.3911

Upper Bound = 121.782 121.7171 0.087505 Lower Bound = 121.6523

Upper Bound = 121.8553 121.7686 0.117108 Lower Bound = 121.6818 121.82 0.095917 Upper Bound =

68

+%0.02

4603 No of VMs = 9206 No of Cloudlets = 9206

Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

121.96 kWh 121.83 kWh 121.75 kWh 121.79 kWh 121.92 kWh 121.68 kWh

121.8911 Lower Bound = 121.7489

Table 3. Shows 15 experiments without the deployment of power aware mechanism Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20 No of Hosts = 20 No of VMs = 40 No of Cloudlets = 40 No of Hosts = 40 No of VMs = 80 No of Cloudlets = 80 No of Hosts = 80 No of VMs = 160 No of Cloudlets = 160 No of Hosts = 160 No of VMs = 320 No of Cloudlets = 320 No of Hosts = 320 No of VMs = 640 No of Cloudlets = 640

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 0.87 kWh 0.88 kWh 0.89 kWh 0.86 kWh 0.90 kWh 0.84 kWh 0.85 kWh 1.83 kWh 1.74 kWh 1.81 kWh 1.78 kWh 1.84 kWh 1.75 kWh 1.77 kWh 3.63 kWh 3.51 kWh 3.56 kWh 3.50 kWh 3.60 kWh 3.63 kWh 3.61 kWh 7.28 kWh 7.31 kWh 7.14 kWh 7.47 kWh 7.19 kWh 7.39 kWh 7.25 kWh 14.39 kWh 14.72 kWh 15.06 kWh 14.50 kWh 14.44 kWh 14.61 kWh 14.33 kWh 29.33 kWh 29.11 kWh 29.78 kWh 29.78 kWh 30.40 kWh 29.44 kWh 30.11 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.886003 Lower Bound = 0.853997

Exp.1

0.87

0.021602

Exp.2

1.788571

0.038914

Upper Bound = 1.817399 Lower Bound = 1.759744

Exp.3

3.577143

0.054685

Upper Bound = 3.617654 Lower Bound = 3.536632

Exp.4

7.29

0.113284

Upper Bound = 7.373921 Lower Bound = 7.206079

Exp.5

14.57857

0.250162

Upper Bound = 14.76389 Lower Bound = 14.39325

Upper Bound = 30.04162 29.70714 0.451505 Lower Bound = 29.37267

Exp.6

69

Exp.7

No of Hosts = 640 No of VMs = 1280 No of Cloudlets = 1280 No of Hosts = 1280 No of VMs = 2560 No of Cloudlets = 2560 No of Hosts = 2560 No of VMs = 5120 No of Cloudlets = 5120 No of Hosts = 3840 No of VMs = 7680 No of Cloudlets = 7680 No of Hosts = 4320 No of VMs = 8640 No of Cloudlets = 8640 No of Hosts = 4590 No of VMs = 9180 No of Cloudlets = 9180 No of Hosts = 4598 No of VMs = 9196 No of Cloudlets = 9196 No of Hosts = 4602 No of VMs = 9204 No of

Exp.8

Exp.9

Exp.10 +%50

Exp.11 +%12.5

Exp.12 +%6.25

Exp. 13 +%0.2

Exp.14 +%0.1

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5

59.33 kWh 59.56 kWh 58.67 kWh 59.56 kWh 59.33 kWh 60.44 kWh 60.67 kWh 117.33. kWh 120.89 kWh 119.56 kWh 118.22 kWh 120.0 kWh 119.56 kWh 120.44 kWh 240.0 kWh 238.22 kWh 250.67 kWh 244.44 kWh 239.11 kWh 241.78 kWh 241.78 kWh 362.67 kWh 360.00 kWh 368.00 kWh 364.00 kWh 357.33 kWh 372.00 kWh 365.33 kWh 409.50 kWh 418.50 kWh 414.00 kWh 406.50 kWh 409.50 kWh 409.50 kWh 408.00 kWh 436.69 kWh 433.50 kWh 428.72 kWh 441.47 kWh 438.28 kWh 436.69 kWh 441.47 kWh 434.26 kWh 431.06 kWh 432.66 kWh 442.24 kWh 432.66 kWh 432.66 kWh 439.05 kWh 442.62 kWh 441.03 kWh 437.83 kWh 442.62 kWh 436.23 kWh

Upper Bound = 60.16162 59.65143 0.688705 Lower Bound = 59.14124

Upper Bound = 120.355 119.4286 1.250632 Lower Bound = 118.5021

Upper Bound = 245.4183 242.2857 4.22864 Lower Bound = 239.1532

Upper Bound = 367.8163 364.19 4.895134 Lower Bound = 360.5637

Upper Bound = 413.824 410.7857 4.101393 Lower Bound = 407.7474

Upper Bound = 440.0279 436.6886 4.507806 Lower Bound = 433.3492

Upper Bound = 437.9852 434.9414 4.108813 Lower Bound = 431.8976 Upper Bound = 442.2549 440.34 2.584944 Lower Bound = 438.4251

70

Exp.15 +%0.02

Cloudlets = 9204 No of Hosts = 4603 No of VMs = 9206 No of Cloudlets = 9206

Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

439.43 kWh 442.62 kWh 431.53 kWh 429.93 kWh 439.52 kWh 441.12 kWh 436.33 kWh 437.92 kWh 434.73 kWh

Upper Bound = 438.9037 435.8686 4.09711 Lower Bound = 432.8334

The mean values, standard deviation and confidence interval have been calculated based on using the built-in functions in Excel sheet. As a sample, Figure 7 below shows a screen shot of the calculation made for this scenario with the deployment of DVFS. The calculations for the remaining scenarios are calculated in the same way.

Figure 7. A sample of using Excel to calculate mean values, standard deviation, and confidence interval.

71

8.8. Appendix H: A Source Code Sample of the CloudSim Java Classes

The following sample is adapted from the example provided in CloudSim by the people from the University of Melbourne [48], and this code was particularly implemented for experiment 1 of scenario 1 with the DVFS mechanism. package org.cloudbus.cloudsim.examples.power; /* * Title: CloudSim Toolkit

* Description: CloudSim (Cloud Simulation) Toolkit for Modeling and Simulation * of Clouds GPL - http://www.gnu.org/copyleft/gpl.html

* Licence: *

* Copyright (c) 2009, The University of Melbourne, Australia */ import java.text.DecimalFormat; import java.util.ArrayList; import java.util.Calendar; import java.util.LinkedList; import java.util.List; import org.cloudbus.cloudsim.Cloudlet; import org.cloudbus.cloudsim.CloudletSchedulerDynamicWorkload; import org.cloudbus.cloudsim.DatacenterBroker; import org.cloudbus.cloudsim.DatacenterCharacteristics; import org.cloudbus.cloudsim.Log; import org.cloudbus.cloudsim.Storage; import org.cloudbus.cloudsim.UtilizationModelStochastic; import org.cloudbus.cloudsim.Vm; import org.cloudbus.cloudsim.VmSchedulerTimeShared; import org.cloudbus.cloudsim.core.CloudSim; import org.cloudbus.cloudsim.power.PowerDatacenter; 72

import org.cloudbus.cloudsim.power.PowerHost; import org.cloudbus.cloudsim.power.PowerPe; import org.cloudbus.cloudsim.power.PowerVmAllocationPolicySingleThreshold; import org.cloudbus.cloudsim.power.models.PowerModelLinear; import org.cloudbus.cloudsim.provisioners.BwProvisionerSimple; import org.cloudbus.cloudsim.provisioners.PeProvisionerSimple; import org.cloudbus.cloudsim.provisioners.RamProvisionerSimple;

/** * An example of a heterogeneous DVFS-enabled data center: the voltage * and clock frequency of the CPU are adjusted ideally according * to current resource requirements. * * Modified and Implemented by Ibrahim Alzamil (IA) */ public class ScenariosWithDVFS { /** The cloudlet list. */ private static List<Cloudlet> cloudletList; /** The vm list. */ private static List<Vm> vmList;

// IA .. here we adjust the settings of these three parameters based on the objective and design for each experiment private static double hostsNumber = 10; private static double vmsNumber = 20; private static double cloudletsNumber = 20;

/** * Creates main() to run this example. * 73

* @param args the args */ public static void main(String[] args) {

// IA .. the next three lines were used when using shell scripting to run the experiments hostsNumber = Double.valueOf(args[0]).doubleValue(); vmsNumber = Double.valueOf(args[1]).doubleValue(); cloudletsNumber = Double.valueOf(args[2]).doubleValue();

Log.printLine("Starting DVFS example...");

try { // First step: Initialize the CloudSim package. It should be called // before creating any entities. We can't run this example without // initializing CloudSim first. We will get run-time exception // error. int num_user = 1; // number of cloud users Calendar calendar = Calendar.getInstance(); boolean trace_flag = false; // mean trace GridSim events

// Initialize the CloudSim library CloudSim.init(num_user, calendar, trace_flag);

// Second step: Create Datacenters // Datacenters are the resource providers in CloudSim. We need at // list one of them to run a CloudSim simulation PowerDatacenter datacenter = createDatacenter("Datacenter_0"); datacenter.setDisableMigrations(true);

74

// Third step: Create Broker DatacenterBroker broker = createBroker(); int brokerId = broker.getId();

// Fourth step: Create one virtual machine vmList = createVms(brokerId);

// submit vm list to the broker broker.submitVmList(vmList);

// Fifth step: Create one cloudlet cloudletList = createCloudletList(brokerId);

// submit cloudlet list to the broker broker.submitCloudletList(cloudletList);

// Sixth step: Starts the simulation double lastClock = CloudSim.startSimulation();

// Final step: Print results when simulation is over List<Cloudlet> newList = broker.getCloudletReceivedList(); Log.printLine("Received " + newList.size() + " cloudlets");

CloudSim.stopSimulation();

printCloudletList(newList);

Log.printLine();

75

// IA .. to check the parameters settings were set correctly when getting the results System.out.println("Number of Hosts are: "+ hostsNumber); System.out.println("Number of VMs are: "+ vmsNumber); System.out.println("Number of Cloudlets are: "+ cloudletsNumber);

Log.printLine(String.format("Total simulation time: %.2f sec", lastClock)); Log.printLine(String.format("Energy consumption: %.2f kWh", datacenter.getPower() / (3600 * 1000))); Log.printLine();

} catch (Exception e) { e.printStackTrace(); Log.printLine("Unwanted errors happen"); }

Log.printLine("DVFS experiment finished!"); }

/** * Creates the cloudlet list. * * @param brokerId the broker id * * @return the cloudlet list */ private static List<Cloudlet> createCloudletList(int brokerId) { List<Cloudlet> list = new ArrayList<Cloudlet>();

long length = 150000; // 10 min on 250 MIPS 76

int pesNumber = 1; long fileSize = 300; long outputSize = 300;

for (int i = 0; i < cloudletsNumber; i++) { Cloudlet cloudlet = new Cloudlet(i, length, pesNumber, fileSize, outputSize, new UtilizationModelStochastic(), new UtilizationModelStochastic(), new UtilizationModelStochastic()); cloudlet.setUserId(brokerId); cloudlet.setVmId(i); list.add(cloudlet); }

return list; }

/** * Creates the vms. * * @param brokerId the broker id * * @return the list< vm> */ private static List<Vm> createVms(int brokerId) { List<Vm> vms = new ArrayList<Vm>();

// VM description int[] mips = { 250, 500, 750, 1000 }; // MIPSRating int pesNumber = 1; // number of cpus int ram = 128; // vm memory (MB) long bw = 2500; // bandwidth 77

long size = 2500; // image size (MB) String vmm = "Xen"; // VMM name

for (int i = 0; i < vmsNumber; i++) { vms.add( new Vm(i, brokerId, mips[i % mips.length], pesNumber, ram, bw, size, vmm, new CloudletSchedulerDynamicWorkload(mips[i % mips.length], pesNumber)) ); }

return vms; }

/** * Creates the datacenter. * * @param name the name * * @return the datacenter * * @throws Exception the exception */ private static PowerDatacenter createDatacenter(String name) throws Exception { // Here are the steps needed to create a PowerDatacenter: // 1. We need to create an object of HostList2 to store // our machine List<PowerHost> hostList = new ArrayList<PowerHost>();

double maxPower = 250; // 250W double staticPowerPercent = 0.7; // 70% 78

int[] mips = { 1000, 2000, 3000 }; int ram = 10000; // host memory (MB) long storage = 1000000; // host storage int bw = 100000;

for (int i = 0; i < hostsNumber; i++) { // 2. A Machine contains one or more PEs or CPUs/Cores. // In this example, it will have only one core. // 3. Create PEs and add these into an object of PowerPeList. List<PowerPe> peList = new ArrayList<PowerPe>(); peList.add(new PowerPe(0, new PeProvisionerSimple(mips[i % mips.length]), new PowerModelLinear(maxPower, staticPowerPercent))); // need to store PowerPe id and MIPS Rating

// 4. Create PowerHost with its id and list of PEs and add them to the list of machines hostList.add( new PowerHost( i, new RamProvisionerSimple(ram), new BwProvisionerSimple(bw), storage, peList, new VmSchedulerTimeShared(peList) ) ); // This is our machine } // 5. Create a DatacenterCharacteristics object that stores the // properties of a Grid resource: architecture, OS, list of 79

// Machines, allocation policy: time- or space-shared, time zone // and its price (G$/PowerPe time unit). String arch = "x86"; // system architecture String os = "Linux"; // operating system String vmm = "Xen"; double time_zone = 10.0; // time zone this resource located double cost = 3.0; // the cost of using processing in this resource double costPerMem = 0.05; // the cost of using memory in this resource double costPerStorage = 0.001; // the cost of using storage in this // resource double costPerBw = 0.0; // the cost of using bw in this resource DatacenterCharacteristics characteristics = new DatacenterCharacteristics( arch, os, vmm, hostList, time_zone, cost, costPerMem, costPerStorage, costPerBw); // 6. Finally, we need to create a PowerDatacenter object. PowerDatacenter powerDatacenter = null; try { powerDatacenter = new PowerDatacenter( // IAA this is the difference from the nonPowerAware name, characteristics, new PowerVmAllocationPolicySingleThreshold(hostList, 1.0), new LinkedList<Storage>(), 5.0); } catch (Exception e) { e.printStackTrace(); } return powerDatacenter; } // We strongly encourage users to develop their own broker policies, to // submit vms and cloudlets according 80

// to the specific rules of the simulated scenario /** * Creates the broker. * @return the datacenter broker */ private static DatacenterBroker createBroker() { DatacenterBroker broker = null; try { broker = new DatacenterBroker("Broker"); } catch (Exception e) { e.printStackTrace(); return null; } return broker; }

/** * Prints the Cloudlet objects. * * @param list list of Cloudlets */ private static void printCloudletList(List<Cloudlet> list) { int size = list.size(); Cloudlet cloudlet; String indent = "\t"; Log.printLine(); Log.printLine("========== OUTPUT =========="); Log.printLine("Cloudlet ID" + indent + "STATUS" + indent + "Resource ID" + indent + "VM ID" + indent + "Time" + indent + "Start Time" + indent + "Finish Time"); 81

DecimalFormat dft = new DecimalFormat("###.##"); for (int i = 0; i < size; i++) { cloudlet = list.get(i); Log.print(indent + cloudlet.getCloudletId());

if (cloudlet.getCloudletStatus() == Cloudlet.SUCCESS) { Log.printLine(indent + "SUCCESS" + indent + indent + cloudlet.getResourceId() + indent + cloudlet.getVmId() + indent + dft.format(cloudlet.getActualCPUTime()) + indent + dft.format(cloudlet.getExecStartTime()) + indent + indent + dft.format(cloudlet.getFinishTime()) ); } } } }

82

8.9. Appendix I: Collected Results of Scenario 2: Increasing the No. Of Hosts DVFS Mechanism
Table 4. Shows 14 experiments with the deployment of DVFS mechanism. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 0.26 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.26 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.23 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.256049

Exp.1

0.25

0.008165 Lower Bound = 0.243951

Exp.2

No of Hosts = 20 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 0.246472 0.242857 0.00488 Lower Bound = 0.239242

Exp.3

No of Hosts = 40 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 0.248457 0.242857 0.007559 Lower Bound = 0.237257

Exp.4

No of Hosts = 80 No of VMs = 20 No of Cloudlets = 20

0.24

0.005774

Upper Bound = 0.244277 Lower Bound = 0.235723

Exp.5

No of Hosts = 160 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 0.248457 0.242857 0.007559 Lower Bound = 0.237257

Exp.6

No of Hosts = 320 No of VMs = 20 No of Cloudlets = 20 No of Hosts = 640 No of VMs = 20

Upper Bound = 0.246541 0.241429 0.006901 Lower Bound = 0.236317 Upper Bound = 0.244229

Exp.7

0.241429

0.00378

83

No of Cloudlets = 20

Exp.8

No of Hosts = 1280 No of VMs = 20 No of Cloudlets = 20

Exp.9

No of Hosts = 2560 No of VMs = 20 No of Cloudlets = 20

Exp.10

No of Hosts = 5120 No of VMs = 20 No of Cloudlets = 20

Exp.11

No of Hosts = 10240 No of VMs = 20 No of Cloudlets = 20

Exp.12

No of Hosts = 20480 No of VMs = 20 No of Cloudlets = 20

Exp.13

No of Hosts = 40960 No of VMs = 20 No of Cloudlets = 20

Exp.14

No of Hosts = 81920 No of VMs = 20 No of Cloudlets = 20

Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh

Lower Bound = 0.238629

Upper Bound = 0.244277 0.24 0.005774 Lower Bound = 0.235723 Upper Bound = 0.246472 0.242857 0.00488 Lower Bound = 0.239242

Upper Bound = 0.244229 0.241429 0.00378 Lower Bound = 0.238629

Upper Bound = 0.241371 0.238571 0.00378 Lower Bound = 0.235771

Upper Bound = 0.244229 0.241429 0.00378 Lower Bound = 0.238629

Upper Bound = 0.244229 0.241429 0.00378 Lower Bound = 0.238629

Upper Bound = 0.246472 0.242857 0.00488 Lower Bound = 0.239242

84

Table 5. Shows 14 experiments without the deployment of DVFS mechanism Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 0.89 kWh 0.89 kWh 0.85 kWh 0.88 kWh 0.88 kWh 0.87 kWh 0.84 kWh 1.78 kWh 1.74 kWh 1.76 kWh 1.71 kWh 1.67 kWh 1.76 kWh 1.74 kWh 3.46 kWh 3.40 kWh 3.47 kWh 3.49 kWh 3.43 kWh 3.58 kWh 3.36 kWh 6.94 kWh 7.28 kWh 6.64 kWh 7.06 kWh 6.78 kWh 7.03 kWh 6.89 kWh 14.22 kWh 13.89 kWh 14.44 kWh 13.83 kWh 14.11 kWh 13.56 kWh 14.28 kWh 27.33 kWh 27.67 kWh 28.22 kWh 27.44 kWh 28.11 kWh 27.67 kWh 28.33 kWh 56.22 kWh 58.00 kWh 56.89 kWh 55.11 kWh 54.89 kWh 54.67 kWh 57.56 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.885887

Exp.1

0.871429

0.019518

Lower Bound = 0.85697

Exp.2

No of Hosts = 20 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 1.764433 1.737143 0.036839 Lower Bound = 1.709852

Exp.3

No of Hosts = 40 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 3.507897 3.455714 0.070441 Lower Bound = 3.403532

Exp.4

No of Hosts = 80 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 7.098724 6.945714 0.206548 Lower Bound = 6.792705

Exp.5

No of Hosts = 160 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 14.27174 14.04714 0.303189 Lower Bound = 13.82254

Exp.6

No of Hosts = 320 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 28.1165 27.82429 0.394456 Lower Bound = 27.53207

Exp.7

No of Hosts = 640 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 57.1858 56.19143 1.342304 Lower Bound = 55.19705

85

Exp.8

No of Hosts = 1280 No of VMs = 20 No of Cloudlets = 20

Exp.9

No of Hosts = 2560 No of VMs = 20 No of Cloudlets = 20

Exp.10

No of Hosts = 5120 No of VMs = 20 No of Cloudlets = 20

Exp.11

No of Hosts = 10240 No of VMs = 20 No of Cloudlets = 20

Exp.12

No of Hosts = 20480 No of VMs = 20 No of Cloudlets = 20

Exp.13

No of Hosts = 40960 No of VMs = 20 No of Cloudlets = 20

Exp.14

No of Hosts = 81920 No of VMs = 20 No of Cloudlets = 20

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

111.11 kWh 112.44 kWh 108.89 kWh 110.22 kWh 106.67 kWh 114.67 kWh 115.11 kWh 220.44 kWh 226.67 kWh 223.11 kWh 226.67 kWh 227.56 kWh 228.44 kWh 221.33 kWh 446.22 kWh 444.44 kWh 451.56 kWh 448.00 kWh 439.11 kWh 428.44 kWh 424.89 kWh 910.22 kWh 896.00 kWh 860.44 kWh 885.33 kWh 917.33 kWh 878.22 kWh 888.89 kWh 1728.00 kWh 1728.00 kWh 1763.56 kWh 1870.22 kWh 1777.78 kWh 1777.78 kWh 1792.00 kWh 3754.67 kWh 3498.67 kWh 3697.78 kWh 3584.00 kWh 3498.67 kWh 3612.44 kWh 3669.33 kWh 7253.33 kWh 7139.56 kWh 7111.11 kWh 7253.33 kWh 7025.78 kWh 6712.89 kWh 7367.11 kWh

Upper Bound = 113.5569 111.3014 3.04467 Lower Bound = 109.0459

Upper Bound = 227.2644 224.8886 3.207093 Lower Bound = 222.5128 Upper Bound = 447.8968 440.38 10.14694 Lower Bound = 432.8632

Upper Bound = 905.1728 890.9186 19.24177 Lower Bound = 876.6643

Upper Bound = 1812.417 1776.763 48.12904 Lower Bound = 1741.109

Upper Bound = 3688.921 3616.509 97.74878 Lower Bound = 3544.097 Upper Bound = 7280.918 7123.301 212.7669 Lower Bound = 6965.684

86

8.10. Appendix J: Collected Results of Scenario 3: Increasing the No. Of VMs DVFS Mechanism
Table 6. Shows 14 experiments with the deployment of DVFS mechanism. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 0.24 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.71 kWh 0.75 kWh 0.73 kWh 0.75 kWh 0.72 kWh 0.74 kWh 0.71 kWh 0.72 kWh 0.73 kWh 0.73 kWh 0.71 kWh 0.71 kWh 0.75 kWh 0.74 kWh 0.74 kWh 0.76 kWh 0.73 kWh 0.73 kWh 0.71 kWh 0.71 kWh 0.74 kWh 0.72 kWh 0.73 kWh 0.77 kWh 0.72 kWh 0.76 kWh 0.75 kWh 0.74 kWh 0.74 kWh 0.72 kWh 0.74 kWh 0.73 kWh 0.75 kWh 0.75 kWh 0.74 kWh 0.74 kWh 0.74 kWh 0.71 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.250758

Exp.1

0.247143

0.00488 Lower Bound = 0.243528

Exp.2

No of Hosts = 10 No of VMs = 40 No of Cloudlets = 20

Upper Bound = 0.742831 0.73 0.017321 Lower Bound = 0.717169

Exp.3

No of Hosts = 10 No of VMs = 80 No of Cloudlets = 20

Upper Bound = 0.738225 0.727143 0.01496 Lower Bound = 0.71606

Exp.4

No of Hosts = 10 No of VMs = 160 No of Cloudlets = 20

Upper Bound = 0.744561 0.731429 0.017728 Lower Bound = 0.718296

Exp.5

No of Hosts = 10 No of VMs = 320 No of Cloudlets = 20

Upper Bound = 0.755887 0.741429 0.019518 Lower Bound = 0.72697

Exp.6

No of Hosts = 10 No of VMs = 640 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 1280 No of Cloudlets =

Upper Bound = 0.746491 0.738571 0.01069 Lower Bound = 0.730652 Upper Bound = 0.734114

Exp.7

0.725714

0.011339

87

20

Exp.8

No of Hosts = 10 No of VMs = 2560 No of Cloudlets = 20

Exp.9

No of Hosts = 10 No of VMs = 5120 No of Cloudlets = 20

Exp.10

No of Hosts = 10 No of VMs = 10240 No of Cloudlets = 20

Exp.11

No of Hosts = 10 No of VMs = 20480 No of Cloudlets = 20

Exp.12

No of Hosts = 10 No of VMs = 40960 No of Cloudlets = 20

Exp.13

No of Hosts = 10 No of VMs = 81920 No of Cloudlets = 20

Exp.14

No of Hosts = 10 No of VMs = 163840 No of Cloudlets = 20

Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.72 kWh 0.72 kWh 0.73 kWh 0.72 kWh 0.73 kWh 0.76 kWh 0.75 kWh 0.77 kWh 0.73 kWh 0.74 kWh 0.74 kWh 0.72 kWh 0.73 kWh 0.73 kWh 0.74 kWh 0.72 kWh 0.74 kWh 0.75 kWh 0.76 kWh 0.74 kWh 0.72 kWh 0.75 kWh 0.75 kWh 0.73 kWh 0.72 kWh 0.74 kWh 0.74 kWh 0.73 kWh 0.73 kWh 0.73 kWh 0.73 kWh 0.75 kWh 0.72 kWh 0.75 kWh 0.73 kWh 0.72 kWh 0.72 kWh 0.73 kWh 0.71 kWh 0.74 kWh 0.74 kWh 0.74 kWh 0.72 kWh 0.74 kWh 0.74 kWh 0.74 kWh 0.73 kWh 0.73 kWh 0.72 kWh 0.76 kWh 0.71 kWh 0.70 kWh 0.75 kWh

Lower Bound = 0.717314

Upper Bound = 0.756914 0.745714 0.015119 Lower Bound = 0.734514

Upper Bound = 0.7411 0.732857 0.011127 Lower Bound = 0.724614

Upper Bound = 0.750229 0.738571 0.015736 Lower Bound = 0.726914

Upper Bound = 0.741543 0.735714 0.007868 Lower Bound = 0.729886

Upper Bound = 0.73514 0.725714 0.012724 Lower Bound = 0.716288

Upper Bound = 0.742743 0.737143 0.007559 Lower Bound = 0.731543

Upper Bound = 0.744244 0.728571 0.021157 Lower Bound = 0.712898

88

Table 7. Shows 14 experiments without the deployment of Power Aware mechanism Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 0.87 kWh 0.85 kWh 0.89 kWh 0.88 kWh 0.91 kWh 0.88 kWh 0.85 kWh 0.86 kWh 0.86 kWh 0.92 kWh 0.86 kWh 0.91 kWh 0.84 kWh 0.88 kWh 0.89 kWh 0.86 kWh 0.87 kWh 0.88 kWh 0.89 kWh 0.91 kWh 0.89 kWh 0.86 kWh 0.89 kWh 0.83 kWh 0.85 kWh 0.86 kWh 0.86 kWh 0.88 kWh 0.91 kWh 0.88 kWh 0.84 kWh 0.91 kWh 0.86 kWh 0.89 kWh 0.87 kWh 0.85 kWh 0.87 kWh 0.88 kWh 0.91 kWh 0.91 kWh 0.82 kWh 0.83 kWh 0.87 kWh 0.88 kWh 0.89 kWh 0.90 kWh 0.88 kWh 0.85 kWh 0.84 kWh 0.85 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.891635

Exp.1

0.875714

0.021492 Lower Bound = 0.859793

Exp.2

No of Hosts = 10 No of VMs = 40 No of Cloudlets = 20

Upper Bound = 0.897463 0.875714 0.029358 Lower Bound = 0.853966

Exp.3

No of Hosts = 10 No of VMs = 80 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 160 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 320 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 640 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 1280 No of Cloudlets = 20 No of Hosts =

Upper Bound = 0.896274 0.884286 0.016183 Lower Bound = 0.872297

Upper Bound = 0.875887 0.861429 0.019518 Lower Bound = 0.84697

Exp.4

Upper Bound = 0.899127 0.88 0.02582 Lower Bound = 0.860873

Exp.5

Upper Bound = 0.893755 0.867143 0.035923 Lower Bound = 0.840531

Exp.6

Upper Bound = 0.888696 0.872857 0.021381 Lower Bound = 0.857018 0.857143 0.017995 Upper Bound =

Exp.7

Exp.8

89

10 No of VMs = 2560 No of Cloudlets = 20

Exp.9

No of Hosts = 10 No of VMs = 5120 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 10240 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 20480 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 40960 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 81920 No of Cloudlets = 20 No of Hosts = 10 No of VMs = 163840 No of Cloudlets = 20

Exp.10

Exp.11

Exp.12

Exp.13

Exp.14

Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.85 kWh 0.86 kWh 0.86 kWh 0.86 kWh 0.89 kWh 0.83 kWh 0.91 kWh 0.87 kWh 0.89 kWh 0.88 kWh 0.93 kWh 0.84 kWh 0.92 kWh 0.89 kWh 0.86 kWh 0.88 kWh 0.89 kWh 0.90 kWh 0.85 kWh 0.88 kWh 0.86 kWh 0.87 kWh 0.88 kWh 0.87 kWh 0.96 kWh 0.89 kWh 0.83 kWh 0.87 kWh 0.89 kWh 0.90 kWh 0.88 kWh 0.86 kWh 0.88 kWh 0.87 kWh 0.87 kWh 0.86 kWh 0.90 kWh 0.85 kWh 0.86 kWh 0.87 kWh 0.85 kWh 0.91 kWh 0.88 kWh 0.88 kWh 0.87 kWh 0.86 kWh 0.91 kWh 0.86 kWh

0.870473 Lower Bound = 0.843812

Upper Bound = 0.91463 0.891429 0.03132 Lower Bound = 0.868227

Upper Bound = 0.891704 0.878571 0.017728 Lower Bound = 0.865439

Upper Bound = 0.909632 0.88 0.04 Lower Bound = 0.850368

Upper Bound = 0.888537 0.878571 0.013452 Lower Bound = 0.868606

Upper Bound = 0.878443 0.865714 0.017182 Lower Bound = 0.852986

Upper Bound = 0.897102 0.881429 0.021157 Lower Bound = 0.865756

90

8.11. Appendix K: Collected Results of Scenario 4: Increasing the No. Of Cloudlets DVFS Mechanism
Table 8. Shows 13 experiments with the deployment of DVFS mechanism. Parameters Setting
No of Host = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 0.25 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.25 kWh 0.24 kWh 0.26 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.24 kWh 0.26 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper bound = 0.256541

Exp.1

0.251429

0.006901 Lower Bound = 0.246317

Exp.2

No of Host = 10 No of VMs = 20 No of Cloudlets = 40

Upper bound = 0.256541 0.251429 0.006901 Lower Bound = 0.246317

Exp.3

No of Host = 10 No of VMs =20 No of Cloudlets = 80

Upper bound = 0.253683 0.248571 0.006901 Lower Bound = 0.243459

Exp.4

No of Host = 10 No of VMs = 20 No of Cloudlets = 160

Upper bound = 0.250758 0.247143 0.00488 Lower Bound = 0.243528

Exp.5

No of Host = 10 No of VMs = 20 No of Cloudlets = 320

Upper bound = 0.254229 0.251429 0.00378 Lower Bound = 0.248629

Exp.6

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 640 No of Hosts = 10 No of VMs = 20

Upper bound = 0.250758 0.247143 0.00488 Lower Bound = 0.243528 Upper bound = 0.250758

Exp.7

0.247143

0.00488

91

No of Cloudlets = 1280

Exp.8

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 2560

Exp.9

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 5120

Exp.10

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 10240

Exp.11

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20480

Exp.12

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 40960

Exp.13 +%50

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 61440

Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.25 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.26 kWh 0.26 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.25 kWh 0.26 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.25 kWh

Lower Bound = 0.243528

Upper bound = 0.256472 0.252857 0.00488 Lower Bound = 0.249242 Upper bound = 0.255237 0.248571 0.008997 Lower Bound = 0.241906

Upper bound = 0.251371 0.248571 0.00378 Lower Bound = 0.245771

Upper bound = 0.248245 0.244286 0.005345 Lower Bound = 0.240326

Upper bound = 0.254277 0.25 0.005774 Lower Bound = 0.245723

0.248571

Upper bound = 0.253683 0.006901 Lower Bound = 0.243459

92

Table 9. Shows 13 experiments without the deployment of Power Aware mechanism Parameters Setting
No of Host = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 0.88 kWh 0.83 kWh 0.87 kWh 0.88 kWh 0.88 kWh 0.85 kWh 0.89 kWh 0.88 kWh 0.87 kWh 0.92 kWh 0.87 kWh 0.83 kWh 0.91 kWh 0.89 kWh 0.86 kWh 0.88 kWh 0.87 kWh 0.84 kWh 0.86 kWh 0.85 kWh 0.87 kWh 0.89 kWh 0.86 kWh 0.84 kWh 0.85 kWh 0.89 kWh 0.88 kWh 0.86 kWh 0.85 kWh 0.86 kWh 0.86 kWh 0.88 kWh 0.86 kWh 0.83 kWh 0.85 kWh 0.88 kWh 0.84 kWh 0.86 kWh 0.86 kWh 0.86 kWh 0.89 kWh 0.85 kWh 0.87 kWh 0.89 kWh 0.85 kWh 0.87 kWh 0.88 kWh 0.92 kWh 0.88 kWh 0.87 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper bound = 0.884244

Exp.1

0.868571

0.021157 Lower Bound = 0.852898

Exp.2

No of Host = 10 No of VMs = 20 No of Cloudlets = 40

Upper bound = 0.903416 0.881429 0.029681 Lower Bound = 0.859441

Exp.3

No of Host = 10 No of VMs =20 No of Cloudlets = 80

Upper bound = 0.871394 0.861429 0.013452 Lower Bound = 0.851463

Exp.4

No of Host = 10 No of VMs = 20 No of Cloudlets = 160

Upper bound = 0.881781 0.867143 0.01976 Lower Bound = 0.852504

Exp.5

No of Host = 10 No of VMs = 20 No of Cloudlets = 320

Upper bound = 0.866914 0.855714 0.015119 Lower Bound = 0.844514

Exp.6

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 640

Upper bound = 0.875483 0.862857 0.017043 Lower Bound = 0.850231

Exp.7

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 1280 No of Hosts = 10

Upper bound = 0.896003 0.88 0.021602 Lower Bound = 0.863997 0.871429 0.017728 Upper bound =

Exp.8

93

No of VMs = 20 No of Cloudlets = 2560

Exp.9

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 5120

Exp.10

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 10240

Exp.11

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20480

Exp.12

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 40960

Exp.13 +%50

No of Hosts = 10 No of VMs = 20 No of Cloudlets = 61440

Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.88 kWh 0.85 kWh 0.87 kWh 0.88 kWh 0.90 kWh 0.85 kWh 0.88 kWh 0.87 kWh 0.84 kWh 0.89 kWh 0.88 kWh 0.87 kWh 0.90 kWh 0.93 kWh 0.85 kWh 0.87 kWh 0.86 kWh 0.84 kWh 0.85 kWh 0.87 kWh 0.89 kWh 0.88 kWh 0.86 kWh 0.87 kWh 0.85 kWh 0.87 kWh 0.90 kWh 0.90 kWh 0.88 kWh 0.86 kWh 0.86 kWh 0.87 kWh 0.84 kWh 0.85 kWh 0.86 kWh 0.87 kWh 0.88 kWh 0.87 kWh 0.85 kWh 0.88 kWh 0.89 kWh

0.884561 Lower Bound = 0.858296

Upper bound = 0.889807 0.875714 0.019024 Lower Bound = 0.861622

Upper bound = 0.889249 0.867143 0.029841 Lower Bound = 0.845037

Upper bound = 0.887014 0.874286 0.017182 Lower Bound = 0.861557

Upper bound = 0.880442 0.865714 0.019881 Lower Bound = 0.850987

Upper bound = 0.881394 0.871429 0.013452 Lower Bound = 0.861463

94

8.12. Appendix L: Collected Results of Scenario 5: Increasing the No. Of Hosts and VMs DVFS Mechanism
Table 10. Shows 10 experiments with the deployment of DVFS mechanism. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.75 kWh 0.73 kWh 0.74 kWh 0.74 kWh 0.73 kWh 0.74 kWh 0.72 kWh 1.48 kWh 1.51 kWh 1.55 kWh 1.52 kWh 1.51 kWh 1.47 kWh 1.57 kWh 3.27 kWh 3.18 kWh 3.08 kWh 3.10 kWh 3.20 kWh 3.02 kWh 3.06 kWh 6.39 kWh 6.39 kWh 6.38 kWh 6.86 kWh 6.16 kWh 6.43 kWh 6.68 kWh 12.90 kWh 12.75 kWh 13.00 kWh 13.00 kWh 12.30 kWh 13.35 kWh 13.05 kWh 25.89 kWh 26.29 kWh 25.08 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.248245

Exp.1

0.244286

0.005345 Lower Bound = 0.240326

Exp.2

No of Hosts = 20 No of VMs = 40 No of Cloudlets = 20

Upper Bound = 0.742944 0.74 0.009759 Lower Bound = 0.728485

Exp.3

No of Hosts = 40 No of VMs = 80 No of Cloudlets = 20

Upper Bound = 1.54203 1.515714 0.035523 Lower Bound = 1.489399

Exp.4

No of Hosts = 80 No of VMs = 160 No of Cloudlets = 20 No of Hosts = 160 No of VMs = 320 No of Cloudlets = 20 No of Hosts = 320 No of VMs = 640 No of Cloudlets = 20 No of Hosts = 640 No of VMs =

Upper Bound = 3.195843 3.13 0.088882 Lower Bound = 3.064157

Upper Bound = 6.639684 6.47 0.229056 Lower Bound = 6.300316

Exp.5

Upper Bound = 13.14664 12.90714 0.323301 Lower Bound = 12.66764 Upper Bound = 25.80636

Exp.6

Exp.7

25.37

0.590972

95

1280 No of Cloudlets = 20 No of Hosts = 1280 No of VMs = 2560 No of Cloudlets = 20 No of Hosts = 2560 No of VMs = 5120 No of Cloudlets = 20 No of Hosts = 3840 No of VMs = 7680 No of Cloudlets = 20

Exp.8

Exp.9

Exp.10 +50%

Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

25.08 kWh 25.69 kWh 24.78 kWh 24.77 kWh 50.08 kWh 49.67 kWh 48.86 kWh 50.08 kWh 51.70 kWh 50.49 kWh 50.28 kWh 102.59 kWh 102.59 kWh 98.93 kWh 100.96 kWh 101.37 kWh 100.56 kWh 104.21 kWh 155.15 kWh 148.44 kWh 153.33 kWh 146.61 kWh 155.77 kWh 153.33 kWh 149.06 kWh

Lower Bound = 24.93078

Upper Bound = 50.8028

50.16571

0.859997 Lower Bound = 49.52863

Upper Bound = 102.8646 101.6014 1.705153 Lower Bound = 100.3383

Upper Bound = 154.3286 151.67 3.588876 Lower Bound = 149.0114

Table 11. Shows 10 experiments without the deployment of power aware mechanism Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 0.84 kWh 0.88 kWh 0.88 kWh 0.84 kWh 0.85 kWh 0.86 kWh 0.94 kWh 1.74 kWh 1.76 kWh 1.74 kWh 1.72 kWh 1.72 kWh 1.77 kWh 1.78 kWh 3.39 kWh 3.49 kWh 3.60 kWh 3.32 kWh 3.39 kWh 3.44 kWh 3.38 kWh 6.86 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.896016

Exp.1

0.87

0.035119 Lower Bound = 0.843984

Exp.2

No of Hosts = 20 No of VMs = 40 No of Cloudlets = 20

Upper Bound = 1.764629 1.747143 0.023604 Lower Bound = 1.729657

Exp.3

No of Hosts = 40 No of VMs = 80 No of Cloudlets = 20 No of Hosts =

Upper Bound = 3.497895 3.43 0.091652 Lower Bound = 3.362105 6.982857 0.14127 Upper Bound =

Exp.4

96

80 No of VMs = 160 No of Cloudlets = 20

Exp.5

No of Hosts = 160 No of VMs = 320 No of Cloudlets = 20 No of Hosts = 320 No of VMs = 640 No of Cloudlets = 20 No of Hosts = 640 No of VMs = 1280 No of Cloudlets = 20 No of Hosts = 1280 No of VMs = 2560 No of Cloudlets = 20 No of Hosts = 2560 No of VMs = 5120 No of Cloudlets = 20 No of Hosts = 3840 No of VMs = 7680 No of Cloudlets = 20

Exp.6

Exp.7

Exp.8

Exp.9

Exp.10 +50%

Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

6.94 kWh 7.06 kWh 6.83 kWh 7.00 kWh 6.94 kWh 7.25 kWh 13.39 kWh 14.11 kWh 13.56 kWh 13.22 kWh 14.00 kWh 13.89 kWh 15.39 kWh 27.22 kWh 27.67 kWh 28.67 kWh 27.89 kWh 27.56 kWh 26.78 kWh 28.11 kWh 56.00 kWh 55.78 kWh 58.00 kWh 54.67 kWh 53.11 kWh 56.00 kWh 57.11 kWh 111.11 kWh 114.67 kWh 113.87 kWh 113.33 kWh 106.67 kWh 110.22 kWh 108.44 kWh 226.67 kWh 223.11 kWh 224.00 kWh 227.56 kWh 230.22 kWh 231.11 kWh 217.78 kWh 334.67 kWh 344.00 kWh 340.00 kWh 326.67 kWh 332.00 kWh 329.33 kWh 334.67 kWh

7.087509 Lower Bound = 6.878205

Upper Bound = 14.46986 13.93714 0.719113 Lower Bound = 13.40443

Upper Bound = 28.15288 27.7 0.611337 Lower Bound = 27.24712

Upper Bound = 56.9871 55.81 1.588962 Lower Bound = 54.6329

Upper Bound = 113.385 111.1871 2.966832 Lower Bound = 108.9893

Upper Bound = 229.1784 225.7786 4.589486 Lower Bound = 222.3787

Upper Bound = 338.91 334.4771 5.983894 Lower Bound = 330.0443

97

8.13. Appendix M: Collected Results of Scenario 6: Increasing the No. Of Hosts and Cloudlets DVFS Mechanism
Table 12. Shows 13 experiments with the deployment of DVFS mechanism. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 0.25 kWh 0.25 kWh 0.26 kWh 0.25 kWh 0.25 kWh 0.25 kWh 0.26 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.23 kWh 0.24 kWh 0.24 kWh 0.24 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.256472

Exp.1

0.252857

0.00488 Lower Bound = 0.249242

Exp.2

No of Hosts = 20 No of VMs = 20 No of Cloudlets = 40

Upper Bound = 0.246541 0.241429 0.006901 Lower Bound = 0.236317

Exp.3

No of Hosts = 40 No of VMs = 20 No of Cloudlets = 80

Upper Bound = 0.246472 0.242857 0.00488 Lower Bound = 0.239242

Exp.4

No of Hosts = 80 No of VMs = 20 No of Cloudlets = 160

Upper Bound = 0.241371 0.238571 0.00378 Lower Bound = 0.235771

Exp.5

No of Hosts = 160 No of VMs = 20 No of Cloudlets = 320

Upper Bound = 0.241371 0.238571 0.00378 Lower Bound = 0.235771

Exp.6

No of Hosts = 320 No of VMs = 20 No of Cloudlets = 640 No of Hosts = 640 No of VMs = 20

Upper Bound = 0.246541 0.241429 0.006901 Lower Bound = 0.236317 Upper Bound = 0.244229

Exp.7

0.241429

0.00378

98

No of Cloudlets = 1280

Exp.8

No of Hosts = 1280 No of VMs = 20 No of Cloudlets = 2560

Exp.9

No of Hosts = 2560 No of VMs = 20 No of Cloudlets = 5120

Exp.10

No of Hosts = 5120 No of VMs = 20 No of Cloudlets = 10240

Exp.11

No of Hosts = 10240 No of VMs = 20 No of Cloudlets = 20480

Exp.12

No of Hosts = 20480 No of VMs = 20 No of Cloudlets = 40960

Exp.13 +50%

No of Hosts = 30720 No of VMs = 20 No of Cloudlets = 61440

Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.23 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.24 kWh 0.25 kWh

Lower Bound = 0.238629

Upper Bound = 0.246472 0.242857 0.00488 Lower Bound = 0.239242

Upper Bound = 0.244277 0.24 0.005774 Lower Bound = 0.235723

Upper Bound = 0.244229 0.241429 0.00378 Lower Bound = 0.238629 Upper Bound = 0.249674 0.245714 0.005345 Lower Bound = 0.241755

Upper Bound = 0.244229 0.241429 0.00378 Lower Bound = 0.238629 Upper Bound = 0.244229 0.241429 0.00378 Lower Bound = 0.238629

99

Table 13. Shows 13 experiments without the deployment of power aware mechanism Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 0.86 kWh 0.89 kWh 0.86 kWh 0.91 kWh 0.86 kWh 0.87 kWh 0.87 kWh 1.79 kWh 1.74 kWh 1.76 kWh 1.73 kWh 1.69 kWh 1.76 kWh 1.80 kWh 3.51 kWh 3.44 kWh 3.68 kWh 3.51 kWh 3.46 kWh 3.58 kWh 3.68 kWh 7.39 kWh 6.72 kWh 6.78 kWh 6.81 kWh 6.64 kWh 6.97 kWh 6.67 kWh 13.83 kWh 13.89 kWh 13.78 kWh 13.89 kWh 13.78 kWh 13.44 kWh 14.67 kWh 28.78 kWh 29.56 kWh 28.33 kWh 28.00 kWh 27.22 kWh 28.89 kWh 27.22 kWh 54.44 kWh 54.67 kWh 55.56 kWh 56.44 kWh 55.33 kWh 54.44 kWh 56.44 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.888378

Exp.1

0.874286

0.019024 Lower Bound = 0.860193

Exp.2

No of Hosts = 20 No of VMs = 20 No of Cloudlets = 40

Upper Bound = 1.780481 1.752857 0.037289 Lower Bound = 1.725234

Exp.3

No of Hosts = 40 No of VMs = 20 No of Cloudlets = 80

Upper Bound = 3.624317 3.551429 0.098392 Lower Bound = 3.47854

Exp.4

No of Hosts = 80 No of VMs = 20 No of Cloudlets = 160

Upper Bound = 7.046934 6.854286 0.260055 Lower Bound = 6.661638

Exp.5

No of Hosts = 160 No of VMs = 20 No of Cloudlets = 320

Upper Bound = 14.17405 13.89714 0.373796 Lower Bound = 13.62024

Exp.6

No of Hosts = 320 No of VMs = 20 No of Cloudlets = 640

Upper Bound = 28.93353 28.28571 0.874488 Lower Bound = 27.6379

Exp.7

No of Hosts = 640 No of VMs = 20 No of Cloudlets = 1280

Upper Bound = 55.9756 55.33143 0.869567 Lower Bound = 54.68726

100

Exp.8

No of Hosts = 1280 No of VMs = 20 No of Cloudlets = 2560

Exp.9

No of Hosts = 2560 No of VMs = 20 No of Cloudlets = 5120

Exp.10

No of Hosts = 5120 No of VMs = 20 No of Cloudlets = 10240

Exp.11

No of Hosts = 10240 No of VMs = 20 No of Cloudlets = 20480

Exp.12

No of Hosts = 20480 No of VMs = 20 No of Cloudlets = 40960

Exp.13 +50%

No of Hosts = 30720 No of VMs = 20 No of Cloudlets = 61440

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

118.67 kWh 109.78 kWh 108.44 kWh 114.67 kWh 110.67 kWh 116.00 kWh 114.22 kWh 219.56 kWh 224.00 kWh 214.22 kWh 222.22 kWh 224.00 kWh 222.22 kWh 229.33 kWh 449.78 kWh 437.33 kWh 444.44 kWh 451.56 kWh 451.56 kWh 455.11 kWh 453.33 kWh 888.89 kWh 910.22 kWh 864.00 kWh 906.67 kWh 888.89 kWh 853.33 kWh 853.33 kWh 1777.78 kWh 1728.00 kWh 1827.56 kWh 1813.33 kWh 1834.67 kWh 1699.56 kWh 1763.56 kWh 2688.00 kWh 2730.67 kWh 2602.67 kWh 2656.00 kWh 2613.33 kWh 2560.00 kWh 2709.33 kWh

Upper Bound = 115.941 113.2071 3.690455 Lower Bound = 110.4733

Upper Bound = 225.6425 222.2214 4.618161 Lower Bound = 218.8003

Upper Bound = 453.57 449.0157 6.147864 Lower Bound = 444.4614

Upper Bound = 898.5434 880.7614 24.00389 Lower Bound = 862.9794

Upper Bound = 1815.767 1777.78 51.27857 Lower Bound = 1739.793

Upper Bound = 2697.472 2651.429 62.15335 Lower Bound = 2605.386

101

8.14. Appendix N: Collected Results of Scenario 7: Increasing the No. Of Hosts, VMs, and Cloudlets - VM Migration Mechanism
Table 14. Shows 9 experiments with the deployment of VM migration mechanism with DVFS enabled. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 0.23 kWh 0.23 kWh 0.22 kWh 0.21 kWh 0.23 kWh 0.21 kWh 0.22 kWh 0.40 kWh 0.42 kWh 0.41 kWh 0.41 kWh 0.41 kWh 0.43 kWh 0.39 kWh 0.74 kWh 0.74 kWh 0.75 kWh 0.77 kWh 0.78 kWh 0.76 kWh 0.76 kWh 1.39 kWh 1.45 kWh 1.40 kWh 1.47 kWh 1.46 kWh 1.42 kWh 1.46 kWh 2.98 kWh 2.90 kWh 2.86 kWh 2.87 kWh 2.98 kWh 3.03 kWh 3.10 kWh 5.91 kWh 5.96 kWh 5.91 kWh 5.85 kWh 6.11 kWh 5.98 kWh 6.21 kWh 12.05 kWh 11.76 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.228094

Exp.1

0.221429

0.008997 Lower Bound = 0.214763

Exp.2

No of Hosts = 20 No of VMs = 40 No of Cloudlets = 40

Upper Bound = 0.419564 0.41 0.01291 Lower Bound = 0.400436

Exp.3

No of Hosts = 40 No of VMs = 80 No of Cloudlets = 80 No of Hosts = 80 No of VMs = 160 No of Cloudlets = 160 No of Hosts = 160 No of VMs = 320 No of Cloudlets = 320 No of Hosts = 320 No of VMs = 640 No of Cloudlets = 640 No of Hosts = 640

Upper Bound = 0.768225 0.757143 0.01496 Lower Bound = 0.74606

Upper Bound = 1.459473 1.435714 0.032071 Lower Bound = 1.411956

Exp.4

Upper Bound = 3.025565 2.96 0.088506 Lower Bound = 2.894435

Exp.5

Upper Bound = 6.083802 5.99 0.126623 Lower Bound = 5.896198 11.87 0.211401 Upper Bound = 12.02946

Exp.6

Exp.7

102

No of VMs = 1280 No of Cloudlets = 1280 No of Hosts = 1280 No of VMs = 2560 No of Cloudlets = 2560 No of Hosts = 2560 No of VMs = 5120 No of Cloudlets = 5120

Exp.8

Exp.9

Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

12.15 kWh 12.00 kWh 11.71 kWh 11.55 kWh 11.89 kWh 22.94 kWh 22.73 kWh 23.45 kWh 24.01 kWh 22.94 kWh 23.39 kWh 23.28 kWh 46.55 kWh 46.44 kWh 46.66 kWh 46.71 kWh 47.09 kWh 46.74 kWh 47.45 kWh

Lower Bound = 11.71625

Upper Bound = 23.56586 23.24857 0.428308 Lower Bound = 22.93128

Upper Bound = 47.06407 46.80571 0.348753 Lower Bound = 46.54736

8.15. Appendix O: Collected Results of Scenario 8: Increasing the No. Of Hosts - VM Migration Mechanism
Table 16. Shows 14 experiments with the deployment of VM migration mechanism with DVFS enabled. Parameters Setting
No of Hosts = 10 No of VMs = 20 No of Cloudlets = 20

Results of Energy Consumption (EC) for each run

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 0.25 kWh 0.23 kWh 0.30 kWh 0.26 kWh 0.27 kWh 0.26 kWh 0.23 kWh 0.18 kWh 0.17 kWh 0.21 kWh 0.22 kWh 0.18 kWh 0.21 kWh 0.20 kWh 0.19 kWh 0.23 kWh 0.20 kWh 0.24 kWh 0.20 kWh 0.21 kWh 0.19 kWh

Mean Value of EC

Standard Deviation of EC

Confidence Interval of EC
Upper Bound = 0.275144

Exp.1

0.257143

0.0243 Lower Bound = 0.239142

Exp.2

No of Hosts = 20 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 0.209807 0.195714 0.019024 Lower Bound = 0.181622

Exp.3

No of Hosts = 40 No of VMs = 20 No of Cloudlets = 20

Upper Bound = 0.22303 0.208571 0.019518 Lower Bound = 0.194113

103

Exp.4

No of Hosts = 80 No of VMs = 20 No of Cloudlets = 20

Exp.5

No of Hosts = 160 No of VMs = 20 No of Cloudlets = 20

Exp.6

No of Hosts = 320 No of VMs = 20 No of Cloudlets = 20

Exp.7

No of Hosts = 640 No of VMs = 20 No of Cloudlets = 20

Exp.8

No of Hosts = 1280 No of VMs = 20 No of Cloudlets = 20

Exp.9

No of Hosts = 2560 No of VMs = 20 No of Cloudlets = 20

Exp.10

No of Hosts = 5120 No of VMs = 20 No of Cloudlets = 20 No of Hosts = 10240 No of VMs = 20 No of Cloudlets = 20

Exp.11

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5

0.18 kWh 0.18 kWh 0.22 kWh 0.22 kWh 0.21 kWh 0.21 kWh 0.18 kWh 0.21 kWh 0.20 kWh 0.19 kWh 0.19 kWh 0.20 kWh 0.21 kWh 0.22 kWh 0.17 kWh 0.22 kWh 0.19 kWh 0.20 kWh 0.21 kWh 0.21 kWh 0.19 kWh 0.18 kWh 0.19 kWh 0.23 kWh 0.19 kWh 0.21 kWh 0.17 kWh 0.21 kWh 0.19 kWh 0.19 kWh 0.22 kWh 0.20 kWh 0.21 kWh 0.20 kWh 0.19 kWh 0.19 kWh 0.20 kWh 0.19 kWh 0.17 kWh 0.20 kWh 0.18 kWh 0.22 kWh 0.18 kWh 0.20 kWh 0.20 kWh 0.20 kWh 0.21 kWh 0.18 kWh 0.24 kWh 0.19 kWh 0.18 kWh 0.19 kWh 0.19 kWh 0.18 kWh

Upper Bound = 0.214185 0.2 0.019149 Lower Bound = 0.185815

Upper Bound = 0.2111 0.202857 0.011127 Lower Bound = 0.194614

Upper Bound = 0.210988 0.198571 0.016762 Lower Bound = 0.186154

Upper Bound = 0.212393 0.20 0.020587 Lower Bound = 0.181892

Upper Bound = 0.208554 0.20 0.011547 Lower Bound = 0.191446

Upper Bound = 0.204736 0.192857 0.016036 Lower Bound = 0.180978

Upper Bound = 0.216507 0.201429 0.020354 Lower Bound = 0.18635 Upper Bound = 0.193683 0.188571 0.006901 Lower Bound = 0.183459

104

Exp.12

No of Hosts = 20480 No of VMs = 20 No of Cloudlets = 20

Exp.13

No of Hosts = 40960 No of VMs = 20 No of Cloudlets = 20

Exp.14

No of Hosts = 81920 No of VMs = 20 No of Cloudlets = 20

Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7

0.20 kWh 0.19 kWh 0.19 kWh 0.21 kWh 0.19 kWh 0.20 kWh 0.18 kWh 0.19 kWh 0.19 kWh 0.21 kWh 0.18 kWh 0.18 kWh 0.19 kWh 0.21 kWh 0.24 kWh 0.19 kWh 0.20 kWh 0.19 kWh 0.19 kWh 0.19 kWh 0.19 kWh 0.20 kWh 0.19 kWh

Upper Bound = 0.199904 0.192857 0.009512 Lower Bound = 0.185811

Upper Bound = 0.216003 0.2 0.021602 Lower Bound = 0.183997

Upper Bound = 0.196472 0.192857 0.00488 Lower Bound = 0.189242

105