SLA-aware Energy-efficient Scheduling Scheme for Hadoop YARN

LI Ping; JU Lei; JIA Zhiping

Chinese︱Feedback︱Save this page

• Elaborating Academic Views 　　　　 • Exchanging Innovative Ideas
• Protecting Intellectual Properties 　　• Fast Sharing Science Papers

Sponsored by the Center for Science and Technology Development of the Ministry of Education
Supervised by Ministry of Education of the People's Republic of China

Home > Papers

SLA-aware Energy-efficient Scheduling Scheme for Hadoop YARN

LI Ping 1,JU Lei 2 * #,JIA Zhiping 2

1.School of Computer Science and Technology, Shandong University, 250101

2.School of Computer Science and Technology, Shandong University, 250101

*Correspondence author

#Submitted by

Subject:

Funding: Research Fund for the Doctoral Program of Higher Education of China （No.RFDP) grant 2012013112003）

Opened online:15 June 2015

Accepted by: none

Citation: LI Ping,JU Lei,JIA Zhiping.SLA-aware Energy-efficient Scheduling Scheme for Hadoop YARN[OL]. [15 June 2015] http://en.paper.edu.cn/en_releasepaper/content/4645579

Apache Hadoop becomes ubiquitous for cloud computing which provide resources as services for multi-tenant applications. YARN (a.k.a. MapReduce 2.0) is one of the key feature in the second-generation Hadoop, which provides resource management and scheduling for large scale MapReduce environments. Two enormous challenges in the YARN scheduler are the abilities to automatically tailor and control resource allocations to different jobs for achieving their Service Level Agreements (SLAs), and minimize energy consumption of the overall cloud computing system. In this work, we propose an SLA-aware energy-efficient scheduling scheme which allocates appropriate amount of resources to MapReduce applications with YARN architecture. We perform job profiling to obtain the performance characteristics for different phases of a MapReduce application, which will be considered during resource provisioning in order to meet the completion deadlines specified by the application's SLA. Furthermore, an online userspace governor based dynamic voltage and frequency scaling (DVFS) scheme is designed in the YARN per-application ApplicationMaster to dynamically change the CPU frequency for upcoming tasks given the slack time between the actual execution time of completed tasks and expected completion time of the application. Experimental evaluation shows that our proposed scheme is both resource and energy efficient compared with the existing MapReduce scheduling policies.

Keywords:computer architecture; Hadoop YARN; SLA; resource allocation; DVFS

For this paper

● PDF (0B)
● Revision 0 　　
● Print this paper
● Recommend this paper to a friend
● Add to my favorite list

Saved Papers

Please enter a name for this paper to be shown in your personalized Saved Papers list

Tags

Add yours

Related Papers

Statistics

PDF Downloaded	42754
Bookmarked	0
Recommend	5
Comments	Array

Submit your papers

Alert Name:
Alerting to:
Authentication email will be sent to your email address in 24 hours
Frequency:
Email Message Format:	Plain text Graphical(HTML)

Complete the form below and we will recommend the selected titles to your friends on your behalf. * Indicates a required field.
Your name*:
Your email address*:
Recipient's name*:
Recipient's email address*:
(multiple recipient's names and email addresses should be separated with semicolons)
Your comments:	I thought you would find the page(s) useful.

Your name:
Your email address:
Recipient's name:
Recipient's email address:
(multiple recipient's names and email addresses should be separated with semicolons)
Your comments:	I thought you would find this page useful.

Disclaimer: This message was sent to your friend using the "Send it to a friend" facility on the Sciencepaper Online’ WWW site, http://www.paper.edu.cn/en. The Sciencepaper Online is not responsible for the content of this email, and anything said in this email does not necessarily reflect the Sciencepaper Online's views.

	Check out RSS, or use RSS reader to subscribe this item

Saved Papers