Improving Real Time Tuning on YARN

It appears your Web browser is not configured to display PDF files. Download adobe Acrobat or click here to download the PDF file.

Click here to download the PDF file.

Creator: 

Navarro Belmonte, Victor Pablo

Date: 

2018

Abstract: 

Big data is becoming a significant part of the operations of modern organizations. Performing analysis of large amounts of data requires computer clusters to run the calculations and analysis. YARN is an internal framework that is responsible for coordinating big data jobs for some popular distributed storage and processing frameworks like Hadoop and Spark. Running YARN with the correct configuration parameters is critical for the good performance of a cluster. KERMIT is an online tuner of YARN configuration parameters that aims to improve cluster performance. The first KERMIT implementation proved the feasibility of the concept. In this study we modified the tuning algorithms inside the KERMIT components; by doing so, we achieved a reduction in the execution time as compared to industry benchmarks for a shallow tuning technique. We also verified that KERMIT can tune Spark; this suggests that Kermit could be used in other YARN-based frameworks.

Subject: 

Computer Science

Language: 

English

Publisher: 

Carleton University

Thesis Degree Name: 

Master of Computer Science: 
M.C.S.

Thesis Degree Level: 

Master's

Thesis Degree Discipline: 

Computer Science

Parent Collection: 

Theses and Dissertations

Items in CURVE are protected by copyright, with all rights reserved, unless otherwise indicated. They are made available with permission from the author(s).