Learning to Query: Deep Learning Based Method for Improving Join Query in Hive

Lixiang Huang; Tong Xu; WeiLi; Lei Zhang

Chinese︱Feedback︱Save this page

• Elaborating Academic Views 　　　　 • Exchanging Innovative Ideas
• Protecting Intellectual Properties 　　• Fast Sharing Science Papers

Sponsored by the Center for Science and Technology Development of the Ministry of Education
Supervised by Ministry of Education of the People's Republic of China

Home > Papers

Learning to Query: Deep Learning Based Method for Improving Join Query in Hive

Lixiang Huang 1,Tong Xu 2 *,WeiLi 2,Lei Zhang 2

1.Institute of Network Technology, Beijing University of Posts and Telecommunications, 100000;Institute of Network Technology, Beijing University of Posts and Telecommunications, 100000;Institute of Network Technology, Beijing University of Posts and Telecommunications, 100000;Institute of Network Technology, Beijing University of Posts and Telecommunications, 100000

*Correspondence author

#Submitted by

Subject:

Funding: none

Opened online:18 December 2018

Accepted by: none

Citation: Lixiang Huang,Tong Xu,WeiLi.Learning to Query: Deep Learning Based Method for Improving Join Query in Hive[OL]. [18 December 2018] http://en.paper.edu.cn/en_releasepaper/content/4746691

Improving join query is crucial for efficiency of processing queries over big data. Hive is a batch-oriented big data management engine that is well suited for data analysis and mining. In this paper, we introduced an innovative method to improve join query in Hive, from the perspective of users. In this solution, users only need to configure the entity attributes, filtering rules and join rules between the entities in the visual operation interface, and our intelligent model can decide the optimal query plan and execute it. We proposed a graph-based SQL generation model to generate Hive-based query plans for complex join query, and use deep learning technique LSTM for query cost prediction so as to pick out the optimal query plan. Moreover, through a comprehensive set of experiments, we demonstrate that our proposed method performs well and significantly improves join query in Hive.

Keywords:Big Data; Join Query; LSTM; SQL Generation Engine; Query Cost Prediction

For this paper

● PDF (0B)
● Revision 0 　　　　　　
● Print this paper
● Recommend this paper to a friend
● Add to my favorite list

Saved Papers

Please enter a name for this paper to be shown in your personalized Saved Papers list

Tags

Add yours

Related Papers

Statistics

PDF Downloaded	102
Bookmarked	0
Recommend	0
Comments	Array

Submit your papers

Alert Name:
Alerting to:
Authentication email will be sent to your email address in 24 hours
Frequency:
Email Message Format:	Plain text Graphical(HTML)

Complete the form below and we will recommend the selected titles to your friends on your behalf. * Indicates a required field.
Your name*:
Your email address*:
Recipient's name*:
Recipient's email address*:
(multiple recipient's names and email addresses should be separated with semicolons)
Your comments:	I thought you would find the page(s) useful.

Your name:
Your email address:
Recipient's name:
Recipient's email address:
(multiple recipient's names and email addresses should be separated with semicolons)
Your comments:	I thought you would find this page useful.

Disclaimer: This message was sent to your friend using the "Send it to a friend" facility on the Sciencepaper Online’ WWW site, http://www.paper.edu.cn/en. The Sciencepaper Online is not responsible for the content of this email, and anything said in this email does not necessarily reflect the Sciencepaper Online's views.

	Check out RSS, or use RSS reader to subscribe this item

Saved Papers