Nonlinear stochastic variance reduction gradient based neural networks and its convergence

WANG Jian; YANG Guo-Ling; ZHANG Hua-Qing; GAO Tao; SUN Zhan-Quan

Chinese︱Feedback︱Save this page

• Elaborating Academic Views 　　　　 • Exchanging Innovative Ideas
• Protecting Intellectual Properties 　　• Fast Sharing Science Papers

Sponsored by the Center for Science and Technology Development of the Ministry of Education
Supervised by Ministry of Education of the People's Republic of China

Home > Papers

Nonlinear stochastic variance reduction gradient based neural networks and its convergence

WANG Jian 1 #, YANG Guo-Ling 1, ZHANG Hua-Qing 1,GAO Tao 2, SUN Zhan-Quan 3

1. College of Science, China University of Petroleum, Qingdao, 266580

2. College of Information and Control Engineering, China University of Petroleum, Qingdao, 266580

3. Shandong Computer Science Center (National Supercomputer Center in Jinan), Shandong Provincial Key Laboratory of Computer Networks, Jinan, 250014

*Correspondence author

#Submitted by

Subject:

Funding: National Natural Science Foundation of China （No.No. 61305075）, Specialized Research Fund for the Doctoral Program of Higher Education of China （No.No. 20130133120014）

Opened online:25 May 2017

Accepted by: none

Citation: WANG Jian, YANG Guo-Ling, ZHANG Hua-Qing.Nonlinear stochastic variance reduction gradient based neural networks and its convergence[OL]. [25 May 2017] http://en.paper.edu.cn/en_releasepaper/content/4733411

A large number of optimization problems are nonconvex optimization problems. Stochastic variance reduced gradient (SVRG) algorithm provides a solution, which can solve the nonconvex optimization problems through training neural networks. In this paper, nonlinear stochastic variance reduced gradient method (NSVRG) is proposed, in which the objective function is nonlinear. Under mild conditions, the monotonicity of the error function is obtained. We then establish the weak convergence property with a constant learning rate. The weak convergence indicates that the gradient of the error function goes to zero. Finally, numerical example is given to substantiate the effectiveness of the theoretical results.

Keywords:feedforward neural network, nonconvex, stochastic variance reduced gradient algorithm, monotonicity, convergence

For this paper

● PDF (0B)
● Revision 0 　　
● Print this paper
● Recommend this paper to a friend
● Add to my favorite list

Saved Papers

Please enter a name for this paper to be shown in your personalized Saved Papers list

Tags

Add yours

Related Papers

Statistics

PDF Downloaded	41
Bookmarked	0
Recommend	0
Comments	Array

Submit your papers

Alert Name:
Alerting to:
Authentication email will be sent to your email address in 24 hours
Frequency:
Email Message Format:	Plain text Graphical(HTML)

Complete the form below and we will recommend the selected titles to your friends on your behalf. * Indicates a required field.
Your name*:
Your email address*:
Recipient's name*:
Recipient's email address*:
(multiple recipient's names and email addresses should be separated with semicolons)
Your comments:	I thought you would find the page(s) useful.

Your name:
Your email address:
Recipient's name:
Recipient's email address:
(multiple recipient's names and email addresses should be separated with semicolons)
Your comments:	I thought you would find this page useful.

Disclaimer: This message was sent to your friend using the "Send it to a friend" facility on the Sciencepaper Online’ WWW site, http://www.paper.edu.cn/en. The Sciencepaper Online is not responsible for the content of this email, and anything said in this email does not necessarily reflect the Sciencepaper Online's views.

	Check out RSS, or use RSS reader to subscribe this item

Saved Papers