Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[R] Piecewise Strong Convexity of Neural Networks

Paper: https://arxiv.org/abs/1810.12805

Video summary: https://www.youtube.com/watch?v=z89BTMQGVn

Earlier related work: https://arxiv.org/abs/1607.04917 (piecewise convexity)

I am not the author. This paper will be presented at NeurIPS this month and exposes some convexity results about piece-wise linear nns under the least squares loss – namely piecewise strong-convexity & the non-existance of differentiable local maxima. The approach is a spectral analysis of the Hessian and weights of the nn. The result is a relatively attractive convergence estimate for sgd.

I guess this provides some more motivation for studying techniques like ADMM which have convergence properties for some classes of piece-wise functions and can exploit lipschitz cts gradients. Nice work!

submitted by /u/i-heart-turtles
[link] [comments]