I just came across this recent post by Andrew Ng’s deeplearning.ai team, talking about the importance of initialization. The interactive demos are fantastic! I haven’t seen it shared anywhere, so thought I would do so here.

http://www.deeplearning.ai/ai-notes/initialization/