Out of curiosity, does anyone know of any results that train deep models on the ImageNet dataset —

• in an online (i.e. one-pass) fashion; and/or
• multi-pass but without any data augmentation?

Think it would be an interesting exploration understanding online vs. batch training in nonconvex models!

(Of course, one should take efforts to push for the best results in both settings, such that those results are comparable.)