Recently, Neural Architecture Search is coming back to the research spotlight. Elsken et al. published a survey on this topic (https://arxiv.org/pdf/1808.05377.pdf ) but the development is fast and many new works are emerging. For example, there is Weight Agnostic Neural Network (WANN) https://arxiv.org/abs/1906.04358 that demonstrates that Neural Architectures can be more significant than the weights of the network. You can read of the list of paper in this topic at https://www.automl.org/automl/literature-on-neural-architecture-search/ . Nevertheless, this type of topic is already researched in 1990 ( https://pdfs.semanticscholar.org/2118/55f1de279c452858177331860cbc326351ab.pdf ), are there still significance in improvement? If so, how much?

Are researchers just making up new Neural Architecture Search methods for publication, or is there really a big difference? Are there any work that focused on a detailed comparison for Neural Architecture Search.