[D] Machine learning, alchemy or real science?
A while ago I’ve seen a video of a talk by Ali Rahimi. It’s worth the 20 minutes for people who haven’t seen it!
I recently rewatched it and tried to look around if I could find papers which go into the direction of focusing more onto smaller ML experiments to build fundamental knowledge. But I’ve not found much aside from stuff on interpretability.
As a side note, I’m not an ML researcher but a physicist who uses ML in various aspects for work. I’d not consider myself an expert on any ML specific field but have an interest and try to keep up to date on things. From my personal experience, I’ve certainly felt (not sure how to describe it but let’s go with annoyed/demotivated) to use ML when it’s not clear to me why certain things work.
So a lot of times it seems more like Alchemy, in that I mix and match certain things until I get a result I like.
But why does it work? No clue.
Will it work for the next problem that is sufficiently different? No clue!
I’d be very interested to hear what other people think or how they feel about this topic.
Also if you know of any paper seeking to build a more fundamental understanding, I’d appreciate a link 🙂