Hi guys, I have what could be a stupid question, but I see that I’m encountering this issue regularly and would like to know your opinion: so yesterday I was trying to improve my ML model in order to improve its accuracy, and found out that it was performing worse. Why? I checked the previous model architecture (saved with Keras plot_model) and saw what I did differently last week. No problem, I will just revert to that architecture and test again. Model overfits in half the epochs now. Damn, I also changed the dataset augmentation pipeline, now I cannot recreate those specific scores.

​

Basically this is my issue, I happen to develop a model for n-days, test it, save it etc. then after a couple of weeks I try to revert to “that good model setup I was having” and I cannot get the same results anymore as I changed too much stuff. I marginally fixed it by saving the model architecture as png using Keras in order to have a quick visual comparison, It’s not the end of the world, but I don’t have a clean way to deal with this issue. How do you guys avoid such problems?

​

Thank you!

Say I was using Bert and trained it on Squad 2.0 (which I have done) and came across:

Question: What is your favorite color?

Possible (Correct) Answer given:Blue

Possible (Wrong) Answer given:Lasagna

The idea would be that a model would predict Blue on the first one and nothing on the second one (implying it was not a feasible answer).

Is there any research or ideas on how (if possible) you could train a model like a Bert to do that? I feel like it should be doable however my current results with training on Squad 2.0 were not extremely promising so I’m not sure if I’m not thinking about it problem correct or if there is some research out there on how to better approach this.

It seems like a lot of high-impact machine learning research has recently involved work in image or text domains (or perhaps this is mostly what I’m exposed to because I read a lot of deep network papers). Are people here familiar with high-impact work/papers outside of these domains? Do you think the work in these areas is actually less frequent or impactful, or is that perhaps a perceptual bias due to the popularity of things like deep network approaches?

DeepMind, the artificial-intelligence company owned by Google parent Alphabet Inc., saw its revenue almost double last year, but gains were dwarfed by losses that increased to hundreds of millions of dollars.

The London-based company also has more than a billion dollars of debt due for repayment this year, according to full-year accounts for the year ended Dec. 31 posted to U.K. business registry Companies House.

Losses for 2018 widened to 470.2 million pounds ($572 million) from 302.2 million pounds in 2017. Revenue rose to 102.8 million pounds, up from 54.4 million pounds. Staff costs also nearly doubled against the year-ago period to 398 million pounds in 2018.

A debt of 1.04 billion pounds due this year includes an 883 million-pound loan from its owner. DeepMind had written assurances it would be financially supported for at least another year.

“Our DeepMind for Google team continues to make great strides bringing our expertise and knowledge to real-world challenges at Google scale, nearly doubling revenue in the past year,” a spokeswoman for the company said in a statement. “We will continue to invest in fundamental research and our world-class, interdisciplinary team, and look forward to the breakthroughs that lie ahead.”

Alphabet Inc. bought DeepMind for 400 million pounds in 2014. The next year, the company began working on health-care research, eventually creating an entire division dedicated to the area.

The company works with the U.K. National Health Service hospitals, researching algorithms that can diagnose eye diseases and spot head and neck cancers from medical imagery, and the U.S. Department of Veterans Affairs on an algorithm that can predict which patients are at risk of sudden deterioration from acute kidney injury and other conditions.

​

https://www.bloomberg.com/news/articles/2019-08-07/alphabet-s-deepmind-takes-on-billion-dollar-debt-as-loss-spirals