Now we have more details about how OpenAI LP is using the proceeds of their “Capped-Profit” investment vehicle 🙂

From the article:

WeWork’s parent company, artificial intelligence company OpenAI and the World Economic Forum have teamed up on a proposal to redevelop and renovate the 22 barracks and buildings of the Presidio’s Fort Scott campus.

The We Company’s space would be split between “mission-driven” companies — those “focused on addressing the significant environmental and/or social challenges of our time,” as specified in the Presidio Trust’s Requests for Proposals — and the EPIC Institute, a nonprofit funded by the California Clean Energy Fund, a private equity and VC firm. The Lela Goren Group, a woman-owned and operated real estate developer, would be the co-developer along with The We Co.

Site plans call for artificial intelligence company OpenAI to lease about 107,000 square feet of space for its offices and labs, many of which are adjacent to two office buildings previously developed by the Presidio Trust and currently occupied by the World Economic Forum.

https://www.bizjournals.com/sanfrancisco/news/2019/03/15/wework-openai-redevelop-presidio-fort-scott.html

I recently read a paper that suggested increasing weight decay, dropout rate, etc. (i.e., regularization parameters) while a deep network was training to avoid overfitting; however, I cannot remember the name of the paper. I tried to search through the literature, but searching using terms like “increase regularization deep learning” hasn’t turned up much (unsurprisingly).

​

I did find Curriculum Dropout, which suggests increasing the dropout rate during training, but I don’t believe this is the paper I had in mind.

​

Anyone happen to know of other papers discussing this subject? Are there any appearing trends surrounding changing regularization parameters during training? Anyone have any experience testing this idea out?

​

Thanks

Hi all, we are super proud to announce the Depth First Learning 2019 Fellows (http://www.depthfirstlearning.com/2019/Announcing-DFL-Fellows).

They are:

Steve Kroon – Stellenbosch (South Africa) –> Variational Inference with Normalizing Flows

Sandhya Prabhakaran – New York (USA) –> Spherical CNNs

Bhairav Mehta – Montreal (Canada) –> Stein Variational Gradient Descent

Vinay Ramasesh, Piyush Patil, and Riley Edmunds – Berkeley (USA) –> Resurrecting the Sigmoid in Deep Learning through Dynamical Isometry

If you’re keen to join in on any of those learning groups, please do apply on our announcement page (http://www.depthfirstlearning.com/2019/Announcing-DFL-Fellows).

(reposting, I guess the first time I didn’t have a tag so it was removed)

Here’s a new paper and new dataset for spoken language understanding (SLU):

Paper: arxiv.org/abs/1904.03670

Code: https://github.com/lorenlugosch/pretrain_speech_model

Data: https://www.fluent.ai/research/fluent-speech-commands/

We use transfer learning (pre-train the model on LibriSpeech) to improve end-to-end SLU models, and we introduce a new speech dataset that can be used for SLU experiments, or more generally audio and sequence classification experiments.

I’m the first author; let me know if you have any questions!

I’m making a ConvNet that classifies my data into one of classes [A, B, C, D, “other”], “other” being anything that doesn’t fit into [A, B, C, D].

I’m thinking of mapping the input x to a vector “c” in R^4, and then transforming “c” to R⁵ (using a few tiny FC layers) before applying softmax to get the output.

My reasoning is that I don’t want the network attempting “learn” anything specific about the “other” class, I just want it to be a sort of “nor” operation. The network should only focus on learning the features of [A, B, C, D]. The variance in the “other” class is very high so I want it to generalize well.

Thoughts? Papers? Prior experience?

Using pre-trained deep learning models like ResNet, Inception, and VGG is easier than ever, but there are implementation details you need to be careful with to avoid subpar performance and errors.

​

Put together this list of implementation details to be cautious of — curious to see if I’m missing anything or if other folks have had similar experiences with discrepancies in performance?

https://medium.com/comet-ml/approach-pre-trained-deep-learning-models-with-caution-9f0ff739010c

MSc dissertation (from Summer 2018) on using the Differentiable Neural Computer in the Reinforcement Learning / Evolution Strategies context: http://blog.adeel.io/2018/09/10/probabilistic-model-based-reinforcement-learning-using-the-differentiable-neural-computer/

“…experiments found that a model learned in a Differentiable Neural Computer outperformed a vanilla LSTM based model, on two gaming environments.”

Would love to hear your thoughts!