“System and method for addressing overfitting in a neural network”

​

2016-08-02

Application granted

2017-10-05

Assigned to GOOGLE LLC

2019-06-25

Application status is Active

A system for training a neural network. A switch is linked to feature detectors in at least some of the layers of the neural network. For each training case, the switch randomly selectively disables each of the feature detectors in accordance with a preconfigured probability. The weights from each training case are then normalized for applying the neural network to test data.

https://patents.google.com/patent/US9406017B2/en

The idea of r/estoration is to restorate old images and remove the little specs, tears, discolorations, etc… And I think that this process could be easily automated for your average image via neural networks, I mean trying to recover data from an image as already been done (denoising autoencoder, super resolution for example), the only thing is that what is one of the most valuable thing (IMO) in Deep Learning is the dataset, plenty of autoencoder/GAN are out there but they are not worth anything if they can’t be trained, hopefully someone or mutiple people (and that obviously will make things faster) reading this is/are motivated to make a dataset.

​

So I am just hoping to spread the word around, if anyone have the patience to make a dataset, maybe even if you have the resources to train a model/host a website for people to use to restore their images plenty of people will be grateful, I really think there is something to be done here.

​

^{Also some people will probably tell me “can’t you do it?”, in theory I can, in practice I don’t have the patience to gather 500+ images and edit them, also I have a r9 390 so I am a bit limited hardware-side, I use plaid-ML+keras on windows 10 for those wondering, plus if I spread the word around hopefully someone with a lot more of ressources/knowledge than me can do something 10x better than what I could.}

^{Speaking of knowledge, I have thought about the architecture of the NN, I think in a first time an autoencoder would be easier (maybe a gan after, a fully convolutionnal one (so that it can accept most images sizes, with strides of 2 instead of pools, that output a 3 layer image that is then added to the input image to produce the output image, so as to reduce blurriness and maybe use dssim as loss, I will just leave that out there in case someone wants a “guide” instead of using a random autoencoder.}

Hello, i’ve recently stumbled upon: Psychopathy Prediction Based on Twitter Usage and i’ve been wondering if there any other cool applications of ml in these fields. Im the most interested in user profiling based on social media activity(papers, blog posts, kaggle competitions previous or active)

​

Thanks!

I am working on a project that aims to predict stock returns using tweet data. I have been playing with an online dataset from here: https://github.com/yumoxu/stocknet-dataset. My aim is to feed, for example, tweets for 30 stocks in a day (variable number of tweets every day), and output a vector of stock return predictions for those 30 stocks. Since each tweet has different length, I was thinking to implement a RNN to feed in the words sequentially. It then seems to me the model will then capture the “temporal structure” of the text, but I am not sure how to capture the time series aspect of the data.

​

My questions can be summarised as follows:

(1) How to incorporate the time series as well as the textual temporal structure in the data I have?

(2) Or I am modelling my problem wrongly?

​

Edit: I have heard of encoder-decoder structures in sophisticated models like BERT, and the use of <EOS> tags to notify the model where to stop for each sentence (tweet). I think that might be something I should look into but it seems a little complicated when I was reading the BERT paper. I am rather amateur in this area so I prefer something a little beginner friendly to start with. Thanks!

​

Any ideas or references will be greatly appreciated. Cheers!