Hi everyone,

I have a few questions on research practices that are generally followed but rarely mentioned in papers.

1. Let’s say I have a train-dev-test split. After finding the best hyperparameters on dev set, should I retrain the model on train+dev set before evaluating it on the test set? Some discussions say yes, others say it depends on you and how much data you have and some say no.
2. Let’s say I’m showcasing results on multiple datasets. Can one change the hyperparameters (learning rate, batch size, etc) from one dataset to another? More importantly, can I change, let’s say, number of units in a layer without adding more layers? Would this count as an architectural change?
3. If yes, how would answer to above question change if the same is done within the dataset itself containing multiple parts?
4. Are we allowed to change a publically available dataset? For example, removing outliers for a regression problem?

In neural network-based conditional density estimation (CDE), classic regularization approaches in the parameter space are mostly ineffective. To address this issue, we develop a model-agnostic noise regularization method for CDE that adds random perturbations to the data during training. We demonstrate that the proposed approach corresponds to a smoothness regularization, we prove its asymptotic consistency and show across 7 datasets and 3 CDE models that this works well. Result: makes neural network-based CDE the preferable method over previous non- and semi-parametric approaches, even when training data is scarce!

Paper: https://arxiv.org/abs/1907.08982
Code: https://github.com/freelunchtheorem/Conditional_Density_Estimation

Hey all, I’m participating in a contest where we have to do sentiment analysis on drug reviews. It has three categories as outputs : positive, negative, neutral.

I’ve done the basics pre- processing, tf-idf vectorizer etc. I’m now on the model application process. Can you suggest some best practices that I might have missed?

I’ve tried out many models like Logistic Regression, LinearSVM, Randomforest etc. LR came out the best. If I improve my f1score by 0.1 I might actually win the contest. Can you please help? It’s currently at 0.54

Hey guys! I’m new to AI and machine learning and I created these 3 laws(similar to 3 laws of thermodynamics) which can govern a self thinking(self-conscious?..maybe in the future?) robot or machine.I would love to hear your input on this.

1st law Law of Efficiency: A Systems efficiency can be determined by how well it handles the entropy present in its environment and the variables given to it.

2nd law Law of Ideal: A system is said to be ideal if it can take into consideration and handle on its own each and every event that happens in its environment and universe.

3rd law Law of Safe: A system is said to be safe if it handles and responds on its own to any event happening in its environment in a universally accepted manner.

What do you think on this?

I built a deep learning workstation to play around with some deep learning models. Nothing to beastly, just a single GTX1060.

What’s the most convenient method to connect to it with my laptop and run experiments on the GPU?

Hi everyone.

I made an AI agent to play the game of Snake using Reinforcement Learning https://youtu.be/yp826Ybh4rU . Here is a link to the code https://github.com/AI-Olympics/Snake-AI . Any suggestions and positive feedback are welcome.

Edit : While making this project I faced a small hurdle i.e the snake AI would perform good and keep increasing it’s average score as training goes on, but after a score/length of near 50, it’s average score would start decreasing and if trained for any longer the AI would become ridiculous at eating food. Is there a way to solve this problem?

Does anyone know what’s going on with AISTATS 2020? The web page https://www.aistats.org/ still directs to AISTATS 2019. Usually, the deadline is in early October, so I’d have expected to see an announcement by now?

I’ve implemented a seq2seq generative character model in pure Numpy on top of CS231n’s assignment code. It’s nothing fancy but I think being able to see things in matrix multiplication level gives good intuitions about concepts in machine learning. Thus this repository contains reference implementations for

• CharRNN (Sequence to Sequence RNN to produce characters with temperature)
• Unit tests with numerical gradients to check implementations
• Character Embedding
• Recurrent Neural Networks
• Temporal Affine Layer
• Temporal Softmax
• Adam optimizer
• Solver (Run training loop, update weights, save model, record logs, etc.)
• Data loading, character mapping, masking non-frequent characters

Hope it will be useful for others, as well!

Repo

https://github.com/kirbiyik/generate-any-text

I am looking to sparse training; that is, during each training step, only a very small fraction of the weights go through the forward and backwards pass. So, only a small fraction of weights need to be loaded into memory at once.

I was thinking of having the weights saved in sqlite or hdf5, and then having a keras layer which copies the relevant weights stored for the forward pass. And then after the update step, those weights are update into the disk.

I know there are ways to directly setting the weights outside training https://stackoverflow.com/questions/51354186/how-to-update-weights-manually-with-keras

But I am wondering if any framework allows manually updating variables, after they have been already updated in a training step.