It has been established that diverse behaviors spanning the controllable subspace of an Markov decision process can be trained by rewarding a policy for being distinguishable from other policies. However, one limitation of this formulation is generalizing behaviors beyond the finite set being explicitly learned, as is needed for use on subsequent tasks. Successor features provide an appealing solution to this generalization problem, but require defining the reward function as linear in some grounded feature space. In this paper, we show that these two techniques can be combined, and that each method solves the other’s primary limitation. To do so we introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide enhanced generalization and fast task inference through the successor feature framework. We empirically validate VISR on the full Atari suite, in a novel setup wherein the rewards are only exposed briefly after a long unsupervised phase. Achieving human-level performance on 14 games and beating all baselines, we believe VISR represents a step towards agents that rapidly learn from limited feedback.

https://arxiv.org/abs/1906.05030

So I’ve used serpent, mss, pil, custom buffer transfer from Windows to numpy and no matter what you’re topping out at 60ish fps. So we have about 60 fps capture running in its own thread. Then I have yolov3 tiny which when capturing from a on disk video can process at 10-20ms per frame. Cool. Also about 60 fps.

I think im losing an additional 15ms when I do expand, and I think it’s resize. So you have an image that is a numpy array and you expand it (which I think just adds a dimension), then you resize it for processing by the model. Those 2 lines are killing me.

For people that are doing things like real time game play, how are you handling your pipeline in?

Edit: too, I’ve read pytorch is faster (I’m dying reimplementing this thing so many times though) would anyone agree?

I am working on a supervised regression problem and I have several different representations of essentially the same input features but with varying formulations and of varying length (model_i). I also have a list of properties that influence the target variable (props), which I am thinking of adding as a stacked layer. Basically I want to proceed in the following way:

​

model_i –> predict props –> predict target variable,

​

where I compute for each separate model “i” and just choose either the best performing result, or take the average over all initial models. Is there a better way to approach this problem? I was thinking of using decision tree methods with this approach. I am not too familiar with neural networks but it seems that they are usually ideal for image/video/audio tasks. Can someone point me in the right direction please. Thank you.

For example, a human learns to crawl, then stand, then walk, then run. Learning to run directly would probably be slower.

I’ve seen plenty of “neural network learns to run” youtube videos, but none show a stand training stage.

Is there a formal name for this kind of training? Is it even used in any meaningful way?

Just wanted to share my PyTorch implementation of Amazon’s paper Robust Universal Neural Vocoding

Repo: https://github.com/bshall/UniversalVocoding

Samples: https://bshall.github.io/UniversalVocoding/

Pretrained Models: https://github.com/bshall/UniversalVocoding/releases/tag/v0.1

​

I found that the model trains relatively quickly with intelligible audio after only 20k steps and decent results after 100k steps. Also seems to work well on out of domain speakers (as advertised in the paper).

Background:

I got introduced to this field around 7 months back, circumstances were that I had no internship opportunities in mechanical engineering and a friend of mine was working on his data analyst profile. I have an upcoming undergrad project under a professor in the field of reinforcement learning. My current intern revolves around deep learning and computer vision. Pretty academically oriented – CGPA 8.5 on a 10 point scale which translates to 3.75+ in most colleges.

​

Current situation:

I mentioned I was a really academically focused student. During my intern here, I have to implement certain computer vision models and stuff related to disparity maps. Issue is, I don’t have the necessary background, coming as a mechanical engineer. I researched and found out statistics is an essential part and a lot of CS background is required as well. I plan on going full ahead to pursue a PhD in this field – it doesn’t tire me, I enjoy it and in the 3 years of my college life I have never wanted to learn more about a field. Definitely determined to go for a PhD.

​

So, my undergrad is unrelated. I have 3 options in front of me now:

• MSc in Statistics : I heard this is a good option, but a lot of what they teach isn’t applied to Data Science which focused on computational statistics. My knowledge in Statistics is limited to the high school probability class, permutations and combinations and I touched a little on the z-scores and p-scores thing casually. Definitely seems like I need formal education in statistics but I’m worried about going too deep into this when it’s not required in Data Science.
• MSc in Computer Science : One of the most sought after degrees right now. I talked to a few batch-mates doing undergrad in CS though, they’re also learning of Operating Systems, Database management and hardware. I don’t think this will be required AT ALL in the field of my interest. But I’d definitely love an opinion from someone with experience in this regard.
• MSc in Data Science : Now this is a rather new branch! I have seen few colleges offering this but I’m skeptical because I read reviews that the colleges are just brushing the topics superficially and not going in depth. If I plan on going for a research I’ll need good foundation – that’s can’t be compromised. It also seems to be a Jack of all trades and a master of none thing. I would have the basic idea of everything – CS and Stats, but a good grasp in neither. Again, would love to hear from someone who is pursuing this.

​

Appreciate any support. I have been researching this for days and I was about to write emails to professors/PhD students directly for guidance but thought of asking here first.

I’m a student researcher looking for literature on neural network parameter optimization where the objective loss is non-smooth. Meaning that that the typical gradient based methods are ruled out and something like proximal gradient methods are employed. Preferably in the context of regression. This condition seems to be commonly ignored in practice.

1. Are non differentialable losses avoided in NN’s?
2. Is there a need for this kind of work from a non theoretical point of view? That is, smoothness conditions are violated, but gradient methods still find empirical success?

I have many more questions, but really any direction or content would be helpful! Thanks!