Hi everyone,

I have created a yet another reinforcement learning library. https://github.com/vwxyzjn/cleanrl

This repository focuses on a clean and minimal implementation of reinforcement learning algorithms. The highlights features of this repo are:

• Most algorithms are self-contained in single files with a common dependency file common.py that handles different gym spaces.
• Easy logging of training processes using Tensorboard and Integration with wandb.com to log experiments on the cloud. Check out https://app.wandb.ai/costa-huang/cleanrltest.
• Easily customizable and being able to debug directly in Python’s interactive shell.
• Convenient use of commandline arguments for hyper-parameters tuning.

Currently I support A2C, PPO, and DQN. If you are interested, please consider giving it a try 🙂

Motivation:

There are two types of RL library on the two ends of the spectrum. The first one is the demo kind that really just demos what the algorithm is doing, only deals with one gym environment and hard to record experiments and tune parameters.

On the other end of the spectrum, we have OpenAI/baselines, ray-project/ray, and couple google repos. My personal experience with them is that I could only run benchmark with them. They try to write modular code and employ good software engineering practices, but the problem is python is a dynamic language without IDE support. As a result, I had no idea what variable types in different files are and it was very difficult to do any kind of customization. I had to see through dozens of files before even able to try some experiments.

That’s why I created this repo that leans towards the first kind, but has more actual experimental support. I support multiple gym spaces (still working on it), command line arguments to tune parameters, and very seamless experiment logging, all of which are essential characteristics for building a pipeline for research I believe.

I want to start building a foundation in machine learning to ultimately do research in the intersection of machine learning and economics/finance. Would The Elements of Statistical Learning (Hastie et. al. 2009) be the best place to start? I fear that the text may be outdated in 2019, but I’m coming to ML with only econometric/statistical knowledge.

Also, would An Introduction to Statistical Learning with Applications in R (Gareth et al 2017) be too basic for my goals? Are there better texts I should start with than Hastie et al (2009)?

Blog post on gradient.pub with quantitative extensive analysis of PyTorch vs TensorFlow adoption in research.

TensorFlow will always have a captive audience within Google/DeepMind, but I wonder whether Google will eventually relax this. Even now, many of the researchers that Google wants to recruit will already prefer PyTorch at varying levels, and I’ve heard grumblings that many researchers inside Google would like to use a framework other than TensorFlow.

In addition, PyTorch’s dominance might start to cut off Google researchers from the rest of the research community. Not only will they have a harder time building on top of outside research, outside researchers will also be less likely to build on top of code published by Google.

https://thegradient.pub/state-of-ml-frameworks-2019-pytorch-dominates-research-tensorflow-dominates-industry/

I am not able to implement the object detection algorithms on code, and need an example where the three main algos are applied on the same dataset, so I can compare their speed myself.

Any help with all this?

Detectron2: A PyTorch-based modular object detection library

• Blog post
• Github page

Improvements in Detectron2

PyTorch: The original Detectron was implemented in Caffe2. PyTorch provides a more intuitive imperative programming model that allows researchers and practitioners to iterate more rapidly on model design and experiments. Because we’ve rewritten Detectron2 from scratch in PyTorch, users can now benefit from PyTorch’s approach to deep learning as well as the large and active community that continually improves PyTorch

Modular, extensible design: In Detectron2, we’ve introduced a modular design that allows users to plug custom module implementations into almost any part of an object detection system. This means that many new research projects can be written in hundreds of lines of code with a clean separation between the core Detectron2 library and the novel research implementation. We continue to refine the modular, extensible design by implementing new models and discovering new ways in which we can make Detectron2 more flexible.

A month ago I wrote a post in this subreddit (here) about a voice conversion and audio style transfer system on unpaired data I had been working on. Many users were quite surprised by the results and recommended me to write a paper about it, despite my completely lack of knowledge in the academic world (which I was very afraid of).

Well, I wrote that paper: https://arxiv.org/abs/1910.03713 (Here the old demo video: https://youtu.be/3BN577LK62Y )

I have no idea if the final product is up to standard or if there are some major holes or mistakes in it (quite likely honestly): this is my first ever paper and I asked advice to multiple subject-related professors in my university but nobody offered to help.

I would love to know what I should now do: I know papers can be published in publications of various level, but the entire process is entirely obscure to me, and I am quite happy to just have it on arXiv.

Finally, I really need to thank all the people that told me that writing a paper was the right idea: I feel like I learned a lot in the process. So, thank you again!

I wanted to share with you this really new project I just stumbled upon from facebook AI research.

Implementing gradient-based hyper-parameter optimization and meta-learning has always been hard because of the non-differentiable optimizers and the stateful, non functional models. This library is supposed to make things easier by replacing existing stateful models with stateless ones automatically at run-time. It also implement differentiable version for most of the the torch.optim optimizers (although you cant use third party ones out of the box).

This means that we can finally differentiate through the usual training loop code with very little changes!

I didn’t try the library myself but it seems really easy to implement and from the stars looks really promising. Let me know what you think.

repo: https://github.com/facebookresearch/higher

Highlights

• [Experimental]: Mobile Support
• [Experimental]: Named Tensor Support
• [Experimental]: Quantization support
• Type Promotion

Full Release Notes