Looking at some recent papers on program synthesis

Neural (Meta) Program Synthesis, Singh {GB}

AlphaNPI twitted that acceptance to NIPS2019 with spotlight

I am wondering if field is still working out good architectures, representations, etc

OR

existing SOTA techniques can be applied if we have x100 more compute, or a massive dataset of input-output pairs, or maybe a long detailed curated curriculum of specs and solutions, etc

Abstract: When processing similar frames in succession, we can take advantage of the locality of the convolution operation to reevaluate only portions of the image that changed from the previous frame. By saving the output of a layer of convolutions and calculating the change from frame to frame, we can reuse previous activations and save computational resources that would otherwise be wasted recalculating convolutions whose outputs we have already observed. This technique can be applied to many domains, such as processing videos from stationary video cameras, studying the effects of occluding or distorting sections of images, applying convolution to multiple frames of audio or time series data, or playing Atari games. Furthermore, this technique can be applied to speed up both training and inference.

Summary of results: Reusing convolutional activations with CPUs is a good way to save computation for both training and inference, and can serve as a viable alternative to training or doing inference on GPUs in some scenarios. It is likely cheaper, sometimes faster, and it will likely have access to more memory. Unfortunately, there is currently not as much incentive to use this method on GPUs, other than possibly saving power. There are many possible application domains for this technique, and there are likely many ways to improve upon it.

Code and result figures: https://github.com/arnokha/reusing_convolutions

Paper link: [Submitted, coming soon]

Hi all!

I’m currently working on a research project, looking to profile some publicly traded ML companies.

Problem is, most ML companies get bought out by IBM, Microsoft, Google, or Amazon before going public.

I’ve got a small list of ones I’ve found, but if you know of any that are doing cool things, I’d love to hear about them!

Thanks in advance 🙂

Sharing with you a project we’ve been working on at Hugging Face: Write With Transformer. It is a web app that hosts most state-of-the-art transformer-based NLP generative models like GPT-2, GPT or XLNet.

You can write a context and trigger completions from the generative model you choose, in a Google Doc-like interface. It also includes one of our fine-tuned models, using GPT-2 as a pretrained model and fine-tuning it on Arxiv papers to get NLP/Deep Learning completions.

It’s built on top of our library pytorch-transformers. Let us know what you think!

This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial conditions. We also introduce a suite of eight tasks that combine these three properties, and show that R2D3 can solve several of the tasks where other state of the art methods (both with and without demonstrations) fail to see even a single successful trajectory after tens of billions of steps of exploration.

Blog post: https://deepmind.com/research/publications/Making-Efficient-Use-of-Demonstrations-to-Solve-Hard-Exploration-Problems

Paper: https://arxiv.org/pdf/1909.01387.pdf

Check out this newly issued patent (filed in Sept. 2014) that appears to shut down the entire discipline of ML as it applies to classifying different machine actions with AI.

https://patentimages.storage.googleapis.com/17/f1/6d/15a1e6f88983c7/US10032117.pdf

I’m paraphrasing / simplifying so it’s probably not completely accurate, but it looks like this tries to patent:

1. Receiving training data from channels on-board (not external to) a mobile machine;

2. Determining training features/data and corresponding labels from the training data;

3. Where the labels/data relate to different machine actions (as opposed to simply active or inactive) each occurring over different time periods;

4. Building the classifier by feeding training features/data and labels into a ML algorithm.

Surely this was known prior to the Sept. 2014 filing date of this patent. Is anyone else looking for a green light to continue to innovate in this space? Would be curious if anyone is aware of, or can find, an earlier public document that overlaps with the main claims of this patent. Thanks!! PS – yes, I’ve googled and found some relevant documents, but have not yet found a single document that expressly includes all 4 items listed above.

In Reinforcement Learning I have noticed a trend in some(1, 2) papers that involve optimizing surrogate clipped functions. Has anyone seen any work that digs deeper into the effects of this? For example, in this paper they dig deeper into the relationship between clipped surrogate functions and trust regions. The above references I gave were from clipped surrogate objectives, but this doesn’t have to be the case(ex: drop the max and only optimize the clipped objective).

Hi, I’m currently working with a dataset of .tiff files and want to feed those to a model using a tf Dataset for performance reasons. However, tf currently does not support loading .tiff files with Dataset.

Now I’m curious how to assess the loss in information if I convert a tiff to e.g. a png. Currently theses tiffs only hold a single image which for me should not make much a difference.

What would be a good approach to assess this?