Trained on CONLL English Dependency, https://github.com/UniversalDependencies/UD_English-EWT. Train set to train, dev and test sets to test.

Stackpointer and Biaffine-attention originally from https://github.com/XuezheMax/NeuroNLP2 written in Pytorch.

Accuracy based on arc, types and root accuracies after 15 epochs only.

1. Bidirectional RNN + CRF + Biaffine, arc accuracy 70.48%, types accuracy 65.18%, root accuracy 66.4%
2. Bidirectional RNN + Bahdanau + CRF + Biaffine, arc accuracy 70.82%, types accuracy 65.33%, root accuracy 66.77%
3. Bidirectional RNN + Luong + CRF + Biaffine, arc accuracy 71.22%, types accuracy 65.73%, root accuracy 67.23%
4. BERT Base + CRF + Biaffine, arc accuracy 64.30%, types accuracy 62.89%, root accuracy 74.19%
5. Bidirectional RNN + Biaffine Attention + Cross Entropy, arc accuracy 72.42%, types accuracy 63.53%, root accuracy 68.51%
6. BERT Base + Biaffine Attention + Cross Entropy, arc accuracy 72.85%, types accuracy 67.11%, root accuracy 73.93%
7. Bidirectional RNN + Stackpointer, arc accuracy 61.88%, types accuracy 48.20%, root accuracy 89.39%

Link to repository, https://github.com/huseinzol05/NLP-Models-Tensorflow#dependency-parser

Discussion

1. Based on 15 epochs only.
2. No dropout here, feel free to do it.
3. BERT cannot implemented in Stackpointer model, stack pointer model required each decoder step.

When you start to humanize (current) AI technologies and describe them in ways you would talk about persons, you can draw all the wrong conclusions. This happens often when we see AI algorithms perform tasks that were previously thought to be off-limits for computers, such as playing Go or detecting cancer or converting text to speech.

Without a reality check on the capabilities and limits of current AI, we tend to have trumped up expectations of what AI can do for us, and become disenchanted when those expectations aren’t met.

https://bdtechtalks.com/2019/01/02/humanizing-ai-deep-learning-alphazero/

Hello everyone,

We have recently made available both the code and paper for our project on visual explanations for (spatio-temporal) 3D-CNNs named Class Feature Pyramids for Video Explanation. Through our method we aim at highlighting the specific time and space locations in videos that different layers and neurons in the network consider informative given a specific class by the discovery of a hierarchical feature association.

ArXiv preprint link Github repo link

This work will also be presented in the ICCV 2019 Workshop on Interpreting and Explaining Visual Artificial Intelligence Models

Abstract

Deep convolutional networks are widely used in video action recognition. 3D convolutions are one prominent approach to deal with the additional time dimension. While 3D convolutions typically lead to higher accuracies, the inner workings of the trained models are more difficult to interpret. We focus on creating human-understandable visual explanations that represent the hierarchical parts of spatio-temporal networks. We introduce Class Feature Pyramids, a method that traverses the entire network structure and incrementally discovers kernels at different network depths that are informative for a specific class. Our method does not depend on the network’s architecture or the type of 3D convolutions, supporting grouped and depth-wise convolutions, convolutions in fibers, and convolutions in branches. We demonstrate the method on six state-of-the-art 3D convolution neural networks (CNNs) on three action recognition (Kinetics-400, UCF-101, and HMDB-51) and two egocentric action recognition datasets (EPIC-Kitchens and EGTEA Gaze+).

After NeurIPS 2018 and the “Neural Ordinary Differential Equations” paper deep learning research has opened up tremendously in this direction and as a result it has been rather difficult to keep up with the latest advancements. To this end I have been collecting relevant papers in the following github repository. Feel free to contribute or suggest changes if you feel something is missing. Hope this is useful to those of you working in this area!

http://rail.eecs.berkeley.edu/deeprlcourse/

Lectures are recorded and live streamed

Material which will be covered:

1. From supervised learning to decision making

2. Model-free algorithms: Q-learning, policy gradients, actor-critic

3. Advanced model learning and prediction

4. Transfer and multi-task learning, meta-learning

5. Exploration

6. Open problems, research talks, invited lectures

There’s a subreddit for this course: r/berkeleydeeprlcourse

What are the best venues to go to (tutorials, conference, workshops, workshop dinner, expo, conference reception)?

Destin posted an video on using Machine Learning to analyze (surveillance) video for guns being held by a person.

Very interesting discussion on training the system, and the challenge of reducing false-positives as people hold cellphones in a similar way to how someone would hold a gun. Also on privacy, regarding the decision to not look for concealed carry.

We built a gun detector using machine learning that works with existing surveillance cameras.

There are many companies working on this type of technology at the moment. We made ours in 2018, and Chad visited my house in January 2019 to film most of this. We didn’t post it for various reasons, but decided now might be a good time. The fidelity of the inference is a function of how you train it…. so obviously the more data points you provide, the better it will perform. This particular type of machine learning is known as object detection, with the computation being performed by a deep neural network “on the edge”. The video stream is processed locally and no internet is needed to find the object in real time. If you’re working on something similar or could use this technology in your systems and would like to collaborate…. (or would simply like to support what we’re doing) please reach out by using this link: https://www.lantern.systems/gun-detector

https://www.youtube.com/watch?v=Lh0x54GC1sw

We try improvements to the existing ‘Hypergradient’ based optimizers proposed in the paper Online Learning Rate Adaptation with Hypergradient Descent.

We expect that the hypergradient based learning rate update could be more accurate and aim to exploit the gains much better by boosting the learning rate updates with momentum and adaptive gradients, experimenting with

1. Hypergradient descent with momentum, and
2. Adam with Hypergradient,

alongside the model optimizers SGD, SGD with Nesterov(SGDN) and Adam.

The new optimizers are compared with their resepective hypergradient-descent baselines and provide advantages such as better generalization and faster convergence for the loss function. The code and the results of our experiments are available at https://github.com/harshalmittal4/Hypergradient_variants.

The notebooks can be found here: https://github.com/quickcoffee/TDF-Starters-RUL

Let me know what you think, it’s my first project with web scraping and neural networks 🙂