I was watching this video, and I actually love the concept of the NEAT algorithm for learning how to achieve goals through neural network evolution. My question is; If a network was trained on a particular level, would it be able to apply those same techniques to beat other levels, or is it just extremely fine-tuned to beat that one level?

Thinking about playing SMW when I was growing up, I died a lot, but after a while I got good enough at the game that I could beat levels I’d never played before in my first attempt. This means I would have built up the experience necessary to deal with unknown game mechanics, rapidly be able to learn how they work, and utilize them to achieve the end goal. Would networks trained via the NEAT algorithm be able to do the same thing? My initial thoughts are: no way…

I’ve been slowly building a collection of pure-NumPy (and a little SciPy) implementations of various ML models + building blocks to use for quick reference. The project has mostly been a fun thing for me to do in my spare time (hence the strange collection of models), though I hope it might also be useful for others interested in bare-bones implementations of particular models / ideas.

https://github.com/ddbourgin/numpy-ml

I’m sure there’s a ton that can be improved / made clearer. Alternatively, if you have models of your own that would be a good fit, PRs are welcome 🙂

Hi, I’m Myeongjun Kim. My major is Computer Vision using Deep Learning. “Stand-Alone Self-Attention in Vision Models” paper published on 13 Jun 2019. This paper was presented by the Google Research Brain Team. I implemented this paper in pytorch. Currently I am experimenting with a CIFAR-10 dataset. (training 30 epoch, Accuracy: 88.4%) The experiment is still in progress. I could see that the learning was very good. Many researches seem to use the Attention Module.

There are some parts that are not currently implemented, and some parts have not yet been experimented. I would appreciate a lot of feedback.

​

[Github URL: https://github.com/leaderj1001/Stand-Alone-Self-Attention]

Thank you so much 🙂

Active Annotation: bootstrapping annotation lexicon and guidelines for supervised NLU learning

​

We present a data annotation paradigm (Active Annotation), which is designed to aid human annotators by means of unsupervised learning. The idea is to set up an iterative process in which instances to be human-labelled are first selected, clustered and automatically labelled, and then passed to the annotator for the final validation of the proposed label or the assignment of a new label. The approach is integrated in a Web tool providing a user interface designed to be easy to use to maximize annotators’ productivity. The approach is evaluated in a natural language understanding scenario, in which annotators had to label with intent information a dataset of booking conversations. In this scenario, active annotation is compared against a baseline approach in which data are annotated instance-by-instance with a “human-only driven” method (in which annotators have to decide, sentence by sentence, whether to validate, replace or skip an automatically produced label). The reported results indicate the effectiveness of active annotation. First, in separate sessions with the same duration, humans were able to annotate a much larger set of instances compared to the baseline approach. Second, systems trained with data annotated with the proposed active annotation paradigm achieve better performance compared to systems trained with data annotated with the baseline approach.

​

— Abstract —

Natural Language Understanding (NLU) models are typically trained in a supervised learning framework. In the case of in-tent classification, the predicted labels are predefined and based on the designed annotation schema while the labeling process is based on a laborious task where annotators manually inspect each utterance and assign the corresponding label. We propose an Active Annotation (AA) approach where we combine an un-supervised learning method in the embedding space, a human-in-the-loop verification process, and linguistic insights to create lexicons that can be open categories and adapted over time. In particular, annotators define the y-label space on-the-fly during the annotation using an iterative process and without the need for prior knowledge about the input data. We evaluate the proposed annotation paradigm in a real use-case NLU scenario.Results show that our Active Annotation paradigm achieves ac-curate and higher quality training data, with an annotation speed of an order of magnitude higher with respect to the traditional human-only driven baseline annotation methodology.

​

— Paper Link —

https://www.academia.edu/39709550/Active_Annotation_bootstrapping_annotation_lexicon_and_guidelines_for_supervised_NLU_learning

​

Feel free to ask

So over the next 10 weeks or so, I’m doing a project on music genre classification for my MSc (it’s a conversion course as I did Philosophy at undergrad). I’m going to be using Python to classify songs based on their genre. My supervisor has suggested two possible project ideas:

1. Compare using SVMs with manual feature engineering (extracting features such as zero crossing rate, MFCCs, etc., from audio files), VS using a deep (probably convolutional) neural network that potentially uses some kind of automated feature learning (not sure how to do this), to see which approach is better.

2. Use a recurrent neural network for the classification, within the ‘reservoir computing’ paradigm. This is something I know much less about, but appanently is relatively new technology, so might be able to do something more novel with this project.

I’m going to mainly be using audio files themselves, but may also be using album artwork as a second form of input if I have time, as some researchers have actually managed to classify albums based on album artwork and use that as an additional set of features that can allow for more accurate classification of an album. I’m aware that music genre classification has been done quite a lot before, which is partly why I thought of adding in the album cover classification as an additional part of the project (as this hasn’t been done as much, so I’d potentially get more marks for originality).

My main question with the first project idea is that I don’t really know how to turn it into a sophisticated project. I have a small dataset of 1000 songs and pre-extracted features (extracted using Librosa) that I used a SVM on (scikit learn) to classify the songs, but the accuracy was only about 0.65. More fundamentally, after the data preprocessing, I only used a few lines of code overall, so to me it just felt really basic. I’ve never written a report like this before so I honestly don’t know how to turn it into something more substantial (or how to make the SVM more accurate). Is part of the low accuracy here potentially down to the dataset being too small?

I also don’t really know how I’d make the neural network part of the project more substantial. Would this involve lots of trial and error – tweaking the hyperparameters, number of layers, activation functions, etc., to get the best result possible?

For the second project idea, I know less about RNNs and even less about reservoir computing, so I don’t know if I’ll have time to learn enough to make it into a substantial, worthwhile project. Does this one have potential?

I would like to get a distinction for the project if I can, so any feedback on my ideas/answers to my questions would be really appreciated. Thanks.

https://youtu.be/zr9fFSYIVYI

Remove your phone from the mirror selfie using deep learning.