YouTube lecture here

As part of Fast.ai’s NLP course, Rachel Thomas has prepared an excellent overview of the existing cases of disinformation. She highlights several notable examples from this past year alone, and covers the potential misuses and abuses of AI and NLP techniques. These include prominent generative methods such as GPT-2 to make fake-but-plausible reddit comments, and StyleGAN portraits to fake social media profile images.

Hi all the MLers,

Usually, we could have labeled data with specific degree of QUALITY from Kaggle or from some workshops. However, if we are in real world usage, how could we ensure the quality of the labeled data(in addition to accuracy, a more crucial point is the reliability on it to train models)?

To be more specific, I am doing a project on Named Entity Recognition(to extract and label some product names from some formal documents describing some products and their relation). We now have a rule-based model to extract the NE by Regular Expression, and we now would like to do the task in a Machine Learning workflow.

So, here’s the question, if we directly take the data labeled by our rule-based model as the training data for our ML model, how could we first ensure the quality of this training data before we feed it into the model.

​

This is my first time to have to generate training data from scratch, so I really appreciate any discussion with you guys. Any ideas and comments are welcome! Thanks ALOT!

It seems there is some variability to diffusion appearance depending on the material/distance/light intensity/object being reflected. For instance if you hold your hand up to a white piece of paper with a bulb behind your hand, you can see your hands shape a few centimeters off the paper. Move your hand back and eventually it just becomes a large blurry dark spot exponentially fading (assuming the scale was kept constant).

I was thinking earlier about this, and we as humans really are only trained to see these pronounced diffused reflections, as anything else is cognitively expensive and probably wasteful. So once the hand no longer represents a hand to us on the white paper, it starts losing its reason to be seen more and more.

Do you think it is possible now, using gans/cnns, to train a model, to learn to see more than we can? Will we now, or ever, be able to decode the diffused reflections of whole/partial objects from vast distances using ML?

Hey guys, I was hoping for someone to help me, been stuck with this problem for a while now.

I have been trying to train DeepSpeech on a Spanish CommonVoice dataset.

However the script barely uses my gpu, if at all.

I seem to have everything installed by now except for the correct version of the CudNN.

I am using an anaconda environment and I installed everything easily, however anaconda doesn’t have the version 7.5.0 in its repository so I cannot install it through “conda install”. I have been searching how to install cudnn into the environment in another way but I haven’t been able to find anything.

Any help would be very much appreciated!!

I am using Ubuntu 18.04 LTS

For example, I am looking for cluster in a dataset of 10000 binary variables. I reduce the number to 50 variables with a PCA, then apply t-sne on it to find clusters in the output.

There are separated shapes in the visualisation of tsne, but I understood that points that are far apart in the output are not always far in the higher dimension space. T-sne can find clusters in a normally distributed dataset.

Can we to prove that data follows a normal distribution in all directions of space? Is there a way to remove variables that would add noise to clustering? Like some kind of variable selection but for clustering?

So, today my company decided to send me and a bunch of people onto one of those machine learning courses that are trendy these days. The subject was on time series prediction, and the structure followed everything I’ve seen in the past few years and that always leaves me baffled.

For every lag they want to use they create a new column with shifted data! To me this seems to be a dealbreaker for 2 reasons. First it effectively multiplies the memory usage by the length of the history we wish to consider. Second it means that we can’t keep history longer than the lags considered, as the internal state doesn’t transfer between lines of the data matrix (I think I remember keras having something to keep this state but it required lining examples between batches). I don’t see how this can possibly work when you have high resolution data and care about microstructure but also need to take into account longer term dynamics.

So, what am I missing here? When I’m working with things like exponentially weighted moving averages I can trivially create a statsmodels model to fit the decay rate and find the optimal history length. Is there really no straightforward way to use tensorflow/pytorch to look back into values earlier in the same data column and skip the whole data duplication issue? If not, is there a good reason for it?

Hi

I trained some Resnet models to estimate tags for anime images.

There has been a lot of cool anime-related projects recently, such as DeepDanbooru and some other cool work with anime face generation, however most use tensorflow and so I wanted a nice pretrained pytorch model to use for transfer learning with downstream tasks.

A TL;DR of the networks:

• Framework: Pytorch
• Network: adapted resnet50, resnet34, and resnet18 models
• Training time : about 12 days (using mixed-precision training and Nvidia V100)
• Dataset: Danbooru2018 dataset, with networks trained to predict top tags of each image

An example output of the resnet50 model

If you have pytorch, you can load up the model and predict on new images in only a few lines using pytorch hub. See this post for a quick getting started

I hope some people find this useful, and that it saves some people training time with other tasks.

For more info on the networks, training, data preparation, and performance metrics of each network, please have a look at the project page, or just ask :). Any comments or feedback is nice.

I recently read a new NLP paper called CNM: An Interpretable Complex-valued Network for Matching (https://arxiv.org/abs/1904.05298). In summary, the paper proposes a framework for NLP explainability using complex-valued vector spaces, borrowing math frameworks from quantum mechanics. It even won an award at this years NAACL. Reading the paper however, I couldn’t help but to feel a bit mislead. There seemed to be a stretching of the applications of quantum mechanics and there were a lot of ideas proposed in the paper which had little evidence to back them up. I was so bothered by it in fact that I wrote a blog post about it. There’s also a great paper I read that talks more about the state of research in the field: https://arxiv.org/abs/1807.03341.

​

I wanted to know people’s thoughts on the matter. Do you think we’re critical enough as a community? Do you feel because of the recent successes of machine learning/deep learning research, people maybe are hesitant to speak out or to ask more questions on why things work?

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using a neural network system that includes a batch normalization layer. One of the methods includes receiving a respective first layer output for each training example in the batch; computing a plurality of normalization statistics for the batch from the first layer outputs; normalizing each component of each first layer output using the normalization statistics to generate a respective normalized layer output for each training example in the batch; generating a respective batch normalization layer output for each of the training examples from the normalized layer outputs; and providing the batch normalization layer output as an input to the second neural network layer.

​

Link: https://patents.google.com/patent/US20160217368A1/en

Now in the latest version of torchbearer:

• A host of regularisers including: Mixup, CutOut, CutMix, Sample Pairing, Label Smoothing and Random Erase
• Built-in integration with PyCM for generating confusion matrices
• Built-in support for LiveLossPlot
• A set of example colab notebooks
• Bug fixes and more listed in the CHANGELOG: https://github.com/pytorchbearer/torchbearer/blob/master/CHANGELOG.md

​

Install: http://www.pytorchbearer.org/#install

Check out the examples at: http://www.pytorchbearer.org/#examples

Find us on GitHub: https://github.com/pytorchbearer/torchbearer

​

Please take a look and let us know what you think!