What if someone wore a brain scanner / portable FMRI machine on their head, as well as a Google Glass type device with a camera, microphone, etc., and then just went about their lives for some significant amount of time. Then, since there would be so much labeled data with a correspondence from their brain imaging to the major sensory inputs they were experiencing at the time, you could create a model that could then reconstruct their real-time brain imaging into a video that others could watch. So, theoretically, you could do things like translate brain activity in someone’s dreams into a video, or otherwise read people’s minds.

Apologies if I’ve been spamming the subreddit with these threads.

When I originally posted my article comparing PyTorch’s and Tensorflow’s growth, some people questioned whether the results only extended to NLP conferences.

Judging from these results at NeurIPS, the answer is no.

Some people still have the narrative that dynamic graphs/PyTorch are only popular in the NLP community. It’s clear that that’s not the case.

https://twitter.com/cHHillee/status/1195568939381776386

PS: Reposted because I had incorrect numbers for TensorFlow originally.

For the past few months I’ve been working on a small introduction on how to hack with and into neural networks.

The entire course (Article + Exercises) can be found here:

https://github.com/Kayzaks/HackingNeuralNetworks

Abstract:

A large chunk of research on the security issues of neural networks is focused on adversarial attacks. However, there exists a vast sea of simpler attacks one can perform both against and with neural networks. In this article, we give a quick introduction on how deep learning in security works and explore the basic methods of exploitation, but also look at the offensive capabilities deep learning enabled tools provide. All presented attacks, such as backdooring, GPU-based buffer overflows or automated bug hunting, are accompanied by short open-source exercises for anyone to try out.

​

The course is more aimed towards Security Experts that want to learn about how they can use/misuse neural networks, rather than ML researchers.

I think the exercises are the best part of the project at the moment. The article itself is fine in my opinion, but the introduction to neural networks isn’t all that great (I’ve been thinking about taking it out completely).

​

Would love to hear what you guys think about it! Any feedback is greatly appreciated.

Hello everyone!

I’m just wondering whether someone had already tried to use a pre-trained classifier or other kind of neural network as part of or as a starting point for the discriminator of a GAN. I thought maybe it could help stabilize or speed up training, specially if you freeze the first layers (it’d be a mix of the perceptual loss and the adversarial loss I guess?)

Tried to search a little on Google Scholar or Arxiv and couldn’t find anything. Has this been done before? What do you think of this idea?

Hi, I made a ML library using only the c++17 standard library over the summer!

Take a look at it on product hunt: https://producthunt.com/posts/l2

And on GitHub: https://github.com/bkkaggle/L2

Let me know what you think

https://github.com/google/XNNPACK

Recently found out about this. It works well and is easy to integrate (unlike Arm ComputeLibrary).

Wondering when pytorch will switch to it because NNPACK is definitely slower.

So basically I have to build a neural network but instead of backpropagation , i should use any search algorithm ( hill climbing , simulated annealing) . I have no idea on how to implement that. Can you guys help me out. Any help is appreciated.

Hello everyone,

I recently trained a tiny bidirectional LSTM model to achieve high accuracy on Stanford’s SST-2 by using knowledge distillation and data augmentation. The accuracy is comparable to BERT after fine-tuning, but the model is small enough to run at hundreds of iterations per second on a laptop CPU core. I believe this approach could be very useful since most user-devices in the world are low-power.

I believe this can also give some insight into the success of huggingface’s DistilBERT, as it seems their success doesn’t stem solely from knowledge distillation but also from the Transformer’s unique architecture and the clever way they initialize its weights.

If you have any questions or insights, please share 🙂

For more details please take a look at the article:

https://blog.floydhub.com/knowledge-distillation/

Am new to ML, background in signal processing and traditional CS. Have been working on classifying raw audio from my custom created datasets and I want to move into prediction and then generation(GANs)

Is there any difference between GANs and prediction ? Like if I wanted to predict the next audio sample how would my current classification CNN change? My thinking is to be able to predict the next sentence from a recorded sentence.

Hey all,

I created a small python repository called Replicate TorontoBookCorpus that one can use to replicate the no-longer-available Toronto BookCorpus (TBC) dataset.

As I’m currently doing research on transformers for my thesis, but could not find/get a copy of the original TBC dataset by any means, my only alternative was to replicate it. I figured I am not the only one with this issue, and thus made and published this small project.

As with the original TBC dataset, it only contains English-language books with at least 20k words. Furthermore, the total number of words in the replica dataset is also slightly over 0.9B. All in all, if you follow the steps outlined in the repository, you end up with a 5Gb text file with one sentence per line (and three blank sentences between books).

PS. If you have a copy of the original TBC dataset, please get in touch with me (I am desperately looking for the original)!