This question isn’t just asking for neural network or RL related approaches, open to everything! Thanks!
submitted by /u/rulerofthehell
[link] [comments]
Hi,
Is there any app/software that analyses thousands of images and has an interface to browse the collection via content-based related images?
I’m trying to find a tutorial to achieve something like this, https://github.com/machinebox/visualsearch but locally and OpenSource
Goal: have a local “google-photos”, that self-organizes my photo archive and I can browse and I get “related suggestions” per each image.
submitted by /u/tanreb
[link] [comments]
Hi folks,
I recently worked on prediction of event streams to improve the customer experience on web sites. I compared simple LSTMs, GRUs and 1D convolution models and found that for my data convNets perform better, but also with a different characteristics.
You can find the article here.
Cheers
submitted by /u/ixeption
[link] [comments]
Original post here: https://www.reddit.com/r/MachineLearning/comments/bvwvoo/p_ai_against_humanity_play_cards_against_humanity/
Thanks for your positive feedback in my last post! 🙂 I kept working on the project, adding new features and cards, and collecting all user choices to train a simple AI you can play against!
Try it out: https://www.aiagainsthumanity.app/
I tried to be super frugal and everything works 100% out of your browser via a static website. A blog post detailing the process is in the works, but in short: I generated BERT embeddings for each of the cards, then applied PCA to reduce the dimension to something a browser / mobile device can easily handle. Then I trained a small neural network that takes a question-answer pair and outputs a probability. This then gets loaded in the browser via TensorFlow.js.
Please let me know what you think or if you run into any trouble. Ideas are welcome as well! And yes, I know multiplayer would be amazing, but then I’d have to set up a real server and – god forbid – actually pay hosting costs 😱
submitted by /u/cpury
[link] [comments]
Since it was introduced a few years ago, Google’s Transformer architecture has been applied to challenges ranging from generating fantasy fiction to writing musical harmonies. Importantly, the Transformer’s high performance has demonstrated that feed forward neural networks can be as effective as recurrent neural networks when applied to sequence tasks, such as language modeling and translation. While the Transformer and other feed forward models used for sequence problems are rising in popularity, their architectures are almost exclusively manually designed, in contrast to the computer vision domain where AutoML approaches have found state-of-the-art models that outperform those that are designed by hand. Naturally, we wondered if the application of AutoML in the sequence domain could be equally successful.
After conducting an evolution-based neural architecture search (NAS), using translation as a proxy for sequence tasks in general, we found the Evolved Transformer, a new Transformer architecture that demonstrates promising improvements on a variety of natural language processing (NLP) tasks. Not only does the Evolved Transformer achieve state-of-the-art translation results, but it also demonstrates improved performance on language modeling when compared to the original Transformer. We are releasing this new model as part of Tensor2Tensor, where it can be used for any sequence problem.
Developing the Techniques
To begin the evolutionary NAS, it was necessary for us to develop new techniques, due to the fact that the task used to evaluate the “fitness” of each architecture, WMT’14 English-German translation, is computationally expensive. This makes the searches more expensive than similar searches executed in the vision domain, which can leverage smaller datasets, like CIFAR-10. The first of these techniques is warm starting—seeding the initial evolution population with the Transformer architecture instead of random models. This helps ground the search in an area of the search space we know is strong, thereby allowing it to find better models faster.
The second technique is a new method we developed called Progressive Dynamic Hurdles (PDH), an algorithm that augments the evolutionary search to allocate more resources to the strongest candidates, in contrast to previous works, where each candidate model of the NAS is allocated the same amount of resources when it is being evaluated. PDH allows us to terminate the evaluation of a model early if it is flagrantly bad, allowing promising architectures to be awarded more resources.
The Evolved Transformer
Using these methods, we conducted a large-scale NAS on our translation task and discovered the Evolved Transformer (ET). Like most sequence to sequence (seq2seq) neural network architectures, it has an encoder that encodes the input sequence into embeddings and a decoder that uses those embeddings to construct an output sequence; in the case of translation, the input sequence is the sentence to be translated and the output sequence is the translation.
The most interesting feature of the Evolved Transformer is the convolutional layers at the bottom of both its encoder and decoder modules that were added in a similar branching pattern in both places (i.e. the inputs run through two separate convolutional layers before being added together).
 |
| A comparison between the Evolved Transformer and the original Transformer encoder architectures. Notice the branched convolution structure at the bottom of the module, which formed in both the encoder and decoder independently. See our paper for a description of the decoder. |
This is particularly interesting because the encoder and decoder architectures are not shared during the NAS, so this architecture was independently discovered as being useful in both the encoder and decoder, speaking to the strength of this design. Whereas the original Transformer relied solely on self-attention, the Evolved Transformer is a hybrid, leveraging the strengths of both self-attention and wide convolution.
Evaluation of the Evolved Transformer
To test the effectiveness of this new architecture, we first compared it to the original Transformer on the English-German translation task we used during the search. We found that the Evolved Transformer had better BLEU and perplexity performance at all parameter sizes, with the biggest gain at the size compatible with mobile devices (~7 million parameters), demonstrating an efficient use of parameters. At a larger size, the Evolved Transformer reaches state-of-the-art performance on WMT’ 14 En-De with a BLEU score of 29.8 and a SacreBLEU score of 29.2.
 |
| Comparison between the Evolved Transformer and the original Transformer on WMT’14 En-De at varying sizes. The biggest gains in performance occur at smaller sizes, while ET also shows strength at larger sizes, outperforming the largest Transformer with 37.6% less parameters (models to compare are circled in green). See Table 3 in our paper for the exact numbers. |
To test generalizability, we also compared ET to the Transformer on additional NLP tasks. First, we looked at translation using different language pairs, and found ET demonstrated improved performance, with margins similar to those seen on English-German; again, due to its efficient use of parameters, the biggest improvements were observed for medium sized models. We also compared the decoders of both models on language modeling using LM1B, and saw a performance improvement of nearly 2 perplexity.
Future Work
These results are the first step in exploring the application of architecture search to feed forward sequence models. The Evolved Transformer is being open sourced as part of Tensor2Tensor, where it can be used for any sequence problem. To promote reproducibility, we are also open sourcing the search space we used for our search and a Colab with an implementation of Progressive Dynamic Hurdles. We look forward to seeing what the research community does with the new model and hope that others are able to build off of these new search techniques!
![[Project] pyTsetlinMachine released. High-level Tsetlin Machine Python API with fast C-extensions.](https://b.thumbs.redditmedia.com/l2GNP2jpsTQXxjOxvOaoU5LlL1UpYymcwtj3FgRttYs.jpg "[Project] pyTsetlinMachine released. High-level Tsetlin Machine Python API with fast C-extensions.") |
I have made a Python library for the Tsetlin Machine. You can now set up, train and evaluate Tsetlin Machines in just three lines of code. I have used C extensions for speed, wrapped in Python. Currently, the Multi-class and Convolutional Tsetlin Machines are available. The Regression Tsetlin Machine follows soon. Will also add more demos and support functions (e.g. binarization). submitted by /u/olegranmo |
From voice assistants like Alexa and Google Maps navigation to Bing’s conversational search, AI has become a part of daily life for many.
These tasks are performing deep learning inference, which might be thought of as AI put into action.
The deep learning neural networks that power AI are trained on massive amounts of data. Putting this training to work in the digital world — to recognize spoken words, images or street signs, or to suggest the shirt you might want to buy or the next movie to watch — is inferencing.
And the breadth of inference applications on GPUs may surprise you. It’s pervasive in everything from the lumber industry to research that delves into reading ancient Japanese texts.
Below are four diverse ways inference running on GPUs is already making a difference.
PayPal is using deep learning inference on GPUs to pinpoint fraudulent transactions — and help ensure they don’t happen again.
The company processes millions of transactions every day. Advances in AI — specifically logistic regression-powered neural network models — have allowed it to filter out deceptive merchants and crack down on sales of illegal products.
The deep learning models also help PayPal optimize its operations by identifying why some transactions fail and spotting opportunities to work more efficiently.
And since the models are always learning, they can personalize user experiences by serving up relevant advertisements based on people’s interests.
Boston-based ClimaCell is working to bring unprecedented speed, precision and accuracy to weather forecasting by listening closely to a powerful voice: Mother Nature’s.
The company uses inference on GPUs to offer so-called “nowcasting” — hyper-local, high-resolution forecasts that can help businesses and people make better decisions about everything from paving projects to wind generation to planning a daily commute to avoid bad weather. The company also offers forecasting and historic data.
To achieve this, the company writes software that turns the signals in existing communication networks into sensors that can analyze the surrounding environment and extract real-time weather data.
ClimaCell’s network quickly analyzes the signals, integrates them with data from the National Oceanic and Atmospheric Administration and then weaves it all together using predictive models run on NVIDIA GPU accelerators.
Mammogram machines are effective at detecting breast cancer, but expensive. In many developing countries, this makes them rare outside of large cities.
Mayo Clinic researcher Viksit Kumar is leading an effort to use GPU-powered inferencing to more accurately classify breast cancer images using ultrasound machines, which are much cheaper and more accessible around the world.
Kumar and his team have been able to detect and segment breast cancer masses with very good accuracy and few false positives, according to their research paper.
The team does its local processing using the TensorFlow deep learning framework container from the NGC registry on NVIDIA GPUs. It also uses NVIDIA V100 Tensor Core GPUs on AWS using the same container.
Eventually, Kumar hopes to use ultrasound images for the early detection of other forms of the disease, such as thyroid and ovarian cancer.
MuseNet is a deep learning algorithm demo from AI research organization OpenAI that automatically generates music using 10 kinds of instruments and a host of different styles — everything from pop to classical.
People can create entirely new tracks by applying different instruments and sounds to music the algorithm generates. The demo uses NVIDIA V100 Tensor Core GPUs for this inferencing task.
Using the demo, you can take spin up twists on your favorite songs. Add guitars, leave out the piano, go big on drums. Or change its style to sound like jazz or classic rock.
The algorithm wasn’t programmed to mimic the human understanding of music. Instead, it was trained on hundreds of thousands of songs so it could learn the patterns of harmony, rhythm and style prevalent within music.
Its 72-layer network was trained using NVIDIA V100 Tensor Core GPUs with the cuDNN-accelerated TensorFlow deep learning framework.
Read more stories about deep learning inferencing.
The post Four Surprising Ways Inference Is Putting AI into Action appeared first on The Official NVIDIA Blog.
So I understand that you don’t need to scale features for a decision tree, since it will just find the right place to split on anyway. Do outliers need to be handled though? Does it matter if I have some features that have a way higher variance within it compared to others? Lastly, for categorical variables, I have my observations labeled in one of 5 labels (this is a feature, not the target). Will it adversely affect the results if the labels are imbalanced? I have one label that makes up roughly 60% of all the observations. Should I try to relabel things so it’s a little more balanced (I can collapse some of the other labels) Btw these questions are for either classification or regression trees.
submitted by /u/mydogissnoring
[link] [comments]
Hello,
I hope to clear a few doubts regarding Variational Autoencoders.
https://papers.nips.cc/paper/5352-semi-supervised-learning-with-deep-generative-models.pdf
According to the above paper, the representations learned by VAEs are used for semi-supervised learning. However, in my experiments I was getting better accuracy results with regular autoencoders than VAEs. I do know that we can generate new samples using a VAE but is there a reason why VAEs are used in the paper instead of regular autoencoders? What can be the advantages of the representations of VAE compared to that of an autoencoder w.r.t. semi-supervised learning?
Any help is appreciated?
submitted by /u/question_hour
[link] [comments]
Hi all, this is my first post here so I hope I do this right…
I’m currently trying to get a CNN to learn the rotation angle of 2D images. I tried to make a post on stackoverflow first, because I didn’t know if the question fits into this subreddit, but so far that thread didn’t gain any traction… see here.
I hope it’s ok when I just quote the question from there:
I am trying to build a CNN (in Keras) that can estimate the rotation of an image (or a 2d object). So basically, the input is an image and the output should be its rotation.
My first experiment is to estimate the rotation of MŃIST digits (starting with only one digit “class”, let’s say the “3”). So what I did was extracting all 3s from the MNIST set, and then building a “rotated 3s” dataset, by randomly rotating these images multiple times, and storing the rotated images together with their rotation angles as ground truth labels.
So my first problem was that a 2d rotation is cyclic and I didn’t know how to model this behavior. Therefore, I encoded the angle as y=sin(ang), x = cos(ang). This gives me my dataset (the rotated 3s images) and the corresponding labels (x and y values).
For the CNN, as a start, i just took the keras MNIST CNN example (https://keras.io/examples/mnist_cnn/) and replaced the last dense layer (that had 10 outputs and a softmax activation) with a dense layer that has 2 outputs (x and y) and a tanh activation (since y=sin(ang), x = cos(ang) are within [-1,1]).
The last thing i had to decide was the loss function, where i basically want to have a distance measurement for angles. Therefore i thought “cosine_proximity” is the way to go.
When training the network I can see that the loss is decreasing and converging to a certain point. However when I then check the predictions vs the ground truth I observe a (for me) fairly surprising behavior. Almost all x and y predictions tend towards 0 or +/-1. And since the “decoding” of my rotation is ang=atan2(y,x) the predictions are usually either +/- 0°, 45°, 90, 135° or 180°. However, my training and test data has only angles of 0°, 20°, 40°, … 360°. This doesn’t really change if I change the complexity of the network. I also played around with the optimizer parameters without any success.
Is there anything wrong with the assumptions:
x,y encoding for angle
tanh activation to have values in [-1,1]
cosine_proximity as loss function
Thanks in advance for any advice, tips or pointing me towards a possible mistake i made!
If this is the wrong place for this question I’m sorry and would be happy if someone could point me to the right forum subreddit!
submitted by /u/the_best_broccoli
[link] [comments]
