Skip to main content

Blog

Learn About Our Meetup

5000+ Members

GO >

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOIN

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

JOBS

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

CONTACT

Author: torontoai

Speed up training on Amazon SageMaker using Amazon FSx for Lustre and Amazon EFS file systems

Written on . Posted in Amazon.

Amazon SageMaker provides a fully-managed service for data science and machine learning workflows. One of the most important capabilities of Amazon SageMaker is its ability to run fully-managed training jobs to train machine learning models. Visit the service console to train machine learning models yourself on Amazon SageMaker.

Now you can speed up your training job runs by training machine learning models from data stored in Amazon FSx for Lustre or Amazon Elastic File System (EFS). Amazon FSx for Lustre provides a high performance file system optimized for workloads, such as machine learning, analytics and high performance computing. Amazon EFS provides a simple, scalable, elastic file system for Linux-based workloads for use with AWS Cloud services and on-premises resources.

Training machine learning models requires providing the training datasets to the training job. Until now, when using Amazon S3 as the training data source in File input mode, all training data had to be downloaded from Amazon S3 to the EBS volumes attached to the training instances at the start of the training job. A distributed file system such as Amazon FSx for Lustre or EFS can speed up machine learning training by eliminating the need for this download step.

In this blog, we will go over the benefits of training your models using a file system, provide information to help you choose a file system, and show you how to get started.

Choosing a file system for training models on SageMaker

When considering whether you should train your machine learning models from a file system the first thing to consider is: where does your training data reside now?

If your training data is already in Amazon S3 and your needs do not dictate a faster training time for your training jobs, you can get started with Amazon SageMaker with no need for data movement. However, if you need faster startup and training times we recommend that you take advantage of Amazon SageMaker’s integration with Amazon FSx for Lustre file system, which can speed up your training jobs by serving as a high-speed cache.

The first time you run a training job, if Amazon FSx for Lustre is linked to Amazon S3, it automatically loads data from Amazon S3 and makes it available to Amazon SageMaker at hundreds of gigabytes per second and submillisecond latencies. Additionally, subsequent iterations of your training job will have instant access to the data in Amazon FSx. Because of this, Amazon FSx has the most benefit to training jobs that have several iterations requiring multiple downloads from Amazon S3, or in workflows where training jobs must be run several times using different training algorithms or parameters to see which gives the best result.

If your training data is already in an Amazon EFS file system, we recommend choosing Amazon EFS as the file system data source. This choice has the benefit of directly launching your training jobs from the data in Amazon EFS with no data movement required, resulting in faster training start times. This is often the case in environments where data scientists have home directories in Amazon EFS, and are quickly iterating on their models by bringing in new data, sharing data with colleagues, and experimenting with which fields or labels to include. For example, a data scientist can use a Jupyter notebook to do initial cleansing on a training set, launch a training job from Amazon SageMaker, then use their notebook to drop a column and re-launch the training job, comparing the resulting models to see which works better.

Getting started with Amazon FSx for training on Amazon SageMaker

  1. Note your training data Amazon S3 bucket and path.
  2. Launch an Amazon FSx file system with the desired size and throughput, and reference the training data Amazon S3 bucket and path. Once created, note your file system id.
  3. Now, go to the Amazon SageMaker console and open the Training jobs page to create the training job, associate VPC subnets, security groups, and provide the file system as the data source for training.
  4. Create your training job:
    1. Provide the ARN for the IAM role with the required access control and permissions policy. Refer to AmazonSageMakerFullAccess for details.
    2. Specify a VPC that your training jobs and file system have access to. Also, verify that your security groups allow Lustre traffic over port 988 to control access to the training dataset stored in the file system. For more details, refer to Getting started with Amazon FSx.
    3. Choose file system as the data source and properly reference your file system id, path, and format.
  5. Launch your training job.

Getting started with Amazon EFS for training on Amazon SageMaker

  1. Put your training data in its own directory in Amazon EFS.
  2. Now go to the Amazon SageMaker console and open the Training jobs page to create the training job, associate VPC subnets, security groups, and provide the file system as the data source for training.
  3. Create your training job:
    1. Provide the IAM role ARN for the IAM role with the required access control and permissions policy
    2. Specify a VPC that your training jobs and file system have access to. Also, verify that your security groups allow NFS traffic over port 2049 to control access to the training dataset stored in the file system.
    3. Choose file system as the data source and properly reference your file system id, path, and format.
  4. Launch your training job.

After your training job completes, you can view the status history of the training job to observe the faster download time when using a file system data source.

Summary

With the addition of Amazon EFS and Amazon FSx for Lustre as data sources for training machine learning models in Amazon SageMaker, you now have greater flexibility to choose a data source that is suited to your use case. In this blog post, we used a file system data source to train machine learning models, resulting in faster training start times by eliminating the data download step.

Go here to start training machine learning models yourself on Amazon SageMaker or refer to our sample notebook to train a liner learner model using a file system data source to learn more.

 

About the Authors

Vidhi Kastuar is a Sr. Product Manager for Amazon SageMaker, focusing on making machine learning and artificial intelligence simple, easy to use and scalable for all users and businesses. Prior to AWS, Vidhi was Director of Product Management at Veritas Technologies. For fun outside work, Vidhi loves to sketch and paint, work as a career coach, and spend time with his family and friends.

 

 

Will Ochandarena is a Principal Product Manager on the Amazon Elastic File System team, focusing on helping customers use EFS to modernize their application architectures. Prior to AWS, Will was Senior Director of Product Management at MapR.

 

 

 

 

[D] Is there an accepted state-of-the-art for Video Action Localisation/Region Suggestion?

Written on . Posted in Reddit MachineLearning.

I am working on a project which involves video action recognition, and am using the fantastic I3D approach.

However, I am now interested in localising the region of video which contains the action being classified (i.e. with a bounding box). For example, if my I3D network classifies a segment of video containing the action of a human “Eating an Apple”, I now want to draw a bounding box over the person eating the apple in each frame of video. Note that I am not interested in drawing a region around “Apple” or “Human”, but instead the region in which the action itself is being performed.

I am familiar with similar approaches for image classification (e.g. YOLO), but am having trouble finding work in the video domain for actions. Can anyone point me to some good papers which cover this if they exist?

submitted by /u/lantern_lol
[link] [comments]

[D] How to create Experience Replay table?

Written on . Posted in Reddit MachineLearning.

Hello I want to create a DQN for a 2 player strategic board game. I read that experience replay is recommended to break correlation. I want to know how I create the experience replay table?

Can I build it using random agent?(Playing random moves). After I have built the table I want to proceed like that: I want to randomly take one state out of experience replay table, take an action according to epsilon greedy policy. Put in the next state into a second neural network and backpropagate the difference Q values(From second neural network and first) in the first neuronal network. After that I randomly choose another state and repeat the step from above. Is this correct? Thank you

submitted by /u/Kralex68
[link] [comments]

Keeping an AI on Damage: Startup Automates Vehicle Condition Inspections

Written on . Posted in NVIDIA.

Anyone familiar with a fender bender knows that the rigmarole of getting a damage estimate is ready for the wrecking yard.

A startup wants to change that.

Ravin is using AI to help automate the process of vehicle inspections and reduce headaches for car rental agencies, car dealers, insurers and all of these companies’ customers.

Based in Haifa, Israel, and London, Ravin applies computer vision and AI to vehicle damage detection and assessment. It has obvious applications such as rental car pickups and returns.

With its app, people can circle a vehicle to take a video and be done. AI does the rest: It calculates the cost of repairs based on the vehicle and damages it identifies.

Ravin co-founders Eliron Ekstein and Roman Sandler formed the company to harness AI to alleviate the problems of tracking damages for car rental businesses and dealerships.

The startup came about as a digital business unit spinout of oil giant Shell. Earlier this year, the founders secured $4 million in seed funding led by Pico Venture Partners and with participation from the Dutch energy giant.

Automated Inspections

Today’s advanced vehicle damage assessment can require a person to take photos up close from multiple angles and another person at an office to assess the damage from the pictures.

There’s an easier way, says Ekstein, the company’s CEO. “We ask the user to walk around the car with a mobile phone (running a video app), or drive through a set of CCTV cameras. We pick up the damages ourselves automatically, and we classify and estimate it for the insurer or fleet owner to make a decision,” he said.

Ravin’s app enables a quick video to be converted into dozens of images around a car and then run through its algorithms in AWS powered by NVIDIA GPUs for damage detection, he said.

The startup’s neural networks were trained on NVIDIA GPUs running in workstations. “We used hundreds of thousands of images to train the models,” said Sandler, the company’s CTO.

Rental Applications

For fleet operators, Ravin can set up fixed camera systems to capture photos of inbound vehicles, automating the process of damage assessments.

Rental company Avis has been using Ravin’s system at Heathrow Airport in London.

“They use it to inspect cars for damage when customers come back, which helps them charge only the right people for the right damage,” said Ekstein.

Ravin’s system can help fleet operators manage their risk on damaged vehicles as well as more quickly move cars to repairs for a faster turnaround to get them back in use, he said.

“We can shorten the lead time to repair and help understand the true cost to repair it,” said Ekstein.

The post Keeping an AI on Damage: Startup Automates Vehicle Condition Inspections appeared first on The Official NVIDIA Blog.

Exploring Weight Agnostic Neural Networks

Written on . Posted in Google.

Posted by Adam Gaier, Student Researcher and David Ha, Staff Research Scientist, Google Research, Tokyo

When training a neural network to accomplish a given task, be it image classification or reinforcement learning, one typically refines a set of weights associated with each connection within the network. Another approach to creating successful neural networks that has shown substantial progress is neural architecture search, which constructs neural network architectures out of hand-engineered components such as convolutional network components or transformer blocks. It has been shown that neural network architectures built with these components, such as deep convolutional networks, have strong inductive biases for image processing tasks, and can even perform them when their weights are randomly initialized. While neural architecture search produces new ways of arranging hand-engineered components with known inductive biases for the task domain at hand, there has been little progress in the automated discovery of new neural network architectures with such inductive biases, for various task domains.

We can look at analogies to these useful components in examples of nature vs. nurture. Just as certain precocial species in biology—who possess anti-predator behaviors from the moment of birth—can perform complex motor and sensory tasks without learning, perhaps we can construct network architectures that can perform well without training. Of course, these natural (and by analogy, artificial) neural networks are further improved through training, but their ability to perform even without learning shows that they contain biases that make them well-suited to their task.

In “Weight Agnostic Neural Networks” (WANN), we present a first step toward searching specifically for networks with these biases: neural net architectures that can already perform various tasks, even when they use a random shared weight. Our motivation in this work is to question to what extent neural network architectures alone, without learning any weight parameters, can encode solutions for a given task. By exploring such neural network architectures, we present agents that can already perform well in their environment without the need to learn weight parameters. Furthermore, in order to spur progress in this field community, we have also open-sourced the code to reproduce our WANN experiments for the broader research community.

Left: A hand-engineered, fully-connected deep neural network with 2760 weight connections. Using a learning algorithm, we can solve for the set of 2760 weight parameters so that this network can perform the BipedalWalker-v2 task. Right: A weight agnostic neural network architecture with 44 connections that can perform the same Bipedal Walker task. Unlike the fully-connected network, this WANN can still perform the task without the need to train the weight parameters of each connection. In fact, to simplify the training, the WANN is designed to perform when the values of each weight connection are identical, or shared, and it will even function if this shared weight parameter is randomly sampled.

Finding WANNs
We start with a population of minimal neural network architecture candidates, each with very few connections only, and use a well-established topology search algorithm (NEAT), to evolve the architectures by adding single connections and single nodes one by one. The key idea behind WANNs is to search for architectures by de-emphasizing weights. Unlike traditional neural architecture search methods, where all of the weight parameters of new architectures need to be trained using a learning algorithm, we take a simpler and more efficient approach. Here, during the search, all candidate architectures are first assigned a single shared weight value at each iteration, and then optimized to perform well over a wide range of shared weight values.

Operators for searching the space of network topologies
Left: A minimal network topology, with input and outputs only partially connected.
Middle: Networks are altered in one of three ways:
(1) Insert Node: a new node is inserted by splitting an existing connection.
(2) Add Connection: a new connection is added by connecting two previously unconnected nodes.
(3) Change Activation: the activation function of a hidden node is reassigned.
Right: Possible activation functions (linear, step, sin, cosine, Gaussian, tanh, sigmoid, inverse, absolute value, ReLU)

In addition to exploring a range of weight agnostic neural networks, it is important to also look for network architectures that are only as complex as they need to be. We accomplish this by optimizing for both the performance of the networks and their complexity simultaneously, using techniques drawn from multi-objective optimization.

Overview of Weight Agnostic Neural Network Search and corresponding operators for searching the space of network topologies.

Training WANN Architectures
Unlike traditional networks, we can easily train the WANN by simply finding the best single shared weight parameter that maximizes its performance. In the example below, we see that our architecture works (to some extent) for a swing-up cartpole task using constant weights:
A WANN performing a Cartpole Swing-up task at various different weight parameters, and also using fine-tuned weight parameters.

As we see in the above figure, while WANNs can perform its task using range of shared weight parameters, the performance is still not comparable to a network that learns weights for each individual connection, as normally done in network training. If we want to further improve its performance, we can use the WANN architecture, and the best shared weight as a starting point to fine-tune the weights of each individual connection using a learning algorithm, like how we would normally train any neural network. Using the weight agnostic property of the network architecture as a starting point, and fine-tuning its performance via learning, may help provide insightful analogies to how animals learn.
Through the use of multi-objective optimization for both performance and network simplicity, our method found a simple WANN for a Car Racing from pixels task that works well without explicitly training for the weights of the network.

The ability for a network architecture to function using only random weights offers other advantages too. For instance, by using copies of the same WANN architecture, but where each copy of the WANN is assigned a different distinct weight value, we can create an ensemble of multiple distinct models for the same task. This ensemble generally achieves better performance than a single model. We illustrate this with an example of an MNIST classifier evolved to work with random weights:

An MNIST classifier evolved to work with random weights.

While a conventional network with random initialization will achieve ~10% accuracy on MNIST, this particular network architecture uses random weights and when applied to MNIST achieves an accuracy much better than chance (> 80%). When an ensemble of WANNs is used, each of which assigned with a different shared weight, the accuracy increases to > 90%.

Even without ensemble methods, collapsing the number of weight values in a network to one allows the network to be rapidly tuned. The ability to quickly fine-tune weights might be useful in continual lifelong learning, where agents acquire, adapt, and transfer skills throughout their lifespan. This makes WANNs particularly well positioned to exploit the Baldwin effect, the evolutionary pressure that rewards individuals predisposed to learn useful behaviors, without being trapped in the computationally expensive trap of ‘learning to learn’.

Conclusion
We hope that this work can serve as a stepping stone to help discover novel fundamental neural network components such as the convolutional network, whose discovery and application have been instrumental to the incredible progress made in deep learning. The computational resources available to the research community have grown significantly since the time convolutional neural networks were discovered. If we are devoting such resources to automated discovery and hope to achieve more than incremental improvements in network architectures, we believe it is also worth searching for with new building blocks, not just their arrangements.

If you are interested to learn more about this work, we invite readers to read our interactive article (or pdf version of the paper for offline reading). In addition to open sourcing these experiments to the research community, we have also released a general Python implementation of NEAT called PrettyNEAT to help interested readers to explore the exciting area of neural network evolution from first principles.

[N] Huawei’s First Commercial AI Chip Doubles the Training Performance of Nvidia’s Flagship GPU

Written on . Posted in Reddit MachineLearning.

Billed as the single chip with the greatest computing density, Ascend 910 delivers performance of up to 256 teraFLOPS under FP16 and 512 teraOPS under IN8 with declared max power consumption of 310W. In comparison, the GPU Tesla V100 delivers up to 125 teraFLOPS with a max power consumption of 300W, while Google’s TPU 2.0 with four ASICs can reach 180 teraFLOPS.

​

Link: https://medium.com/syncedreview/huaweis-first-commercial-ai-chip-doubles-the-training-performance-of-nvidia-s-flagship-gpu-86e4d0078f6f

submitted by /u/Empty_Lecture
[link] [comments]

[P] I applied Mark Zuckerberg’s face to Facebook emojis

Written on . Posted in Reddit MachineLearning.

[P] I applied Mark Zuckerberg's face to Facebook emojis

Seeing the post on photorealistic emojis reminded me of a project I did last year: Zuckerberg Emojis

​

Sad Mark

Why? Well, facebook forces us to use quite specific representation of emotions to react to things. In a way, these emojis become our facial expression. So it would only fair to apply the same expression to Zuckerberg’s face.

I used CNNMRF, Deep Image Analogy and jcjohnsons neural style in sequence to apply the face and upscale it to a good resolution.

1.Original 2.CNNMRF result 3. Deep Image Analogy output 4.Upscaled with Neural-style

The full write-up with all emojis is here: https://rybakov.com/blog/zuckerberg_emojis/

submitted by /u/rybakovcom
[link] [comments]

[P] JAXnet (preview) – An alternative to TensorFlow2/Keras/PyTorch for more concise, robust and optimized deep learning code

Written on . Posted in Reddit MachineLearning.

On GitHub

JAXnet is a deep learning library based on Google’s JAX. JAXnet’s functional API provides unique benefits over TensorFlow2, Keras and PyTorch, while maintaining user-friendliness, modularity and scalability:

  • More robustness through immutable weights, no global compute graph.
  • GPU-compiled numpy code for networks, training loops, pre- and postprocessing.
  • Regularization and reparametrization of any module or whole networks in one line.
  • No global random state, flexible random key control.

Your feedback is welcome!

submitted by /u/juliuskunze
[link] [comments]