Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

Author: torontoai

[D] Trust t-SNE without PCA verification?

Hi all,
for my dataset t-SNE produces beautiful clusters with some transition in between clusters and a plot that overall is just very exciting. PCA on the other hand just produces very boring results.

Now I’m aware that t-SNE will try much harder to cluster stuff than PCA, so I’m not sure what to make of it.

Can I somehow verify I’m not seeing artefacts that are based on the workings of t-SNE?

I can’t share the data, but here are some crudely drawn examples <3 https://imgur.com/a/7gQPrMA

Thanks!

submitted by /u/Seiteshyru
[link] [comments]

[DISCUSSION] Bert Token Embeddings

From Paper is easy to understand that BERT input is composed by Token Embeddings, Positional Encode, Sentence Encode. The last two are well-defined in BERT paper and in “Attention is all you need”. But Token embeddings is not clear how are build. Reading on Internet I found different opinions. For sure tokenization is performed using WordPiece Tokens and it’s easy understand how it splits words. But once you have the token id how BERT converts it in a Embedding?

submitted by /u/lor_v
[link] [comments]

[R] Weakly Supervised Disentanglement with Guarantees

We build a theoretical framework for analyzing disentanglement in the weakly supervised regime. We provide new definitions for disentanglement (sorry) that can be measured in a weakly supervised manner, and use these definitions as the cornerstone for developing a calculus and theory of disentanglement. We then analyzed several weak supervision techniques and proved (and empirically demonstrated) their disentanglement guarantees (or lack thereof).

We hope that the concepts developed in this paper will help researchers frame their discussion and analysis of weakly supervised disentanglement in future work.

Paper: https://arxiv.org/abs/1910.09772

Cute gif: https://twitter.com/i/status/1187507675258486784

submitted by /u/approximately_wrong
[link] [comments]

[P] 10K Downloads Special 🎉: gpt2-client accepting all feature requests!

[P] 10K Downloads Special 🎉: gpt2-client accepting all feature requests!

Hey everyone 👋🏻👋🏻!

First off, I want to thank all of you for your amazing support. gpt2-client just reached 10K+ downloads!!

Being my first open-source project, it’s touching to see the positive experiences you share with me via email/DM. I’ve noticed a trend where many of you are using it for your NLP research and some of you for your side-projects. No matter what you do, I’d love to know how I can improve it, either in terms of functionality, extendability, modularity, efficiency. You name it.

We did it y’all! 10K in the bag 😀

The Good Stuff: You’re in control now

As a way of giving back, I’d love to hear what you’d want to see in gpt2-client. It can be any bombastic feature request!!! We could discuss this on any platform (or you can open a feature request here https://github.com/rish-16/gpt2client/issues/new/choose). This could tug on any aspect of gpt2-client that you feel should belong inside the module.

————————————

If you still aren’t sure what gpt2-client is, I urge you to check out https://github.com/rish-16/gpt2client/ and if you like what you’re seeing, do drop a ⭐. It means a lot to me and motivates me to continue building open-source technology.

Express your creativity down below in the comments!!! Grateful for your continuing support 🤘🏻

Cheers!

submitted by /u/rish-16
[link] [comments]

[R] Distributed self-supervising capsule network

One month ago I post an introduction to what I was working on, and someone suggested I should “create *something*, anything that people can look at to try and understand “. So I have written an article now. This is the link.

This article proposes a self-supervising machine learning architecture which is actually a two-step model. The first step is to construct a causal representation model, and the second step is to promote intentions for it to complete tasks. The output cannot be supervised directly but it is action control signals sparsely coding information. In these situations, the end-to-end supervised learning is not applicable anymore.

Here is the former post a month ago.

submitted by /u/tobby_liu
[link] [comments]

[R] Attenchilada: Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis

tl;dr: Using location-relative attention mechanisms allows Tacotron-based TTS systems to generalize to very long utterances.

Abstract:
Despite the ability to produce human-level speech for in-domain text, attention-based end-to-end text-to-speech (TTS) systems suffer from text alignment failures that increase in frequency for out-of-domain text. We show that these failures can be addressed using simple location-relative attention mechanisms that do away with content-based query/key comparisons. We compare two families of attention mechanisms: location-relative GMM-based mechanisms and additive energy-based mechanisms. We suggest simple modifications to GMM-based attention that allow it to align quickly and consistently during training, and introduce a new location-relative attention mechanism to the additive energy-based family, called Dynamic Convolution Attention (DCA). We compare the various mechanisms in terms of alignment speed and consistency during training, naturalness, and ability to generalize to long utterances, and conclude that GMM attention and DCA can generalize to very long utterances, while preserving naturalness for shorter, in-domain utterances.

Paper: https://arxiv.org/abs/1910.10288

Audio Examples: https://google.github.io/tacotron/publications/location_relative_attention

submitted by /u/animus144
[link] [comments]

Your guide to artificial Intelligence and machine learning at re:Invent 2019

With less than 40 days to re:Invent 2019, the excitement is building up and we are looking forward to seeing you all soon! Continuing our journey on artificial intelligence and machine learning, we are bringing a lot of technical content this year, with over 200 breakout sessions, deep-dive chalk talks, hands-on exercises with workshops featuring Amazon SageMaker, AWS DeepRacer, and deep learning frameworks such as TensorFlow, PyTorch, and more. You’ll hear from many customers including Vanguard, BBC, Autodesk, British Airways, Fannie Mae, Thermo Fisher, Intuit, and many more. We are also hosting the Machine Learning Summit again this year, where you will hear from researchers and entrepreneurs about the latest breakthroughs today and the future possibilities tomorrow.

To get you started on planning, here are a few highlights for the AI and ML sessions from the re:Invent 2019 session catalog. The reserved seating is now open, so get your seats in advance for your favorite sessions.

Getting started

If you are new to AI and ML, we have some sessions for you to get started and learn these concepts. These sessions cover the basics including overviews and demos for Amazon SageMaker, the different AI services for many applications, and the popular AWS DeepLens and AWS DeepRacer to help you learn, while having fun.

Leadership session: Machine Learning (Session AIM218-L)

As we embark on the golden age of machine learning, we are seeing the constraints and blockers disappear, and the value extending across different industries. In this leadership session, learn about the latest machine learning offerings from AWS as we explore the democratization of machine learning. We will discuss the breadth and depth of our machine learning services and you will hear from customers who are partnering with AWS on this journey.

Amazon SageMaker deep dive: A modular solution for machine learning (Session AIM307)

Amazon SageMaker is a fully managed service enabling all developers and data scientists with every aspect of the machine learning workflow. In this session, we will discuss the technical details of Amazon SageMaker to help you with your machine learning journey to get your ML models from experimentation to production at scale. We will also discuss practical deployments through real-world customer examples.

Starting the enterprise machine learning journey (Session AIM205)

Amazon has been investing in machine learning for more than 20 years, innovating in areas such as fulfillment and logistics, personalization and recommendations, forecasting, fraud prevention, and supply chain optimization. During this session, we take this expertise and show you how to identify business problems that can be solved with machine learning. We discuss considerations including selecting the right use case for a machine learning pilot, nurturing skills, and measuring the success of such pilots.

Finding a needle in a haystack: Use AI to transform content management (Session AIM206)

Finding digital content, from documents to media, can be frustrating and time-consuming. Across your employees or customers, this challenge can waste hours, derail projects, and create poor experiences. In this breakout session, learn how to use language and vision AI services to extract data, insights, and trends from all of your digital content, with a focus on how to more effectively manage your documents and find what you need.

Get started with AWS DeepRacer (Workshop AIM207)

Get behind the keyboard for an immersive experience with AWS DeepRacer. Developers with no prior machine learning experience learn new skills and apply their knowledge in a fun and exciting way. With the help of the AWS pit crew, build and train a reinforcement learning model that you can race on the tracks and win special AWS prizes, in this one of many workshops for AWS DeepRacer. See the “Advanced topics in machine learning” section for an advanced version of this workshop.

Start using computer vision with AWS DeepLens (Workshop AIM229)

If you’re new to deep learning, this workshop is for you. Learn how to build and deploy computer-vision models using the AWS DeepLens deep-learning-enabled video camera. Also learn how to build a machine learning application and a model from scratch using Amazon SageMaker. Finally, learn to extend that model to Amazon SageMaker to build an end-to-end AI application. See the “Advanced topics in machine learning” section for an advanced version of this workshop.

Improve machine learning model quality in response to changes in data (Session AIM213)

Machine learning models are typically trained and evaluated using historical data. But the real-world data may not look like the training data, especially as models age over time and the distribution of data changes. This gradual variance of the model from the real world is known as model drift, and it can have a big impact on prediction quality. This session explores techniques you can use to monitor prediction quality in production, as well as effective corrective actions such as auditing and iterative retraining.

Practical applications of machine learning

The biggest value for machine learning is its applicability across different industries. In these sessions, chalk talks, and workshops, we will dive deep into the practical aspects of machine learning for specific industries including finance, healthcare, retail, media and entertainment, manufacturing, and more.

Transforming Healthcare with AI (Session AIM210)

Improving patient care, making treatment decisions, managing clinical trials, and more are all moving into a new age due to advancements in AI. In this session, we cover AI solutions specific to the Healthcare industry, from extracting relevant medical information from patient records and clinical trial reports to automating the clinical documentation process with automatic speech recognition. Hear directly from our customers and come away with answers on how to get started immediately.

ML in retail: Solutions that add intelligence to your business (Session AIM212)

Machine learning is ranked the number-one “game changer” for the retail market segment by chief experience officers (CXOs), yet it’s only number eight on top spending priorities. So which scenarios are real? In this session, we dive into how AWS puts machine learning in the hands of every developer, without the need for deep machine learning experience. Learn about personalized product recommendations, inventory forecasting, new in-store experiences, and more. Learn from our experience at Amazon.com and hear from our customers today.

AI document processing for business automation (Session AIM211)

Millions of times per day, customers from the Finance, Healthcare, public, and other sectors rely on information that is locked in documents. Amazon Textract uses artificial intelligence to “read” such documents as a person would, to extract not only text but also tables, forms, and other structured data without configuration, training, or custom code. In this session, we demonstrate how you can use Amazon Textract to automate business processes with AI. You also hear directly from our customers about how they accelerated their own business processes with Amazon Textract.

Predict future business outcomes using Amazon Forecast (Session AIM312)

Based on the same technology used at Amazon.com, Amazon Forecast uses machine learning and time-series data to build accurate business forecasts. In this session, learn how machine learning can improve accuracy in demand forecasting, financial planning, and resource allocation while reducing your forecasting time from months to hours.

Build accurate training datasets with Amazon SageMaker Ground Truth (Session AIM308)

Successful machine learning models are built on high-quality training datasets. Typically, the task of data labeling is distributed across a large number of humans, adding significant overhead and cost. This session explains how Amazon SageMaker Ground Truth reduces cost and complexity using techniques designed to improve labeling accuracy and reduce human effort. We will walk through best practices for building highly accurate training datasets and discuss how you can use Amazon SageMaker Ground Truth to implement them.

Build predictive maintenance systems with Amazon SageMaker (Chalk Talk AIM328)

Across a wide spectrum of industries, customers are starting to use prediction maintenance models to proactively fix problems before they impact production. The result is an optimized supply chain and improved working conditions. In this session, learn how to use data from equipment to build, train, and deploy predictive models. We dive deep into the architecture for using the turbofan degradation simulation dataset to train the model to recognize potential equipment failures and share details.

Build a fraud detection system with Amazon SageMaker (Workshop AIM359)

In this workshop, we will explore the new AWS Fraud Detection Solution. We show you how to build, train, and deploy a fraud detection machine learning model. The fraud detection model recognizes fraud patterns, and is self-learning that enables it to adapt to new, unknown fraud patterns. We will show you how to execute automated transaction processing, and how to the Fraud Detection solution flags that activity for review.

Delight your customers with ML-based personalized recommendations (Session AIM323)

Recommendation engines make targeted marketing campaigns, re-ranking of items, personalized notifications, and personalized search possible. In this session, we deep-dive into using Amazon Personalize to create and manage personalized recommendations efficiently, letting you focus on the real value of the data for your business. We discover how these deep learning techniques have a direct impact on the bottom line of your business by increasing engagement, click-through, satisfaction, and revenue. Learn from customer examples and dive into some live demonstrations.

Accelerate time-series forecasting with Amazon Forecast (Workshop AIM335)

Based on the same technology used at Amazon.com, Amazon Forecast uses machine learning to combine time-series data with additional variables to build up to 50% more accurate forecasts. In this workshop, prepare a dataset, build models based on that dataset, evaluate a model’s performance based on real observations, and learn how to evaluate the value of a forecast compared with another. Gain the skills to make decisions that will impact the bottom line of your business.

Build a content-recommendation engine with Amazon Personalize (Workshop AIM304)

Machine learning is being used increasingly to improve customer engagement by powering personalized product and content recommendations. Amazon Personalize lets you easily build sophisticated personalization capabilities into your applications, using machine learning technology perfected from years of use on Amazon.com. In this workshop, you build your own recommendation engine by providing training data, building a model based on the algorithm of your choice, testing the model by deploying your Amazon Personalize campaign, and integrating it into your own application.

Advanced topics in machine learning

We have a number of sessions that will dive deep into the technical details of machine learning across our service portfolio as well as deep learning frameworks including TensorFlow, PyTorch, and Apache MXNet. These code-level sessions and hands-on workshops will enable the advanced developer or data scientist in you to customize, integrate, and solve many challenges with deep technical solutions.

Deep learning with TensorFlow (Session AIM410, Workshop AIM401)

TensorFlow is of the most popular open-source deep learning frameworks used in machine learning development. The advanced breakout session will dive deep into training machine learning models with TensorFlow using Amazon SageMaker, including distributed training, cost-effective inference, and workflow management. The code-level workshop will include hands-on exercises where we will train and deploy TensorFlow models, apply automatic model tuning using Amazon SageMaker, and make predictions in production.

Deep learning with PyTorch (Session AIM412, Workshop AIM402)

PyTorch is rapidly gaining popularity in the industry as a deep learning framework used to transition seamlessly from research prototyping to production deployment. In the breakout session, you will lern how to develop deep learning models with PyTorch using Amazon SageMaker for multiple use cases including using a BERT model and instance segmentation for fine-grain computer vision. In the workshop, you will build a natural language processing model to analyze text.

Deep learning with Apache MXNet (Session AIM411, Workshop AIM403)

Apache MXNet has been a widely used deep learning framework on diverse applications such as computer vision, speech recognition, and natural language processing (NLP). The breakout session will discuss on building computer vision and NLP models using MXNet to automatically extract information from documents. In the workshop, we will build a computer vision model using MXNet and train the model for high accuracy, and finally deploy it to production using Amazon SageMaker.

Deep dive on Project Jupyter (Session AIM413)

Amazon SageMaker offers fully managed Jupyter notebooks that you can use in the cloud so you can explore and visualize data and develop your machine learning model. In this session, we explain why we picked Jupyter notebooks, and how and why AWS is contributing to Project Jupyter. We dive deep into our overall strategy for Jupyter and explain different use cases for Jupyter, including data science, analytics, and simulation.

Under the hood of AWS DeepRacer: Advanced RL driving course (Workshop AIM428)

This technical deep dive is suitable for advanced machine learning developers looking to learn more complex reinforcement learning concepts using AWS DeepRacer and Amazon SageMaker RL. AWS data scientists help you build models that require innovations in neural network architecture, expand the algorithms, and help you customize your AWS DeepRacer model for performance. We also dive deep into the technology under the hood that powers the AWS DeepRacer car.

Optimize deep learning models for edge deployments with AWS DeepLens (Workshop AIM405)

In this workshop, learn how to optimize your computer vision pipelines for edge deployments with AWS DeepLens and Amazon SageMaker Neo. Also learn how to build a sample object detection model with Amazon SageMaker and deploy it to AWS DeepLens. Finally, learn how to optimize your deep learning models and code to achieve faster performance for use cases where speed matters.

Take an ML model from idea to production using Amazon SageMaker (Workshop AIM427)

Come build the most accurate text-classification model possible with Amazon SageMaker. This service lets you build, train, and deploy ML models using built-in or custom algorithms. In this workshop, learn how to leverage Keras/TensorFlow deep-learning frameworks to build a text-classification solution using custom algorithms on Amazon SageMaker. We walk you through packaging custom training code in a Docker container, testing it locally, and then using Amazon SageMaker to train a deep-learning model. You then try to iteratively improve the model to achieve high accuracy. Finally, you deploy the model in production so applications can leverage the classification service.

Implement ML workflows with Kubernetes and Amazon SageMaker (Session AIM326)

Until recently, data scientists have spent much time performing operational tasks, such as ensuring that frameworks, runtimes, and drivers for CPUs and GPUs work well together. In addition, data scientists needed to design and build end-to-end machine learning (ML) pipelines to orchestrate complex ML workflows for deploying ML models in production. With Amazon SageMaker, data scientists can now focus on creating the best possible models while enabling organizations to easily build and automate end-to-end ML pipelines. In this session, we dive deep into Amazon SageMaker and container technologies, and we discuss how easy it is to integrate such tasks as model training and deployment into Kubernetes and Kubeflow-based ML pipelines.

Security for ML environments with Amazon SageMaker (Session AIM327)

Amazon SageMaker is a modular, fully managed platform that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale. In this session, we dive deep into the security configurations of Amazon SageMaker components, including notebooks, training, and hosting endpoints. Vanguard joins us to discuss the company’s use of Amazon SageMaker and its implementation of key controls in a highly regulated environment, including fine-grained access control, end-to-end encryption in transit, and comprehensive audit trails for resource and data access. If you want to build secure ML environments, this session is for you.

Machine Learning Summit

Whether you are a data scientist, machine learning practitioner, or business professional, you’ll enjoy the Machine Learning Summit at this year’s re:Invent, which will showcase advances in machine learning as well as the emerging trends. From disaster management to pediatrics, from fighting fake news to indoor farming, you will hear experts share their knowledge and perspectives.

Some of the sessions include:

Deep Learning for Disaster Management and Response
Cornelia Caragea, Associate Professor, Science and Engineering Offices,
Computer Science, University of Illinois at Chicago

Fighting Fake News and Deep Fakes with Machine Learning
Delip Rao, Vice President of Research at the AI Foundation

Deep Learning in Deep Nets: Helping Fish Farmers Feed the World
Bryton Shang, Founder and CEO, Aquabyte

Big Data for Tiny Patients: Applying ML to Pediatrics
Dr. Judith Dexheimer, Associate Professor, UC Department of Pediatrics,
Cincinnati Children’s Hospital Medical Center

Machine Learning and Society: Bias, Fairness and Explainability
Pietro Perona, Amazon Fellow, AWS

From Seed to Store: Using AI to Optimize the Indoor Farms of the Future
Henry Sztul, SVP, Science and Technology, Bowery Farming

The Machine Learning Summit will inform you about what’s on the horizon for machine learning. The event is scheduled for Tuesday, December 3, 2019, from 1:30 PM to 6 PM at the Venetian Theater. Visit the summit home page and register today.

 


About the Author

Shyam Srinivasan is on the AWS Machine Learning marketing team. He cares about making the world a better place through technology and loves being part of this journey. In his spare time, Shyam loves to run, travel, and have fun with his family and friends.

 

 

 

 

[R] AI Benchmark: All About Deep Learning on Smartphones in 2019

[R] AI Benchmark: All About Deep Learning on Smartphones in 2019

[arXiv Abstract]: The performance of mobile AI accelerators has been evolving rapidly in the past two years, nearly doubling with each new generation of SoCs. The current 4th generation of mobile NPUs is already approaching the results of CUDA-compatible Nvidia graphics cards presented not long ago, which together with the increased capabilities of mobile deep learning frameworks makes it possible to run complex and deep AI models on mobile devices. In this paper, we evaluate the performance and compare the results of all chipsets from Qualcomm, HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware acceleration for AI inference. We also discuss the recent changes in the Android ML pipeline and provide an overview of the deployment of deep learning models on mobile devices. All numerical results provided in this paper can be found and are regularly updated on the official project website: http://ai-benchmark.com

Performance evolution of mobile AI accelerators rs: image throughput for the float Inception-V3 model.

The paper discusses the following topics:

  1. Four generations of mobile NPUs
  2. Hardware acceleration resources for AI inference on each of Android mobile SoC platforms
  3. Android ecosystem for running deep learning models
  4. Quantized and Floating-point performance of all generations of mobile NPUs
  5. Performance comparison of FP inference on mobile NPUs vs. Intel CPUs vs. Nvidia GPUs.

The full paper is available on arXiv: https://arxiv.org/pdf/1910.06663.pdf

submitted by /u/aiff22
[link] [comments]