Skip to main content

Blog

Learn About Our Meetup

5000+ Members

GO >

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOIN

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

JOBS

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

CONTACT

Author: torontoai

Evolution of Deep Learning Symposium recap blog post

Written on . Posted in Toronto AI Organizations, Vector Institute.

AI community celebrates Dr. Geoffrey Hinton at Evolution of Deep Learning Symposium

International AI talent gathered in Toronto last week to share perspectives on how research and applications are evolving, and how researchers can continue momentum in the field based on historic accomplishments in deep learning. 

The two-day Evolution of Deep Learning Symposium, hosted by the Vector Institute, was in celebration of Dr. Geoffrey Hinton’s leadership and foresight in the field. His four decades of work earned him the 2018 ACM A.M. Turing Award, alongside colleagues Dr. Yoshua Bengio and Dr. Yann LeCun. Dr. Hinton is Chief Scientific Advisor at the Vector Institute, Vice President and Engineering Fellow at Google, and Professor Emeritus at the University of Toronto. 

Researchers from throughout the deep learning community gathered to celebrate Dr. Hinton’s contributions and achievements, presented posters showcasing  the impact of his research across a broad range of topics, and heard from speakers spanning four decades of Dr. Hinton’s career and collaborators: 

The line-up of talks given at the event was a testament to Dr. Hinton’s enduring influence in the field. All are all colleagues, collaborators or former students of Dr. Hinton’s who have since gone on to make their own significant contributions to deep learning. Each spoke at length about Dr. Hinton’s convictions, curiosity , and friendship.

  • Radford Neal, Professor emeritus at the University of Toronto
  • George Dahl, research scientist with Google Brain
  • Terrence Sejnowski, Professor and Director of the Computational Neurobiology Laboratory at the Salk Institute and Distinguished Professor at UC San Diego
  • Max Welling, research chair in Machine Learning at the University of Amsterdam, VP technologies at Qualcomm and Senior Fellow at CIFAR
  • Ilya Sutskever, cofounder and chief scientist at OpenAI

Dr. Hinton himself presented his Turing lecture, and remarked that “Turing is amazing because he spanned both connectionist and symbolic fields.”

The Symposium’s second day was capped off with an industry panel featuring a discussion of the benefits and challenges of “productization” in Canada, and opportunities created for researchers in industry labs to test solutions at scale. 

Panelists

  • Foteini Agrafioti, Chief Science Officer at RBC and Head of Borealis AI
  • Andrew Brown, Senior Director of Data Science and AI Research at CIBC 
  • Raquel Urtasun, Uber ATG Chief Scientist, Professor University of Toronto, and Co-Founder of the Vector Institute

Moderator

  • Graham Taylor, Associate Professor of Engineering at the University of Guelph, CIFAR Azrieli Global Scholar, and Academic Director of NextAI

 

Celebrating fundamental discovery and the power of research groups

The symposium’s highlight was a candid discussion between Dr. Hinton and Eric Schmidt, Technical Advisor at Alphabet Inc. and former CEO and Executive Chairman at Google. At a reception sponsored by TD Layer 6, the pair discussed how they’ve seen the field evolve and what excites them about the future, such as neutral networks with improved interpretability – especially in health care – as well as understanding the limitations of current techniques and pursuing capabilities such as self-supervised learning.

Both reminded the audience that Canada made its mark by supporting fundamental research with long-term gains and abilities to tackle rare diseases, crop science, financial fraud, and more. 

Dr. Hinton credits the funding he received for full-time research as the reason he was able to establish a foothold and make strides in neural network research. He also spoke to the power of a strong research community, encouraging Ph. D. candidates in the crowd to pursue their research alongside trustworthy colleagues who will be honest and push their work further by asking the right questions.

Leaders of the Vector Institute’s industry sponsors and special guests gathered after the fireside chat for a celebratory dinner. 

Left to right: Ed Clark, Chair of the Board, Vector Institute; Anthony Viel, Chief Executive Officer, Deloitte LLP; Jim Smith, President and Chief Executive Officer, Thomson Reuters; Garth Gibson, President and Chief Executive Officer, Vector Institute; Eric Schmidt, Technical Advisor, Alphabet Inc.; Elio Luongo, Chief Executive Officer and Senior Partner, KPMG LLP; Aaron Regent, Chairman of the Board, Scotiabank; Brian Levitt, Chairman of the Board, TD Bank Group; Raquel Urtasun, Uber ATG Chief Scientist, Professor University of Toronto, Co-Founder of the Vector Institute; Cameron Schuler, Chief Commercialization Officer and VP Industry Innovation, Vector Institute; Ilya Sutskever, Co-founder and Chief Scientist, Open AI

University of Toronto faculty commitments honour Hinton’s work

Dr. Hinton’s advocacy for strong research communities was fitting, as the symposium was also the launchpad for an exciting announcement to expand Toronto’s deep learning community. In recognition of Dr. Hinton’s work, Vector is working with the University of Toronto to recruit three new tenure-stream faculty positions in deep learning.

That news came on the heels of Vector cross-appointing eight new faculty members and 29 faculty affiliates from universities across Canada. These researchers now have access to a collaborative research community based in Toronto’s MaRS Discovery District, along with computing resources to catalyze both foundational research and specific applications. 

These academic appointments will not only enhance the University of Toronto’s already established reputation as an AI leader, but also support the growth of this dynamic field that Dr. Hinton has helped champion for decades.

AI community celebrates Dr. Geoffrey Hinton at Evolution of Deep Learning Symposium

Written on . Posted in Toronto AI Organizations, Vector Institute.

Geoffrey Hinton delivered his Turing Lecture to a crowd of researchers and professionals at the Vector Institute’s Evolution of Deep Learning Symposium on October 16th.

International AI talent gathered in Toronto last week to share perspectives on how research and applications are evolving, and how researchers can continue momentum in the field based on historic accomplishments in deep learning. 

The two-day Evolution of Deep Learning Symposium, hosted by the Vector Institute, was in celebration of Dr. Geoffrey Hinton’s leadership and foresight in the field. His four decades of work earned him the 2018 ACM A.M. Turing Award, alongside colleagues Dr. Yoshua Bengio and Dr. Yann LeCun. Dr. Hinton is Chief Scientific Advisor at the Vector Institute, Vice President and Engineering Fellow at Google, and Professor Emeritus at the University of Toronto. 

Researchers from throughout the deep learning community gathered to celebrate Dr. Hinton’s contributions and achievements, presented posters showcasing the impact of his research across a broad range of topics, and heard from speakers spanning four decades of Dr. Hinton’s career and collaborators.

The line-up of talks given at the event was a testament to Dr. Hinton’s enduring influence in the field. Colleagues, collaborators, and former students of Dr. Hinton’s who have since gone on to make their own significant contributions to deep learning each spoke at length about Dr. Hinton’s convictions, curiosity, and friendship.

  • Radford Neal, Professor Emeritus at the University of Toronto
  • George Dahl, research scientist with Google Brain
  • Terrence Sejnowski, Professor and Director of the Computational Neurobiology Laboratory at the Salk Institute and Distinguished Professor at UC San Diego
  • Max Welling, Research Chair in Machine Learning at the University of Amsterdam, VP technologies at Qualcomm and Senior Fellow at CIFAR
  • Ilya Sutskever, Co-founder and Chief Scientist at OpenAI

Dr. Hinton himself presented his Turing lecture, and remarked that “Turing is amazing because he spanned both connectionist and symbolic fields.”

The Symposium’s second day was capped off with an industry panel featuring a discussion of the benefits and challenges of productizing AI in Canada, and opportunities created for researchers in industry labs to test solutions at scale.

Panelists

  • Foteini Agrafioti, Chief Science Officer at RBC and Head of Borealis AI
  • Andrew Brown, Senior Director of Data Science and AI Research at CIBC 
  • Raquel Urtasun, Uber ATG Chief Scientist, Professor University of Toronto, and Co-Founder of the Vector Institute

Moderator

  • Graham Taylor, Associate Professor of Engineering at the University of Guelph, CIFAR Azrieli Global Scholar, and Academic Director of NextAI

Celebrating foundational discovery and the power of good collaborators

The symposium’s highlight was a candid discussion between Dr. Hinton and Eric Schmidt, Technical Advisor at Alphabet Inc. and former CEO and Executive Chairman at Google. At a reception sponsored by TD Layer 6, the pair discussed how they’ve seen the field evolve and what excites them about the future, such as neutral networks with improved interpretability – especially in health care – as well as understanding the limitations of current techniques and pursuing capabilities such as self-supervised learning.

Left to right: Katherine Gorman, Geoffrey Hinton, Eric Schmidt

Both guests reminded the audience that Canada made its mark by supporting foundational research with long-term gains and abilities to tackle challenges such as rare diseases, crop science, financial fraud, and more.

Dr. Hinton credits the funding he received for full-time research as the reason he was able to establish a foothold and make strides in neural network research. He also spoke to the power of a strong research community, encouraging PhD candidates in the crowd to pursue their research alongside trustworthy colleagues who will be honest and further their work further by asking the right questions.

Leaders of the Vector Institute’s industry sponsors and special guests gathered after the fireside chat for a celebratory dinner. 

Left to right: Ed Clark, Chair of the Board, Vector Institute; Anthony Viel, Chief Executive Officer, Deloitte LLP; Jim Smith, President and Chief Executive Officer, Thomson Reuters; Garth Gibson, President and Chief Executive Officer, Vector Institute; Eric Schmidt, Technical Advisor, Alphabet Inc.; Elio Luongo, Chief Executive Officer and Senior Partner, KPMG LLP; Aaron Regent, Chairman of the Board, Scotiabank; Brian Levitt, Chairman of the Board, TD Bank Group; Raquel Urtasun, Uber ATG Chief Scientist, Professor University of Toronto, Co-Founder of the Vector Institute; Cameron Schuler, Chief Commercialization Officer and VP Industry Innovation, Vector Institute; Ilya Sutskever, Co-founder and Chief Scientist, Open AI

University of Toronto faculty commitments honour Dr. Hinton’s work

Dr. Hinton’s advocacy for strong research communities was fitting, as the symposium was also the launchpad for an exciting announcement to expand Toronto’s deep learning community. In recognition of Dr. Hinton’s work, Vector Institute will work with the University of Toronto to recruit three new tenure-stream faculty positions in deep learning.

That news came on the heels of Vector appointing eight new faculty members and 29 faculty affiliates who also hold appointments at universities across Canada. These researchers now have access to a collaborative community based in Toronto’s MaRS Discovery District, along with computing resources to catalyze both foundational research and specific applications.

Altogether, these academic appointments will enhance Canada’s leadership in AI and support the growth of this dynamic field that Dr. Hinton has championed for decades.

 

Deep Learning Detects Brain Hemorrhages with Accuracy Rivaling Experts

Written on . Posted in NVIDIA.

There’s a maxim in stroke treatment: “time is brain.”

It’s a reminder that during a stroke, human nervous tissue is lost at an alarming rate. From the onset of a brain injury and the start of medical treatment, every moment matters.

Researchers at UC Berkeley and UCSF School of Medicine are working on a deep learning model to reduce the time it takes to diagnose intracranial hemorrhage (bleeding in the skull) on a CT scan. With more than 80 million CT scans performed annually in the United States alone, AI could increase radiologists’ efficiency amidst an overwhelming volume of images.

“When someone gets into a car accident, or there’s a fall or other trauma that involves the head — a doctor will order a head CT scan,” said Weicheng Kuo, lead author on the paper, published this week in the leading scientific journal PNAS.

It’s critical to detect hemorrhages, even tiny ones. The analysis requires a high degree of focus by specially trained radiologists.

A neural network that assesses CT scans could reduce the burden on these specialists. Kuo and his collaborators expect a significant increase in radiologists’ productivity with their deep learning system, PatchFCN.

The researchers used NVIDIA V100 Tensor Core GPUs through Amazon Web Services for both the training and inference of their AI model, which segments the hemorrhage area, and identifies brain hemorrhages with 99 percent accuracy. The neural network also performs automated measurements of abnormalities on the CT scans, a  time-consuming step that radiologists manually perform today.

Every Minute Matters

Radiologists often have a large stack of sans to go through. Depending on how busy the day is, the turnaround time for reading a scan and reporting results to the emergency department can be half an hour or more.

If an abnormal case is at the bottom of the stack, the delay in diagnosis can adversely affect patients. AI can close this gap, processing a scan within a second, on average, using a single NVIDIA GPU for inference.

Trained on a dataset of more than 4,000 head CT scans from UCSF-affiliated hospitals, the PatchFCN performance rivals that of experienced radiologists, the study showed.

The volume of medical imaging studies is on the rise. Tools like PatchFCN could help radiologists manage larger workloads and boost their efficiency, Kuo says.

Looking Patch by Patch

Many convolutional neural networks analyze a whole image at once to come up with a result. And that makes sense: in the world of digital data, it’s often assumed that more information is better.

But, instead, the team found that splitting a CT scan into smaller patches improved the neural network’s results. They experimented with the patch size to achieve the best performance.

Neural networks can be set at different levels of recall, or sensitivity. The researchers believe that for this clinical application, the system should operate at the highest possible recall level, since the consequences of missing a brain hemorrhage could be catastrophic.

With this high-recall setting, the model had an average precision of 99 percent for detecting hemorrhages, the highest classification accuracy to date.

Rather than providing only a “yes” or “no” result, the neural network also provides a detailed tracing of each hemorrhage. The ability to highlight abnormalities directly on the image is essential for clinical use, because neurosurgeons must visually confirm the locations of hemorrhages on a head CT exam to judge the need and approach for surgical intervention.

The post Deep Learning Detects Brain Hemorrhages with Accuracy Rivaling Experts appeared first on The Official NVIDIA Blog.

[R]Research Guide: Image Quality Assessment for Deep Learning

Written on . Posted in Reddit MachineLearning.

The quality of images is relevant in building compression and image enhancement algorithms. Image Quality Assessment (IQA) is divided into two main areas; reference-based evaluation and no-reference evaluation.

In this guide, we’ll look at how deep learning has been used in image quality analysis.

https://heartbeat.fritz.ai/research-guide-image-quality-assessment-c4fdf247bf89

submitted by /u/mwitiderrick
[link] [comments]

[D] Feature Loss vs. GANs – what are the trade offs?

Written on . Posted in Reddit MachineLearning.

I’m doing a bit of reading on the speech enhancement problem, where you have an audio signal containing human speech plus some noise, and you want extract just the human speech. It’s pretty analogous to image denoising or “super-resolution”, and a lot of the techniques from the image domain are being borrowed and re-applied to audio quite successfully (eg. repurposing the U-Net architecture from image processing to spectrograms and then raw audio). It’s all pretty cool.

There’s some interesting work being done with loss functions this space and I’m looking for some clarification as to why you’d choose one approach over another. You want to compare a target image, or audio waveform, with a predicted sample, and you need to define a loss function which measures how “close” they are. The Related work – Loss functions (1.1.3) section of this paper gives a pretty good overview of the different approaches, which I’ll try to summarize here.

  • Mean squared error loss: A pretty standard regression loss as far as I know, but it’s limited to only considering one pixel at a time: “minimizing MSE encourages finding pixel-wise averages of plausible solutions which are typically overly-smooth and thus have poor perceptual quality”.
  • Feature loss: This is where you pre-train a network on a similar problem, such as image classification, and then you freeze the weights. For both the target and predicted sample, you run each through the classification network, then grab some internal activations from that network and call them “features”. You compute some distance between these feature vectors to get your loss. The key idea is that the classification network is able to capture important features that MSE loss cannot (more detail here).
  • GAN loss: A discriminator network trains in-tandem with the generator network, where the job of the discriminator is to classify whether its input is “real” or “generated”. Like the feature loss network, it can detect features that MSE loss cannot, but it can also punish identifiable quirks of the generator network, whereas feature loss can potentially be “hacked” by the generator network.

So my questions are:

  • Have I characterised these approaches well?
  • Why would you ever choose feature loss over using a discriminator network (ie. GAN)?
    • Discriminators can punish the generator for being predictably wrong (ie. common artifacts)
    • Pre-trained feature loss networks may better represent image features, if they have been trained for longer, on larger data sets
    • Apparently GANs can have stability issues when training
  • The SRGAN paper suggests using both feature loss and a GAN for their loss function – is this the best known approach?

submitted by /u/The_Amp_Walrus
[link] [comments]

Quantum Supremacy Using a Programmable Superconducting Processor

Written on . Posted in Google.

Posted by John Martinis, Chief Scientist Quantum Hardware and Sergio Boixo, Chief Scientist Quantum Computing Theory, Google AI Quantum

Physicists have been talking about the power of quantum computing for over 30 years, but the questions have always been: will it ever do something useful and is it worth investing in? For such large-scale endeavors it is good engineering practice to formulate decisive short-term goals that demonstrate whether the designs are going in the right direction. So, we devised an experiment as an important milestone to help answer these questions. This experiment, referred to as a quantum supremacy experiment, provided direction for our team to overcome the many technical challenges inherent in quantum systems engineering to make a computer that is both programmable and powerful. To test the total system performance we selected a sensitive computational benchmark that fails if just a single component of the computer is not good enough.

Today we published the results of this quantum supremacy experiment in the Nature article, “Quantum Supremacy Using a Programmable Superconducting Processor”. We developed a new 54-qubit processor, named “Sycamore”, that is comprised of fast, high-fidelity quantum logic gates, in order to perform the benchmark testing. Our machine performed the target computation in 200 seconds, and from measurements in our experiment we determined that it would take the world’s fastest supercomputer 10,000 years to produce a similar output.

Left: Artist’s rendition of the Sycamore processor mounted in the cryostat. (Full Res Version; Forest Stearns, Google AI Quantum Artist in Residence) Right: Photograph of the Sycamore processor. (Full Res Version; Erik Lucero, Research Scientist and Lead Production Quantum Hardware)

The Experiment
To get a sense of how this benchmark works, imagine enthusiastic quantum computing neophytes visiting our lab in order to run a quantum algorithm on our new processor. They can compose algorithms from a small dictionary of elementary gate operations. Since each gate has a probability of error, our guests would want to limit themselves to a modest sequence with about a thousand total gates. Assuming these programmers have no prior experience, they might create what essentially looks like a random sequence of gates, which one could think of as the “hello world” program for a quantum computer. Because there is no structure in random circuits that classical algorithms can exploit, emulating such quantum circuits typically takes an enormous amount of classical supercomputer effort.

Each run of a random quantum circuit on a quantum computer produces a bitstring, for example 0000101. Owing to quantum interference, some bitstrings are much more likely to occur than others when we repeat the experiment many times. However, finding the most likely bitstrings for a random quantum circuit on a classical computer becomes exponentially more difficult as the number of qubits (width) and number of gate cycles (depth) grow.

Process for demonstrating quantum supremacy.

In the experiment, we first ran random simplified circuits from 12 up to 53 qubits, keeping the circuit depth constant. We checked the performance of the quantum computer using classical simulations and compared with a theoretical model. Once we verified that the system was working, we ran random hard circuits with 53 qubits and increasing depth, until reaching the point where classical simulation became infeasible.

Estimate of the equivalent classical computation time assuming 1M CPU cores for quantum supremacy circuits as a function of the number of qubits and number of cycles for the Schrödinger-Feynman algorithm. The star shows the estimated computation time for the largest experimental circuits.

This result is the first experimental challenge against the extended Church-Turing thesis, which states that classical computers can efficiently implement any “reasonable” model of computation. With the first quantum computation that cannot reasonably be emulated on a classical computer, we have opened up a new realm of computing to be explored.

The Sycamore Processor
The quantum supremacy experiment was run on a fully programmable 54-qubit processor named “Sycamore.” It’s comprised of a two-dimensional grid where each qubit is connected to four other qubits. As a consequence, the chip has enough connectivity that the qubit states quickly interact throughout the entire processor, making the overall state impossible to emulate efficiently with a classical computer.

The success of the quantum supremacy experiment was due to our improved two-qubit gates with enhanced parallelism that reliably achieve record performance, even when operating many gates simultaneously. We achieved this performance using a new type of control knob that is able to turn off interactions between neighboring qubits. This greatly reduces the errors in such a multi-connected qubit system. We made further performance gains by optimizing the chip design to lower crosstalk, and by developing new control calibrations that avoid qubit defects.

We designed the circuit in a two-dimensional square grid, with each qubit connected to four other qubits. This architecture is also forward compatible for the implementation of quantum error-correction. We see our 54-qubit Sycamore processor as the first in a series of ever more powerful quantum processors.

Heat map showing single- (e1; crosses) and two-qubit (e2; bars) Pauli errors for all qubits operating simultaneously. The layout shown follows the distribution of the qubits on the processor. (Courtesy of Nature magazine.)

Testing Quantum Physics
To ensure the future utility of quantum computers, we also needed to verify that there are no fundamental roadblocks coming from quantum mechanics. Physics has a long history of testing the limits of theory through experiments, since new phenomena often emerge when one starts to explore new regimes characterized by very different physical parameters. Prior experiments showed that quantum mechanics works as expected up to a state-space dimension of about 1000. Here, we expanded this test to a size of 10 quadrillion and find that everything still works as expected. We also tested fundamental quantum theory by measuring the errors of two-qubit gates and finding that this accurately predicts the benchmarking results of the full quantum supremacy circuits. This shows that there is no unexpected physics that might degrade the performance of our quantum computer. Our experiment therefore provides evidence that more complex quantum computers should work according to theory, and makes us feel confident in continuing our efforts to scale up.

Applications
The Sycamore quantum computer is fully programmable and can run general-purpose quantum algorithms. Since achieving quantum supremacy results last spring, our team has already been working on near-term applications, including quantum physics simulation and quantum chemistry, as well as new applications in generative machine learning, among other areas.

We also now have the first widely useful quantum algorithm for computer science applications: certifiable quantum randomness. Randomness is an important resource in computer science, and quantum randomness is the gold standard, especially if the numbers can be self-checked (certified) to come from a quantum computer. Testing of this algorithm is ongoing, and in the coming months we plan to implement it in a prototype that can provide certifiable random numbers.

What’s Next?
Our team has two main objectives going forward, both towards finding valuable applications in quantum computing. First, in the future we will make our supremacy-class processors available to collaborators and academic researchers, as well as companies that are interested in developing algorithms and searching for applications for today’s NISQ processors. Creative researchers are the most important resource for innovation — now that we have a new computational resource, we hope more researchers will enter the field motivated by trying to invent something useful.

Second, we’re investing in our team and technology to build a fault-tolerant quantum computer as quickly as possible. Such a device promises a number of valuable applications. For example, we can envision quantum computing helping to design new materials — lightweight batteries for cars and airplanes, new catalysts that can produce fertilizer more efficiently (a process that today produces over 2% of the world’s carbon emissions), and more effective medicines. Achieving the necessary computational capabilities will still require years of hard engineering and scientific work. But we see a path clearly now, and we’re eager to move ahead.

Acknowledgements
We’d like to thank our collaborators and contributors — University of California Santa Barbara, NASA Ames Research Center, Oak Ridge National Laboratory, Forschungszentrum Jülich, and many others who helped along the way.