Skip to main content

Blog

Learn About Our Meetup

5000+ Members

GO >

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOIN

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

JOBS

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

CONTACT

Author: torontoai

[D] Positional Encoding in Transformer

Written on . Posted in Reddit MachineLearning.

Hey all,

I was reading up the transformer paper https://arxiv.org/abs/1706.03762. This architecture uses positional encoding which the attention layers ignore.

I don’t understand two things –

  1. Why use Sin & Cos as positional embeddings , why not any other function?
  2. They also talk about training these positional embeddings, how do you go about training such embeddings. As in how do you let the model know that these embeddings are for the position

Thanks !

submitted by /u/amil123123
[link] [comments]

[D] Is neural architecture search race to beat ImageNet actually relevant anymore

Written on . Posted in Reddit MachineLearning.

We’ve seen the limit of training a 2D CNN on RGB images, resulting in texture bias, exploiting regularities, etc.

  1. CNN ImageNet is bag-of-features (https://openreview.net/forum?id=SkfMWhAqYQ)
  2. CNN ImageNet actually learns to classify texture instead of learning 3D shapes https://openreview.net/forum?id=Bygh9j09KX
  3. Backprop on CIFAR-10 exploit Surface Statistical Regularities to get good test accuracy https://arxiv.org/abs/1711.11561

Is there any meaning for race to find best neural architecture search (NAS) on ImageNet? We are hitting limits of training with monocular RGB images with unknown arbitrary camera poses and intrinsics (focal length, skew, etc). In the end what we get is powerful monocular texture classifier but easily duped by adversarial attacks.

And the found architecture hyperparams is easily overfit to one dataset. In my experience, using Imagenet EfficientNet-B0 to train CIFAR-10 from scratch (not transfer learning like the official paper), resulting accuracy worse than Resnet.

Is there ongoing work to create pose-aware “3D ImageNet”? The closest I can found is probably ShapeNet and various robotics datasets like Princeton SUN-RGBD. But the scale and domain is too small and narrow.

submitted by /u/tsauri
[link] [comments]

[D] What should I read/do to get into machine learning?

Written on . Posted in Reddit MachineLearning.

I’m currently in high school looking to get into machine learning and I was wondering what I should read or what courses I should take.

I am planning on taking the MITx Python course and reading the 2nd edition of Aurélian Géron’s book sometime after it is released. I am also currently eyeing Andriy Burkov’s “The Hundred-Page Machine Learning Book”.

submitted by /u/thetylerwolf
[link] [comments]

[D] Most outlandish application of Transformer Architechture

Written on . Posted in Reddit MachineLearning.

I’m conducting some independent research on the effectiveness of Transformer, Attention, GPT, BERT structure on tasks outside the domain of Language. I was curious to know what the most outlandish implementation you have done may be or the coolest cross-domain application you can think of. Lets see how much we can Transform the Transformer!

submitted by /u/ThatAi_guy
[link] [comments]

How AI Is Helping Protect Taiwan’s Endangered Leopard Cats

Written on . Posted in NVIDIA.

There’s no mistaking why the leopard cat of Taiwan got its name. While only about the size of domestic felines, it sports a beautiful, flower-spotted pattern on its fur.

There’s also no debate about why the leopard cat, the only remaining native wild cat species in Taiwan, is on the edge of extinction.

Fewer than 500 of the leopard cats live in a natural habitat that overlaps with many development projects in the central regions of the island. In an otherwise rural area, the cats are often victims of roadkill due to increased traffic.

To preserve leopard cat populations, the Taiwanese government, animal protection organizations, researchers and AI experts have been working together to save the species.

DT42, a Taiwan-based deep learning startup, and a research team led by Ya-Yu Chiang, assistant professor of mechanical engineering at National Chung Hsing University (NCHU), are collaborating on an AI project initiated by Taiwan’s Directorate General of Highways to detect leopard cats when they near roads and keep them out of harm’s way, reducing roadkills.

Spotting Roadside Leopard Cats

One of the primary challenges of conserving the leopard cat in Taiwan stems from a lack of resources and network infrastructure in the field. Building the network required for cloud-based AI detection isn’t feasible in the animal’s rural habitat.

Traffic signs meant to warn drivers to be cautious of wildlife are in place, but haven’t reduced the number of wildlife collisions. Edge AI systems could provide a more effective way to warn drivers of nearby leopard cats.

DT42, a member of the NVIDIA Inception program, developed a user-friendly, GPU-powered cloud platform through Amazon Web Services to help NCHU researchers train an AI model that identifies leopard cats. Deployed on NVIDIA Jetson TX2 edge devices, the image recognition model can detect leopard cats at wildlife hotspots.

When one of the devices spots a feline getting too close to the road, it sets off a mechanical warning. The alert system plays sounds designed to keep the animals away from passing cars. Additionally, flashing lights on the road also attract the attention of the animals to prevent them from getting on the road.

“After considering all the factors — size, heat dissipation, price, device stability and flexibility — the Jetson TX2 was the best hardware choice on which to deploy our AI model,” said Tammy Yang, DT42’s founder and CEO. “For training, the GPU resources in the AWS cloud platform are easy to use, allowing anyone to upload leopard cat images to help train and refine the neural networks and improve recognition accuracy.”

The company optimized its algorithms for inference at the edge using the NVIDIA Jetson TX2, shrinking the time to detect fast-moving leopard cats to less than half a second. A short response time is critical to spot the animals and sound the alarm before one runs into the road.

Continuing the Conservation Conversation

The average leopard cat roadkill rate from 2015 to 2018 was about one feline killed a month. In the three months since the AI system was deployed in a test area in central Taiwan, there’s been just one leopard cat-related collision — and the animal survived. Earlier this month, the system marked its first recorded instance of deterring a crossing.

Based on these initial results, NCHU researchers and the Taiwanese government hope to roll out additional AI-powered developments.

“Following the success of the leopard cat project, we are going to broaden the monitoring field, and are in discussions with the government to initiate new projects to continuously support leopard cat preservation,” said Chiang.

The researchers also plan to expand the project to other wildlife, including the endangered Chinese ferret-badger and masked palm civet.

“We are devoted to using deep learning to make contributions to the world,” Yang said. “We’re looking forward to seeing more people and organizations joining meaningful conservation projects like this.”

The leopard cat protection project was recently featured in a broadcast by the Taiwan Public Television Service, attracting the government’s attention and sparking discussions about the need for leopard cat conservation laws.

The post How AI Is Helping Protect Taiwan’s Endangered Leopard Cats appeared first on The Official NVIDIA Blog.

[D] Numenta (neurocortical theory group behind the Thousand Brain Theory of Intelligence) is doing an AMA on /r/neuroscience.

Written on . Posted in Reddit MachineLearning.

Link here.

Joining us is Matt Taylor (/u/rhyolight), who is /u/Numenta‘s community manager. He’ll be answering the bulk of the questions here, and will refer any more advanced neuroscience questions to Jeff Hawkins, Numenta’s Co-Founder.

We are on a mission to figure out how the brain works and enable machine intelligence technology based on brain principles. We’ve made significant progress in understanding the brain, and we believe our research offers opportunities to advance the state of AI and machine learning.

Despite the fact that scientists have amassed an enormous amount of detailed factual knowledge about the brain, how it works is still a profound mystery. We recently published a paper titled A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex that lays out a theoretical framework for understanding what the neocortex does and how it does it. It is commonly believed that the brain recognizes objects by extracting sensory features in a series of processing steps, which is also how today’s deep learning networks work. Our new theory suggests that instead of learning one big model of the world, the neocortex learns thousands of models that operate in parallel. We call this the Thousand Brains Theory of Intelligence.

The Thousand Brains Theory is rich with novel ideas and concepts that can be applied to practical machine learning systems and provides a roadmap for building intelligent systems inspired by the brain. I am excited to be a part of this mission! Ask me anything about our theory, code, or community.

Relevant Links:

  • Past AMA:
    /r/askscience previously hosted Numenta a couple of months ago. Check for further Q&A.
  • Numenta HTM School:
    Series of videos introducing HTM Theory, no background in neuro, math, or CS required.

submitted by /u/blueneuronDOTnet
[link] [comments]