Embedding layer is essentially a lookup into an array while fully connected layer is full matrix multiplication.
However, for one-hot encoding, there is one 1 value in the whole input layer, so when you matmul, you are getting different weights for every input neuron. Therefore, I believe it should be the same as an array lookup. Is this true?
submitted by /u/happyhammy
[link] [comments]
An accurate diagnosis is key to treating cancer — a disease that kills 600,000 people a year in the U.S. alone — and AI can help.
Common forms of the disease, like breast, lung and prostate cancer, can have good recovery rates when diagnosed early. But diagnosing the tumor, the work of pathologists, can be a very manual, challenging and time-consuming process.
Pathologists traditionally interpret dozens of slides per cancer case, searching for clues pointing to a cancer diagnosis. For example, there can be more than 60 slides for a single breast cancer case and, out of those, only a handful may contain important findings.
AI can help pathologists become more productive by accelerating and enhancing their workflow as they examine massive amounts of data. It gives the pathologists the tools to analyze images, provide insight based on previous cases and diagnose faster by pinpointing anomalies.
Paige.AI is applying AI to pathology to increase diagnostic accuracy and deliver better patient outcomes, starting with prostate and breast cancer. Earlier this year, Paige.AI was granted “Breakthrough Designation” by the U.S. Food and Drug Administration, the first such designation for AI in cancer diagnosis.
The FDA grants the designation for technologies that have the potential to provide for more effective diagnosis or treatment for life-threatening or irreversibly debilitating diseases, where timely availability is in the best interest of patients.
To find breakthroughs in cancer diagnosis, Paige.AI will access millions of pathology slides, providing the volume of data necessary to train and develop cutting-edge AI algorithms.
To make sense of all this data, Paige.AI uses an AI supercomputer made up of 10 interconnected NVIDIA DGX-1 systems. The supercomputer has the enormous computing power of over 10 petaflops necessary to develop a clinical-grade model for pathology and, for the first time, bridge the gap from research to a clinical setting that benefits future patients.
One example of how NVIDIA’s technology is already being used is a recent study by Paige.AI that used seven NVIDIA DGX-1 systems to train neural networks on a new dataset to detect prostate cancer. The dataset consisted of 12,160 slides, two orders of magnitude larger than previous datasets in pathology. The researchers achieved near perfect accuracy on a test set consisting of 1,824 real-world slides without any manual image-annotation.
By minimizing the time pathologists spend processing data, AI can help them focus their time on analyzing it. This is especially critical given the short supply of pathologists.
According to The Lancet medical journal, there is a single pathologist for every million people in sub-Saharan Africa and one for every 130,000 people in China. In the United States, there is one for rohly every 20,000 people, however, studies predict that number will shrink to one for about every 30,000 people by 2030.
AI gives a big boost to computational pathology by enabling quantitative analysis of the study of structures seen under a microscope and cell biology. This advancement is made possible by combining novel image analysis, computer vision and machine learning techniques.
“With the help of NVIDIA technology, Paige.AI is able to train deep neural networks from hundreds of thousands of gigapixel images of whole slides. The result is clinical-grade artificial intelligence for pathology,” said Dr. Thomas Fuchs, co-founder and chief scientific officer at Paige.AI. “Our vision is to help pathologists improve the efficiency of their work, for researchers to generate new insights, and clinicians to improve patient care.”
Feature image credit: Dr. Cecil Fox, National Cancer Institute, via Wikimedia Commons.
The post Paige.AI Ramps Up Cancer Pathology Research Using NVIDIA Supercomputer appeared first on The Official NVIDIA Blog.
Only a few years ago, web search was simple. Users typed a few words and waded through pages of results.
Today, those same users may instead snap a picture on a phone and drop it into a search box or use an intelligent assistant to ask a question without physically touching a device at all. They may also type a question and expect an actual reply, not a list of pages with likely answers.
These tasks challenge traditional search engines, which are based around an inverted index system that relies on keyword matches to produce results.
“Keyword search algorithms just fail when people ask a question or take a picture and ask the search engine, ‘What is this?’” said Rangan Majumder, group program manager on Microsoft’s Bing search and AI team.
Of course, keeping up with users’ search preferences isn’t new — it’s been a struggle since web search’s inception. But now, it’s becoming easier to meet those evolving needs, thanks to advancements in artificial intelligence, including those pioneered by Bing’s search team and researchers at Microsoft’s Asia research lab.
“The AI is making the products we work with more natural,” said Majumder. “Before, people had to think, ‘I’m using a computer, so how do I type in my input in a way that won’t break the search?’”
Microsoft has made one of the most advanced AI tools it uses to better meet people’s evolving search needs available to anyone as an open source project on GitHub. On Wednesday, it also released user example techniques and an accompanying video for those tools via Microsoft’s AI lab.
The algorithm, called Space Partition Tree And Graph (SPTAG), allows users to take advantage of the intelligence from deep learning models to search through billions of pieces of information, called vectors, in milliseconds. That, in turn, means they can more quickly deliver more relevant results to users.
Vector search makes it easier to search by concept rather than keyword. For example, if a user types in “How tall is the tower in Paris?” Bing can return a natural language result telling the user the Eiffel Tower is 1,063 feet, even though the word “Eiffel” never appeared in the search query and the word “tall” never appears in the result.
Microsoft uses vector search for its own Bing search engine, and the technology is helping Bing better understand the intent behind billions of web searches and find the most relevant result among billions of web pages.
Essentially a numerical representation of a word, image pixel or other data point, a vector helps capture what a piece of data actually means. Thanks to advances in a branch of AI called deep learning, Microsoft said it can begin to understand and represent search intent using these vectors.
Once the numerical point has been assigned to a piece of data, vectors can be arranged, or mapped, with close numbers placed in proximity to one another to represent similarity. These proximal results get displayed to users, improving search outcomes.
The technology behind the vector search Bing uses got its start when company engineers began noticing unusual trends in users’ search patterns.
“In analyzing our logs, the team found that search queries were getting longer and longer,” said Majumder. This suggested that users were asking more questions, over-explaining because of past, poor experiences with keyword search, or were “trying to act like computers” when describing abstract things — all unnatural and inconvenient for users.
With Bing search, the vectorizing effort has extended to over 150 billion pieces of data indexed by the search engine to bring improvement over traditional keyword matching. These include single words, characters, web page snippets, full queries and other media. Once a user searches, Bing can scan the indexed vectors and deliver the best match.
Vector assignment is also trained using deep learning technology for ongoing improvement. The models consider inputs like end-user clicks after a search to get better at understanding the meaning of that search.
While the idea of vectorizing media and search data isn’t new, it’s only recently been possible to use it on the scale of a massive search engine such as Bing, Microsoft experts said.
“Bing processes billions of documents every day, and the idea now is that we can represent these entries as vectors and search through this giant index of 100 billion-plus vectors to find the most related results in 5 milliseconds,” said Jeffrey Zhu, program manager on Microsoft’s Bing team.
To put that in perspective, Majumder said, consider this: A stack of 150 billion business cards would stretch from here to the moon. Within a blink of an eye, Bing’s search using SPTAG can find 10 different business cards one after another within that stack of cards.
The Bing team said they expect the open source offering could be used for enterprise or consumer-facing applications to identify a language being spoken based on an audio snippet, or for image-heavy services such as an app that lets people take pictures of flowers and identify what type of flower it is. For those types of applications, a slow or irrelevant search experience is frustrating.
“Even a couple seconds for a search can make an app unusable,” noted Majumder.
The team also is hoping that researchers and academics will use it to explore other areas of search breakthroughs.
“We’ve only started to explore what’s really possible around vector search at this depth,” he said.
The post As search needs evolve, Microsoft makes AI tools for better search available to researchers and developers appeared first on The AI Blog.
Hello all! Nakamoto & Turing Labs is developing a shared distributed computing platform hosted on a blockchain network, with a focus for Machine Learning and other computationally intense projects. We are conducting initial Market Research and would love to hear your thoughts! Please take a look at the survey below. https://jamesmcconchie.typeform.com/to/SrpuOw
submitted by /u/NakamotoTuringLabs
[link] [comments]
![[R] COCO-GAN: Generation by Parts via Conditional Coordinating](https://b.thumbs.redditmedia.com/oO1be2eDsTgDvOKOsgfxgNtsUErF8xyP5hPQ3pY43_I.jpg "[R] COCO-GAN: Generation by Parts via Conditional Coordinating") |
https://i.redd.it/savexxzumdy21.png https://i.redd.it/2j0btvnvmdy21.gif TL;DR We show that it is possible to generate images by parts with a conditional coordinate mechanism. Our model preserves the state-of-the-art FID score and provides multiple interesting applications. Project page: https://hubert0527.github.io/COCO-GAN/ Paper:
We will release our code soon here: https://github.com/hubert0527/COCO-GAN Welcome for any question and comment 🙂 submitted by /u/hubert0527 |
I’m a bit confused by the derivation of a lemma in the appendix of the InfoGAN paper (Appendix A lemma). I have added the link to the paper here (https://arxiv.org/abs/1606.03657)
In particular, between third and fourth line, f(x,y) turns into f(x’,y). How does that work?
submitted by /u/mellow54
[link] [comments]
For example, if you wanted to encode months as real numbers, you could encode january as 0 and december as 11, but that doesn’t capture that december is closely related to january. How would you encode these kind of features? Months are just a toy example, but imagine a real numbered feature that has cyclical relationsip. e.g. range of (-1, 1) where -1 = 1.
submitted by /u/happyhammy
[link] [comments]
