Hi everyone. I recently trained an image classifier on a Japanese character dataset called KMNIST and achieved 97% validation accuracy within a few minutes using a suite of modern deep learning tools and techniques.

I explain all the techniques my blog post, published on the weights and biases site.

I kept this article relatively short and straightforward, so it should be quite accessible to beginners and is likely to improve the performance of your deep learning models.

Primarily, I used the learning rate finder and 1cycle learning rate policy taught by Jeremy Howard in the fast.ai practical deep learning for coders course along with visualization and monitoring tools from a library called Weights & Biases.

Hope you enjoy (and your poor GPU that’s been computing for days) it!

https://medium.com/ai%C2%B3-theory-practice-business/ai-scholar-towards-the-internet-of-robotic-things-f6ca55859691

Abstract: Internet of Things (IoT) and robotics cannot be considered two separate domains these days. Internet of Robotics Things (IoRT) is a concept that has been recently introduced to describe the integration of robotics technologies in IoT scenarios. As a consequence, these two research fields have started interacting, and thus linking research communities. In this paper we intend to make further steps in joining the two communities and broaden the discussion on the development of this interdisciplinary field. The paper provides an overview, analysis and challenges of possible solutions for the Internet of Robotic Things, discussing the issues of the IoRT architecture, the integration of smart spaces and robotic applications.

Hey all

Since Bert is a language model, by default do we obtain sentence or word embedding?

I actually plan to use these embeddings for various NLP related tasks like Sentence Similarity, NMT, Summarization etc.

​

Also :

• If it by default gives Sentence Level Embedding then what is the process to get Word Embedding ( any refer might help here ).
• If we obtain Word Embeddings then do we just simply do Mean/Max pooling to get Sentence embedding or are there better approaches?

Hey,

So I’m about to build my first ever PC, which I’m going to use mainly for NLP tasks. My budget is around €1100 ($1200). I’ve read Tim Dettmers’ blog post about GPUs, however he didn’t update it yet and RTX Super cards got released.

At this point I’m thinking about going with an RTX 2070 Super, I’m not sure however if the price differences between the 2060 Super, 2070 and the 2070 Super are even worth it. A quick comparison:

​

Source ​

Also, this is what I came up with so far. If anyone would add any suggestions on this as well, it would be greatly appreciated.

​

PCPartPicker Part List

Link: https://arxiv.org/pdf/1907.02711.pdf

​

In this paper, we introduce the concept of Prior Activation Distribution (PAD) as a versatile and general technique to capture the typical activation patterns of hidden layer units of a Deep Neural Network used for classification tasks. We show that the combined neural activations of such a hidden layer have class-specific distributional properties, and then define multiple statistical measures to compute how far a test sample’s activations deviate from such distributions. Using a variety of benchmark datasets (including MNIST, CIFAR10, Fashion-MNIST & notMNIST), we show how such PAD-based measures can be used, independent of any training technique, to (a) derive fine-grained uncertainty estimates for inferences; (b) provide inferencing accuracy competitive with alternatives that require execution of the full pipeline, and (c) reliably isolate out-of-distribution test samples.

Intel’s ultra-efficient AI chips can power prosthetics and self-driving cars They can crunch deep learning tasks 1,000 times faster than CPUs.

https://www.engadget.com/2019/07/15/intel-neuromorphic-pohoiki-beach-loihi-chips/

Even though the whole 5G thing didn’t work out, Intel is is still working on hard on its Loihi “neuromorphic” deep-learning chips, modeled after the human brain. It unveiled a new system, code-named Pohoiki Beach, made up of 64 Loihi chips and 8 million so-called neurons. It’s capable of crunching AI algorithms up to 1,000 faster and 10,000 times more efficiently than regular CPUs for use with autonomous driving, electronic robot skin, prosthetic limbs and more.

The Loihi chips are installed on a “Nahuku” board that contains from 8 to 32 Loihi chips. The Pohoiki Beach system contains multiple Nahuku boards that can be interfaced with Intel’s Arria 10 FPGA developer’s kit, as shown above.

Pohoiki Beach will be very good at neural-like tasks including sparse coding, path planning and simultaneous localization and mapping (SLAM). In layman’s terms, those are all algorithms used for things like autonomous driving, indoor mapping for robots and efficient sensing systems. For instance, Intel said that the boards are being used to make certain types of prosthetic legs more adaptable, powering object tracking via new, efficient event cameras, giving tactile input to an iCub robot’s electronic skin, and even automating a foosball table.

The Pohoiki system apparently performed just as well as GPU/CPU-based systems, while consuming a lot less power — something that will be critical for self-contained autonomous vehicles, for instance. ” We benchmarked the Loihi-run network and found it to be equally accurate while consuming 100 times less energy than a widely used CPU-run SLAM method for mobile robots,” Rutgers’ professor Konstantinos Michmizos told Intel.

Intel said that the system can easily scale up to handle more complex problems and later this year, it plans to release a Pohoiki Beach system that’s over ten times larger, with up to 100 million neurons. Whether it can succeed in the red-hot, crowded AI hardware space remains to be seen, however.

It is well known that, when we train a word embedding using one of various models, we can perform a vector arithmetic that reflects semantics. Is this just an empirical result, or does a model and/or a training loss guarantees such an embedding?