Hey everyone,

I’ve put up a new blog post, Understanding the Neural Tangent Kernel, that aims to distill the ideas behind the neural tangent kernel that is making waves in recent theoretical deep learning research. A large portion of the talks in the recent Workshop on Theory of Deep Learning at the Institute for Advanced Study were based on ideas related to the neural tangent kernel. This is a slightly long post, as it involves a fair bit of math (you can skip some of the proofs though). A bit of linear algebra background is necessary to fully grasp what is going on here, but I hope that my visualizations can help with that.

Code for the experiments and animations: https://github.com/rajatvd/NTK

Feedback and suggestions are welcome!

So apparently Colab has P100 GPUs now. I can’t find any announcements and/or discussions about this, seems like google decided to do this without any notice?

I am aware of the famous example of Embedding(King) – Embedding(Man) + Embedding(Woman) = Embedding(Queen). From this example, we can say that the characteristic of “royalty” has been understood.

I guess in a way I am trying to interpret the hidden layer neurons which might not always have meaning.

I have looked into techniques like SHAP and LIME but I’m still to plug the concepts together.

My last post asked for help with SSH and training a model on a remote server.

In this post can I just get a confirmation for what I think I need to do?

1) build docker image for project

2) transfer image to server using scp

3) log in to server

4) select a machine to compute on

5) use linux command to run docker image

Did I get this right? If not, can someone please tell me what I got wrong? Thank you for your help everyone!

Often a Generalized Linear Model is developed using a neural network with one or more Fully Connected layers, but I can’t understand why.

Here you can find an example:
https://github.com/quertenmont/GLMPerf/blob/master/Airline%20Delay%20with%20a%20GLM%20in%20Keras.ipynb

How are GLMs and neural network with Fully Connected layers related to each other?

I can’t sleep until I see the result……………. plz…..

There are some really good tools for tagging data and creating datasets for ML like doccano. Most of these are web GUIs though which I find hard and annoying to extend.

Some time back, I made a system for tagging within Emacs which then got extended into a sort of framework. If you live inside Emacs and are willing to spend some time creating a fast tagging workflow, you can try tog which lets you create custom data taggers by writing a few rendering and parsing functions. I have been using it personally for the following:

1. NER tagging
2. Audio/text intent tagging
3. Transcribing
4. Voicing texts or parses
5. Triplet-ish song similarity tagging

It’s all Emacs Lisp, so you can extend everything. A recent example, I hit an active learning backend on each save and get next to-tag data points which are hard according to the then tagged dataset + model.

Hello,

While working with Kaldi for speech recognition, we found relatively few serving solutions, most of which haven’t worked out nicely for us so we have been building a simple server here.

Although, the program should not be considered stable, is missing many features we want (like supporting on-the-fly HLCG creation for providing speech contexts like in gspeech) and we are going relatively slow on it, it can be used without unreasonable effort if you have regular trained chain models.

Feel free to suggest changes, improvements, PRs etc.

And what are my chances of being accepted?

SAT: 1500-1550 2 sat subject tests

5 ap courses; 3 5s and 2 4s

Admissions avg around the 95% mark

I have several silver and bronze medals medals on kaggle and I’m ranked in in the top 500 of all kagglers.

Several large ml projects, as well as contributions to major ml open-source projects

What should I do to get a better chance of getting in? I don’t have any publications at any conferences so maybe I should start there?