So as we know for MAML-like meta learning, many works inner-update the top layers while meta-update the backbone. Is there any paper showing the results about the difference for inner-updatting different number of top layers?

Paper: Open access in Science Advances

Code: Available on GitHub

Lately, there has been a lot of cross-pollination of ideas between different areas of physical and numerical science and the field of machine learning. This has lead to interesting demonstrations of optimizing physical models using machine learning frameworks, but also to the development of a number of exciting new machine learning models (e.g. neural ODEs, Hamiltonian neural networks, etc) that borrow concepts from physics.

My group has been particularly interested in the view point that physics itself can be used as a computational engine. In other words, we’re interested in physical systems that can serve as hardware accelerators or as specialized analog processors for fast and efficient machine learning computations.

In our paper that was recently published in Science Advances (open access) we have shown that the physics of waves map directly into the time dynamics of recurrent neural networks (RNNs). Using this connection, we demonstrated that an acoustic / optical system (through a numerical model developed in PyTorch) could be trained to accurately classify vowels from recordings of human speakers. Essentially, we launched the vowel waveforms into the physical model and allowed the optimizer to add and remove material at 1000’s of individual points within the domain, essentially acting as the weights of the model.

Because this machine learning model actually corresponds to a physical system, it means that we could take the trained material distribution and “print it” into a real physical device. The result would be something like an ASIC (application specific integrated circuit), but for a specific RNN computation. We’re really excited about these results because they point to being able to perform complex recurrent machine learning calculations completely passively, with no energy consumption, aside from the energy carried by the pulse itself.

Hi everyone, I made a video on how to record or validate clips for the Common Voice project, you guys can check it out at end of this post, the project has come a long way, have fun.

​

https://voice.mozilla.org

​

https://www.youtube.com/watch?v=s9TF4jXxwFM

I ran an experiment comparing both the latency and cost of serving realtime inference with GPT-2 on AWS using GPUs and CPUs. I also added a bit of context for how latency correlates to real world product requirements:

https://towardsdatascience.com/how-much-difference-do-gpus-make-in-model-serving-c40b885ac096?

It’s no secret that ML and AI can deliver a great value in Life Sciences.

I have spent past 10 years as a researcher in the field of Computational Biology working very close to wetlab.

Now, together with a lovely bunch of peops, I am building virtual community dedicated to AI in Life Sciences in the format of a not-for-profit organization. We now unite several people working in the field in the UK and Switzerland. The mission: share tech expertise, code, links and papers.

For the sake of experiment at the moment we are running fully from a telegram channel, come have a look at https://ails.institute and join us! All ideas are welcome!

I want to deeply understand maths related to deep learning and machine learning. I am mentioning my background so someone can suggest me some materials for learning purpose.

Backgroud: Undergrad completed.I have completed deep learning specialization. I can calculate derivation of different things like sigmoid and thanks to deep learning specialization, i also know how actually neural network work. I have also implemented few research paper on my own and open sourced the code.

But, now i want to re-learn calculus, linear algebra and probabilities for better understanding of dl and ml methods. Why we choose sigmoid or tanh, how actually relu is implemented and how actually auto-grad works. things like that.

I searched for some book on calculus and linear algebra but recommendations are to off. Many people recommended Spivak and i started reading it. but starting chapter looks too much dull and only explains theoretically. I want materials where i can understand thing pratically. If possible, then with programming exercises.

please suggest for calculus, LA and Stats.