Hi everyone. I have an interesting question: how would you modify a high-performance neural net to an entirely new architecture.

​

For example, Tesla probably uses some sort of neural net based on convolutions (at least, I think they do; correct me if I’m wrong). On the Tesla autonomy day, Andrej Karpathy mentioned something along the lines of throwing away data after training on it (a system that Elon Musk referred to as Dojo, but the details were not provided).

​

Suppose that in the future, we realize that some new kind of architecture (transformers, for instance) perform significantly better than convolution-based models. How would Tesla (or anyone that’s using convents, for that matter) adapt their self-driving system to the new technology?

​

This shouldn’t be a problem if you keep your training data, but what would companies like Tesla, who have training data on a scale that’s infeasible to store, do? Is there any existing technique to “transfer” weights across completely different architectures?

What is the current SOTA for real time 3d avatars and is there a open source code implementation

http://www.hao-li.com/publications/papers/siggraphAsia2018PAGAN.pdf

really interested in this paper and ones referenced in there

As a recent master’s grad that just entered into the ML/robotics industry, I’m drafting all the observations and lessons I learned throughout this journey. I hope that anyone else reading this doesn’t repeat my mistakes.

NOTE: This might not be applicable to everyone, for obvious reasons.

A little bit about me – A recent master’s grad with almost non-existent industry experience, apart from one summer internship with my advisor at a robotics company she worked for (I did not end up there). My undergrad was in something different altogether and shifted to CV/ML/DL over the past 2 years. I graduated from a top 5 university in robotics/ML. I took few but highly impactful classes throughout my masters and focussed more on research. I did not publish anything but plan to soon. I was always fascinated about autonomous vehicles and am now working at a promising mid-sized company.

Firstly, there is an extreme dearth of good research engineers and companies are willing to shell out a butt-load of money and stocks to lure you in.

Resume – I had put GitHub hyperlinks to my project code (GitHub) and reports (gDrive) which surprisingly caught a lot of traction. When interviewing for the company I work for now, the interviewers took the time to read through a project report I had done last year (I made it a point to read through all my reports before any interview), grilled me on it for one hour and it was the most fun interview I ever had.

Midway through all the interviews, I started tracking each application through an app like Trello. By ‘tracking’ I mean every single technical or coding question asked, how did I answer or approach it and what could have been done better. It goes without saying that the initial interviews were horrible. Having an answer within 5 sec of the question is not what I was conditioned for. I tend to think for a long time (15-20 sec) before I can spew out an answer. However, by tracking each application, I observed that all companies would pretty much ask the same technical questions and before each interview going over that question bank got me through some rounds that I would have otherwise never been able to crack. The link is here. Feel free to add more questions that you’ve come across.

The dreaded coding rounds – At first, they were daunting! Speaking your mind while you try to come up with a working logic, code and test it in 45-60 min is not humanly possible without a ton a practice. That said, my confidence did grow with time and I noticed that almost every question asked was from the easy and medium collection from this question bank. I had to go through all the questions in that collection TWICE before I could muster this round. If you’re interested in the autonomous driving or robotics industry, C++ skills and knowledge is an extreme necessity. The C++ modules in Geeksforgeeks.com was a life-saver. Specifically, you should be able to understand and incorporate in your code – templates, inheritance, pointers, references, std::vector, std::unordered_map, std::move, std::undordered_set, constructor, destructors, virtual functions and have an understanding of how they work behind the scenes. Again, all coding questions are listed in the same doc as the technical questions one (above).

My unemployed friends and I practiced a few coding rounds amongst ourselves. That helped.

Write pseudo-code before diving into code! I like the pseudo-code to be fairly detailed but that’s up to you. Even though you might not be able to finish coding the solution in time, the interviewer has some data points to look at and also makes sure you and the interviewer are on the same page.

Some other points to consider- Towards the end of the interview, ask meaningful questions. What do they work on, what challenges are they facing currently, how did they tackle a previous challenge, etc.

After the interview, please make sure to send a thank you mail to all your interviewers. It goes a long way.

As always, if you have any questions, I’d be glad to answer

Smerity on Twitter proposes red teams for AI companies to protect their ML products from malicious attacks. The context is YouTube, which was found by one researcher to have recommended the RussiaToday video on the Mueller report to an extreme degree over others

https://twitter.com/Smerity/status/1121647755208744963

While I know about all the major network types (CNN, RNN, Transformer, etc.) I can’t really wrap my head around how I would go about answering to a message while keeping the messages before that in consideration, so it would have a conversation with a human instead of just message – answer scheme.

It’s probably pretty obvious but my thought process is stuck right now. Thanks!

A few days ago, I was reminded of this video: https://www.youtube.com/watch?v=Vt4Dfa4fOEY which is an attempt to grasp how e.g. English sounds like to non-speakers. My question is: has there been any attempt to visualise this through how text looks like? For example, how does English look like to Cyrillic readers?

A possible method of replicating this that I have thought of is mapping the differences and nuances of how the language is displayed graphically and in terms of grammar and then reapplying that transformation to create an entirely new language. Is this possible? And if so, has anyone done it?

Any responses will be appreciated.

If I’m working on an ML project that is tackling an issue that has not been addressed before (at least not directly), is it possible to release a paper without benchmarking against other models since there will not be any other models that can be directly benchmarked against? I have barely seen any examples of this so I am unsure what is the correct approach in this situation.

What are current state-of-the-art models or architectures for sentiment analysis/classification of text? For example, classifying if a sentence has a positive or negative sentiment.

Some recent work by frequent participant of this subreddit /u/alexmlamb

Didn’t know he was in Japan doing ML for Japanese work

From the blog post:

MNIST, a dataset built before neural networks were able to read the handwritten numbers on bank checks. It was a difficult task when it was introduced in 1998, but now, in the words of Mila PhD student Alex Lamb, it is “done to death.” Because so many programs can solve it with greater than 99% accuracy, it is no longer useful for showing whether a new program advances the state of the art or not. As a result, researchers have started creating harder spinoff tasks with the same standard conditions, such as EMNIST (a mixture of upper- and lower-case letters along with digits) and FashionMNIST (pictures of clothing items, to be classified as shoes, shirts, etc.) Alex wants to add another criterion to these spinoffs: instead of just making new versions of MNIST which are harder to solve, why can’t we make ones which are useful outside of our own research community?

Alex admits that machine learning systems which can only read the 10 types of characters included in KMNIST would be of little value to literature scholars, but he calls this task “a gateway drug,” expressing the hope that models (and researchers) trained on KMNIST would be competent to move on to the other datasets his team has assembled, like Kuzushiji-49, which contains the 49 most common characters, and Kuzushiji-Kanji, which contains 3,832 rare characters and stands as a credible replacement for the popular Omniglot dataset, introduced for few-shot learning in 2015 and beginning to suffer from the same overuse as MNIST. The final step is to read raw pages of these pre-modern books, which brings the added problems of distinguishing text from illustration and moving between the columns of text in the proper order.

https://medium.com/mila-quebec/choosing-ai-benchmark-tasks-to-benefit-other-fields-90f3a861886e