Here’s the video: https://www.youtube.com/watch?v=1zZZjaYl4AA

It seems like the Siraj saga is continuing this time with Siraj posting a video titled “My Apology”.

In the video, he doesn’t talk about plagiarising papers, rather his history of not crediting developers whose code he’s “re-uploaded”.

He briefly outlines his process:

what my practice has been… is I will spend all my time on github.com, which is where code lives – I love Github, and I’ll just be analyzing and looking for code that I think is cool and useful and then once I find some I’ll download it, re-upload it to my Github and then put the name of the author at the very bottom of the Github README. I’ll then make a YouTube video about that code to get more people interested in it, but inside the video itself I’ll be like “hey this is the code” I’ll be very ambiguous I won’t name the author and then sometimes – here’s the embarrassing part – sometimes I’ll literally say “this is my code”

as well as trying to justify his actions:

the reason I did that is selfishness and ego pretty much and I just convinced myself that it wasn’t that way because I was still doing good for other people

In the middle of the video, he speaks about how people who open-source code on Github are like family

I’m your promoter, I’m your, I’m the guy who is trying to uplift you because you’re the one who is actually doing the work. If you like to code and you share that code with people on github, to me you’re like family. To me, I like, I love you I would, I’m not you I ant to be you. But my whole job is to promote you

before reading a list of developers whose code he’s “re-uploaded”.

He then talks about wanting to be a person that promotes and uplifts developers instead of stealing credit

I’m gonna focus, moving forward I’m gonna focus on promoting other developers, crediting them in the video, uplifing other researchers and developers and having just a huge focus on that, rather than trying to be the guy, you know the king who’s the one who’s doing all of this stuff, you know.

Finally, he mentions that he’s not striving to be famous anymore

but now it’s like I don’t care about you know, being famous at all seriously, like obviously I don’t

​

I had a hard time believing his apology, especially considering the ease with which he lied and the lengths he went to to ensure you couldn’t find the original author of the code (i.e. not clicking the fork button that links to the original repository). Hopefully nonetheless it results in more authors being credited.

Hello world;

I want to optimize my model based on two contraints. My goal is to find the W matrix that minimize this abs(W*x – Y). This can be done using gradient descent or some other models. In addition i want the model to be biased to some points of (x , y) based on a contraint that i will give. such as x*2 = y …. or something else, which is available just for some samples in the data.

Overall, i want that the model will be regularized by a second constraint that will keep it biased to some samples but not overfitted on under fitted on the other samples.

Which cost-function or optimization algorithm should i look for?

I had a little thought.

If we were to construct a godel numbering of a language then we can be sure that any random program we generate is valid.

There is such a language called Jot. It consists of a binary string which maps to the combinatory logic.

https://www.nyu.edu/projects/barker/Iota/

A compiler taking us from unlambda to jot exists:

https://tromp.github.io/cl/lazy-k.html

So one could take all the programs written in unlambda and use them as training material, rather than using a custom domain-specific language. Also, it is possible to map from various other languages to unlamda. So we have a pipe all the way down to a representation that has no possible syntax errors.

Where every string represents a valid program we do not have to worry about invalid syntax. We can simply evaluate the program for a limited time and examine the output.

As well as applying this technique to the existing program synthesis techniques you could also construct a GAN.

Then use the manifold of pre-existing programs constructed by it to inform the search for useful programs.

I’m sure this is all pie in the sky, but I have to let these ideas out onto the internet or I start to self destruct!

Hi, I attempted to recreate the results from the multi-modal self supervised learning paper – “Ambient Sound Provides Supervision for Visual Learning” by Owens, et. al.

Here is the code along with my detailed report on it – https://github.com/rowhanm/ambient-sound-self-supervision

Some key things I learned during this:

1. Sound is an interesting supervision signal for image/scene recognition.
2. Representation of sound matter quite a lot. I tried using MFCCs along with the Statistical sound summaries proposed in the original paper and saw some increase in the downstream task evaluation.
3. I don’t know what makes an optimal sound representation. Perhaps these can be learned.
4. Visualizing the top activated images from learned by using sound as supervision, we see that the model somewhat understands the context in which an object occurs (like fish splashing in water/ man holding some kind of fish) and not the details of the object itself.
5. Can we combine both audio and images as supervisory signals? What are some good papers on this?

Would love to hear some comments/criticisms/thoughts on this.

Thanks!

We are organizing a workshop in which people will download their chat logs (using Chatistics), and train a GPT2 model that talk like them.

But not everyone may be comfortable working with such data, or maybe some people don’t use IM. So as a backup, we are looking for other sources of one-to-one conversations.

We thoughts about using “celebrity” interview, eg a journalist talking with Donald Trump, or with Tom Cruise, or with Richard Feynman – it would be quite interesting to see how their GPT2 models would sounds like!

Any pointer to such datasets?

Quote of this post can be found here:

https://twitter.com/notmisha/status/1208569457511030784

The tweet is referring to this article:

“This AI researcher is trying to ward off a reproducibility crisis”

https://www.nature.com/articles/d41586-019-03895-5

To give some context: Joelle Pineau’s group authored a paper called “Deep Reinforcement Learning that Matters” (https://arxiv.org/abs/1709.06560) which performed a meta-analysis of how reproducible papers in the deep RL community were. They found that reproducing results were surprisingly difficult and the methods these papers used to report results were questionable.

I personally think Joelle handled the topic very diplomatically and in her talk made efforts not to isolate any individual or group. Despite her efforts, the work of one or two groups did stand out.

I thought I would start a discussion on the merits of reproducibility and reflect on how important it really is. If it is important, what makes it important? Is it over emphasized over the bigger picture? What should we strive for in the future?

I am looking for resources regarding people tracking with good occlusion handling for a research project I have.

In a recent project I’ve found that I need to use gradient clipping to stop my model from suddenly NaNing out after training for a while, and it also seems to have a positive regularizing effect. I’ve read that the clipping cutoff should be trained as a hyperparameter, but is there any rule of thumb or results on setting the learning rate when using gradient clipping? It seems like if the gradient clipping cutoff is too low, most gradient steps will be of the same magnitude.

Hi, I’m new to machine learning. I am wondering if it is possible to run MuZero on my mac. Or I need to download a software for this?

Thank you.

So I have been recently admitted to a relatively new Masters in Data Science and Artificial Intelligence at Saarland University to start in April of next year. I have read great things about the CS departments at Saarland, and the presence of the Max Planck Institute in Informatics + the German Research Center for Artificial Intelligence is really attractive for someone interested in developing an ML career, specially to someone interested in the Computer Vision + ML area like me, since the machine learning laboratory is very focused on this.

Besides applying to Saarland, I was planning to apply to Tuebingen’s Masters in Machine Learning when applications open next year. However, since Saarland admitted me this early, I will have to make a decision before I hear about the results of Tuebingen.

If I accept the offer at Saarland, I would be looking to work as a Research assistant in one of these institutes (ideally with Prof. Bernt Schiele in the Computer Vision and Machine Learning lab) and after graduating, I would search for Machine Learning engineer jobs or if I really enjoy my experience with research, maybe pursue a PhD.

The thing is that at the moment, I see myself more in the future in an industry environment rather than in academia. Because of this, what attracts me from Tuebingen is the presence of Amazon, Bosch, the Max Planck Institute for Intelligent Systems, and all of the current hype it is gaining recently from the “Cyber Valley” consortium (https://cyber-valley.de/), all of which make Tuebingen look like a place where I could get an internship in one of these companies and then more easily transition into industry. In fact, some professors that previosly worked

Both universities seem to be really strong in ML research with amazing professors and opportunities in their respective laboratories, despite this, I can’t help but feel that if I choose Saarland, I may be missing out on internships in Industry such as the ones I mentioned. Am I overthinking the weight of the nature internships in a Masters? (ML Research in laboratories vs ML Industry)

It is also important for me to mention that I am from Venezuela, and while it may sound easy for some people to just defer the admission for the next semester to get an answer from Tuebingen and make a better decision, for me it can be a risk since the country can get worse economically and politically at any moment. Going to Saarland means starting in April, going to Tubingen (If I were to get accepted) means starting in October.

I don’t know if this is the right place to ask but since my doubts are based on the quality of both ML programs and the opportunities available at each university are in ML positions I thought I’d raise my concerns here.