Does anyone know of a list of co-located events to NeurIPS? So far, I’ve seen:

• OPT2019
• MLCB
• Queer in AI
• LatinX in AI
• WiML
• Black in AI
• EMC2

What did I miss?

(reposted because the tag was in the wrong place in the title)

I’m studying to become a video game designer, and these advances in machine learning are proooobably gonna affect my chosen career at some point. I know there’s stuff already, like AlphaStar being a Pretty Darn Good SC2 AI, and this paper where character face sliders are preselected based off of a photo.

There’s one application I can already see happening in the short term: right now, in story driven games, characters awkwardly avoid referring to the player by their name (with one notable exception in Fallout 4). But, I could easily see AIs trained off of a game’s voice actors adjusting in-game recorded dialogue to include synthesized clips of the player’s name. Even on lower end machines, these clips could be generated in the background so they’re ready when needed.

I listened to a very interesting talk at MAIS 2019 last Friday about a novel approach to learn DAG using neural networks (all the details in this paper here: arXiv:1906.02226). It’s far from my actual discipline of sensor design and data processing, but I still sent to speak with the author at the poster session afterwards.

We didn’t go into the details of the technique, instead we had a discussion about how there doesn’t seem to exist a usable definition of causality in terms of graph analysis. He said that causality is something we all kind of agree on, but that we can’t define. For example, the direction of the causality arrow between the average temperature and the altitude of a city is clear. If we magically changed the altitude, the average temperature would change, while if we magically changed the average temperature, the altitude wouldn’t change. Therefore, the direction of causality is from “altitude” to “average temperature”.

From my readings in cosmology and thermodynamics, I realized there seems to be a very similar concept that would benefit from being shared here. At least I hope so, it’s sometimes hard to know the exact boundaries 😉

​

Here is a proposed definition for causality: a causal relationship R from set A to set B is a function transforming A into B such that information that was available in A is lost when working with the set B.

It means that a system that can be uniquely described in A cannot be uniquely described in B and it is impossible to know exactly which element from A was mapped to an element from B. In that sense, the set A has a greater information content than the set B and the function R reduces the amount of information available in the set.

In the case of large scale phenomenon where classical physics tells us that each process is deterministic (ie: maps one unique state to one other unique state), but we must also take into account the passage of time. The chronological order of the events dictates the direction of causality. This is where it gets interesting in my opinion: the arrow of time as defined by physicist Sean Carroll (book, multiple articles) is deeply linked to the evolution of the entropy of the universe. The entropy itself is closely related to information content, from the definition of the Shannon Entropy.

It all comes back to the fact that causality points from a set containing more information to a set containing less information, and not the other way around.

I hope it makes sense and there’s probably a better way to write it all and make the explanation clearer, but I feel like there’s something useful about that.
For example, if we find a causal link between two variables that seems to go against the above definition, it probably means that we are missing some information about the first set, or that the second set is not described in a very “compact” way and has redundant information.

Thanks for your comments!

Hello r/MachineLearning,

>Link to the Twitter Plot Bot<

I’ve seen a post about the Trump bot and I don’t know why it has been removed. But I’d like to share the bot that u/Schnox have created by fine-tuning GPT-2 (774M) with r/WritingPrompts. Then we fed it with IMDB plotlines of the top 250 movies to create new narratives for all the best movies we like and love.
The results are not chosen, this is unfiltered output of the algorithm.

We’d appreciate your feedback on the project! Feel free to post questions or check out the code at github.com/hansbambel/storytelling_gpt2

Abstract – Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard data-sets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.

Page – https://arxiv.org/abs/1905.02161

PDF – https://arxiv.org/pdf/1905.02161.pdf

Has anyone read the paper and experienced robustness issues with deployment of Batchnorm models in the real world?

While reading the other thread (https://www.reddit.com/r/MachineLearning/comments/d1ooem/d_when_the_ai_professor_leaves_students_suffer/), someone linked to a H1B database of professor salaries which showed assistant professors are getting paid ~120K max at a top school like CMU (unless in the business school).

Is this really true? Is there variance among schools? I am very surprised since these candidates could easily make 300K-500K as a research scientist at the big tech companies. Granted, there is still old school prestige attached to being a professor, but it seems like they are leaving a lot of money on the table.

See my repository below, it’s just plain python that will work in any 3.x version as far as I know, and requires no extra installs (uses only the base python libraries, all you need is python installed).

It’s a simple feed forward network, but hopefully it can shed light on how the forward and backward pass works.

https://github.com/sean-gribben/pynet