So I’m premed, but I’m also doing undergrad research in machine learning. Unfortunately, my university is very sparse in terms of machine learning research (there really isn’t any going on).

I found an MD (physician) who happens to have a research role at a Big N company. I checked his LinkedIn, and he’s only been there for 2 years. Because he’s a doctor, I would assume he’s a bit further removed from most standard administrative and workplace stuff than the average Big N employee (am I correct to assume this?). I guess you could consider him to be more of an “adjunct” there – though I really have no idea how it works for a doctor at a tech company.

Anyway, I emailed him saying “if I do really well on the MCAT, and do [list of tasks to learn/practice ML]”, will you put me past the HR resume filter for an interview?

This was his first response: “Sounds reasonable to me. Internships are pretty competitive for some of the teams at [Big N] and some but not all look for students in graduate studies. As long as you enjoy your studies and are passionate about what you are doing, you will be on the right path.”

Me: “It’s my understanding that someone at Big N reaches out to HR, and from there I’m put in touch with a recruiter who starts the interview process. If I pass the interview, HR then confirms with the initial employee/team that there’s an open spot for an intern in the group. If I meet the goals I’ve outlined, would you or someone else on your team be comfortable with reaching out to HR to start the process?”

Him: “i’m not sure if internal referrals change the process for intern selection, which generally has its own application process. feel free to reach out again as you get closer to applying and we can see”

So I’ve met one component of the “goals I’ve outlined” (did really well on the MCAT), and I’ll easily be able to accomplish the rest. Do his responses sound like he was just being polite/he very likely won’t be able to put me past the resume filter? I’m not sure if I should actually consider this an “in” or not hahaha

This is a place to share machine learning research papers, journals, and articles that you’re reading this week. If it relates to what you’re researching, by all means elaborate and give us your insight, otherwise it could just be an interesting paper you’ve read.

Please try to provide some insight from your understanding and please don’t post things which are present in wiki.

Preferably you should link the arxiv page (not the PDF, you can easily access the PDF from the summary page but not the other way around) or any other pertinent links.

Previous weeks :

Most upvoted papers two weeks ago:

/u/MasterScrat: Online Batch Selection for Faster Training of Neural Networks

/u/YoungStellarObject: Layer-Wise Relevance Propagation paper

Besides that, there are no rules, have fun.

I took Keras Tuner for a spin. This post: https://www.curiousily.com/posts/hackers-guide-to-hyperparameter-tuning/ contains examples of how to tune your models using Random Search & Bayesian Optimization (Google Colab notebook included).

Wanted to hear people’s opinions on this as I’ve heard both sides. Specifically in the case where we have a box we can fill in with comments that only the ACs and PCs will see.

A blog post that reviews applying transformers and attention (more broadly) to time series data. Also investigates a bit into why self-attention works in NLP and the necessary modifications to for it to work with time series data.

Setting: I have a convolution of Image I and kernel w, J=conv2D(I,w). I is an image I of size nxn with c channels, while w is a cxcx3x3 matrix, therefore we have c 3×3 filters that can be applied to the image. We use zero-padding so that J has the same dimensionality of I.

What properties does w have to fulfill for the convolution to be invertible? It is clear that at least one such w exists, because 1×1 convolutions are a subset of 3×3 convolutions and since we have c filters, we can encode the identity, which is invertible. It is not enough for the filters to be independent, because as the input has 9c dimensions, we can still loose information.

The Seeker youtube channel put out this video from Columbia University today titled How These Self-Aware Robots Are Redefining Consciousness. Seems like they’re basically rebranding model-based rl as self-modeling. I couldn’t find a full-length paper of their Task-agnostic Self-modeling Machines paper. Also no links to a full-length paper in their citation to their github for this project.

In their paper they use the term “action-sensation pair”. What does this mean? I don’t follow DRL as closely, but I don’t recall ever coming across this term and they didn’t define it here. I assume it’s a tuple of a random action and signal from the environment that corresponds to the robot’s body.

A few quotes from the paper:
“by separating the self from the task, every future experience can be used to refine a common self-model, leading to continuous self-monitoring”

What does that mean?

“As an alternative to self-modeling, many robotic systems do without a model altogether by using end-to-end training for a specific task, applying techniques such as model-free reinforcement learning (4).”

So is this just model-based DRL? Is this just more ml hype?

Hey everyone! 🙂

As the title says, I am looking for both success stories and disappointing failures of applications of modern unsupervised document embedding techniques on actual problems (as opposed to academic benchmarks, toy datasets, academic evaluation tasks, etc.). The main focus is naturally on industry uses for business/product problems, but I would also love to hear about cases from government bodies, non-profits, use in research (with empirical measurement and where document embedding is one of the tools, not the subject of research) and any other “real life” use. I would love to hear about your experience, but connecting me to people you know or even hinting me towards companies or projects you know used these techniques (or tried to) would also be of tremendous help.

What’s in it for you? Well, I’m preparing a talk for the data science track of the CodeteCON #KRK5 conference based on my literature review-y blog post on document embedding techniques, and while I feel I have a pretty good overview of the academic papers, benchmarks and SOTA status up until the most recent stuff to come out in the field at this point in time, I can’t say the same for uses in the industry; I have a partial view from my experience in one ongoing project to actually use this, and experience shared by some of my data scientist friends (all in Israel, naturally) – most of it, so far, by the way, is that averaging (good) word embeddings is a very tough “baseline” to beat.

This is why I thought reaching out to get a better sense of things in the industry world-wide, and enriching my talk with the status of actual successes and industry applications will give people attending my talk more value, and will serve my attempt to make my talk a status report on the topic.

And (coming back to WIIFM) naturally (I think), I intend to share any (share-able) knowledge I accumulate not only in my talk, but also by adding a section dedicated to it to the aforementioned blog post, and maybe even by writing an extended post around it (if enough interesting trends and issues come up). So, hopefully, if you are (like me) interested in this, we might also end up getting, together, a nice overview of where the industry stands at the moment.

What modern techniques (so no variants of bag-of-words or topic modeling techniques) am I talking about? These are the ones that I know of (I’d love to hear about others!):

• n-gram embeddings
• Averaging word embeddings (including all variants, e.g. SIF)
• Sent2Vec
• Paragraph vectors (doc2vec)
• Doc2VecC
• Skip-thought vectors
• FastSent
• Quick-thought vectors
• Word Mover’s Embedding (WME)
• Sentence-BERT (SBERT)
• GPT/GPT2 (can also be supervised)
• Universal Sentence Encoder (can also be supervised)
• GenSen (can also be supervised)

Thank you and cheers,
Shay 🙂