What’s up with LDC pay walling every corpus up to 8,000$? Look at the surreal price of how a year subscription is worth. It is out of anybody’s league. Are there any open alternatives to speech recognition?

​

We have a lot of open-sourced datasets for other domains, computer vision, image processing, medical scans. Only audio datasets are being kept behind a paywall. According to another Reddit post, they had plenty of public funding, and yet nothing changed.

​

They have old stuff as of 1992 and nothing is released for free. Is it common for US unis to have access to this kind of material? They sound more like a corporation than a consortium. Not having an updated and big corpora hinders the whole field from discoveries, because each time there is a research in the domain, there will be time wasted building another dataset.

The idea that the brain learns generative models of the world has been widely promulgated. Most approaches have assumed that the brain learns an explicit density model that assigns a probability to each possible state of the world. However, explicit density models are difficult to learn, requiring approximate inference techniques that may find poor solutions. An alternative approach is to learn an implicit density model that can sample from the generative model without evaluating the probabilities of those samples. The implicit model can be trained to fool a discriminator into believing that the samples are real. This is the idea behind generative adversarial algorithms, which have proven adept at learning realistic generative models. This paper develops an adversarial framework for probabilistic computation in the brain. It first considers how generative adversarial algorithms overcome some of the problems that vex prior theories based on explicit density models. It then discusses the psychological and neural evidence for this framework, as well as how the breakdown of the generator and discriminator could lead to delusions observed in some mental disorders.

​

PDF: http://gershmanlab.webfactional.com/pubs/GenerativeAdversarialBrain.pdf

Some commentary from Smerity about yesterday’s cash infusion from MS into OpenAI:

What is OpenAI? I don’t know anymore. A non-profit that leveraged good will whilst silently giving out equity for years prepping a shift to for-profit that is now seeking to license closed tech through a third party by segmenting tech under a banner of pre/post “AGI” technology?

The non-profit/for-profit/investor partnership is held together by a set of legal documents that are entirely novel (=bad term in legal docs), are non-public + unclear, have no case precedence, yet promise to wed operation to a vague (and already re-interpreted) OpenAI Charter.

The claim is that AGI needs to be carefully and collaboratively guided into existence yet the output of almost every other existing commercial lab is more open. OpenAI runs a closed ecosystem where they primarily don’t or won’t trust outside of a small bubble.

I say this knowing many of the people there and with past and present love in my heart—I don’t collaborate with OpenAI as I have no freaking clue what they’re doing. Their primary form of communication is high entropy blog posts that’d be shock pivots for any normal start-up.

Many of their blog posts and spoken positions end up influencing government policy and public opinion on the future of AI through amplified pseudo-credibility due to Open, Musk founded, repeatedly hyped statements, and a sheen from their now distant non-profit good will era.

I have mentioned this to friends there and say all of this with positive sum intentions: I understand they have lofty aims, I understand they need cash to shovel into the forever unfurling GPU forge, but if they want any community trust long term they need a better strategy.

The implicit OpenAI message heard over the years: “Think of how transformative and dangerous AGI may be. Terrifying. Trust us. Whether it’s black-boxing technology, legal risk, policy initiatives, investor risk, …—trust us with everything. We’re good. No questions, sorry.”

We’ll clarify our position in an upcoming blog post.

I remember him making a huge announcement about it, but I can’t find the post in this subreddit. It seems to have been removed.

If anyone has any information about this, it’d be great to know. Thanks!

I am semi-self taught in machine learning (my degree covered most of the math) and have done a few projects with ML. I think the next step to prove myself to employers would be to work in an enviroment which is a little more professional, and more likely to emulate the ML enviroment at a company.

Since I am not in school anymore, I do not qualify for most internships.

Another thought would be to find some very early stage startups, and offer to work for really cheap, I would be willing to work for minimum wage for a while.

Hi all,

I’ve come across a bit of a problem, and my attempts at coding a solution seem to have been unsuccessful. I could do this quite easily in numpy, but since the calculations have to be done on the Keras.Backend tensor objects, I can’t figure it out.

Sidenote – how do you guys debug code in these backend functions? You can’t print() or step through…

The problem:

For my models, I’m mostly interested in precision and recall, because those are what directly impact a real world application of the model. I typically calculate them as such:

from keras import backend as K def pos_precision_acc(y_true, y_pred): interesting_class_id = 1; class_id_true = K.argmax(y_true, axis=-1) class_id_preds = K.argmax(y_pred, axis=-1) # Replace class_id_preds with class_id_true for recall here accuracy_mask = K.cast(K.equal(class_id_preds, interesting_class_id), 'int32') class_acc_tensor = K.cast(K.equal(class_id_true, class_id_preds), 'int32') * accuracy_mask class_acc = K.sum(class_acc_tensor) / K.maximum(K.sum(accuracy_mask), 1) return class_acc 

So, if I have a one-hot encoded binary problem with labels as [1 0] for negative and [0 1] for positive, the above code would return the precision for the positive class.

However, in a multi-label problem, my classes are multi-hot-encoded – for example a GT label [1 1 0 0] would indicate that the first two are positive, and the last two are negative. The pos_precision_acc() func above would fail at this, because the argmax call limits the output to a single result per instance.

I tried modifying the above function in several ways, but I’m not having any luck. argmax() is out for sure, because it only returns the idx of the first max value, whereas I expect to have multiple. I also tried to .equal after .flatten() on the tensors, but that didnt help either.

Anyone familiar with Keras.backend tensor operations willing it help out? It should actually be pretty easy…I’m just not having much luck.

Note – I could just calculate the precision and recall on the final model (in fact, I do), but I want to use these outputs as a monitored measure in a parameter search (for performance as well as early stopping), so having it work as a typical custom accuracy metric would be ideal.

Hi all,

​

I’m a self-taught ML practitioner, working in the industry. However, I feel that my lack of formal education is hurting me, especially when working with research folks (stats & math heavy topics).

​

I’m quite good at self-learning, so I’d like to revisit all the foundations in the next 12 months. I’m looking to have solid, comprehensive grasp of the most important topics in ML, so that I can at least understand conversations around me.

​

Would appreciate suggestions on how improve the reading list I cooked up (what to add / remove / replace).

Do you think 12 months is a reasonable timeline for the following?

Basics

Calculus

1. Thomas’ Calculus
2. The matrix Calculus you need for deep learning

Linear Algebra

1. No bullshit guide to linear algebra

2. Linear Algebra Done Right

Probability

1. Introduction to Probability

2. MIT RES.6-012 Introduction to Probability, Spring 2018

Information Theory

1. Information Theory: A Tutorial Introduction

2. Information Theory, Inference, and Learning Algorithms

Statistics

1. All of Statistics

2. Casella, G. and Berger, R.L. (2001). “Statistical Inference”

Bayesian Statistics

1. Bayesian Data Analysis

Optimization

1. Introduction to Linear Optimization

2. Convex Optimization

ML

Foundations

1. A Course in Machine Learning

2. Machine Learning: a Probabilistic Perspective

Specifics

Causality

1. The Book of Why: The New Science of Cause and Effect

2. Causality: Models, Reasoning and Inference

DL

1. Deep Learning

2. EPFL Course

Gaussian Processes

1. Gaussian Processes for Machine Learning

NLP

1. Eisenstein’s Notes

2. A Primer on Neural Network Models for Natural Language Processing

Reinforecement Learning

1. Reinforcement Learning: An Introduction

Graphical Models

1. Probabilistic Graphical Models

Recommender Systems

1. Recommender Systems

Probabilistic Programming

1. PPLs