I recently got turned on to the idea of Topic modeling by way of Correlation Explanation (CorEx) vis-a-vis this post:

https://medium.com/pew-research-center-decoded/overcoming-the-limitations-of-topic-models-with-a-semi-supervised-approach-b947374e0455

I found an implementation in Python here:

https://github.com/gregversteeg/corex_topic

For those familiar with LDA (Latent Dirichlet Allocation), oftentimes the resulting topics don’t make very much sense. Often the beta probabilities (word-to-topic) are so similar that any classification is arbitrary at best, and more often, simply meaningless.

CorEx provides the ability to “anchor” topics to specific terms, providing a semi-supervised approach to topic modeling. Sounds exciting. Has anyone worked with this algorithm before? Any good results?

Also: has anyone found (or made) an implementation of this CorEx algorithm in R yet?

I’m interested in reconstructing networks/graphs where information is distributed via a gossip/epidemic/rumour protocol where each peer/node sends information to all of its neighbours, so that any piece of information eventually floods the network. Any peer can propagate new information. The number of neighbours of each peer should be much less than the total number of peer.

By connecting to all peers, we can measure the time that each peer sends a piece of information. Hopefully we can use that data to reconstruct the network, i.e. work out which peers are connected to which other peers.

Is anyone aware of research that has been done in this area? I’ve found quite a bit on locating the source of the information given this setup, but I’m really interested in reconstructing the graph.

To appear at NeurIPS 2019.

Modular and programmable architecture search framework that allows you to implement your own search spaces and search algorithms through a consistent API. Reading the Twitter thread will give you a pretty good idea of the main ideas.

Paper: https://arxiv.org/abs/1909.13404

Github: https://github.com/negrinho/deep_architect

Documentation: https://deep-architect.readthedocs.io/en/latest/

Twitter thread: https://twitter.com/rmpnegrinho/status/1179132702819266560

Blog post: https://negrinho.github.io/2019/07/26/introducing-deep-architect.html

Looking to get a few initial users and feedback. Happy to provide support.

I found this notebook to be a really useful summary of some of the new features in TensorFlow 2.0, in particular the gradient tape, tf.function, and custom gradients. I hope the community will find it useful too.

There’s also this crash course notebook for researchers that’s more focused on model building.

I’m building an open source project that combines TensorFlow Serving, ONNX Runtime, and Kubernetes to automate deploying models as autoscaling web APIs on AWS (GitHub). It supports TensorFlow, Keras, PyTorch, Scikit-learn, XGBoost, and other frameworks.

I started working on this when I realized that while there’s been a lot of recent innovation on machine learning libraries like TensorFlow and PyTorch, actually building and shipping production applications is hard. My colleagues and I see a lot of data scientists and developers without DevOps backgrounds struggling to build model serving infrastructure with tools like Docker, Kubernetes, Flask, TensorFlow Serving, ONNX Runtime, and various AWS services. So we decided to combine these tools in an effort to improve the developer experience. It’s available for anyone to download and self-host on their AWS account for free.

I’d love to hear from anyone who has experience deploying models to production. Especially around the tooling and workflows that work well for you.

The latest Lookahead optimizer paper, co-authored by Geoffrey Hinton, used AdamW as its base optimizer, and noted it performing superior to plain Adam. To the best of my knowledge, no complete implementation of AdamW in Keras existed – until now, by me:

Keras AdamW

It includes NadamW and SGDW, and their WR (Warm Restart) counterparts – with cosine annealing learning rate schedule, and per layer learning rate multipliers (useful for pretraining). All optimizers are well-tested, and for me have yielded 3-4% F1-score improvements in already-tuned models for seizure classification. Up to date with Keras 2.3.0.

I recommend giving it a go. Any feedback is welcome.

https://www.theregister.co.uk/2019/09/27/youtube_ai_star/

I found this comment on the article hilarious

Why aren’t you writing these articles slamming universities? I am currently a software engineer in a data science team producing software that yields millions of dollars in revenue for our company. I did my undergraduate in physics and my professors encouraged us to view MIT Open Courseware lectures alongside their subpar teaching. I learned more from those online lectures than I ever could in those expensive classes. I paid tens of thousands of dollars for that education. I decided that it was better bang for my buck to learn data science than in would every be to continue on in the weak education system we have globally. I paid 30 dollars month, for a year, to pick up the skills to get into data science. I landed a great job, paying a great salary because I took advantage of these types of opportunities. If you hate on this guy for collecting code that is open to the public and creating huge value from it, then you can go get your masters degree for $50-100k and work for someone who took advantage of these types of offerings. Anyone who hates on this is part of an old school, suppressive system that will continue to hold talented people down. Buck the system and keep learning!

I’ve been experimenting with unfiltering and de-distortion using GANs and have since produced the same result which can be seen here. This seems strange to me, considering the audio is completely maxed out and inaudible when played back.

Has anyone experienced this with GANs before?

Does it resemble the effects of non-convergence?

Thanks