To the more experienced ML professionals in the community – I want to hear about what you use to build scalable ML pipelines at your work. I’ve been building models for a while now for research purposes. However, I’m totally in the dark about the other side of things, namely how to engineer and deploy data/ML pipelines that are scalable and provide reproducible results (whatever that may be in this context).

I’ve looked at scikit-learn pipelines, but they seem a bit clunky while handling pandas dataframes (although workarounds do seem to exist). Another sentiment I hear is that they don’t scale well to large datasets.

Care to part with your wisdom? Thanks!

[Posted this yesterday, but i think it got removed for some reason]

I have data that has equal variance along each of the target dimensions, but if i analyze the results of training i notice that that my trained model does not have the same error variances along each of the dimensions, which means some dimensions are more erroneous than others.

Does this say anything about me missing some sort of regularization, or is this not something I should be fretting over? Is there a way I can regularize to make sure the errors along different dimensions have the same variance?

I want to apply DQN in environment where action space is continuous. I even don’t know is it possible or not, is it good idea or not ? If yes then what should be my function approximator’s (neural net) output dimensions ? Thank you in advance.

I have been working with sequence to sequence models for a while now. And the attention mechanism is a cornerstone to most seq2seq setups. It is typically added as an explicit part of the network, which has its advantages if one might want to modify how attention is computed without messing the encoder or the decoder code.

However, I am wondering if there would be ways to compute attention weights through introspection of the weights of RNN cells. In very coarse terms, an LSTM cell for instance has a forget gate. Therefore, if we can see when what part of the sequence is forgotten, this could give us an indication of what the attention weights might.

Now, if such a mechanism existed, here are my assumptions of the properties it would need to have: – the RNN would need to be bidirectional – the RNN cells would need to have a forget gate

I also think that the biggest challenge would be to correlate changes in the RNN state and explicit parts of the sequence being analyzed.

Is there work being done on this? Or are there reasons why it cannot work?

It’s been 6 months since I started learning deep learning. Finally last week I implemented DQN for atari games. It is in its simplest form with 3 conv layers, 2 dense layers, replay memory and fixed targets. This week I upgraded my gpu from a gtx 950 to rtx2060, and the training speed is only increased like 10-20%. I know it is a simple code for maybe higher gpu utilization, but it is kind of huge for me, and honestly I was expecting it to scale similar with its fp32 calculation capabilities(x3.5-4). Obviously I can’t utilize my gpu, and I’d like to learn if there is something I can do to improve my code in the future outside of just increasing batch size.

This post is not referring necessarily to any particular machine translation technique; rather, I wanted to see on what criteria we trust any translation human or otherwise. My rationale is that translation is 100% lossless, and to trust models particularly black-box ones such as neural networks, we should first look at what makes a translation trustworthy in general. Obviously, this is deeply embedded in the context of a translation. The criteria for trusting a UN translator will be different than those for more ad-hoc situations. How can evaluate a criteria that we hold humans to such as cultural knowledge?

At first glance, RL appears more expensive and require orders of magnitude more training steps, however computer vision for object localization and classification may require larger and more fine-tuned architectures given the problems are a bit more mature.

Of the two, which would you say is more computationally demanding? Or is it a tie? Or is it the ever common “it depends” answer?

Recently I worked on a binary classification problem. The input data is a high dimension (>100) series. I tried PCA to lower the input to a much smaller dimension (<10) then applied Gradient Boosting on it and this seems to give good result. However I want to improve the results by replacing the PCA part since the classifier is not necessarily linear.

​

I tried both tSNE and UMAP and they can bring out clusters even in 2D. However I don’t know what to do next:

1. Should I use clustering algorithms like DBSCAN to do the binary classification? How should I do that? One of the issue is that although I can see a cluster of positives, there are also clusters of mixed positives and negatives that I couldn’t label;
2. I tried to put UMAP results to Gradient Boosting and to my surprise, it actually give poorer classification than PCA + Gradient Boosting. One issue I believe is that I only tried tSNE and UMAP at 2 or 3 dimensions because the computation time involved. So is there a way (in tSNE or UMAP) to know the intrinsic dimension of a input dataset, like the explained variance or factor loadings in PCA?

​

I tried to read many articles on how to use tSNE/UMAP properly but it seems most of them focused on visualization and clustering.