Hi, I’m currently working with a dataset of .tiff files and want to feed those to a model using a tf Dataset for performance reasons. However, tf currently does not support loading .tiff files with Dataset.

Now I’m curious how to assess the loss in information if I convert a tiff to e.g. a png. Currently theses tiffs only hold a single image which for me should not make much a difference.

What would be a good approach to assess this?

Danny Hillis “Co-evolving parasites improve simulated evolution as an optimization procedure “, 1990.

https://archive.org/details/06Kahle001316/page/n3

​

…there are two independent gene pools, each evolving according to the selection/mutation/recombination sequence outlined above. One population, the “hosts”, represents sorting networks, while the other population, the “parasites”, represents test cases. (These two populations might also be considered as “prey” and “predator”, since their evolution rates are comparable.) Both populations evolve on the same grid, and their interaction is through their fitness functions. The sorting networks are scored according to the test cases provided by the parasites at the same grid location. The parasites are scored according to how well they find flaws in sorting networks. Specifically, the phenotype of each parasite is a group of 10 to 20 test cases, and its score is the number of these tests that the corresponding sorting network fails to pass. The fitness functions of the host sorting networks and the parasitic sets of test patterns are complementary in the sense that a success of the sorting network represents a failure of the test pattern and vice versa.

Online demo

arXiv link

Code

We recently released our demo for our EMNLP paper “Neural Linguistic Steganography”, hiding secret messages in natural language via arithmetic coding and GPT-2. Arithmetic coding is a powerful entropy coding technique that is optimal for random sequences. Using arithmetic coding in reverse enables extremely efficient steganography, and when combined with modern language models like GPT-2 it allows for convincing cover text generations that encode information.

Here I compare 11 Speech-to-Text models using Tensorflow, 100% jupyter notebook and simplify. Accuracy based on character position.

80% of the dataset to train, 20% of the dataset to test.

1. Tacotron, test accuracy 77.09%
2. BiRNN LSTM, test accuracy 84.66%
3. BiRNN Seq2Seq + Luong Attention + Cross Entropy, test accuracy 87.86%
4. BiRNN Seq2Seq + Bahdanau Attention + Cross Entropy, test accuracy 89.28%
5. BiRNN Seq2Seq + Bahdanau Attention + CTC, test accuracy 86.35%
6. BiRNN Seq2Seq + Luong Attention + CTC, test accuracy 80.30%
7. CNN RNN + Bahdanau Attention, test accuracy 80.23%
8. Dilated CNN RNN, test accuracy 31.60%
9. Wavenet, test accuracy 75.11%
10. Deep Speech 2, test accuracy 81.40%
11. Wav2Vec Transfer learning BiRNN LSTM, test accuracy 83.24%

Link to repository, https://github.com/huseinzol05/NLP-Models-Tensorflow#speech-to-text

Link to dataset, https://tspace.library.utoronto.ca/handle/1807/24487, also included a notebook how to download the dataset.

Discussion

1. Dataset is not that really big, only 286MB.
2. Transfer learning Wav2Vec accuracy is not that high, maybe need more dataset.
3. I use my own hyperparameters for Wav2Vec, use original hyperparameters caused my GPU sync problem, sequence is too long.
4. I need to use bigger dataset.

I’d like to start a thread about a small surge of recent papers studying machine learning from the perspective of functional programming/category theory. Plenty of interesting things are happening that most people don’t seem to be aware of, unless they are in these tight circles!

Category theory is a very general and rigorous mathematical theory of compositionality that seems have become a powerful unifying force in all the mathematics and very recently sciences. Its main concerns are those alike to in deep learning: finding compositional structure in data, such that the created abstractions are non-leaky and as general as possible.

Alongside the many of the papers I linked below, the Symposium on Compositional Structures that’s happening this week has two talks on abstract mathematical generalizations of machine learning.

Note: unlike most ML papers which are focused on experiments, almost all of these are biased heavily on theory and disentangling of some of the existing structure, rather than providing new ad-hoc design choices in neural network architectures. They don’t have a SOTA result or any immediate benefit you can implement right now, but are more focused on the long term understanding of the relevant structures underlying neural networks.

I’ve compiled a list of these papers below. To me all these things are cool and I thought it might be useful for people to see these new approaches, as they might show us a shape of things to come.

Backprop As Functor

The simple essence of automatic differentiation

Lenses and Learners

Compositional Deep Learning

Generalized convolution and efficient language recognition

Towards formalizing and extending differential programming using tangent categories

Learning as change propagation with delta lenses

From open learners to open games

EDIT: Disclaimer: I am the author of the fourth paper “Compositional deep learning”

Just a bit of poking fun at ML culture.

https://youtu.be/MIEA8azwu1k

Content Update in PyTorch NLP Tutorial repo.

– Text Classification, with simple annotation.

• Dataset: HuffPost news corpus including corresponding category.
• Pre-trained word vectors: How pre-trained word representations affect model performance (via ablation study)

The model trained on this dataset identify the category of news article based on their headlines and descriptions.

link : https://github.com/lyeoni/nlp-tutorial/tree/master/news-category-classifcation

Hi everyone! I found this post on HN and would like to know your opinion/professional experience.

“Experienced machine learning professionals – How do you create scalable, deployable and reproducible data/ML pipelines at your work?”