I recall reading in the last month a paper with the topic described in the title, but despite quite a lot of searching, have not been able to find it. If I recall correctly, the authors found that they could produce just about any sentence with an appropriate initial state.

Alternately, if anyone is aware of other work on this topic, references would be appreciated.

I am aware of this work, and don’t think that it’s what I’m thinking of.

*or non gaussian

I get the point in having your features be of the same scaling but wouldn’t min max scaling (x-xmin/xmax-xmin) work better? Every tutorial Ive seen about data cleaning seems to use the StandardScaler() without looking at the underlying data.

Why use fixed length sequences in transformer ? In what way and why does it effect the performance and training of transformer ? Why did they not use sequences of length <= some number ?

Any paper regarding this?

Also, while reading the paper on Transformer-XL (Dai et. al, 2019) they say,

“We propose a novel neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence”

Why can’t we learn dependencies with a normal transformer(Vaswani et. al) beyond a fixed length without disrupting temporal coherence?

I think temporal coherence gets disturbed when the input length becomes comparable to the length of embedding used for a single word/character because the embedding then doesn’t contain enough information to link the word embedding to all the previous length of this input sequence . Am i right ?

In this guide, we’ll look at a couple of papers and research efforts that have been put forth to tackle this challenge.

​

https://heartbeat.fritz.ai/research-guide-data-augmentation-for-deep-learning-7f141fcc191c

I’m looking for a figure to cite in one of my projects. I have seen it once before, but didn’t save the source, and have no luck finding it again. The effect of the figure is to show that on average, successful theories were invented very early, but good results only follow the release of good datasets.

This is achieved by listing a number of tasks (e.g. image classification). For each task, the figure lists which technique has been used to successfully tackle the problem (e.g. CNNs) and the year it was first proposed. Additionally, it lists the year that a significant dataset for this problem (e.g. ImageNet) was released. Finally, the last column displays the year that some performance threshold was reached on the given technique.

I hope my description is clear. It would be great if someone could find the actual figure!

In general, do you think the claim made here is valid? Or is it simplistically aggregating too much information, and missing the point?

I made an implementation of MixMatch (paper) in PyTorch, thought I’d share for those who are interested. Works as an installable package which you can use to create a dataloader that implements the mixmatch algorithm, as well as construct the appropriate loss function.

https://github.com/FelixAbrahamsson/mixmatch-pytorch

Feedback and comments are appreciated!

Congestive Heart Failure (CHF) is a severe pathophysiological condition associated with high prevalence, high mortality rates, and sustained healthcare costs, therefore demanding efficient methods for its detection. Despite recent research has provided methods focused on advanced signal processing and machine learning, the potential of applying Convolutional Neural Network (CNN) approaches to the automatic detection of CHF has been largely overlooked thus far. This study addresses this important gap by presenting a CNN model that accurately identifies CHF on the basis of one raw electrocardiogram (ECG) heartbeat only, also juxtaposing existing methods typically grounded on Heart Rate Variability. We trained and tested the model on publicly available ECG datasets, comprising a total of 490,505 heartbeats, to achieve 100% CHF detection accuracy. Importantly, the model also identifies those heartbeat sequences and ECG’s morphological characteristics which are class-discriminative and thus prominent for CHF detection. Overall, our contribution substantially advances the current methodology for detecting CHF and caters to clinical practitioners’ needs by providing an accurate and fully transparent tool to support decisions concerning CHF detection.

(emphasis mine)

Press release: https://www.surrey.ac.uk/news/new-ai-neural-network-approach-detects-heart-failure-single-heartbeat-100-accuracy

Paper: https://www.sciencedirect.com/science/article/pii/S1746809419301776