I’ve been pondering this question and wanted to get some of your thoughts on it.

Kernel functions finds distances between two inputs relative to each other in some transformed space. Neural networks on the other hand finds the exact location of of the input in its transformed space. Are there benefit and downsides between the two transformations? Why are kernel functions used instead of specifying the direct transformation from input to transformed space

In my view, overfitting does not necessarily imply lack of generalization, just as well as generalization cannot be directly associated to degree of overfitting.

​

An overfit model is a model that is tuned to generate the highest performance (e.g. lowest loss) on the dataset it was trained with. This can be tested by the difference between the losses on the validation set and on the training set. In order to test for overfitting, training and validation sets should have similar distributions. If that’s the case, an overfit model will deviate in performance on the validation set from the training performance. This is because, even if the distributions are similar, the model is tuned to pick up correctly only the samples it has seen on the training set.

​

As for generalization, it can only be evaluated between datasets (test and training) that have different distributions. Ideally, the test distribution will be the most heterogeneous of them all. In my opinion, this is the only way to really assess generalization: the difference between the losses on training versus testing set.

​

TLDR: Overfitting is indicated by when model underperforms on unseen data with similar distributions to seen data. Generalization, on the other hand, is indicated by the performance differences between seen and unseen data with different distributions, where the unseen data ideally represents real world distributions.

​

I think this is a misconception most have, even in industry.

​

What are your thoughts?

I’m in the process of trying to get a handle on some datasets. I know there are identical entries spread across several files, but I’d like to find a way to visualize those connections, either in a map or even just a table.

My immediate task just has three smallish CSV’s so I could easily write an R script to pull out the matches, but I’d prefer to more visual tool that can operate across larger bodies of data.

I remember seeing a Defcon presentation where a similar tool was described for matching metadata, so I’m going back through old videos to try and find that, but I’m hoping someone here might know some good suggestions.

Thanks!

My project is classification of ultrasound 2D images, the size of the full data set is approximately 1000 images. For this analysis 250 features were handcrafted by calculating different parameters of the whole images, or horizontal slices of the images. For features selection Kbest with chi2 is used to select the best 50 features. To calculate balanced accuracy I am using sklearn.model_selection.cross_val_score, and Random Forest and Extra Trees (1000 trees). What confuses me is that when I split the data with train_test_split randomly with 9:1 ratio, and use cross_val_score only on 90% of the data the highest accuracy score is 80% with random forest, and 85% with extra trees. But when I don’t apply train_test_split and calculate balanced accuracy score on the full data set, the highest score is not higher than 60%. I expected to get better results when I included more data, but opposite happened. I would appreciate any advice or idea, how to improve the accuracy score.

I’m currently taking a Machine Learning class and need to create a critique of a published work. I was looking for some published papers in the field that weren’t really based on a valid foundation and/or made poor decisions in designing their model. Any suggestions or links would be appreciated.

Nanophotonic neural networks are an exciting emerging technology which promises low-energy, ultra high-throughput machine learning systems implemented purely optically. Our lab has previously done work on these devices, and our new paper which extends programmable photonics to the quantum domain is now on arXiv!

In this paper, we describe a photonic architecture for a quantum programmable gate array (QPGA) which can be dynamically reprogrammed to perform any quantum computation. We show how to exactly prepare arbitrary quantum states and operators on the device, and we apply machine learning techniques to automatically implement highly compact approximations to important quantum circuits.

Below is an animation of a simulated QPGA being trained to implement a quantum Fourier transform on five qubits. Supplementary materials and the TensorFlow code for the quantum circuit optimization section of the paper can be found in the GitHub repository for the paper.

Paper: arxiv.org/abs/1910.10141

GitHub repo: github.com/fancompute/qpga

​

Simulated QPGA learning to implement a 5-qubit quantum Fourier transform