Hi everyone!

Recently a friend and I have been working on a new Python library for machine learning with Spiking Neural Networks (SNNs), called PySNN, which is built on top of PyTorch. We feel it is time to share it with more people, and hopefully get good feedback and contributions:)!

Our goal for PySNN was to make a truly modular framework for machine learning with SNNs, while staying as close to PyTorch as possible. All of the existing frameworks either operate more like simulators for neuroscientific research, or use relatively fixed network and training/evaluation loop designs. PySNN, on the other hand, consists of building blocks for neurons, connections, and learning rules which the user can combine in their desired way. It even allows for mixing learning rules or training only specific parts of the network.

Furthermore, since PySNN consists of just the basic elements, the framework is lightweight and allows for easy extension. Because of its tight integration with PyTorch it fully supports GPU acceleration, batching of samples, and supports tools like the jit compiler and graph tracing for TensorBoard.

There are still many improvements and extensions that can be made, so feel free to have a look and send out a pull request! We will be very active in helping with any issues! https://github.com/BasBuller/PySNN

We are looking forward to your comments and suggestions!:)

I want to build a model for Chess/Go/Shogi that is trained and tested on real players, and I want it to pass the Turing test. I don’t want my model to play the best move in a position, I want it to play the move that a person would play (of a certain strength, time control, etc..).

​

It’s easy to make this a classification problem and train a CNN on a one-hot encoded policy of actual moves played. The only problem is, without some kind of look-ahead algorithm (MCTS for example) the model fails to learn sequences that require multiple moves, such as tactics.

​

However, current MCTS/alpha-beta/minimax models require evaluation of leaf nodes. I don’t have a way to shape the reward to an evaluation of a leaf node. So my question: how would I incorporate a look-ahead algorithm in an imitation learning problem like this?

A satirical piece I wrote on the parallels between machine learning engineers and Wall Street traders:

https://towardsdatascience.com/the-wolf-of-silicon-valley-150e5f501216

As per subject, wasn’t there a thread on that yesterday? I can’t find it anymore. Was it mowed down by moderators?

Hey all,

Working at a small startup, and we have extracted 33 million text messages from our users. We plan to create a model to classify different types of sms relevant to us.

First step is to create a Word 2 Vector dictionary for EDA and clustering and possibly to use these embeddings for classification further down the line .

Just wanted some guidance about the hyperparameters for the gensim’s Word2Vec.

The corpus is 33 million sms, average sms length is 16 words and the vocab size is 1.5 million.

I used the following hyperparameters and obtained decent results but just wanted to know if I’m doing anything wrong that could be hampering the model from performing even better:

Cbow, window = 4, vector size = 125, iterations =10, workers = 5, min_count= 4.

Furthermore does anyone have any tips on how to evaluate the embeddings ( other than checking that the similarity for a small set of words makes sense) so that I can fine-tune these hyperparameters?

And final question ( I promise) Would it possible or recomendable to take a pre trained Word2Vec model and improve on it by giving it the sms data so that it learns new words like slang and typos without losing its overall knowledge of the language?

Thanks so much for your time in reading.

I would like to share my pet project, Hypertunity, a Python library for black-box hyperparameter optimisation. It’s main features are: * Bayesian Optimisation using Gaussian process regression by wrapping GPyOpt; * Native support for random and grid search; * Visualisation of the results in Tensorboard using the HParams plugin; * Scheduled, parallel execution of experiments using joblib; * Also possible to schedule jobs on Slurm.

For the full set of features, check out the docs.

Your feedback is very much appreciated!

Basically, so far, I have been trying to train BERT on a very long document by cutting start, middle , and end sections of article so it could be fit into the limited input dimension of 512. However; the performance has been dismal for most of the time. So far, I am not sure if using LSTM+GRU was a better approach than this. But are there other ways to train it than just cutting up the article? When I googled for an alternative approach, I couldn’t find much…

AISTATS 2020 reviews are marked for release on Nov 24, 2019. Here’s a thread to discuss this year’s reviews. Godspeed, everyone!

Link to paper: https://arxiv.org/abs/1901.10514

Link to my reimplementation: https://github.com/Abhishaike/HyperProtoNetReproduce

This is a Pytorch reimplementation of the NeurIPS 2019 paper ‘Hyperspherical Prototype Networks’ in Pytorch. This paper proposes an extension to Prototype Networks, in which the prototypes are placed a priori with large margin separation, and remain unchanged during the training/testing process of the model. The paper suggests that this extension allows for more flexible classification, regression, and joint multi-task training of regression/classification, and with higher accuracy compared to typical Prototype Networks.

This repo includes reproduced benchmarks for most of their datasets. Largely the same accuracy/error, but quite off on CIFAR-100 (not ImageNet-200 though for some reason), so it’s possible this is an issue on my end.

I also found their use of SGD for prototype creation to be unusual, considering that, the way they phrased the prototype problem, it seems like a job more for a constrained optimization algorithm. Alongside the SGD implementation (which are used for the included benchmarks), I added in two other optimization algorithms, one unconstrained (BFGS) and one constrained (SLSQP). These didn’t seem to change the results much.

This is my first reimplementation of a paper, so any critiques would be great!