https://snag.gy/XkzUBd.jpg

The dataset is based on the methodology described in this paper https://arxiv.org/abs/1905.07695 by Gidiotis, Tsoumakas which describe using the sections of a structured abstract as the gold standard summaries of their corresponding sections of the paper.

https://snag.gy/YmGADV.jpg

The biggest dataset has ~11 million data points from ~4.3 million papers.

The datasets are in parquet.gz files and can be easily read in python pandas parquet (no need to unzip)

import pandas as pd df = pd.read_parquet( file.parquet.gz ) 

Furthermore, processing the data and setting up training can shave off of few hours in your, many more if you’re unfamiliar with the libraries/repos. So I forked the repos and set up Colab notebook that do all of the heavily lifting, so that you can start training within a few minutes using one of the state of the art architectures for summarization.

For a quick start, here is a link to the main dataset (there are several others, check out the link at the bottom.

https://drive.google.com/open?id=1AH3HEDDs08e-xVRLjAev7K902R0eBrcl

Download it into your drive, then use one of the following notebooks that process the dataset and start training on it

Pointer Generator

https://colab.research.google.com/drive/14-hIiDmUE_qmVK0UHVTjyluHoM1yVKnE

Bert Extractive (BertSum)

https://colab.research.google.com/drive/1IEHBsryjAjddS0jv7oJOi25_TxjVfA4F

Transformer, using Tensor2Tensor

https://colab.research.google.com/drive/1JEfZ2cCJc8Dz_LQMS9_rGgtMgecfXJDG

Here is a link to the full details, including a few other scientific datasets I have created.

https://github.com/Santosh-Gupta/ScientificSummarizationDataSets/blob/master/ReadMe.md

If you have any trouble, feel free to type a comment or open an issue on Github. I am hoping people can make some pretty effective scientific summarizers using the data.

So my native tongue, Persian, is one of those languages in which you can create complex words with the preexisting vocabulary base. Like, imagine this example: A few years ago after reading a dissertation on color correction I downloaded Da Vinci Resolve and realized that color correction is science more than an art and I wanted to market myself as a color corrector. But there were no color correctors in Iran, so I had to come up with a name for the service I was offering myself. So I came up with the title of “rangband”. Rang=color, band=from bastan, closing, also means “setter”. So rangband=colorsetter. Turns out I was an idiot for thinking that I, someone who lacks even a spec of artistic thinking (I do make nice machine learning and signal processing concept videos on my blog though) could do color correction, so I gave up on it — Just like I’ve given up on anything that I’ve ever done in my life. However, a few days ago I was thinking that this feature in the Persian language can be harvested into a nice and dandy classification model:

​

1- Pair up words and make complex words.

​

2- Train the model with the complex words in the dictionary.

​

3- Let the model classify if the word has any meaning.

​

4- Test it with the complex words in the dictionary.

​

This is a binary choice, a complex word either has meaning, or not. So we can use SVMs, something which I love, to get the job done.

​

So thoughts? I wanna write an article and I need a professor of linguistics’ help, so tell me if I’m not wasting my time, and in turn, his time, with it. Thank you.

​

Tl;dr: wanna pair up preexisting words, use dictionary to make sure they are meaningful. Possible?

​

PS; Please tell me if this thread is best fitted for /r/learnmachinelearning. I’m not sure, I’m not that good in machine learning, but I write my own algorithms and that sub is mostly about libraries and modules and whatnot.

We all know about eigenfaces, alright? How about we take a picture of some people whom we know everyone agrees on their attractiveness, make an eigenface for both men and women amongst them, then ask people to send in a driver’s license photo (WHAT ARE THEY CALLED?) and when they do, match it with the eigenface. If they are attractive, the alpha or whatever will be positive. Hence only allowing attractive people to become a member of that website. It will also serve as a website to help people know if they are attractive. Of course there are intricacies, such as race and age, but we’ll overcome them. This algorithm can be used by casting agents to “poll” if someone is beautiful and attractive. It has millions of uses.

​

I think the worst problem we face is fiducial point detection… I don’t personally know how to automatically detect fiducial points, and if we let people do it themselves they might cheat.

​

So what do you think guys? Doable?

Hi,

I’m a game developer, and with my team at the studio we are doing a bit of r&d to find the fittest (see what I did there) solution to have an AI system for our enemies in in a top down 2D game that would feel organic and unpredictable. We are trying to see if ml agents in Unity could be better suited than the heuristic approach. My main question is:

We are trying to setup the input layer, and we hesitate on how much data we can feed to the ml agents of unity, here is our data:

Potential inputs:

• 10 raycasts shot radially in front of the player, giving distance to the walls (only) it touches
• Bullet angle of the closest bullets (*4)
• Bullet distance of the closest bullets (*4)
• Bullet orientation (0 is 90° up from enemy – 0.5 is dead on, 1 is 90° down) (*4)
• Player angle (*6)
• Player distance (*6)
• Player shield (*6)
• Player life (*6)
• Player combat mode (4 different modes, sword/shield – gun/shield (etc), 4 modes, input values increasing by 0.25 (*6)

We are trying to figure out what’s more important, and what to add

By ‘angle’, I’m implying the angle between the nose of the enemy and the object (player/bullet)

that means 46 potential inputs for our NN.

Context:

the player(s) will fight in a coop games, with up to 6 / 8 enemies on screen using non heuristic brains, and other classical AI.

that must be able to run real time with average graphic cards. The game is 2D and doesn’t require too much power from cpu/gpu.

Question/ TL;DR:

• 30~40 inputs * 8 brains in real time on average graphic cards: too much?
• How can we hope to go?
• Any tips on how to think about sizing our NNs ?
• Are you having a good day?

Thank you for the read =)

I have general knowledge about ML/DL, and good knowledge of Deep Reinforcement Learning. I now want to dive into image segmentation. More specifically, I want to take a stab at this Kaggle challenge: SIIM-ACR Pneumothorax Segmentation

What are be the best recent resources? I am mainly looking for review papers and strong blog posts – ideally written resources, which are more efficient to consume than videos.

I have started with this review: Understanding Deep Learning Techniques for Image Segmentation, and will review the relevant part from the fast.ai course, since a lot of strong participants seem to rely on it on Kaggle.

Hello r/ML,

I am going to join Facebook as research scientist soon, and I’ve been told I will start with a bootcamp to match with a team.

Did anyone here go through this process? I was wondering how feasible it is to join a team within AML from there? From reading online, my understanding is that FAIR has its separate process and hiring pipeline, and it is impossible to join it after being hired in a general purpose ML role, but I haven’t read anything about the hiring process of the Applied Machine Learning department. Do they have their own hiring pipeline as well, or are they just one of the possible teams you can match with during bootcamp?

I have a PhD and some postdoc experience, not directly in machine learning but with a lot of ML applications in my research, so I think I’d be a pretty good fit for a more research oriented team, but I don’t know if that’s even an option, and how competitive it is to get into one of those teams.

Hi everyone, one of our writers wrote an article on FloydHub blog. This article is an ensemble of tips and tricks collected from the industry experts (some personal ones too), code walk-throughs and many things more but all heavily focused around the data part – the fuel that drives machine learning projects.

Did we miss anything? Is there something you would like to see in-depth in the next articles?

I hope you enjoy it!