So, I’m a big fan of Lex Fridman’s deep learning podcast. A big ago I watched one he did with Ian Goodfellow.

At the start of the interview Goodfellow describes how deep learning methods are distinguished by the fact that it involves a bunch of computations done in sequence rather than in parallel. (You can watch the video to get a better idea of what he was talking about).

Does anyone have any other examples of machine learning techniques that you feel fit his description of being deep? Just curious about this.

I am working on creating something that can detect and ideally extract information from a job posting.

I have some questions around the data I am using. I currently crawl websites and take screenshots of their career pages. These screenshots vary in dimensions due to the length of the website.

Disclaimer, I am not a ML Pro. I am self taught everything and currently using Google’s AutoML Services for training my model.

My Questions:

1. Should I use these long/large images? Or is it better to cut them in half and then feed it to the AI. With the large images when I zoom in I can see everything fine for labeling. When not zoomed in, it can be hard to make things out.
2. How small should labels be? Google allows the smallest to be 8 pixels by 8 pixels. If they can be big I can use the large images and just zoom in?
3. Is there a way to give context to the classifier/object detector? I realized when I evaluate a job posting I get context from the url and other words on the page that it doesn’t get since it only sees a screenshot.
4. Should I try to label every element on the page? if yes, In a high level way or granular?
5. Any other hints or tips I should think about to solve this problem?

My Attempts/Approaches

Attempt 1: Object Detection

My first attempt was to perform object detection on screen shots that were cut down to ~2,000 pixels. I then labeled most of the content on the page with labels like: Header, Footer, Section, Heading, SubHeading, Job Title, Job Posting, Paragraph, Section Heading, Section SubHeading.

Results :
Total images: 183
Test items: 17
Total objects: 244
Object to image avg: 14.35
Precision: 91.43% (Using a score threshold of 0.508)
Recall: 13.11% (Using a score threshold of 0.508)
Average precision: 0.171 at 0.5 IoU

Conclusion: Object detection needs many more images, also the labels I provided were not concrete enough. Looking back I found the definitions for certain things to be vague. For example I was using the label heading, subheading and job title. Well sometimes the heading is also a job title, but I would only mark it as job title. Thinking about it from the computers perspective how will it know a heading from a job title? There is not much there visually for it to grab onto. This lead me to cut the images down to a height of 2,000 pixels so I could see each element more clearly.

The problem here is do I try to label every HTML element?

Attempt 2 Object Classification

My second try was to use image classification to determine if I was on a job posting page, then if true use another model to extract the data.

My first model1 results
Total images: 85
Test items: 9
Precision: 77.78%
Recall: 77.78%

My second model2 results
Total images: 484
Test items:55
Precision: 90.7%
Recall: 70.91%

These results were more in-line with what I had thought. When looking at the overall page there over and over there becomes a familiar pattern with what a job posting looks like.

Final Attempt – Object Detection:
I am now trying again with an object detection model, that is trained only on job posting’s, I think this will do better as it only has 3 labels, Job Title, Job Location and Apply Button. I wanted to include a label for: Responsibilities, Qualifications, skills, bonus, ect… but came back to the fact that there is not much for it to grab onto…as I find these in the posting by reading.

Model currently in training…

Final Notes
I believe the correct way for me to do this problem would be to train the AI on the html code, but I am using google’s automl services so I dont know how/if that is possible. I was thinking about using/combining different types of data/techniques since there is information in the URL and code that I’m not leveraging. Perhaps apply NLP to the URLS?

Thanks for checking out my project any thoughts are appreciated.

First off I am not an ML engineer. I am am embedded software engineer working mostly in safety critical systems. So if there are some dumb assumptions here don’t crucify me. One of the biggest things that strikes me about ML is it’s black box nature. We can’t ask the machine how it made a descision, in fact I’ve heard claims that we shouldn’t because it would inject human bias into the system. For things like data scraping and image recognition that seems fine, but I can’t imagine having a conversation at my work go like this:

“X failed, people died. Go figure out how and fix it.

Sorry boss I can retrain the model with this new outcome but I can’t tell you why it broke or guarantee to any degree of certainty it won’t happen again”

That just wouldn’t fly. Is there something I’m missing?

Hello everyone!

I work with spreadsheets a lot, doing tasks manually that are just a bit too complex for rules, but I believe they certainly fall into what ML can handle. In a nutshell, I spend 2+ hours a day going through company names, removing legal terms like “LLC” or “Limited”, and humanizing them.

For instance, I have a spreadsheet with company names and emails.

These would become (currently by hand):

What we’re doing here is:

1. Shorting names to their essence
2. Removing legal terms and words
3. Looking at domain names (in email addresses) as a clue for the “most human name”

Now, I very well believe this is something Google Cloud has capabilities for. Given the lack of programming involved with Google Cloud ML (and its potential integration with Google Sheets), I’d imagine it’s the best vehicle for this tool.

Some questions before I embark upon this journey:

1. Would your recommend I use Google Cloud ML or another tool?
2. How much data would you imagine would be necessary to train this tool? (uncleaned spreadsheets and cleaned spreadsheets)
3. Am I critically misunderstanding something here? This is pretty much my first time practically applying ML.

Thank you very much for all your help!

Hi,

I wonder why we are still trying to mimic photorealistic world by counting every reflection, polygon, tracing every ray and so on. Shouldn’t it be done in such manner that AI is just doing the job basing on photos and low polygon input like here https://assetstore.unity.com/packages/3d/characters/animals/poly-art-forest-set-128568 Also all other games like Zelda BOTW, Team Fortress 2 or even Fortnite could be easily turned by AI into photorealistic env. Shouldn’t we start thinking about doing AI accelerators (like first 3dfx cards) for enriching low polygonic world’s generated easily by most commodity hardware? I guess even ray tracing could be made by ML. I believed that future belongs to generating world by AI not by tricky mathematic graphics algorithms. Especially that in future it is easier to go from such trained networks into environment where instead of heaving an output on display, the output would be “drawn” directly in human brain through neural-connectivity. Also AI is able to properly handle cases where object is moving fast or turning around.

Cheers, Alexa

I’m writing my master’s thesis on image generation and was wondering: which other methods were/are used to synthesize images, aside from GANs? Any time I look up “image synthesis” I only find GAN tutorials, and excluding GANs seems to bring up photon scanning.

I thought this project from FB Research is a really cool work:

Abstract

In order to provide an immersive visual experience, modern displays require head mounting, high image resolution, low latency, as well as high refresh rate. This poses a challenging computational problem. On the other hand, the human visual system can consume only a tiny fraction of this video stream due to the drastic acuity loss in the peripheral vision. Foveated rendering and compression can save computations by reducing the image quality in the peripheral vision. However, this can cause noticeable artifacts in the periphery, or, if done conservatively, would provide only modest savings. In this work, we explore a novel foveated reconstruction method that employs the recent advances in generative adversarial neural networks. We reconstruct a plausible peripheral video from a small fraction of pixels provided every frame. The reconstruction is done by finding the closest matching video to this sparse input stream of pixels on the learned manifold of natural videos. Our method is more efficient than the state-of-the-art foveated rendering, while providing the visual experience with no noticeable quality degradation. We conducted a user study to validate our reconstruction method and compare it against existing foveated rendering and video compression techniques. Our method is fast enough to drive gaze-contingent head-mounted displays in real time on modern hardware. We plan to publish the trained network to establish a new quality bar for foveated rendering and compression as well as encourage follow-up research.

Project page / paper / video / code: https://research.fb.com/publications/deepfovea-neural-reconstruction-for-foveated-rendering-and-video-compression-using-learned-statistics-of-natural-videos/

Are there papers that do so? ImageNet images have various resolutions. Since we already have adaptive pooling techniques such as global average pooling, attention-based pooling, etc. that support variable resolution, should it be possible?

Random fixed size crop and resize seems hacky.

Hi Everyone!

New to reddit, but I’d like to share a project I’ve been working on called NBoost. It’s essentially a proxy for search APIs (e.g. Elasticsearch) that reranks search results using finetuned models (e.g. BERT).

Check out our medium article or github to learn more!

It’s main features include: – Super easy to set up (you can just pip install nboost) – Easy, non-invasive integration with Elasticsearch and potentially other search APIs. – Finetuned models are plugins (you can swap them in and out). – Fast and scaleable (written at the lowest level possible)