I’ve been super intrigued by image super resolution problems. Reading online, I found the SRGAN paper to be interesting, especially how the PSNR and SSIM metrics are unreliable when compared to human perception of quality. I wanted to create a faster version of the SRGAN, so I decided to use a MobileNet as the generator. This idea is somewhat inspired by Realtime Image Enhancement, Galteri et al. I want to use it to upsample low quality videos, for scenarios when you may not have access to high speed internet. You can leverage the GPU to do synthetic super resolution. I would appreciate any ideas towards increasing speed/quality of this project.

Here is the implementation in Tensorflow 2.0: Fast-SRGAN. Take a look, and feedback is really appreciated!

https://arxiv.org/abs/1910.07000

The most interesting part is that based on the question, it will look up documents, and based on the question and information in the first set of retrieved documents, it’ll generate new queries to look up and find the exact document which as the answer. The concept itself isn’t new; it’s been a goal for the NLP/ML community for a while, but Stanford was able to do it by creating a dataset (not sure if that’s the entirely right word, they used ‘query generation supervision signal’) of these generated queries.

They generated the gold candidate queries by finding overlap of the content of the first set of retrieved content, and content of the the text that contains the answer. In their own words (and I think this is the most important part of the paper):

“ computing the longest common string/sequence between the current retrieval context and the title/text of the intended paragraph ignoring stop words, then taking the contiguous span of text that corresponds to this overlap in the retrieval context.”

Final thoughts: I love this paper. I’m really interested in dataset generation using very accurate / robust heuristics and models. I think these datasets can be used to trained some very effective language models for information retrieval. I am currently working on a project like this; I’m currently processing a dataset for research paper retrieval.

One of the most counterintuitive developments in ML research is that, despite huge amounts of resources and brain power being poured into field, state-of-the-art neural architecture search algorithms still can’t outperform pure random search.

This fact is so jarring that I’m surprised it’s not being talked about more often.

What exactly does this mean? Are we just putting out ineffective automl algorithms, or has the power of random search been completely overlooked?

A followup to original post (pasted shortened below), with major changes; release v1.1:

• Run-based weight decay normalization scheme, normalizing over arbitrary # of iterations independent of LR scheduler (e.g. over all epochs)
• Full compatibility with TensorFlow 2.0.0 and Keras 2.3.0 (keras + tensorflow.keras)
• Full compatibility with TensorFlow 1.14.0 and Keras 2.2.5 (keras + tensorflow.keras)
• Also compatible w/ TF 1.13.0 & 1.15.0, Keras 2.2.3-2.2.4

For a complete list of changes, see release notes. Optimizers here.

The latest Lookahead optimizer paper, co-authored by Geoffrey Hinton, used AdamW as its base optimizer, and noted it performing superior to plain Adam.

NadamW and SGDW included, along their WR (Warm Restart) counterparts – with cosine annealing learning rate schedule, and per layer learning rate multipliers (useful for pretraining). All optimizers are well-tested, and for me have yielded 3-4% F1-score improvements in already-tuned models for seizure classification.

“We hope that the community can use this dataset to improve developer tools generally, which may include semantic code search. We hope that the state of the art with regards to representation learning of code is advanced because researchers and practitioners now have a common dataset and a forum in which to discuss results. We also hope that the uniqueness of the dataset will inspire the community to uncover new approaches and techniques for code and natural language understanding.”

That’s a quote from the one of the authors of CodeSearchNet – datasets, tools, and benchmarks for representation learning of code. This research on semantic code search has been posted here before as news, but I thought some people here might be interested to know some of the details behind what goes into a project like this at a big company. I interviewed Hamel Husain, a machine learning engineer at GitHub about how the project started and evolved into a wider open source effort to involve the ML research community. Hope there are useful takeaways for people here.

Here’s a link to the interview: https://sourcesort.com/interview/hamel-husain-on-semantic-code-search

And here’s a link to the original paper on arXiv: https://arxiv.org/abs/1909.09436

Hello,

Pretty self-explanatory, I checked on AWS and Google Cloud and I did not find any cloud instance equipped with GPUs with higher than 16Go RAM. Please do correct me if I’m wrong for those two platforms and indicate other services where I could find GPUs with lots of VRAM.

Thanks !

Hi,

Lately I have been trying to control a robot with a puppet (kinda like a voodoo doll) and want to estimate the keypoints on the puppet to recognize the orientation of the head and the wheels so that I can transfer that as commands to the robot. I couldn’t find anything online about estimating keypoints on custom objects.

So that made me think that if a model could exist that can estimate the structural skeleton of objects, what could be it’s potential applications?

One application I can think of is recognizing the orientation of objects and not just detecting them in an image with a bounding box. What more could it be used for?

What are your opinions/thoughts on this?

p.s. I am thinking of this as my thesis for a research paper but just wanted to make sure it is something worth spending time for and knowing its potential applications.

In a project I am working on currently, team got into a discussion over shall we go for Dense MLP or CNN? That discussion sort of made me wonder the question, “Can Dense MLP work as good as any other architecture (CNN, GCN) for every task?” A proper way of putting it will be, given we are able to properly train a huge dense network with enough expressive power for the task, and we have enough data for proper training, can a dense network perform as good as other network architecture, in theory? From what I understand, different architectures are just different ways of pooling/sharing information and feature extraction. The functions that can be realised by any network should also be realisable by some configuration of Dense network.