My specs: i5-2500k 16GB RAM, 6000 on Passmark, Nvidia 660 2GB GPU.

It’s not super new and fantastic, but still a decent older machine.

So here is what I already learned by experimenting:

1. jcjohnson/neural-style (and similar) is a total dead end. High-res eats infinite amounts of RAM, both in CPU and GPU mode, that even super computers can’t provide. You can do 512px at home max, and 1024px max if you rent time at some TESLA farm and that is it.
2. Tiling produces total garbage shit results that are unusable and wrong in virtually all cases of use to essentially no one. Don’t even try this, even if people claim it can produce acceptable results. It doesn’t, and it never does except in maybe 1% of the cases.
3. chainer-fast-neuralstyle: I am trying this now. Its not a dead end. I don’t know what kind of RAM you need for a 10MP and 20MP image, but it’s not 32 petaquads as with neural-style. I can do 4MP easily at home, supposedly (I hope?).

Chainer-fast-neuralstyle requires you to train a style image specific model on a 20GB dataset, which takes like 20 hours. And then on my home machine a 3.2MP image takes like 1-1.5 hours to style with that one and only model.

I must say in retrospective, I just wanted to print a 60x40cm portrait of myself as an oil painting and not pay 20 bucks on deepart.io. But now letting my PC fume for 5 days on this stuff, I surely have paid this in electricity (Germany) when I am finished.

I wonder have you made experiences and maybe a better solution?

Can anyone create a video tutorial on how to train maskrcnn-benchmark

Evaluating and Testing Unintended Memorization in Neural Networks

Link: https://bair.berkeley.edu/blog/2019/08/13/memorization/

For example, we show that given access to a language model trained on the Penn Treebank with one credit card number inserted, it is possible to completely extract this credit card number from the model.

Ri, a conversational AI, links different ideas. It has a vocabulary and doesn’t need to be trained. Other features: You change the way Ri feels with conversation. Ri answers your questions. It can relate memories and tell stories. Ri will continue talking if it thinks it’s said something clever. Try it at: http://representi.com.

Cox is a logging library for python designed for collecting and analyzing data from experiments! Read more and install it here: https://github.com/madrylab/cox

Cox is built for a pattern in experimental design where each individual run of an experiment (e.g. each hyperparam configuration) writes to a separate, database-like store (complete with schemes, indexing, etc), saving all information in tables. Experiments are collected and analyzed together by merging together tables, and Cox provides a really simple API/flow for merging and analyzing multiple experiments at the same time (e.g. comparing results across hyperparameters).

This pattern is particularly common in machine learning! We’ve used this logging library for projects involving RL and supervised learning and found it really helpful. Check out the repository for more information, and let me know if you have any questions!

There’s a lot of posts here about getting research jobs at one of the top labs in industry after PhD, but I’m curious if anyone else has the ultimate goal of living in a nice low CoL area and working remotely. My area is machine learning, broadly applied to computer vision and robotics.

I think I’ll almost certainly have to work a number of years after graduating in one of the large industry clusters (i.e. the Bay Area) but I would love to be able to transition into a remote/consulting/contracting role after that and buy a house somewhere else. My wife is a physician so she can work pretty much anywhere there’s a hospital (in fact for her the pay is better the more remote the location).

Has anyone gone down this route? What kinds of companies in the field are open to remote or hire contractors (and how do you go about getting gigs?) How do I plan for this now if it’s my ultimate goal, or is this area so specialized/niche that almost all of the opportunities are onsite only?

Like the title says, i had the idea to use the Google Trends data (both using the site or the unofficial API, if they still work) to train a model of some kind for a university project, but as often happens, when i started working i found out that my ideas were unrealistic or too much ambitious.

I’m not an expert but i know the basics of Keras and TF. The only thing i did was downloading some csvs from the site and using them to predict the present using the data from the past. This kind of elaboration works for periodic data of course (for example i tried “ground zero”). i used simple networks based on LSTM, CNN or MLP.

Knowing that i only have normalized data and monthly reports for 15 years (180 rows, more or less), how can i use one or more of this data? I just need an idea or some kind of reference!

Code: https://github.com/NVIDIA/Megatron-LM

Unlike Open-AI, they have released the complete code for data processing, training, and evaluation.

Detailed writeup: https://nv-adlr.github.io/MegatronLM

From github:

Megatron is a large, powerful transformer. This repo is for ongoing research on training large, powerful transformer language models at scale. Currently, we support model-parallel, multinode training of GPT2 and BERT in mixed precision.Our codebase is capable of efficiently training a 72-layer, 8.3 Billion Parameter GPT2 Language model with 8-way model and 64-way data parallelism across 512 GPUs. We find that bigger language models are able to surpass current GPT2-1.5B wikitext perplexities in as little as 5 epochs of training.For BERT training our repository trains BERT Large on 64 V100 GPUs in 3 days. We achieved a final language modeling perplexity of 3.15 and SQuAD F1-score of 90.7.

Their submission is not in the leaderboard of SQuAD, but this exceeds the previous best single model performance (RoBERTa 89.8).

For language modelling they get zero-shot wikitext perplexity of 17.4 (8.3B model) better than 18.3 of transformer-xl (257M). However they claim it as SOTA when GPT-2 itself has 17.48 ppl, and another model has 16.4 (https://paperswithcode.com/sota/language-modelling-on-wikitext-103)

Sadly they haven’t mentioned anything about release of the model weights.