Paper: http://proceedings.mlr.press/v97/behrmann19a.html
Code: https://github.com/jhjacobsen/invertible-resnet

Abstract:
We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple normalization step during training, already available in standard frameworks. Invertible ResNets define a generative model which can be trained by maximum likelihood on unlabeled data. To compute likelihoods, we introduce a tractable approximation to the Jacobian log-determinant of a residual block. Our empirical evaluation shows that invertible ResNets perform competitively with both state-of-the-art image classifiers and flow-based generative models, something that has not been previously achieved with a single architecture.

Decaying learning rate is a popular practice even for adaptive optimizers such as Adam. Increasing batchsize was also shown to have the same effect.
But there are other hyperparameters with similar nature.
– Does it make sense to decay/increase them?
– Have anyone tried decaying momentum, or decaying droprate, or increasing L2 regularization?
– Are there other hyperparameters that need tuning like this?

So I’ve got a little issue and I’m looking for some advice please.

I’m new as a researcher in the field, I am a masters student with zero publications so far and I’ve been accepted for an internship at another research lab to work on more theoretical things in ML. My thesis work has been entirely applied, where I worked on some clinical applications of ML, and I was hoping this internship now will help me to broaden my knowledge by working with other experts.

When I signed up I was asked what my research plan there was, and I gave a vague statement that I am interested in the theme of their research with nothing specific in mind. The issue came about when after I joined the team, I was discussing with some of the post-docs what their own projects were, and one post-doc seemed excited to share his projects and have me help out with various aspects. Then, this other post-doc, who seemed a little annoyed at me, tells me I need to find my own project and not to ask others for a project. Ok.. fine.. I was taken aback honestly, because I don’t know what research culture is like, and how people collaborate. From my perspective, there is an insane amount of publications in every niche of ML, so for me as a masters student to just waltz in and be expected to have my own specific idea ready to research it, seems quite silly and a waste of time, but maybe that is the norm? I honestly don’t know.

Sorry for the long read, its more of a vent. But my questions are:

1) Is it out of line to join a research team and expect to be given a problem statement or at least some specific ideas?

2) If yes then, how the hell is it possible for someone who has not worked in the field for a long time to come up with an original problem statement? Is this expectation realistic at all??

Anyone with similar experiences, stories, discussion, advice is appreciated!

Hi everyone,

We are from TomTom Autonomous Driving from Amsterdam and Berlin, and we are working on HD map making for automated driving and navigation.

Being able to build a highly accurate map is essential for precise localization and navigation for an automated vehicle.

To create maps that are precise and detailed enough, we need to efficiently analyse incoming data from diverse on-car sensors such as LiDAR and cameras using state of the art machine learning techniques.

We use a lot of deep learning especially pertaining to computer vision. Published papers from our group include work on structured prediction for lane detection, as well as domain adaptation:

• EL-GAN: Embedding Loss Driven Generative Adversarial Networks for Lane Detection
• Dynamic Adaptation on Non-Stationary Visual Domains

The following users will be answering your questions from 10am-12pm CEST, and from 4pm-6pm CEST:

/u/kmhofmann, /u/pierluigi_tomtom, /u/AlessioColombo, /u/fkariminejadasl, /u/jvvugt, /u/ml_steve

So Ask us Anything!

Paper: https://arxiv.org/abs/1906.01083

Blog (with samples): https://sjvasquez.github.io/blog/melnet/

Many more samples: https://audio-samples.github.io/

Abstract: Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps. While long-range dependencies are difficult to model directly in the time domain, we show that they can be more tractably modelled in two-dimensional time-frequency representations such as spectrograms. By leveraging this representational advantage, in conjunction with a highly expressive probabilistic model and a multiscale generation procedure, we design a model capable of generating high-fidelity audio samples which capture structure at timescales that time-domain models have yet to achieve. We apply our model to a variety of audio generation tasks, including unconditional speech generation, music generation, and text-to-speech synthesis—showing improvements over previous approaches in both density estimates and human judgments.

Here is the link, best deal I could find for a deep learning pc for under $1000 https://www.amazon.com/SkyTech-Azure-Computer-Desktop-GeForce/dp/B07NV1YJK9/ref=cm_cr_arp_d_product_top?ie=UTF8

Since I have been playing around with CNNs quite a bit I was really hoping to find a cheap RTX 2060, and this is by far the cheapest I’ve been able to find. The 16GB of RAM is solid and the 500G NVMe PCI e SSD should be all I need for awhile. The benchmarks on the Ryzen 5 1600 processor seem very good as well. Am I missing something or is this just a really good bargain for a consumer-grade machine learning pc?

Hi,

I am a student who is researching the topic of neural networks. I am posting a video of 12 games being played by the neural network versus the human player in Battleship, and require your help in this project.

​

Please observe the video and answer which board (A or B) is being played by the human for each of the 12 games.

​

Link: https://youtu.be/1tJvdbX0efo

A new efficient, anchor-free and end-to-end trainable 3D object detection method that achieves SOTA results on ScanNet and S3DIS and is approx. 10x faster than most existing methods…

Paper and code: https://arxiv.org/pdf/1906.01140.pdf

Hey everyone! I am a n00b here, A quick question.

We plan to submit results to a conference that due in 4 months. Our study currently have good results, in case of other labs are working on the same problem, we would like to archive it on arXiv as soon as possible. However, there is possibilities that other research groups can look into our archived paper, improve upon, and submit to same conference in between. If that is the case, can the reviewers of the conference simply reject our paper and prefer the new one?

​

Thanks in advance!