Has anybody here used it at work/for a project? If so, what for? What advantages does VW have over other frameworks (e.g. PyTorch or scikit-learn), and do those advantages justify the steeper learning curve?

VW’s speed and benchmarks look very attractive, but I’m not sure what tasks it would be the right tool for.

https://github.com/VowpalWabbit/vowpal_wabbit

There has been a push to have more reproducible code that one submits to conferences like ICML, NeurIPS, etc given the call that was made for this at NeurIPS 2018 and from the wider ML community. And in general in working on projects that span several months and different developer and data science teams.

I’ve looked into tools like Pachyderm https://www.pachyderm.io/ and DVC https://dvc.org/ (I’ve found them to be a bit heavyweight in terms of setup, especially for some of my colleagues who come from more of a research and less of a software engineering background/experience)

Are there any other tools that you use to achieve this for your research and development? Would be great to mention pros and cons of each.

Hi all,

At my university, I am taking a grad course in which we’re select a deep learning project and work on it by the end of the course.I had previously developed a set of tools for my M.Sc project where I need to manipulate sets of medical images in different formats (DICOM, Niftii, Nrrd), pre-process them using SimpleITK and feed them into a deep learning pipeline.I figured I could execute that course project using those tools and put it up on github for everyone to use and for me to showcase my skills, as I will finish school very soon and that will give me some visibility for my upcoming job search.

The platform relies on visdom for visualization capabilities, I had implemented functions that allow you to visualize your network’s computation graph, the experiment options and hyperparameters, your loss and accuracy curves, the gradient flow graphs through your networks as well as histograms for weight distribution in your layers. The python classes offer different functions to easily extend functionality where you can sample images during training if you’re working with images and display them on the browser.

The repository comes with implementations of UNets and ResNets as well as many GAN loss functions such as Wasserstein GAN with gradient penalty. The same class can also be set in non-GAN mode so that it uses a simple cross-entropy loss function.

I had used this code to segment vertebrae from MRIs, reaching a 0.87 dice coefficient.

The code is generic enough to be used for tasks other than image segmentation if you wish to play around with it.

I hope this will be helpful to you.

Please find it in the link down below:

https://github.com/Roulbac/GanSeg

Here is an example of segmentation on unseen data:

https://i.ibb.co/0MS00s6/7-E4-BEE89-FF20-4446-B6-F0-894-BDBEE7-C97.jpg

These contents sum up recent work about extracting style and content features from images. They can be successfully used to generate art pieces!

https://beta.flow-app.io/?path_id=15#/path?section=Overview&modelHead=path_15

Ref. https://openreview.net/pdf?id=SkVhlh09tX and https://github.com/pytorch/fairseq/blob/master/examples/pay_less_attention_paper/README.md.

Looking for a TF implementation of LightConv and/or DynamicConv (at least the layers, don’t need full model/repro)–is anyone aware of one?

Exists well-documented state in Fairseq’s repo (https://github.com/pytorch/fairseq/blob/master/fairseq/models/lightconv.py being the core). I could re-implement in TF, but there are clearly some tricks-of-the-trade which make it a little hairy, so would rather not if could avoid (see both pytorch and paper commentary: “Implementation. Existing CUDA primitives for convolutions did not perform very well to implement LightConv and we found the following solution faster on short sequences: We copy and expand the normalized weights… We then reshape and transpose the inputs … and perform a batch matrix multiplication to get the outputs. We expect a dedicated CUDA kernel to be much more efficient.”).

Plus, last I checked, depthwise conv support has been a little wacky in tf (cf. https://github.com/tensorflow/tensorflow/issues?utf8=%E2%9C%93&q=is%3Aissue+depthwise), although perhaps those have been cleaned up?

Thanks…

I wasn’t sure if there is a consensus on this. Of course, there is widespread use of SGD with momentum, Adam, RMSProp, Adagrad, Adadelta, and probably others — but is there an optimizer that is considered SOTA for DNNs “most of the time”? Or is it basically accepted that there is a collection of “good” optimizers whose efficacy varies depending on the task and architecture?

Hi, I’m fairly familiar with audio processing and deep learning but not deeply experienced with speech/ASR/Kaldi. I might need to work on ASR for speech data analysis (e.g., topic modelling etc) as an application, not as research (it’s a company project). Should I Kaldi or not? I’m hesitating to say ‘yes’ to myself because my shallow impression is that it’d take me quite a while to learn how to use, but I’m not sure speech will be my very topic in the future.

​

FYI, if these are relevant, I don’t think we have lots of annotated dataset by ourselves, so I probably should rely on public datasets. At this moment there’s no demand for multiple languages.