[N] Open-Unmix for Music Separation
It is our great pleasure to announce the release of Open-unmix, a MIT-licensed python implementation for DNN-based music separation.
In the recent years, deep learning-based systems could break a long-standing crystal ceiling, and finally allow high-quality music separation. This provoked a raising interest from both the industry and the machine learning community (like /r/ML)
However, until now, no open-source implementation was available that matches the performance of the best systems proposed more than four years ago. This lead to a waste of time from both the points of view of sheer performance optimization and scientific comparison with the state of the art. Not being able to reproduce state of the art performance makes it difficult to clearly identify the sources for discrepancies and rooms for improvement.
In this context, we release Open-Unmix (UMX) as closing this gap by providing a reference implementation for DNN-based music separation. It serves two main purposes. First, it is intended to academic researchers for serving as a baseline method that is easy to compare to and build upon. Second, the availability of a pre-trained model allows bringing music separation to the enthusiastic end users and artists.
Open-unmix is presented in a paper that has just been published in the Journal of Open Source Software. You may download the paper PDF here
Open-unmix comes in several DNN frameworks:
- we provide extend documentation and further demos on the sigsep website.
Open-unmix has been especially designed to combine well with the following datasets:
- MUSDB18 has become one of the most popular dataset in Source Separation and MIR. We provide full lengths music tracks (~10h duration) of different genres along with their isolated drums, bass, vocals and others stems.
- MUSDB18-HQ: together with Open-Unmix, we also released an additional flavor of the dataset for models that aim to predict high bandwidth of up to 22 kHz. Other than that, MUSDB18-HQ is identical to MUSDB18.
=> Both datasets are available at https://sigsep.github.io/datasets/musdb.html
- Open-unmix also offers a variety of template dataset structures that should be appropriate for many other use cases
If you want to compare separation models to existing source separation literature or if want compare to SiSEC 2018 participants, please use the standard MUSDB18 dataset, instead.
We provide pre-trained models trained on both
MUSDB18-HQ that reach state-of-the-art performance of 6.32 dB SDR (median of medians) on vocals on MUSDB18 test data. This significantly outperforms any model we are aware of that was trained on MUSDB18 only.
The pre-trained models are automatically bundled/downloaded when using the pytorch implementation.
Further information for both models such as evaluation scores can be downloaded from zenodo:
Open-unmix was recently proposed during a tutorial held at EUSIPCO 2019. This features:
- A recent overview into current source separation method with a focus on deep learning
- A lecture on spectrogram models and wiener filtering
- Visualizations and results of Open-Unmix compared to state-of-the-art
The slides of the tutorial as well as self-contained colab notebooks can be found on the tutorial site.
Open-unmix is part of a whole ecosystem enabling easy research on source separation for Python users. Several distinct and independent projects were released in the recent years in this effort to make it possible for researchers to reproduce state of the art performance in this domain.
A reliable python package that implements the multichannel wiener filter and related filtering methods.
We released the new version 0.3.0 of our popular musdb tools. This releases makes it simpler to use musdb inside your data loading framework thus we pro
museval makes it easy to compare the performance of any new method under investigation to both Open-unmix and the participants of SiSEC18.
Please note that we are also working on some version of open-unmix that has been trained on a significantly larger dataset and that achieves unprecedented separation performance. Please feel free to contact us for demonstrations / industrial collaborations / licensing on this matter.
We look forward to your feedback and we hope that you will find Open-unmix useful!