We’ve just open-sourced our library imagededup, a Python package that simplifies the task of finding exact and near duplicates in an image collection.

​

It includes:

🧮 Several hashing algorithms (PHash, DHash etc) and convolutional neural networks

🔎 An evaluation framework to judge the quality of deduplication

🖼 Easy plotting functionality of duplicates

⚙️ Simple API

​

We’re really excited about this library because finding image duplication is a very important task in computer vision and machine learning. For example, severe duplicates can create extreme biases in your evaluation of your ML model (check out the CIFAR-10 problem). Please try out our library, ⭐️ it on Github and spread the word! We’d love to get feedback.

​

🔤 Code: https://github.com/idealo/imagededup

📕 Docs: https://idealo.github.io/imagededup/

​

https://i.redd.it/8jgr7j0tuiq31.png