Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

Author: torontoai

How to evaluate ML models using confusion matrix?

Model Evaluation using Confusion Matrix

Model evaluation is a very important aspect of data science. Evaluation of a Data Science Model provides more colour to our hypothesis and helps evaluate different models that would provide better results against our data.

What Big-O is to coding, validation and evaluation is to Data Science Models.

Photo by Leon Koye on Unsplash

When we are implementing a multi-class classifier, we have multiple classes and the number of data entries belonging to all these classes is different. During testing, we need to know whether the classifier performs equally well for all the classes or whether there is bias towards some classes. This analysis can be done using the confusion matrix. It will have a count of how many data entries are correctly classified and how many are misclassified.

Let’s take an example. There is a total of ten data entries that belong to a class, and the label for that class is “Class 1”. When we generate the prediction from our ML model, we will check how many data entries out of the ten entries get the predicted label as “Class 1”. Suppose six data entries are correctly classified and get the label “Class 1”. In this case, for six entries, the predicted label and True(actual) label is the same, so the accuracy is 60%. For the remaining data entries (4 entries), the ML model misclassifies them. The ML model predicts class labels other than “Class 1”. From the preceding example, it is visible that the confusion matrix gives us an idea about how many data entries are classified correctly and how many are misclassified. We can explore the class-wise accuracy of the classifier.

Source: ML Solutions

For more learning on similar topics, the ML solutions book provides good explanations.

For more such answers to important Data Science concepts, please visit Acing AI.

Subscribe to our Acing AI newsletter, I promise not to spam and its FREE!

Acing AI Newsletter – Revue

Thanks for reading! 😊 If you enjoyed it, test how many times can you hit 👏 in 5 seconds. It’s great cardio for your fingers AND will help other people see the story.


How to evaluate ML models using confusion matrix? was originally published in Acing AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

[D] Help on how to extract specific text from image with printed and handwritten text

Hi there,

I have a task where I need to extract specific elements (name, surname, company name) from text present in images which are forms and other administrative documents (example here), but there are many types of documents and each are structured differently.

Sometimes the required elements will be handwritten, sometimes not. They also will be placed differently on the page depending on the document.

And the data is not annotated. (do I need to annotate it myself before, or can it be made unsupervised ?)

So I’m at a real loss here on what to do. (I’m a beginner in the field so please bear with me :.))

I’m really not an expert on NLP nor text extraction or detection. Mostly worked on images with classical datasets, but with neural achitectures mostly.

Classic OCR techniques I tested like Tesseract don’t work at all for handwritten text (maybe there’s appropriate configuration for this ?).

Since the data is so unstructured (to me) I thought of using neural networks and especially a Faster RCNN model which I would train to detect only the three elements I mentioned before (after having annotated the data myself) as if they were specific objects like for classical Faster RCNN models. Paper (use hub of science) on researchers successfully doing it.

But someone I talked to told me it was easy to do even with the constraints I told you, and that neural networks weren’t even needed, nor was annotating.

My guess at the beginning was that I needed to find a way to extract the whole text from the image (with the handwritten text, and well positioned in relation to the printed text) and then find a way to give my algorithm insight on how to distinguish names from the rest.

How could I do this with simple machine learning ? Is there a dataset of names and surnames that I can train my model on so that it would understand their intrinsic characteristics be able to distinguish them from other text ?

Thanks a lot in advance for your help !

submitted by /u/Atralb
[link] [comments]

[R] Style Transfer with GANs on HD Images

I have been recently exploring how to make GANs work with HD images, without the need of expensive GPUs and long computation times.

This is my result:

https://towardsdatascience.com/style-transfer-with-gans-on-hd-images-88e8efcf3716

It surely isn’t anything new, but I have never heard of this super easy solution being applied to this kind of goal.

I would love to know if there are other methods in literature similar to this one.

submitted by /u/artika_labs
[link] [comments]

[P] Lyft releases self-driving research dataset

Announcement

We’re thrilled to share a comprehensive, large-scale dataset featuring the raw sensor camera and LiDAR inputs as perceived by a fleet of multiple, high-end, autonomous vehicles in a bounded geographic area. This dataset also includes high quality, human-labelled 3D bounding boxes of traffic agents, an underlying HD spatial semantic map.

With this, we aim to empower the community, stimulate further development, and share our insights into future opportunities from the perspective of an advanced industrial Autonomous Vehicles program.

Link to the dataset’s page with more technical information and examples.

There’s also going to be a competition at NeurIPS this year, which I assume to be the recently added “AI Driving Olympics” at the very bottom of the competitions page…

submitted by /u/chisai_mikan
[link] [comments]

[P] Sobert, a neural network chat bot

I’ve been working on a small project for a few years, and I finally made a simple web interface. Sobert is a chat bot using a 4-layer, 113 million parameter byte-level Grid GRU language model trained on ~370MB of chat data collected over the years.

Go to https://sobert.hammy.sk/, click connect and say hello! It’s one shared chat, so don’t post anything private – everyone in the chat can see it.

Be careful, Sobert understands many topics, and not all of them are SFW. The web version is also new and not well tested, and won’t be able to handle too many users, but the Telegram version should be quite reliable.

Code for all the related parts

submitted by /u/DevigForager
[link] [comments]

[P] Final Project ANN

Hello, I’m Hendra. I’m an undergraduate students who is working on final project so I can graduate. My lecturer asked me to use ANN for energy disaggregation system. I’m using keras machine learning with tensor flow as the backend. Lately I have a problem on making a correct model for my case and got bad accuracy score. I have 4 lamps with various power consumption and I want to predict the power consumption of each lamp using ANN. My dataset is on link. On the dataset, 1 means ON and 0 means OFF. I’m using jupyter notebook as its interface. Can anyone give me some idea about what models I can use for this type of problem? I’m currently stumped. Any kind of help is really appreciated. Thank you….. Here is the dataset : https://drive.google.com/file/d/1ZrW083twfEIHZzqGuaDYA7jjO2s5P44p/view?usp=sharing

submitted by /u/hendraahalimm
[link] [comments]

[D] Is there a SOTA model or standardized dataset with a focus on detection/classification of object *attributes*, not just object type?

Hi, pretty new to computer vision here. I’ve seen a lot of standardized datasets for CV such as MNIST, CIFAR-10, ImageNet, etc. ImageNet in particular stood out to me because of how specific a lot of the image classifications were, such as “pug” or “dalmation”. However, is there a model out there which focuses on classifying attributes of the object e.g. dog color, or a standardized dataset that would enable that analysis?

Pointing me towards the relevant literature would be really appreciated! I’ve done my Googling but I think I might not know the specific terms used in this domain to describe stuff like this, because I keep running into generic object classification and detection without the object attributes.

submitted by /u/toadsofbattle
[link] [comments]