Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] lMetrics for similar classes classification

Hello,

I am having trouble finding right metrics for my problem.

I am predicting tags from text (multiclass classification) and I need to come up with metric that allows me to evaluate models.

I am taking top n classes with biggest probability as model output. I can’t monitor just precision/recall/f1 because I have similar classes. For example lets say that text X have classes [ocean, fish, boat] and my predictions are [sea, shark, boat]. I would say that this is correctly classified because sea and ocean are very similar, same for shark and fish. I am currently using word2vec and taking mean cos similarity between each prediction and label which gives maximum cos with given prediction. The problem with this metric is (I suspect) that its value decrease as I increase n (top predictions).

I think that good metrics would be something that combine cos metrics with precision.

Thanks

submitted by /u/matej1408
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.