Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] Baselines for recommendation systems

Recommendation systems are evaluated on a variety of tasks.

  • Top-N Prediction. N items are predicted for the user. This paper finds that a bunch of existing neural network approaches that use this task are outperformed by simple baselines or not reproducible.
  • Rating Prediction. The rating of items are predicted. This apparently has fallen out of favour. Despite being featured on most introductory tutorials to recommendation systems.
  • Sequential Prediction. The next item that a user will interact with is predicted. This is featured in some deep neural network approaches that process sequential data.

  • And more…

According to the reproducibility paper linked above accuracy on datasets like MovieLens is not informative. However, this is the dataset used in most papers, including the spotlight repository that implements deep algorithms. Many recent papers instead prioritize diversity.

So these are my basic questions, for a baseline system:

  • What is a standard reliable dataset?
  • What are some good evaluation metrics?
  • Which tasks should the system be evaluated on?

I am really struggling to get answers from the literature, as they are quite diverse in all three of these aspects. What do you guys think?

submitted by /u/ThomasAger
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat

 


Toronto AI is a social and collaborative hub to unite AI innovators of Toronto and surrounding areas. We explore AI technologies in digital art and music, healthcare, marketing, fintech, vr, robotics and more. Toronto AI was founded by Dave MacDonald and Patrick O'Mara.