Skip to main content


Learn About Our Meetup

5000+ Members



Join our meetup, learn, connect, share, and get to know your Toronto AI community. 



Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.



Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] looking for subjective evaluation tools or web service

Some of us work in image synthesis / translation problems where we can’t use Amazon Mturk.

We need to show people some images and we want them to score / rank each picture or pair or panel etc

Our current approach to this is to create a stack of images or PDF and share it along with an excel etc.

Question 1: is there a tool that we can use to have users see a panel of images and rank/score from a drop down etc? I know we can write our own gui / interface but seems a bit redundant. Is there a simpler way?

Question 2: you can see how the above won’t scale to many different users . Are you aware of any web service / tool instead? (*)

  • one time I did this as a Google form and send it to our specialist pool and worked out wonders but I had to hand do it. It seems that Google api has a scripting language to automate things (which means we may be able to automate part) but Google forms has restrictions in the width of images that can be shown so it affects the range of applications.

submitted by /u/da_g_prof
[link] [comments]