Some of us work in image synthesis / translation problems where we can’t use Amazon Mturk.

We need to show people some images and we want them to score / rank each picture or pair or panel etc

Our current approach to this is to create a stack of images or PDF and share it along with an excel etc.

Question 1: is there a tool that we can use to have users see a panel of images and rank/score from a drop down etc? I know we can write our own gui / interface but seems a bit redundant. Is there a simpler way?

Question 2: you can see how the above won’t scale to many different users . Are you aware of any web service / tool instead? (*)

• one time I did this as a Google form and send it to our specialist pool and worked out wonders but I had to hand do it. It seems that Google api has a scripting language to automate things (which means we may be able to automate part) but Google forms has restrictions in the width of images that can be shown so it affects the range of applications.