Link To TurkPrime

How to run successful experiments and get the most out of Amazon's Mechanical Turk

Showing posts with label autoapproval. Show all posts
Showing posts with label autoapproval. Show all posts

Friday, December 2, 2016

Dynamic Secret Completion Codes for SurveyMonkey

TurkPrime Supports Dynamic Completion Code for SurveyMonkey

Users of Qualtrics and Google Forms have long enjoyed the ability to integrate dynamic secret codes for each participant in their TurkPrime study. Dynamic codes ensure that each MTurk worker participating in your study receives a unique code completely eliminating the possibility that workers share secret codes.  In addition, with auto-approval enabled, those workers are automatically approved without any need for researchers to manually check the worker supplied secret codes and approve workers.

Now, TurkPrime users who host their studies on SurveyMonkey can also use dynamic completion codes as follows:
  1. Check off "Dynamic Completion Code For Qualtrics" when you design your TurkPrime study
  2. Redirect participants at the end of your SurveyMonkey study to https://www.turkprime.com/Router/DynamicCode

Friday, February 19, 2016

New Safety Feature: Assignment Rejections are not Automated and Require Manual User Action

Many researchers set up their studies to use the TurkPrime AutoApprove feature so that they do not need to manually approve worker assignments based on the secret code that workers enter. On occasion, a researcher may set up his study incorrectly which results in many worker assignments getting automatically rejected. This was a significant cause for distress among the Mechanical Turk workers who received rejections for their work which then went unpaid and also lowered their MTurk approval rating.  This was also a sore point for TurkPrime researchers who had to deal with upset workers and correspond with them, and often reverse their rejections

To alleviate this issue, TurkPrime will give researchers greater control of the rejection process when AutoApprove is used: 

Only worker assignments that have the correct secret codes will be auto-approved while workers with invalid secret codes will require manual rejection. (If a rejection is made in error by a researcher, it may still be reversed.) This manual rejection is made using the same interface as is available for non-AutoApproved workers as shown below.

Researchers will see the secret codes the workers entered and whether they were correct. they will then have the option to approve, reject or leave their status undecided.

The manual rejection process must be completed within the Amazon Approval window (default of 7 days). Otherwise, Mechanical Turk will automatically approve the pending assignments that were not resolved buy that time.