Association for Psychological Science Symposium

Please join us this Friday, May 21st at 1pm at our APS symposium in New York City. We will be presenting talks on TurkPrime and how to make the most of Mechanical Turk.

Reverse Rejections

Problem:

How can a Requester who rejected an assignment in error undo his mistake? A rejected assignment affects the Worker negatively and will often impact the Requester with negative feedback which can damage the Requester's online reputation which lowers Worker participation in future HITs. What can a Requester do to reverse the rejection?

Solution:

Reversing a rejection in TurkPrime is as simple as using the "Reverse Rejection" feature. Select the WorkerIds you wish to reverse, add an optional message. and you are done! No programming or installations are needed.

Maximizing HIT Participation

Problem:

How can you increase Amazon Mechanical Turk HIT Worker participation rates and speed completion of a HIT? This is particularly an issue with HITs that have a large number of required participants or have Qualifications that limit the number of qualified Workers

Solution:

By monitoring the participation rates of hundreds of HITs we have observed the following patterns that increase participation significantly:

Qualtrics (and other platforms) Mechanical Turk Integration With TurkPrime

Run Qualtrics Surveys on MTurk using TurkPrime

Running Qualtrics surveys on TurkPrime streamlines two major pain-points that Requesters have dealt with:

1. Dynamic secret key that Workers must enter to verify completion of the survey where the code is unique per user and, therefore, un-shareable.
2. Auto approval and rejection of worker assignments based on the dynamic secret key
3. Passing the Mechanical Turk WorkerId and AssignmentId into Qualtrics so that Workers can be uniquely identified and matched up with MTurk.

The following describes these basic features and how to use them.

The TurkPrime Design Survey Form is intuitive and similar to the MTurk Requester HIT Design form except that it adds many useful features like exclude Workers, target specific workers and much more. Below is a portion of the TurkPrime Design Survey form where you specify the Survey Hyperlink.

Qualtrics Surveys With Workers Specific Secret Keys on MTurk using TurkPrime

Now, you can also run Qualtrics surveys with TurkPrime Worker Specific Secret Keys. That means that each worker will be assigned a unique secret key to demonstrate that they completed the linked Qualtrics survey. There is no possibility that workers are sharing a secret key and not completing the linked survey.

Other Platform Integration

The Dynamic Secret Completion Code can be easily integrated (and has been by many researchers) into other non-Qualtrics systems, as well. TurkPrime calculates the Secret Code based on query string parameters that TurkPrime adds to the url of your survey. These parameters are named:

• a
• b
• c

As long as your survey platform ensures that those query string values are preserved in the url when a participant reaches the last page of your survey that contains the TurkPrime iframe, then a completion code should get displayed. 

This can be achieved by capturing the parameters within your study. Nearly all platforms support this functionality. Please contact us if you have questions..

Exclude Workers With One Click

Problem:

Suppose you're running a Mechanical Turk survey and need to exclude workers who took a previous survey. How can you quickly set this up. 

Some of the currently used solutions require following multiple steps to set things up and are not turnkey solutions and others require Workers to enter their Worker ID, which may self-filter workers and limit the number of workers taking your survey. 

Solution:

Exclude Workers Feature 

Create your surveys using TurkPrime.com's "Exclude Workers" feature. When your HIT launches it will have a Qualification Requirement that will limit your HIT to only the Workers not in your exclude list. All excluded workers will be unqualified from taking your HIT.

Longitudinal and Follow up Surveys on Mechanical Turk

Problem:

Suppose you need to run a survey on Mechanical Turk and follow up with the same workers a week, month or year later. A few issues come up:

1. How can I limit the follow up surveys to survey takers who completed the first study - without the worker needing to follow the link and enter their Worker ID? 
2. Can I notify those workers of my follow up survey?
3. Can I set up a follow up survey so that the workers who take that survey do not know why they were selected?

Solution:

Include Workers Feature 

Create your surveys using TurkPrime.com's "Include Workers" feature. When your HIT launches it will have a Qualification Requirement that will limit your HIT to only the Workers you allowed. All other workers will be unqualified from taking your HIT.