We are conducting an academic study on the perception of the technical quality of images.
Your task is to choose the quality of images using our IQAVi web tool.

Introduction

Technical image quality refers to the level of annoyance caused by visible defects in images. In contrast to aesthetics, it is not concerned with the composition of the photograph, the pleasantness of the content or artistic intent.

Please have a look at the illustrations of common defects in the figure below:

Requirements

The accuracy of your annotation is essential. In order to achieve this:

• Use the Chrome web browser.
• Switch your browser to full-screen during the experiment.
  (click the button in the lower right corner of the UI)
• Use a device with a screen resolution greater than 1366 x 960 pixels​.
• Use a screen size larger than 11 inches (28 cm) measured along the diagonal.​
  
• Accurately specify the system information when requested.

​
Abiding by the requirements will ensure a correct submission and trouble-free compensation for your efforts.

How it works

Please follow these instructions carefully to ensure a proper configuration of IQAVi on your computer.

1. Setup

The IQAVi landing page requires you to enter the following information:

• Your Worker ID, as provided on the Amazon MTurk website.
• Screen width and screen height in pixels.
• Screen diagonal in inches.

Please look up your system information if in doubt.

Click [Start Experiment] to continue.

---

2. Annotate images

The main interface of IQAVi.

The task works as follows:

• You will be presented with a sequence of images, one at a time.
• Please use the slider below the image to choose an adequate quality rating from Bad (1) to Excellent (100).
• Click the arrow on the right to proceed to the next image, finally click [Submit Experiment] when you are done.
• Copy and paste your hash-code as specified in the project description to validate your submission.

Important

Extensively review and familiarise yourself with the common image degradation types. Please review the examples below before starting the job. This will greatly improve your ability to complete the task and achieve the required performance.

Examples

You have to rate images on a continuous scale ranging from bad to excellent (use the slider). We provide examples of each range below. Please click to preview each image to get a feel for the quality range displayed in this task.

---

Excellent quality images

Images in this range show minimal or no defects. 

Click on each image to preview at the correct size.

---

Good quality images

Images show some defects, but they are generally not annoying, such as minor exposure problems, blurs, noise, etc.

Click on each image to preview at the correct size.

---

Fair quality images

Images show annoying defects.

Click on each image to preview at the correct size.

---

Poor quality images

Images are strongly degraded, however the subject is generally identifiable.

Click on each image to preview at the correct size.

---

Bad Images

Images show extreme degradations, making the subject often not clearly discernible.

Click on each image to preview at the correct size.

---

Scale vs Quality

Please note that a larger image with a higher resolution isn't necessarily better than a smaller image!

---

Tips

There are no clearly defined boundaries between these image quality classes. As part of the experiment, we will initially provide a few training examples with adequate bounds. Your performance will be monitored throughout the experiment, by automatically checking your answers against pre-defined gold-standard ratings.

However, we are aware that this task depends somewhat on your viewing device and subjective preference. Please provide honest ratings that are as consistent as possible.

---

Quick recap: required steps to complete the task

1. Open IQAVi in the Chrome browser, enter your ID and screen information.
2. ​Select an appropriate quality score for each image.
3. Submit your results at the end.

It takes about 3 seconds to decide on a quality values for each image. 
​Thank you for participating in our study!