Diversity sampled user generated videos in-the-wild

Subjective video quality assessment (VQA) strongly depends on semantics, context, and the types of visual distortions. A lot of existing VQA databases cover small numbers of video sequences with artificial distortions. When testing newly developed Quality of Experience (QoE) models and metrics, they are commonly evaluated against subjective data from such databases, that are the result of perception experiments. However, since the aim of these QoE models is to accurately predict natural videos, these artificially distorted video databases are an insufficient basis for learning. Additionally, the small sizes make them only marginally usable for state-of-the-art learning systems, such as deep learning. In order to give a better basis for development and evaluation of objective VQA methods, we have created a larger datasets of natural, real-world video sequences with corresponding subjective mean opinion scores (MOS) gathered through crowdsourcing.
​
We took YFCC100m as a baseline database, consisting of 793436 Creative Commons (CC) video sequences, filtered them through multiple steps to ensure that the video sequences are representative of the whole spectrum of available video content, types of distortions, and subjective quality. The resulting 1200 videos are available to download, alongside the subjective data and evaluation of the best-performing techniques available for multiple video attributes. Namely, we have evaluated blur, colorfulness, contrast, spatial information, temporal information and video quality.

Cite us

KoNViD-1k is freely available to the research community. If you use our database in your research, please cite both references:

@misc{konvid1k,
title = {The Konstanz Natural Video Database},
author = {Hosu, Vlad and Hahn, Franz and Jenadeleh, 
Mohsen and Lin, Hanhe and Men, Hui and Szir{\'a}nyi,
Tam{\'a}s and Li, Shujun and Saupe, Dietmar},
year = {2017},
url = {http://database.mmsp-kn.de}}

@inproceedings{hosu2017konstanz,
title = {The Konstanz natural video database 
(KoNViD-1k)},
author = {Hosu, Vlad and Hahn, Franz and Jenadeleh, 
Mohsen and Lin, Hanhe and Men, Hui and Szir{\'a}nyi,
Tam{\'a}s and Li, Shujun and Saupe, Dietmar},
booktitle = {2017 Ninth International Conference on 
Quality of Multimedia Experience (QoMEX)},
pages = {1--6},
year = {2017},
organization = {IEEE}}
​

Alternative download

Downloads

videos (.zip)

2.3 Gb download, 8 second each

meta-data (.zip)

3 Mb download

browse Videos

The meta-data file

•  flickr_id : the video ID as identified on Flickr.com
•  unit_id : crowdflower unit ID
•  created_at : time when the unit was created at
•  golden : if its a normal work item or a test question
•  id : judgement ID
•  quality : quality score from 1 to 5 (individual responses from each crowdworker)
•  user_data : javascript capture user browser related information
•  quality_gold : valid test question answers
•  quality_gold_desired : favored answer (not used when testing)
•  quality_gold_reason : reason displayed to crowdworker on failure
•  file_name : 8s video file name

     Here's more information about other crowdflower.com (now figure-eight.com) fields.