Movie Annotations

Here you will find word and face annotations for all the movies screened for the NNDb.

Words and Faces were annotated in the movies using fully automated approaches. For Words, the audio track and the subtitle track were extracted from each movie. The audio file was input into ‘Amazon Transcribe’ from Amazon Web Services. To estimate the on and offset times of the words not transcribed, a script was written that first uses dynamic time warping (DTW) to align word onsets from the speech-to-text transcript to corresponding subtitle words in each individual subtitle page. Subtitle words that matched or that were similar to the transcriptions during the DTW procedure inherited the timing of the transcriptions. Remaining subtitle words not temporally labeled were then estimated, with different degrees of accuracy. For Faces we used the AWS ‘Amazon Rekognition’ API to obtain ML-based faces annotations, with no script modifications.

Please note that although accuracy is very high, there is still room for improvement. For example, Words have four accuracy levels, in order:

- "Matched" means that the word is 100% accurate

- "Continuous" means the word is in between two perfect matches, so still highly accurate

- "Partial" means that more than one word was found in between two perfect matches, so on/offset is somewhat accurate

- "Full" means that the word timing was completely estimated, therefore low accuracy If you are interested in analysing stimuli with ML, please do get in touch!

12 Years a Slave

First 20 entries for Words and Faces annotations for the movie "12 Years a Slave".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

500 Days of Summer

First 20 entries for Words and Faces annotations for the movie "500 Days of Summer".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

Back to the Future

First 20 entries for Words and Faces annotations for the movie "Back to the Future".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

Citizenfour

First 20 entries for Words and Faces annotations for the movie "Citizenfour".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

Little Miss Sunshine

First 20 entries for Words and Faces annotations for the movie "Little Miss Sunshine".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

Pulp Fiction

First 20 entries for Words and Faces annotations for the movie "Pulp Fiction".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

Split

First 20 entries for Words and Faces annotations for the movie "Split".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

The Prestige

First 20 entries for Words and Faces annotations for the movie "The Prestige".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

The Shawshank Redemption

First 20 entries for Words and Faces annotations for the movie "The Shawshank Redemption".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.

The Usual Suspects

First 20 entries for Words and Faces annotations for the movie "The Usual Suspects".

Words

For the whole Words dataset, click the Download button below.

Faces

For the whole Faces dataset, click the Download button below.