TRECVID 2020 ActEV: Activities in Extended Video

New TRECVID ActEV Deadline: Friday, November 06 2020: 1:00 PM EST, But we will keep the leaderboard running till Dec 15, 2020.
June 09, 2020: We’ve made new partitions for the TRECVID 2020 ActEV data sets and added the to the actev-data-repo.
To get the updates, do a ‘git pull’. There are three new partitions:

• partitions/ActEV20-TRECVID-eval-20200604
• partitions/ActEV20-TRECVID-train-20200604
• partitions/ActEV20-TRECVID-validate-20200604

The leaderboard will soon be set up with the new eval partition. We will inform you when it is available.

ActEV is an series of evaluations to accelerate development of robust, multi-camera, automatic activity detection algorithms for forensic and real-time alerting applications. Each evaluation will challenge systems with new data, system requirements, and/or new activities. For more details about the previous evaluations and example videos see the main ActEV page and click on the example videos tab

An ActEV activity is defined to be “one or more people performing a specified movement or interacting with an object or group of objects”. Activity detection technologies process extended video streams, such as those from a IP camera, and automatically detects all instances of the activity by: (1) identifying the type of activity, (2) producing a confidence score indicating the presence of instance, (3) temporally localizing the instance by indicating the begin and end times, and (4) optionally, detecting and tracking the objects (people, vehicles, objects) involved in the activity.

The ActEV evaluation is being conducted to assess the robustness of automatic activity detection for a multi-camera streaming video environment.

NIST invites all organizations, particularly universities and corporations, to submit their results using their technologies to the ActEV evaluation server. The evaluation is open worldwide. Participation is free. NIST does not provide funds to participants.

To take part in the ActEV evaluation you need to register on the actev.nist.gov website and acknowledge that you have read and accepted the data license to download the data.

In the TrecVID ActEV 2020 evaluation, there is one Activity Detection (AD) task for detecting and localizing of activities
For the Activity Detection task, given a target activity, a system automatically detects and temporally localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the type of activity must be correct and the temporal overlap must fall within a minimal requirement as described in the Evaluation Plan.

The TrecVID ActEV 2020 evaluation is only based on the VIRAT V1 and V2 dataset

The evaluation will be based on 35 activities from the activities listed in the activities tab and the names have been updated for the evaluation. The data is provided in MPEG-4 format . You can download the public VIDEO dataset for free at viratdata.org; and more info about the datasets is on the data tab

Submitted activity detection systems must give a confidence score for each activity they detect. Detected activities are then thresholded based on this confidence score. Varying the threshold makes a trade-off between being sensitive enough to identify true activity instances (low threshold) vs. not making false alarms when no activity is present (high threshold). Submitted systems are scored on both of these, measured by Probability of Missed Detection (Pmiss) and Time-based False Alarm (TFA). Pmiss is the portion of activities where the system did not detect the activity for at least 1 second. TFA is the portion of time that the system detected an activity when in fact there was none. Submitted systems are scored for Pmiss and TFA at multiple thresholds, creating a detect error tradeoff (DET) curve. The leaderboard ranking of a system is based on a summary of its DET curve: the area under the DET Curve across the TFA range between 0% to 20% divided by 0.2 to normalize the value to [0:1]. Lower numbers are better, as they reflect fewer errors.

See details in the TRECVID ActEV 2020 evaluation plan or Scoring software for the TRECVID Activities in Extended Video (ActEV) evaluation

ActEV NIST team (ActEV-nist@nist.gov)