Skip to main content

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

TRECVID 2019 ActEV: Activities in Extended Video


ActEV 2019 Leaderboard

Download TRECVID 2019 Oct 1 Results Snapshot here.


TRECVID

Leaderboard will be available soon.
Updated: 2019-11-06 16:51:19 -0500
A new report is being generated at this time. Please reload the page in a few minutes. You are being shown the last available report.

RANK SCORING REQUEST NAME SCORING_REQUEST_ID EVALUATION_NAME SCORING_PROTOCOL TEAM NAME SYSTEM NAME ?COLUMN? PARTIAL AUDC* PRIZE_ELIGIBLE MEAN-P_MISS@0.15TFA MEAN-W_P_MISS@0.15RFA
1 ActEV-2018_AD_TRECVID_SYS-00299_MUDSML_20191001-094021-7819.sr-20191001-094021-8591 4637 ActEV-2018 ActEV_SDL_V1 MUDSML MMVG-INF-Etrol "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.48407 0.39152 0.79790
2 ActEV-2018_AD_TRECVID_SYS-00311_MUDSML_20191001-091108-3032.sr-20191001-091108-3722 4636 ActEV-2018 ActEV_SDL_V1 MUDSML MMVG-AlibabaAIC-Etrol-P "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.48615 0.41542 0.78032
3 ActEV-2018_AD_TRECVID_SYS-00271_team-arnet_20191001-064833-8241.sr-20191001-064833-8639 4626 ActEV-2018 ActEV_SDL_V1 team-arnet team-arnet-P "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.49099 no 0.38580 0.70228
4 ActEV-2018_AD_TRECVID_SYS-00305_team-arnet_20191001-074203-5294.sr-20191001-074203-5599 4628 ActEV-2018 ActEV_SDL_V1 team-arnet team-arnet - S3 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.49148 no 0.38781 0.70162
5 ActEV-2018_AD_TRECVID_SYS-00304_team-arnet_20191001-130916-2238.sr-20191001-130916-2726 4644 ActEV-2018 ActEV_SDL_V1 team-arnet team-arnet - S2 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.49216 no 0.38718 0.70334
6 ActEV-2018_AD_TRECVID_SYS-00303_team-arnet_20191002-115559-7760.sr-20191002-115559-8177 4721 ActEV-2018 ActEV_SDL_V1 team-arnet team-arnet - S1 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.49297 no 0.39629 0.70723
7 ActEV-2018_AD_TRECVID_SYS-00296_MUDSML_20190930-234117-4412.sr-20190930-234117-5058 4615 ActEV-2018 ActEV_SDL_V1 MUDSML MMVG "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.49595 0.42705 0.76920
8 ActEV-2018_AD_TRECVID_SYS-00260_MUDSML_20191001-084452-5431.sr-20191001-084452-6025 4635 ActEV-2018 ActEV_SDL_V1 MUDSML MMVG-AlibabaAIC-Etrol "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.50911 0.44783 0.84777
9 ActEV-2018_AD_TRECVID_SYS-00293_BUPT-MCPRL_20191013-225650-8429.sr-20191013-225650-8911 4913 ActEV-2018 ActEV_SDL_V1 BUPT-MCPRL MCPRL_S1 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.52196 0.43032 0.74639
10 ActEV-2018_AD_TRECVID_SYS-00306_BUPT-MCPRL_20191013-225015-7293.sr-20191013-225015-7756 4911 ActEV-2018 ActEV_SDL_V1 BUPT-MCPRL MCPRL_S0 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.52283 0.43228 0.75058
11 ActEV-2018_AD_TRECVID_SYS-00301_BUPT-MCPRL_20191001-114332-0195.sr-20191001-114332-0587 4643 ActEV-2018 ActEV_SDL_V1 BUPT-MCPRL MCPRL_S3 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.52408 0.43280 0.74914
12 ActEV-2018_AD_TRECVID_SYS-00295_BUPT-MCPRL_20191013-225339-3197.sr-20191013-225339-3624 4912 ActEV-2018 ActEV_SDL_V1 BUPT-MCPRL MCPRL_S2 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.54147 0.45985 0.74639
13 ActEV-2018_AD_TRECVID_SYS-00274_Hitachi_20191021-053430-0091.sr-20191021-053430-0615 4981 ActEV-2018 ActEV_SDL_V1 Hitachi primary "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.59470 0.50496 0.82238
14 ActEV-2018_AD_TRECVID_SYS-00280_Hitachi_20191001-071718-4772.sr-20191001-071718-5143 4627 ActEV-2018 ActEV_SDL_V1 Hitachi fourth "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.59889 0.50995 0.82406
15 ActEV-2018_AD_TRECVID_SYS-00291_NTT-CQUPT_20191001-033229-7342.sr-20191001-033229-7784 4621 ActEV-2018 ActEV_SDL_V1 NTT_CQUPT p-NTT-CQUPT "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.60058 0.51122 0.87254
16 ActEV-2018_AD_TRECVID_SYS-00278_vireoJD-MM_20190815-015818-3558.sr-20190816-114358-2538 2826 ActEV-2018 ActEV_SDL_V1 vireoJD-MM Pipeline1 "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.60120 no 0.54827 0.72846
17 ActEV-2018_AD_TRECVID_SYS-00275_Hitachi_20191001-041244-0442.sr-20191001-041244-0815 4624 ActEV-2018 ActEV_SDL_V1 Hitachi secondary "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.60221 0.51508 0.82456
18 ActEV-2018_AD_TRECVID_SYS-00294_NTT-CQUPT_20191001-083642-1950.sr-20191001-083642-2365 4634 ActEV-2018 ActEV_SDL_V1 NTT_CQUPT p2_NTT_CQUPT "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.60396 0.51677 0.87168
19 ActEV-2018_AD_TRECVID_SYS-00297_NTT-CQUPT_20191001-094407-4183.sr-20191001-094407-4642 4638 ActEV-2018 ActEV_SDL_V1 NTT_CQUPT system2 "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.60524 0.51755 0.87381
20 ActEV-2018_AD_TRECVID_SYS-00279_Hitachi_20190930-225009-4325.sr-20190930-225009-4695 4612 ActEV-2018 ActEV_SDL_V1 Hitachi third "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.63338 0.56872 0.82467
21 ActEV-2018_AD_TRECVID_SYS-00289_vireoJD-MM_20190817-095405-8047.sr-20190827-105515-7169 3667 ActEV-2018 ActEV_SDL_V1 vireoJD-MM SecondarySystem "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.69360 no 0.66054 0.77857
22 ActEV-2018_AD_TRECVID_SYS-00292_IVP_20190822-050341-6908.sr-20190822-050341-7297 3119 ActEV-2018 ActEV_SDL_V1 IVP p_NTT-CQUPT "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.71621 0.64146 0.76520
23 ActEV-2018_AD_TRECVID_SYS-00287_Fraunhofer-IOSB-Team_20190828-090644-8466.sr-20190828-090644-8849 3698 ActEV-2018 ActEV_SDL_V1 Fraunhofer IOSB Team primary "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.82702 0.77475 0.84748
24 ActEV-2018_AD_TRECVID_SYS-00257_NIST-TEST_20190503-142054-2674.sr-20190816-114351-3114 2789 ActEV-2018 ActEV_SDL_V1 NIST-TEST Test System I "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.85649 no 0.82473 0.87757
25 ActEV-2018_AD_TRECVID_SYS-00310_hsmw-tuc_20191001-094849-3836.sr-20191001-094849-4201 4639 ActEV-2018 ActEV_SDL_V1 hsmw_tuc hsmw_tuc "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.94064 0.93551 0.94371
26 ActEV-2018_AD_TRECVID_SYS-00313_Activity-Detector_20191105-100957-0380.sr-20191105-100957-0933 5682 ActEV-2018 ActEV_SDL_V1 Activity Detector Semantic Network "a9e2fd733f22c00174cb410e43ce05eb046f9cd2--2019-09-11T13:37:01+00:00" 0.96354 0.94959 0.94843
27 ActEV-2018_AD_TRECVID_SYS-00269_MKLab_20190826-120336-4636.sr-20190826-120336-4991 3637 ActEV-2018 ActEV_SDL_V1 MKLab Secondary System 2 "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.96424 0.95781 0.96180
28 ActEV-2018_AD_TRECVID_SYS-00268_MKLab_20190709-063034-0057.sr-20190816-114352-7616 2798 ActEV-2018 ActEV_SDL_V1 MKLab SecondarySystem "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.96424 0.95781 0.96180
29 ActEV-2018_AD_TRECVID_SYS-00267_MKLab_20190709-084820-9193.sr-20190816-114352-8847 2799 ActEV-2018 ActEV_SDL_V1 MKLab 1rstSystem "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.96424 0.95781 0.96180
30 ActEV-2018_AD_TRECVID_SYS-00273_MKLab_20190716-074250-4015.sr-20190816-114353-7049 2802 ActEV-2018 ActEV_SDL_V1 MKLab 3rdSystem "24b83a354c79f848e0524d7beaba97229d6af0bb--2019-08-29T21:43:42+00:00" 0.97356 0.96845 0.97061
Download CSV

TRECVID

Leaderboard will be available soon.
Updated:
A new report is being generated at this time. Please reload the page in a few minutes. You are being shown the last available report.

*Partial AUDC is the area under the DET curve between a Time-based False Alarm rate of 0 and 0.2. Value of a perfect system is 0.

Contact Us

For ActEV 2019 Evaluation information (data, evaluation code, etc.) please email: actev-nist@nist.gov

For ActEV 2019 Evaluation Discussion Please Visit our Google Group. https://groups.google.com/a/list.nist.gov/forum/#!forum/trecvid.actev

Activities and Tasks

List of Activities per Task


The table below provides a summary of the activity list and the required objects for the TrecVID ActEV 2019 evaluation. The eighteen target activities for TrecVID ActEV 2019 are the same as for ActEV-PC Leaderboard. The target objects for the TrecVID ActEV 2019 evaluation are P={Person} and V={Construction_Vehicle, Vehicle}.

ActEV-PC LeaderBoard Eval TrecVID ActEV 2019
Closing (P, V) or (P)
Closing_trunk (P, V)
Entering (P, V) or (P)
Exiting (P, V) or (P)
Loading (P, V)
Open_Trunk (P, V)
Opening (P, V) or (P)
Transport_HeavyCarry (P, V)
Unloading (P, V)
Vehicle_turning_left (V)
Vehicle_turning_right (V)
Vehicle_u_turn (V)
Pull (P)
Riding (P)
Talking (P)
activity_carrying (P)
specialized_talking_phone (P)
specialized_texting_phone (P)
 Closing (P, V) or (P)
Closing_trunk (P, V)
Entering (P, V) or (P)
Exiting (P, V) or (P)
Loading (P, V)
Open_Trunk (P, V)
Opening (P, V) or (P)
Transport_HeavyCarry (P, V)
Unloading (P, V)
Vehicle_turning_left (V)
Vehicle_turning_right (V)
Vehicle_u_turn (V)
Pull (P)
Riding (P)
Talking (P)
activity_carrying (P)
specialized_talking_phone (P)
specialized_texting_phone (P)
Task for the TrecVID ActEV 2019 Evaluation
In the TrecVID ActEV 2019 evaluation, there is one Activity Detection (AD) task for detecting and localizing of activities
Activity Detection (AD)

For the Activity Detection task, given a target activity, a system automatically detects and temporally localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the type of activity must be correct and the temporal overlap must fall within a minimal requirement as described in the Evaluation Plan.
TRECVID ActEV 2019 Evaluation
Updates
New TRECVID ActEV Deadline: October 01, 2019: 1:00 PM EST, But we will keep the leaderboard running till Nov 15.
Summary
ActEV is an series of evaluations to accelerate development of robust, multi-camera, automatic activity detection algorithms for forensic and real-time alerting applications. Each evaluation will challenge systems with new data, system requirements, and/or new activities. For more details about the previous evaluations and example videos see the main ActEV page and click on the example videos tab
What is Activity Detection in Videos ?
An ActEV activity is defined to be “one or more people performing a specified movement or interacting with an object or group of objects”. Activity detection technologies process extended video streams, such as those from a IP camera, and automatically detects all instances of the activity by: (1) identifying the type of activity, (2) producing a confidence score indicating the presence of instance, (3) temporally localizing the instance by indicating the begin and end times, and (4) optionally, detecting and tracking the objects (people, vehicles, objects) involved in the activity.
What
The ActEV evaluation is being conducted to assess the robustness of automatic activity detection for a multi-camera streaming video environment.
Who
NIST invites all organizations, particularly universities and corporations, to submit their results using their technologies to the ActEV evaluation server. The evaluation is open worldwide. Participation is free. NIST does not provide funds to participants.
How
To take part in the ActEV evaluation you need to register on the actev.nist.gov website and acknowledge that you have read and accepted the data license to download the data.
Evaluation Task
In the TrecVID ActEV 2019 evaluation, there is one Activity Detection (AD) task for detecting and localizing of activities
Activity Detection (AD)
For the Activity Detection task, given a target activity, a system automatically detects and temporally localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the type of activity must be correct and the temporal overlap must fall within a minimal requirement as described in the Evaluation Plan.
Data
The TrecVID ActEV 2019 evaluation is only based on the VIRAT V1 and V2 dataset

The Multiview Extended Video with Activities (MEVA) will be used in a future evaluation, but we would like the participants to help us annotate the data for the 38 activities as defines in the annotation guide (coming soon) . The data will be provided in MPEG-4 and AVI format. Please click on the data tab for information on how to download the data.
Metrics
The main scoring metrics will be based on detection, temporal localization using evaluation measures that include the Probability of Missed Detection and Time-Based False Alarm” (TFA) measure. See details in the evaluation plans. NIST maintains the ActEV Scoring Software on the Scoring software for the Activities in Extended Video (ActEV) evaluation GitHub repo. (e.g., Probability of Missed Detection, Time-Based False Alarm” (TFA)) (see: Evaluation Plan).
Scoring software for the TRECVID Activities in Extended Video (ActEV) evaluation
Evaluation Plan
ActEV-PC Evaluation Plan
Task coordinator
ActEV NIST team (ActEV-nist@nist.gov)
News
15April
Register for ActEV 2019
01May
ActEV 2019 Evaluation starts.
01Oct
ActEV 2019 Evaluation now ends.
TRECVID ActEV 2019 Evaluation Schedule
May 01, 2019: Start of ActEV 2019 Leaderboard Evaluation

October 01, 2019: 1:00 PM EST : End of ActEV 2019 Leaderboard Evaluation (Updated Deadline)
But we will keep ActEV Evaluation Leaderboard running till November 15th at 11:00 am.

November 12-13, 2019: NIST present the ActEV leaderboard evaluation results at TRECVID'19


TRECVID 2019 ActEV dataset

This GIT Repo is the data distribution machanism for the ActEV evaluation. The repo presently consists of a collection of corpora (plural for corpus) and partition defintion files to be used for evaluations. Future additions will include source annotations and donated data/annotations. The repo contains textual data but not the large-sized corpora (videos, etc.). For these large corpora, the repo contains the necessary information to download a given corpus giving the user control of the downloaded content. Access credentials provided during signup (see below)

  • Create a login account by registering (using the link above) for the TrecVID 2019 ActEV
  • During account registration, you will:
  • You will receive the username and password for the video data download. If there is any issue please email us at actev.nist@nist.gov

Rules for Leaderboard Evaluation Schedule
Durning the TRECID ActEV 2019 evaluation, you can create a maximum of four systems and submit maximum of two results per day and maximum of 70 results in total for the AD task

The challenge participant can train their systems or tune parameters using any data complying with applicable laws and regulations. In the event that external limitations preclude sharing such data with others, participant are still permitted to use the data, but they must inform NIST that they are using such data, and provide appropriate detail regarding the type of data used and the limitations on distribution.

The challenge participant agree not to probe the test videos via manual/human means such as looking at the videos to produce the activity type and timing information from prior to the evaluation period to end of leaderboard evaluation.

All machine learning or statistical analysis algorithms must complete training, model selection, and tuning prior to running on the test data. This rule does not preclude online learning/adaptation during test data processing so long as the adaptation information is not reused for subsequent runs of the evaluation collection.

The only VIRAT data that may be used by the systems are the ActEV provided training and validation sets, associated annotations, and any derivatives of those sets (e.g. additional annotations on those videos). All other VIRAT data and associated annotations may not be used by any of the systems for the ActEV Leaderboard Evaluation.



If you have any question, please email to actev-nist@nist.gov