ActEV Sequestered Data Leaderboard (SDL) Evaluation
Updates
- March 01, 2020: The ActEV 2020 SDL opens with the expanded MEVA Test3 dataset
- March 01, 2020: ActEV 2020 SDL is a guest task under ActivityNet CVPR'20 ActivityNet workshop
-
May 17, 2020
May 10, 2020at 12:00 noon EST: New Deadline for CLI submissions to be included in ActEV SDL ActivityNet rankings. Top two submissions to be announced on June 1, 2020. - June 14, 2020: CVPR'20 ActivityNet workshop ActEV SDL guest task presentations
- April 28th, 2020: CLI submissions deadline extended to May 17th, 2020 at 12:00 noon EST
Summary
The ActEV (Activities in Extended Video) Sequestered Data Leaderboard is an ongoing ranking
of software systems that watch lengthy videos and detect activities of interest. Anyone can
submit their system to NIST, which will then run the system on sequestered data, score the
results and post the score to the leaderboard.
The sequestered data is from the MEVA dataset, which contains hours of videos, including indoor and outdoor scenes, night and day, crowds and individuals, and videos are from both EO (Electro-Optical) and IR (Infrared) sensors. Hours can go by with no activities, but then multiple activities happen simultaneously. The data is multi-camera, in that multiple cameras may be pointed at the same scene at the same time. There are also two separate leaderboards for EO and IR videos.
The sequestered data is from the MEVA dataset, which contains hours of videos, including indoor and outdoor scenes, night and day, crowds and individuals, and videos are from both EO (Electro-Optical) and IR (Infrared) sensors. Hours can go by with no activities, but then multiple activities happen simultaneously. The data is multi-camera, in that multiple cameras may be pointed at the same scene at the same time. There are also two separate leaderboards for EO and IR videos.
What
Build and submit a software system to watch videos and detect if and when an
activity of interest occurs. System runtime
must be 1 times the video length on the designated
evaluation hardware.
Who
Everyone. Anyone who registers
can submit to the evaluation server.
How
Register here and then
follow the instructions on the
Algorithm Submission tab above.
Systems must follow a NIST-defined command line interface and automatically
run on NIST’s servers, both of which are described in the instructions.
Evaluation Task
Detect if and when an activity occurs. Given a target activity type and a set
of videos, submitted systems must automatically detect all instances of the activity in the
videos. While different activities instances have different durations, a submitted system
will be considered to have detected an activity if it correctly identifies at least 1 second
of the activity.
Data
The data is from the Multiview Extended Video
with Activities (MEVA) dataset and the videos are from both EO (Electro-Optical) and
IR (Infrared) sensors. The data used for SDL evaluation from March 01, 2020 is the the
expanded MEVA Test3 dataset. The public MEVA dataset includes hundreds of hours of data from
the same cameras at the same facility, which can be used for training. You can download the
public MEVA dataset for free at mevadata.org; and more
info about the datasets is on the data tab. We
also provide annotations for 20 hours of MEVA data, and instructions on how to make and
share activity annotations are at mevadata.org.
Metrics
Submitted activity detection systems must give a confidence score for each
activity they detect. Detected activities are then thresholded based on this confidence
score. Varying the threshold makes a trade-off between being sensitive enough to identify
true activity instances (low threshold) vs. not making false alarms when no activity is
present (high threshold). Submitted systems are scored on both of these, measured by
Probability of Missed Detection (Pmiss) and Time-based False Alarm (TFA). Pmiss is the
portion of activities where the system did not detect the activity for at least 1 second.
TFA is the portion of time that the system detected an activity when in fact there was none.
Submitted systems are scored for Pmiss and TFA at multiple thresholds, creating a
detect error tradeoff
(DET) curve. The leaderboard ranking of a system is based on a summary of its DET
curve: the area under the DET Curve across the TFA range between 0% to 20% divided by 0.2 to normalize the value to [0:1]. Lower numbers
are better, as they reflect fewer errors. See details in the
SDL evaluation plan
or check out the ActEV Scoring Software GitHub repo.
Evaluation Plan
Task coordinator
ActEV NIST team (ActEV-nist@nist.gov)
News
01March
ActEV 2020 SDL starts with expanded MEVA Test3 dataset
17May
New Deadline to submit for ActivityNet task
ActEV SDL Evaluation Schedule
March 01, 2020: ActEV 2020 SDL opens with MEVA Test3.
May 17 May 10 , 2020 at 12 noon EST : New Deadline for CLI submissions to be included in ActEV
guest task under CVPR'20 ActivityNet workshop.
June 01, 2020 : We will invite the top two teams on the ActEV 2020 SDL
leaderboard to give ActEV guest task oral presentations at the CVPR'20 ActivityNet workshop
based on the CLI submission deadline (May 17th May 10th , 2020).
June 14, 2020: CVPR'20 ActivityNet workshop
ActEV SDL guest task presentations.
Leaderboard Remains Open: Leaderboard will continue to remain open
to allow participants to show continued progress on this challenging problem.
ActEV SDL Dataset
Multiview Extended Video with Activities (MEVA)
The ActEV SDL evaluation is based on the Multiview Extended Video with Activities (MEVA) dataset (mevadata.org) collected at the Muscatatuck Urban Training Center with a team of over 100 actors performing in various scenarios. The data was built by the Intelligence Advanced Research Projects Activity (IARPA) Deep Intermodal Video Analytics (DIVA) program to support activity detection in multi-camera environments for both DIVA performers and the broader research community.
The MEVA dataset has two parts: the public training and development data and sequestered evaluation data used only by NIST to test systems. The data is accompanied by activity annotations.
Public Training and Development Data
The Multiview Extended Video with Activities (MEVA) dataset website mevadata.org is to share the public MEVA video dataset and annotations. The size of the public MEVA dataset is 333 hours of ground-camera and UAV video. The size of the provided annotated video dataset is 28 hours. ActEV participants are encouraged to annotate the MEVA KF1 dataset for the 37 activities as described at mevadata.org.
The MEVA data GIT repo is the data distribution mechanism for MEVA Related annotations and documentation. The repo presently consists of schemas for the activity annotations https://gitlab.kitware.com/meva/meva-data-repo.
The ActEV data GIT repo, is the data distribution mechanism for the ActEV evaluation. The repo presently consists of a collection of corpora and partition definition files to be used for the evaluations https://gitlab.kitware.com/actev/actev-data-repo.
Sequestered Evaluation Data
As of March 2020, NIST is using a 140-hour collection of annotated MEVA data for sequestered data evaluations. The data set consists of both EO and IR cameras, public cameras (examples of which are in the public data set). The leaderboard presents results on the full 140-hour collection reporting separately by EO and IR data. Developers receive additional scores by activity for the EO_subset1 and the IR_subset1. Both subsets consist of data from public cameras.
ActEV SDL Leaderboard
Update June 3, 2020: Reported scores for all submissions on the ActEV2020 leaderboards were revised using the ActEV_SDL_V2 scoring protocol which slightly decreased all error rates.
SDL20-scoring-EO
Updated: 2020-12-09 14:46:36 -0500
| RANK | SCORING_SUBMISSION_ID | SCORING REQUEST NAME | TEAM NAME | SUBMISSION ID | SUBMISSION DATE | SYSTEM NAME | SYSTEM ID | SCORING PROTOCOL | PARTIAL AUDC* | TIME LIMITED PARTIAL AUDC* | RELATIVE PROCESSING TIME | DETECTED ACTIVITY TYPES† | PROCESSED FILES‡ | MEAN-P_MISS@0.04TFA | TIME LIMITED MEAN-P_MISS@0.04TFA |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 21907 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200802-111248-0247.sr-20200802-111248-6519 | INF | submission_description/21756|21756 | 2020-07-27 | set3 | 290 | ActEV_SDL_V2 | 0.35042 | 0.570 | 100% | 100% | 0.43284 | ||
| 2 | 22213 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00346_TeamVision2_20200917-045128-0151.sr-20200917-045129-0454 | TeamVision2 | submission_description/22212|22212 | 2020-09-14 | dummy_test | 346 | ActEV_SDL_V2 | 0.35362 | 0.590 | 100% | 100% | 0.46523 | ||
| 3 | 21562 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200723-223132-7371.sr-20200723-223133-0801 | IBM-MIT-Purdue | submission_description/20923|20923 | 2020-07-16 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.35548 | 0.941 | 100% | 100% | 0.46655 | ||
| 4 | 19372 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200626-225528-9522.sr-20200626-225529-2904 | UCF | submission_description/19025|19025 | 2020-06-19 | UCF-P | 270 | ActEV_SDL_V2 | 0.36126 | 0.722 | 100% | 100% | 0.42337 | ||
| 5 | 22080 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200813-111436-3293.sr-20200813-111443-6136 | UMD | submission_description/18450|18450 | 2020-06-07 | UMD+UCF | 326 | ActEV_SDL_V2 | 0.36655 | 0.49867 | 1.486 | 94% | 100% | 0.42493 | 0.53357 |
| 6 | 16601 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00328_VUS_20200511-132633-4864.sr-20200515-001742-2385 | VUS | submission_description/16259|16259 | 2020-05-08 | VUS-V1 | 328 | ActEV_SDL_V2 | 0.40599 | ** | 1.344 | 100% | 100% | 0.46931 | ** |
| 7 | 22204 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00349_UMCMU_20200908-213841-1038.sr-20200908-213842-3874 | UMCMU | submission_description/22202|22202 | 2020-09-05 | UMCMU-T | 349 | ActEV_SDL_V2 | 0.43836 | 0.64138 | 2.779 | 100% | 100% | 0.53971 | 0.68352 |
| 8 | 15300 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200416-142124-7336.sr-20200519-235029-4101 | NIST-TEST | submission_description/13021|13021 | 2020-03-05 | NIST Test | 263 | ActEV_SDL_V2 | 0.46563 | 0.822 | 97% | 100% | 0.55441 | ||
| 9 | 17319 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200519-012627-9123.sr-20200519-235030-8540 | Team_Vision | submission_description/16275|16275 | 2020-05-08 | STARK | 282 | ActEV_SDL_V2 | 0.52993 | 1.002 | 100% | 100% | 0.62047 | ||
| 10 | 17477 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00337_vireoJD-MM_20200520-200345-0804.sr-20200526-131616-1250 | vireoJD-MM | submission_description/17093|17093 | 2020-05-16 | vireo3 | 337 | ActEV_SDL_V2 | 0.53876 | 0.149 | 100% | 96% | 0.67389 | ||
| 11 | 17913 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00338_BUPT-MCPRL_20200529-112230-6942.sr-20200529-112231-8566 | BUPT-MCPRL | submission_description/17732|17732 | 2020-05-26 | bupt-mcprl | 338 | ActEV_SDL_V2 | 0.61532 | 0.969 | 100% | 100% | 0.63203 | ||
| 12 | 22210 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00335_CIS-JHU_20200910-111927-6409.sr-20200910-111939-5650 | CIS_JHU | submission_description/17471|17471 | 2020-05-20 | jhu_stmpgnn | 335 | ActEV_SDL_V2 | 0.62911 | 0.77784 | 4.520 | 94% | 69% | 0.70393 | 0.79736 |
| 13 | 22200 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20200903-202020-0140.sr-20200903-202032-7226 | DIVA TE Baseline | submission_description/18234|18234 | 2020-06-04 | RC3D | 276 | ActEV_SDL_V2 | 0.93925 | 0.94669 | 1.327 | 89% | 100% | 0.95265 | 0.95662 |
SDL20-scoring-EO
Updated: 2020-09-21 09:23:39 -0400
SDL20-scoring-EO
Updated: 2020-12-09 14:46:25 -0500
| RANK | SCORING_SUBMISSION_ID | SCORING REQUEST NAME | TEAM NAME | SUBMISSION ID | SUBMISSION DATE | SYSTEM NAME | SYSTEM ID | SCORING PROTOCOL | PARTIAL AUDC* | TIME LIMITED PARTIAL AUDC* | RELATIVE PROCESSING TIME | DETECTED ACTIVITY TYPES† | PROCESSED FILES‡ | MEAN-P_MISS@0.04TFA | TIME LIMITED MEAN-P_MISS@0.04TFA |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 21907 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200802-111248-0247.sr-20200802-111248-6519 | INF | submission_description/21756|21756 | 2020-07-27 | set3 | 290 | ActEV_SDL_V2 | 0.35042 | 0.570 | 100% | 100% | 0.43284 | ||
| 2 | 22213 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00346_TeamVision2_20200917-045128-0151.sr-20200917-045129-0454 | TeamVision2 | submission_description/22212|22212 | 2020-09-14 | dummy_test | 346 | ActEV_SDL_V2 | 0.35362 | 0.590 | 100% | 100% | 0.46523 | ||
| 3 | 21562 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200723-223132-7371.sr-20200723-223133-0801 | IBM-MIT-Purdue | submission_description/20923|20923 | 2020-07-16 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.35548 | 0.941 | 100% | 100% | 0.46655 | ||
| 4 | 22092 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00346_TeamVision2_20200818-120721-8627.sr-20200818-120722-9978 | TeamVision2 | submission_description/22087|22087 | 2020-08-17 | dummy_test | 346 | ActEV_SDL_V2 | 0.35861 | 0.165 | 100% | 100% | 0.46920 | ||
| 5 | 21720 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200727-100139-3414.sr-20200727-100139-7088 | IBM-MIT-Purdue | submission_description/21563|21563 | 2020-07-24 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.35903 | 0.164 | 100% | 100% | 0.46970 | ||
| 6 | 21558 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200721-224447-0979.sr-20200723-115701-6484 | IBM-MIT-Purdue | submission_description/20925|20925 | 2020-07-16 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.36026 | 0.933 | 100% | 100% | 0.47363 | ||
| 7 | 19372 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200626-225528-9522.sr-20200626-225529-2904 | UCF | submission_description/19025|19025 | 2020-06-19 | UCF-P | 270 | ActEV_SDL_V2 | 0.36126 | 0.722 | 100% | 100% | 0.42337 | ||
| 8 | 17885 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200528-094551-9941.sr-20200528-094553-2266 | UCF | submission_description/17152|17152 | 2020-05-17 | UCF-P | 270 | ActEV_SDL_V2 | 0.36452 | 0.684 | 100% | 100% | 0.42292 | ||
| 9 | 22080 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200813-111436-3293.sr-20200813-111443-6136 | UMD | submission_description/18450|18450 | 2020-06-07 | UMD+UCF | 326 | ActEV_SDL_V2 | 0.36655 | 0.49867 | 1.486 | 94% | 100% | 0.42493 | 0.53357 |
| 10 | 22186 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200903-134945-6697.sr-20200903-135004-3443 | UMD | submission_description/19562|19562 | 2020-06-30 | UMD+UCF | 326 | ActEV_SDL_V2 | 0.36941 | 0.51425 | 1.582 | 94% | 100% | 0.43018 | 0.54987 |
| 11 | 19469 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200629-055542-1886.sr-20200629-055542-4982 | UCF | submission_description/19365|19365 | 2020-06-26 | UCF-P | 270 | ActEV_SDL_V2 | 0.36969 | 0.688 | 100% | 100% | 0.42401 | ||
| 12 | 17151 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200516-215958-2007.sr-20200519-235025-5233 | UCF | submission_description/16743|16743 | 2020-05-12 | UCF-P | 270 | ActEV_SDL_V2 | 0.37281 | 0.581 | 94% | 100% | 0.42796 | ||
| 13 | 21246 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200719-014924-2963.sr-20200719-014924-6450 | IBM-MIT-Purdue | submission_description/20951|20951 | 2020-07-16 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.37300 | 0.939 | 100% | 100% | 0.49878 | ||
| 14 | 22066 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200807-122924-7589.sr-20200807-122938-3167 | IBM-MIT-Purdue | submission_description/21386|21386 | 2020-07-20 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.37434 | 0.134 | 100% | 100% | 0.49792 | ||
| 15 | 21560 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200723-065908-6567.sr-20200723-115906-1667 | IBM-MIT-Purdue | submission_description/21401|21401 | 2020-07-20 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.37434 | 0.165 | 100% | 100% | 0.49790 | ||
| 16 | 21363 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00345_IBM-MIT-Purdue_20200720-100421-3618.sr-20200720-100421-7590 | IBM-MIT-Purdue | submission_description/20924|20924 | 2020-07-16 | IBM-Purdue | 345 | ActEV_SDL_V2 | 0.38039 | 0.945 | 100% | 100% | 0.50065 | ||
| 17 | 17057 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20200516-013223-2420.sr-20200526-131614-0410 | INF | submission_description/16309|16309 | 2020-05-08 | speed1x | 298 | ActEV_SDL_V2 | 0.38699 | 0.498 | 97% | 100% | 0.46352 | ||
| 18 | 20991 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200717-020850-9110.sr-20200717-020851-7129 | INF | submission_description/20674|20674 | 2020-07-14 | INF_MEVA1 | 283 | ActEV_SDL_V2 | 0.39638 | 0.569 | 100% | 100% | 0.50412 | ||
| 19 | 20788 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200715-135050-7837.sr-20200715-135051-1502 | IBM-MIT-Purdue | submission_description/20639|20639 | 2020-07-14 | Purdue | 312 | ActEV_SDL_V2 | 0.40001 | 0.446 | 100% | 100% | 0.52175 | ||
| 20 | 21214 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200718-161337-9332.sr-20200718-161338-3634 | IBM-MIT-Purdue | submission_description/20790|20790 | 2020-07-15 | Purdue | 312 | ActEV_SDL_V2 | 0.40279 | 0.105 | 100% | 100% | 0.53051 | ||
| 21 | 16601 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00328_VUS_20200511-132633-4864.sr-20200515-001742-2385 | VUS | submission_description/16259|16259 | 2020-05-08 | VUS-V1 | 328 | ActEV_SDL_V2 | 0.40599 | ** | 1.344 | 100% | 100% | 0.46931 | ** |
| 22 | 20522 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200713-020327-1299.sr-20200713-020327-5401 | IBM-MIT-Purdue | submission_description/20365|20365 | 2020-07-12 | Purdue | 312 | ActEV_SDL_V2 | 0.40889 | 0.094 | 100% | 100% | 0.53662 | ||
| 23 | 16051 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200505-093715-5641.sr-20200519-235025-8703 | INF | submission_description/15846|15846 | 2020-05-01 | meva_inf2 | 288 | ActEV_SDL_V2 | 0.40897 | 0.927 | 97% | 100% | 0.48110 | ||
| 24 | 20535 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200713-140322-3163.sr-20200713-140322-6513 | IBM-MIT-Purdue | submission_description/20390|20390 | 2020-07-12 | Purdue | 312 | ActEV_SDL_V2 | 0.40987 | 0.094 | 100% | 100% | 0.53775 | ||
| 25 | 20364 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200711-232330-7159.sr-20200711-232331-0498 | IBM-MIT-Purdue | submission_description/20207|20207 | 2020-07-10 | Purdue | 312 | ActEV_SDL_V2 | 0.41002 | 0.093 | 100% | 100% | 0.53802 | ||
| 26 | 15414 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20200421-115525-8081.sr-20200519-235026-1773 | INF | submission_description/15132|15132 | 2020-04-13 | INF_MEVA_IOD | 307 | ActEV_SDL_V2 | 0.41143 | 0.848 | 97% | 98% | 0.48364 | ||
| 27 | 22206 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200910-111409-6010.sr-20200910-111427-9528 | UMD | submission_description/15747|15747 | 2020-04-30 | UMD+UCF | 326 | ActEV_SDL_V2 | 0.41450 | 0.51127 | 1.253 | 100% | 100% | 0.50159 | 0.57103 |
| 28 | 17326 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00334_VUS_20200519-030816-7908.sr-20200526-131615-6056 | VUS | submission_description/16908|16908 | 2020-05-14 | VUS-V1-FAST | 334 | ActEV_SDL_V2 | 0.41493 | 0.829 | 100% | 100% | 0.48191 | ||
| 29 | 22207 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200910-111512-8994.sr-20200910-111526-9232 | UMD | submission_description/16984|16984 | 2020-05-15 | UMD+UCF | 326 | ActEV_SDL_V2 | 0.41646 | 0.50800 | 1.226 | 97% | 100% | 0.49014 | 0.56142 |
| 30 | 22190 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200903-143511-7879.sr-20200903-143525-5231 | UMD | submission_description/21909|21909 | 2020-08-03 | UMD | 281 | ActEV_SDL_V2 | 0.41865 | 0.43988 | 1.057 | 97% | 100% | 0.49523 | 0.51203 |
| 31 | 14577 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200317-124405-8174.sr-20200519-235026-8006 | INF | submission_description/14480|14480 | 2020-03-16 | INF_MEVA1 | 283 | ActEV_SDL_V2 | 0.42496 | 0.527 | 97% | 98% | 0.49599 | ||
| 32 | 19647 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200702-000615-6292.sr-20200702-000616-3752 | INF | submission_description/19451|19451 | 2020-06-28 | meva_inf2 | 288 | ActEV_SDL_V2 | 0.43139 | 0.610 | 100% | 100% | 0.54264 | ||
| 33 | 14857 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200319-153022-2982.sr-20200519-235027-0971 | UCF | submission_description/14760|14760 | 2020-03-18 | UCF-P | 270 | ActEV_SDL_V2 | 0.43526 | 0.349 | 100% | 100% | 0.53063 | ||
| 34 | 14963 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200320-041131-0946.sr-20200519-235027-3712 | UCF | submission_description/14858|14858 | 2020-03-19 | UCF-P | 270 | ActEV_SDL_V2 | 0.43555 | 0.346 | 100% | 100% | 0.52764 | ||
| 35 | 20361 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200711-122139-6464.sr-20200711-122140-0051 | IBM-MIT-Purdue | submission_description/20206|20206 | 2020-07-10 | Purdue | 312 | ActEV_SDL_V2 | 0.43758 | 0.093 | 100% | 100% | 0.56295 | ||
| 36 | 22191 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200903-144008-5507.sr-20200903-144024-9839 | UMD | submission_description/19565|19565 | 2020-06-30 | UMD | 281 | ActEV_SDL_V2 | 0.43796 | 0.45792 | 1.055 | 97% | 100% | 0.52779 | 0.54450 |
| 37 | 22204 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00349_UMCMU_20200908-213841-1038.sr-20200908-213842-3874 | UMCMU | submission_description/22202|22202 | 2020-09-05 | UMCMU-T | 349 | ActEV_SDL_V2 | 0.43836 | 0.64138 | 2.779 | 100% | 100% | 0.53971 | 0.68352 |
| 38 | 15986 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200504-164235-0111.sr-20200519-235027-6683 | UCF | submission_description/15544|15544 | 2020-04-27 | UCF-P | 270 | ActEV_SDL_V2 | 0.43888 | 0.356 | 97% | 100% | 0.50250 | ||
| 39 | 16558 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200510-225902-1029.sr-20200519-235027-9601 | UCF | submission_description/16212|16212 | 2020-05-07 | UCF-P | 270 | ActEV_SDL_V2 | 0.43937 | 0.317 | 94% | 100% | 0.51352 | ||
| 40 | 22192 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200903-144542-8180.sr-20200903-144554-4687 | UMD | submission_description/18557|18557 | 2020-06-09 | UMD | 281 | ActEV_SDL_V2 | 0.44296 | 0.44346 | 1.002 | 100% | 100% | 0.52413 | 0.52440 |
| 41 | 22166 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20200824-120056-4943.sr-20200824-120111-4317 | INF | submission_description/22076|22076 | 2020-08-11 | speed1x | 298 | ActEV_SDL_V2 | 0.44860 | 0.759 | 100% | 100% | 0.55369 | ||
| 42 | 22160 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00349_UMCMU_20200822-025139-4835.sr-20200822-025140-6360 | UMCMU | submission_description/22100|22100 | 2020-08-20 | UMCMU-T | 349 | ActEV_SDL_V2 | 0.44974 | 0.48978 | 1.142 | 100% | 100% | 0.55711 | 0.58248 |
| 43 | 22164 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20200824-114239-5597.sr-20200824-114253-9849 | INF | submission_description/22079|22079 | 2020-08-12 | INF_MEVA_IOD | 307 | ActEV_SDL_V2 | 0.45250 | 0.531 | 100% | 98% | 0.52495 | ||
| 44 | 22158 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200821-140731-1281.sr-20200821-140759-0942 | INF | submission_description/22084|22084 | 2020-08-16 | INF_MEVA1 | 283 | ActEV_SDL_V2 | 0.45250 | 0.540 | 100% | 98% | 0.52526 | ||
| 45 | 17599 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200523-181607-2999.sr-20200605-094432-1913 | INF | submission_description/17472|17472 | 2020-05-20 | set3 | 290 | ActEV_SDL_V2 | 0.46181 | 0.915 | 100% | 100% | 0.57696 | ||
| 46 | 16257 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200508-110705-2778.sr-20200519-235028-2659 | UCF | submission_description/15949|15949 | 2020-05-03 | UCF-P | 270 | ActEV_SDL_V2 | 0.46326 | 0.480 | 100% | 100% | 0.56739 | ||
| 47 | 18749 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200611-125827-6623.sr-20200611-125828-0367 | UMD | submission_description/18150|18150 | 2020-06-01 | UMD | 281 | ActEV_SDL_V2 | 0.46455 | 0.884 | 100% | 100% | 0.57492 | ||
| 48 | 14856 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200319-133322-4135.sr-20200519-235029-1164 | UMD | submission_description/13739|13739 | 2020-03-11 | UMD | 281 | ActEV_SDL_V2 | 0.46562 | 0.684 | 97% | 100% | 0.55473 | ||
| 49 | 15300 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200416-142124-7336.sr-20200519-235029-4101 | NIST-TEST | submission_description/13021|13021 | 2020-03-05 | NIST Test | 263 | ActEV_SDL_V2 | 0.46563 | 0.822 | 97% | 100% | 0.55441 | ||
| 50 | 14483 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20200316-112137-9583.sr-20200519-235028-5577 | INF | submission_description/13770|13770 | 2020-03-11 | speed1x | 298 | ActEV_SDL_V2 | 0.46565 | 0.613 | 97% | 100% | 0.54743 | ||
| 51 | 14202 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20200313-132531-6559.sr-20200519-235028-8388 | INF | submission_description/13315|13315 | 2020-03-10 | speed1x | 298 | ActEV_SDL_V2 | 0.46566 | 0.615 | 97% | 100% | 0.54743 | ||
| 52 | 14819 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200319-091913-4675.sr-20200519-235029-6961 | UMD | submission_description/14296|14296 | 2020-03-13 | UMD | 281 | ActEV_SDL_V2 | 0.46662 | 0.703 | 100% | 100% | 0.55902 | ||
| 53 | 15026 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200320-131358-4621.sr-20200519-235029-9885 | UCF | submission_description/13949|13949 | 2020-03-11 | UCF-P | 270 | ActEV_SDL_V2 | 0.46704 | 0.333 | 97% | 100% | 0.53785 | ||
| 54 | 14669 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200318-053708-4851.sr-20200519-235030-2750 | UCF | submission_description/14578|14578 | 2020-03-17 | UCF-P | 270 | ActEV_SDL_V2 | 0.46704 | 0.296 | 97% | 100% | 0.53785 | ||
| 55 | 16557 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200510-225720-2108.sr-20200519-235030-5862 | INF | submission_description/16142|16142 | 2020-05-06 | set3 | 290 | ActEV_SDL_V2 | 0.47294 | 0.559 | 97% | 100% | 0.53440 | ||
| 56 | 22093 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00351_UMCMU_20200818-155914-5142.sr-20200818-155917-4156 | UMCMU | submission_description/22088|22088 | 2020-08-17 | UMCMU-SF | 351 | ActEV_SDL_V2 | 0.47326 | 0.50639 | 1.096 | 100% | 100% | 0.58134 | 0.60281 |
| 57 | 19586 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200701-042026-6851.sr-20200701-042026-9738 | UCF | submission_description/19448|19448 | 2020-06-28 | UCF-P | 270 | ActEV_SDL_V2 | 0.47688 | 0.515 | 100% | 100% | 0.51324 | ||
| 58 | 22081 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00349_UMCMU_20200813-201514-5379.sr-20200813-201515-6990 | UMCMU | submission_description/22072|22072 | 2020-08-11 | UMCMU-T | 349 | ActEV_SDL_V2 | 0.47947 | 0.50746 | 1.084 | 100% | 100% | 0.58884 | 0.60606 |
| 59 | 17346 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200519-111655-5218.sr-20200526-131613-0088 | UCF | submission_description/16386|16386 | 2020-05-09 | UCF-P | 270 | ActEV_SDL_V2 | 0.48113 | 0.305 | 94% | 100% | 0.57069 | ||
| 60 | 18071 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20200530-125344-8660.sr-20200530-125346-3964 | INF | submission_description/17726|17726 | 2020-05-26 | INF_MEVA_IOD | 307 | ActEV_SDL_V2 | 0.48144 | 0.824 | 97% | 97% | 0.57248 | ||
| 61 | 22157 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00351_UMCMU_20200821-120111-4361.sr-20200821-120116-5860 | UMCMU | submission_description/22095|22095 | 2020-08-19 | UMCMU-SF | 351 | ActEV_SDL_V2 | 0.48268 | 0.51874 | 1.108 | 100% | 100% | 0.59095 | 0.61479 |
| 62 | 17744 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200526-170134-8475.sr-20200526-170136-0564 | UMD | submission_description/17521|17521 | 2020-05-22 | UMD | 281 | ActEV_SDL_V2 | 0.48305 | 0.905 | 97% | 100% | 0.57807 | ||
| 63 | 19367 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200626-155016-8498.sr-20200626-155017-1824 | IBM-MIT-Purdue | submission_description/19119|19119 | 2020-06-24 | Purdue | 312 | ActEV_SDL_V2 | 0.48664 | 0.090 | 100% | 100% | 0.62999 | ||
| 64 | 22096 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200819-093352-6433.sr-20200819-093353-7810 | INF | submission_description/22090|22090 | 2020-08-18 | meva_inf2 | 288 | ActEV_SDL_V2 | 0.48932 | 0.486 | 100% | 100% | 0.55322 | ||
| 65 | 18555 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200609-113657-6904.sr-20200609-113658-4373 | INF | submission_description/18247|18247 | 2020-06-05 | meva_inf2 | 288 | ActEV_SDL_V2 | 0.49394 | 0.991 | 100% | 100% | 0.61605 | ||
| 66 | 18229 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200603-135148-1770.sr-20200603-135149-1523 | INF | submission_description/18151|18151 | 2020-06-02 | INF_MEVA1 | 283 | ActEV_SDL_V2 | 0.49492 | 0.950 | 100% | 98% | 0.61675 | ||
| 67 | 20297 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200711-022734-9206.sr-20200711-022735-2782 | IBM-MIT-Purdue | submission_description/20129|20129 | 2020-07-10 | Purdue | 312 | ActEV_SDL_V2 | 0.49857 | 0.063 | 100% | 100% | 0.63895 | ||
| 68 | 22101 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200820-033210-9803.sr-20200820-033212-8662 | INF | submission_description/22097|22097 | 2020-08-19 | INF_MEVA1 | 283 | ActEV_SDL_V2 | 0.50539 | 0.461 | 100% | 100% | 0.56714 | ||
| 69 | 18235 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200605-095636-6760.sr-20200605-095637-0432 | IBM-MIT-Purdue | submission_description/18073|18073 | 2020-05-30 | Purdue | 312 | ActEV_SDL_V2 | 0.50541 | 0.366 | 102% | 100% | 0.65305 | ||
| 70 | 17319 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200519-012627-9123.sr-20200519-235030-8540 | Team_Vision | submission_description/16275|16275 | 2020-05-08 | STARK | 282 | ActEV_SDL_V2 | 0.52993 | 1.002 | 100% | 100% | 0.62047 | ||
| 71 | 22208 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200910-111621-4009.sr-20200910-111632-2445 | Team_Vision | submission_description/16060|16060 | 2020-05-05 | STARK | 282 | ActEV_SDL_V2 | 0.52993 | 0.53006 | 1.003 | 100% | 100% | 0.62047 | 0.62012 |
| 72 | 17090 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200516-094544-8275.sr-20200519-235031-3894 | IBM-MIT-Purdue | submission_description/16902|16902 | 2020-05-14 | Purdue | 312 | ActEV_SDL_V2 | 0.53528 | 0.080 | 97% | 100% | 0.66782 | ||
| 73 | 22174 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200826-171114-9686.sr-20200826-171116-0653 | INF | submission_description/22170|22170 | 2020-08-25 | set3 | 290 | ActEV_SDL_V2 | 0.53531 | 0.518 | 100% | 100% | 0.59422 | ||
| 74 | 17477 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00337_vireoJD-MM_20200520-200345-0804.sr-20200526-131616-1250 | vireoJD-MM | submission_description/17093|17093 | 2020-05-16 | vireo3 | 337 | ActEV_SDL_V2 | 0.53876 | 0.149 | 100% | 96% | 0.67389 | ||
| 75 | 22194 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200903-162052-0365.sr-20200903-162102-4232 | Team_Vision | submission_description/20960|20960 | 2020-07-16 | STARK | 282 | ActEV_SDL_V2 | 0.54007 | 0.54326 | 1.021 | 100% | 100% | 0.63513 | 0.63803 |
| 76 | 22173 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00349_UMCMU_20200826-133244-4242.sr-20200826-133245-6193 | UMCMU | submission_description/22165|22165 | 2020-08-24 | UMCMU-T | 349 | ActEV_SDL_V2 | 0.54015 | 0.973 | 100% | 100% | 0.61852 | ||
| 77 | 16135 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00329_vireoJD-MM_20200506-100639-0175.sr-20200519-235031-6711 | vireoJD-MM | submission_description/15735|15735 | 2020-04-30 | Vireo | 329 | ActEV_SDL_V2 | 0.54107 | 0.160 | 100% | 96% | 0.67380 | ||
| 78 | 22195 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200903-170336-8983.sr-20200903-170348-6197 | Team_Vision | submission_description/20532|20532 | 2020-07-13 | STARK | 282 | ActEV_SDL_V2 | 0.54918 | 0.55134 | 1.014 | 100% | 100% | 0.64655 | 0.64749 |
| 79 | 22161 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200822-124810-1592.sr-20200822-124811-3286 | INF | submission_description/22159|22159 | 2020-08-22 | meva_inf2 | 288 | ActEV_SDL_V2 | 0.54966 | 0.520 | 100% | 100% | 0.60651 | ||
| 80 | 17008 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00331_vireoJD-MM_20200515-163029-9498.sr-20200519-235031-9723 | vireoJD-MM | submission_description/16445|16445 | 2020-05-09 | vireo2 | 331 | ActEV_SDL_V2 | 0.55665 | 0.163 | 100% | 96% | 0.68832 | ||
| 81 | 17387 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200519-233350-8538.sr-20200526-131614-5743 | IBM-MIT-Purdue | submission_description/16909|16909 | 2020-05-14 | Purdue | 312 | ActEV_SDL_V2 | 0.56720 | 0.080 | 97% | 100% | 0.69927 | ||
| 82 | 19993 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200708-194028-8463.sr-20200708-194029-1870 | Team_Vision | submission_description/19871|19871 | 2020-07-07 | STARK | 282 | ActEV_SDL_V2 | 0.57102 | 0.993 | 97% | 100% | 0.65497 | ||
| 83 | 22209 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200910-111813-0426.sr-20200910-111827-9542 | Team_Vision | submission_description/16705|16705 | 2020-05-12 | STARK | 282 | ActEV_SDL_V2 | 0.61351 | 0.63166 | 1.096 | 100% | 100% | 0.70557 | 0.71890 |
| 84 | 17913 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00338_BUPT-MCPRL_20200529-112230-6942.sr-20200529-112231-8566 | BUPT-MCPRL | submission_description/17732|17732 | 2020-05-26 | bupt-mcprl | 338 | ActEV_SDL_V2 | 0.61532 | 0.969 | 100% | 100% | 0.63203 | ||
| 85 | 22210 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00335_CIS-JHU_20200910-111927-6409.sr-20200910-111939-5650 | CIS_JHU | submission_description/17471|17471 | 2020-05-20 | jhu_stmpgnn | 335 | ActEV_SDL_V2 | 0.62911 | 0.77784 | 4.520 | 94% | 69% | 0.70393 | 0.79736 |
| 86 | 22198 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200903-200540-0013.sr-20200903-200553-6396 | Team_Vision | submission_description/18959|18959 | 2020-06-18 | STARK | 282 | ActEV_SDL_V2 | 0.62962 | 0.62956 | 1.005 | 100% | 100% | 0.71889 | 0.71876 |
| 87 | 15224 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200415-114050-1365.sr-20200519-235032-2671 | IBM-MIT-Purdue | submission_description/15129|15129 | 2020-04-10 | Purdue | 312 | ActEV_SDL_V2 | 0.64260 | 0.148 | 100% | 100% | 0.72913 | ||
| 88 | 22175 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20200827-105401-5621.sr-20200827-105403-0050 | INF | submission_description/22167|22167 | 2020-08-24 | INF_MEVA_IOD | 307 | ActEV_SDL_V2 | 0.68832 | 0.551 | 100% | 100% | 0.71984 | ||
| 89 | 14009 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200312-162922-5840.sr-20200519-235032-5355 | Team_Vision | submission_description/13117|13117 | 2020-03-08 | STARK | 282 | ActEV_SDL_V2 | 0.69971 | 0.685 | 97% | 100% | 0.76985 | ||
| 90 | 15027 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200320-133708-7523.sr-20200519-235032-8099 | Team_Vision | submission_description/13533|13533 | 2020-03-10 | STARK | 282 | ActEV_SDL_V2 | 0.70162 | 0.681 | 97% | 98% | 0.77027 | ||
| 91 | 15413 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200421-105807-5524.sr-20200519-235033-1130 | IBM-MIT-Purdue | submission_description/15320|15320 | 2020-04-18 | Purdue | 312 | ActEV_SDL_V2 | 0.74163 | 0.047 | 100% | 100% | 0.85468 | ||
| 92 | 16614 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200511-173821-3000.sr-20200519-235033-4110 | IBM-MIT-Purdue | submission_description/15562|15562 | 2020-04-28 | Purdue | 312 | ActEV_SDL_V2 | 0.74383 | 0.047 | 100% | 100% | 0.85239 | ||
| 93 | 20127 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200710-095000-2256.sr-20200710-095000-5705 | Team_Vision | submission_description/19750|19750 | 2020-07-06 | STARK | 282 | ActEV_SDL_V2 | 0.81637 | 0.918 | 97% | 100% | 0.84071 | ||
| 94 | 22185 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00349_UMCMU_20200902-115359-4665.sr-20200902-115400-5492 | UMCMU | submission_description/22181|22181 | 2020-08-31 | UMCMU-T | 349 | ActEV_SDL_V2 | 0.82685 | 0.85776 | 1.262 | 43% | 100% | 0.85385 | 0.87633 |
| 95 | 22171 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200826-035642-4842.sr-20200826-035643-6712 | INF | submission_description/22169|22169 | 2020-08-25 | INF_MEVA1 | 283 | ActEV_SDL_V2 | 0.85228 | 0.076 | 100% | 100% | 0.85927 | ||
| 96 | 22200 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20200903-202020-0140.sr-20200903-202032-7226 | DIVA TE Baseline | submission_description/18234|18234 | 2020-06-04 | RC3D | 276 | ActEV_SDL_V2 | 0.93925 | 0.94669 | 1.327 | 89% | 100% | 0.95265 | 0.95662 |
| 97 | 22199 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00341_DIVA-TE-Baseline_20200903-201905-7262.sr-20200903-201917-7742 | DIVA TE Baseline | submission_description/18498|18498 | 2020-06-08 | RC3D-kw-annotations | 341 | ActEV_SDL_V2 | 0.96674 | 0.97091 | 1.236 | 75% | 100% | 0.97908 | 0.98109 |
| 98 | 19032 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20200623-035629-8422.sr-20200623-035630-2041 | INF | submission_description/18968|18968 | 2020-06-18 | INF_MEVA_IOD | 307 | ActEV_SDL_V2 | 0.97072 | 0.523 | 97% | 3% | 0.97095 | ||
| 99 | 18855 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200612-101533-8300.sr-20200612-101534-1008 | NIST-TEST | submission_description/18753|18753 | 2020-06-11 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.031 | 0% | 100% | 1.00000 | ||
| 100 | 22153 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200820-163504-5674.sr-20200820-163505-5395 | NIST-TEST | submission_description/22067|22067 | 2020-08-10 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.096 | 0% | 100% | 1.00000 | ||
| 101 | 22182 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200831-172129-2294.sr-20200831-172130-1681 | NIST-TEST | submission_description/22180|22180 | 2020-08-31 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.029 | 0% | 100% | 1.00000 | ||
| 102 | 18863 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200612-174502-8142.sr-20200612-174503-0813 | NIST-TEST | submission_description/18565|18565 | 2020-06-09 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.031 | 0% | 100% | 1.00000 | ||
| 103 | 22184 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200901-234020-6634.sr-20200901-234021-9031 | NIST-TEST | submission_description/22183|22183 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.030 | 0% | 100% | 1.00000 | |||
| 104 | 22155 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200821-104029-2935.sr-20200821-104030-2742 | NIST-TEST | submission_description/22154|22154 | 2020-08-20 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.030 | 0% | 100% | 1.00000 | ||
| 105 | 18859 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200612-105746-5020.sr-20200612-105746-7748 | NIST-TEST | submission_description/18238|18238 | 2020-06-05 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.031 | 0% | 100% | 1.00000 | ||
| 106 | 19360 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20200626-094820-3881.sr-20200626-094820-6286 | NIST-TEST | submission_description/19117|19117 | 2020-06-23 | NIST Test | 263 | ActEV_SDL_V2 | 1.00000 | 0.030 | 0% | 100% | 1.00000 |
SDL20-scoring-EO
Updated: 2020-09-21 09:23:38 -0400
SDL19-scoring-EO
Updated: 2020-06-24 08:44:02 -0400
| RANK | TEAM NAME | SUBMISSION ID | SUBMISSION DATE | SYSTEM NAME | PARTIAL AUDC* | TIME LIMITED PARTIAL AUDC* | RELATIVE PROCESSING TIME | DETECTED ACTIVITY TYPES† | PROCESSED FILES‡ | MEAN-P_MISS@0.04TFA | TIME LIMITED MEAN-P_MISS@0.04TFA |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | UMD | 11045 | 2020-01-24 | UMD+UCF | 0.41585 | 0.65517 | 1.154 | 100% | 100% | 0.49360 | 0.67902 |
| 2 | UCF | 10640 | 2020-01-21 | UCF-P | 0.43810 | 0.362 | 100% | 100% | 0.52356 | ||
| 3 | UCF | 12650 | 2020-02-25 | UCF-P | 0.44047 | 0.364 | 97% | 100% | 0.54081 | ||
| 4 | UCF | 9105 | 2020-01-15 | UCF-P | 0.44056 | 0.424 | 97% | 100% | 0.52513 | ||
| 5 | UCF | 12222 | 2020-02-11 | UCF-P | 0.44198 | 0.395 | 100% | 100% | 0.53514 | ||
| 6 | UCF | 8328 | 2020-01-10 | UCF-P | 0.44360 | 0.403 | 100% | 100% | 0.52972 | ||
| 7 | UCF | 10821 | 2020-01-22 | UCF-P | 0.44473 | 0.407 | 100% | 100% | 0.54228 | ||
| 8 | UCF | 11849 | 2020-02-02 | UCF-P | 0.45165 | 0.378 | 100% | 100% | 0.51752 | ||
| 9 | UMD | 5595 | 2019-12-06 | UMD | 0.47596 | 0.725 | 97% | 100% | 0.54479 | ||
| 10 | UMD | 12468 | 2020-02-18 | UMD | 0.47611 | 0.849 | 97% | 100% | 0.55945 | ||
| 11 | UMD | 8222 | 2020-01-10 | UMD | 0.47668 | 0.68909 | 1.260 | 100% | 100% | 0.54492 | 0.71689 |
| 12 | UMD | 8198 | 2020-01-09 | UMD | 0.48037 | 0.68601 | 1.134 | 97% | 100% | 0.54825 | 0.71585 |
| 13 | UCF | 7728 | 2020-01-04 | UCF-P | 0.48290 | 0.438 | 94% | 100% | 0.51235 | ||
| 14 | UMD | 7992 | 2020-01-08 | UMD | 0.48333 | 0.68298 | 1.098 | 97% | 100% | 0.55028 | 0.71410 |
| 15 | INF | 11911 | 2020-02-06 | set3 | 0.48978 | 0.646 | 97% | 100% | 0.55928 | ||
| 16 | UMD | 10000 | 2020-01-19 | UMD | 0.49061 | 0.803 | 100% | 100% | 0.56996 | ||
| 17 | UMD | 7472 | 2020-01-02 | UMD | 0.49689 | 0.61793 | 1.042 | 97% | 93% | 0.55515 | 0.65471 |
| 18 | UCF | 7235 | 2019-12-30 | UCF-P | 0.51676 | 0.347 | 97% | 100% | 0.55164 | ||
| 19 | UCF | 6499 | 2019-12-18 | UCF-P | 0.52068 | 0.333 | 97% | 100% | 0.57885 | ||
| 20 | UMD | 4809 | 2019-11-15 | UMD | 0.53144 | 0.965 | 100% | 98% | 0.60011 | ||
| 21 | UMD | 3666 | 2019-10-10 | UMD | 0.53147 | 0.970 | 100% | 98% | 0.60101 | ||
| 22 | UMD | 6285 | 2019-12-12 | UMD | 0.53459 | 0.723 | 100% | 98% | 0.59596 | ||
| 23 | INF | 12068 | 2020-02-09 | speed1x | 0.53654 | 0.642 | 97% | 94% | 0.59701 | ||
| 24 | INF | 11850 | 2020-02-03 | meva_inf2 | 0.59044 | 0.960 | 94% | 97% | 0.64252 | ||
| 25 | UCF | 5851 | 2019-12-09 | UCF-P | 0.60431 | 0.293 | 97% | 100% | 0.64157 | ||
| 26 | Edge-Intelligence | 8003 | 2020-01-08 | Edge-Intelligence | 0.62842 | 0.939 | 97% | 100% | 0.75488 | ||
| 27 | IBM-MIT-Purdue | 6513 | 2019-12-18 | Purdue | 0.64182 | 0.272 | 100% | 100% | 0.73321 | ||
| 28 | IBM-MIT-Purdue | 11346 | 2020-01-26 | Purdue | 0.64182 | 0.128 | 100% | 100% | 0.73320 | ||
| 29 | IBM-MIT-Purdue | 11345 | 2020-01-26 | Purdue | 0.64182 | 0.125 | 100% | 100% | 0.73320 | ||
| 30 | Edge-Intelligence | 7845 | 2020-01-06 | Edge_Intelligence | 0.64737 | 0.887 | 94% | 100% | 0.76268 | ||
| 31 | UMD | 2074 | 2019-08-16 | UMD | 0.65325 | 0.744 | 97% | 98% | 0.73860 | ||
| 32 | IBM-MIT-Purdue | 6815 | 2019-12-20 | Purdue | 0.65351 | 0.315 | 100% | 100% | 0.73730 | ||
| 33 | IBM-MIT-Purdue | 5004 | 2019-11-25 | Purdue | 0.65503 | 0.124 | 100% | 98% | 0.73745 | ||
| 34 | IBM-MIT-Purdue | 5005 | 2019-11-25 | Purdue | 0.65913 | 0.124 | 100% | 98% | 0.73971 | ||
| 35 | IBM-MIT-Purdue | 6501 | 2019-12-18 | Purdue | 0.66055 | 0.442 | 100% | 100% | 0.74490 | ||
| 36 | IBM-MIT-Purdue | 4660 | 2019-11-11 | Purdue | 0.66383 | 0.087 | 100% | 98% | 0.74381 | ||
| 37 | IBM-MIT-Purdue | 11343 | 2020-01-26 | Purdue | 0.66394 | 0.038 | 100% | 98% | 0.74397 | ||
| 38 | IBM-MIT-Purdue | 11344 | 2020-01-26 | Purdue | 0.66394 | 0.035 | 100% | 98% | 0.74397 | ||
| 39 | IBM-MIT-Purdue | 11268 | 2020-01-26 | Purdue | 0.66496 | 0.124 | 100% | 100% | 0.74147 | ||
| 40 | IBM-MIT-Purdue | 6303 | 2019-12-12 | Purdue | 0.66900 | 0.096 | 100% | 100% | 0.75098 | ||
| 41 | IBM-MIT-Purdue | 6547 | 2019-12-18 | Purdue | 0.67114 | 0.099 | 100% | 100% | 0.75100 | ||
| 42 | UMD | 1907 | 2019-08-13 | UMD | 0.67831 | 0.727 | 0.74917 | ||||
| 43 | IBM-MIT-Purdue | 4999 | 2019-11-25 | Purdue | 0.67912 | 0.098 | 100% | 98% | 0.76349 | ||
| 44 | INF | 11670 | 2020-01-28 | INF_MEVA1 | 0.67983 | 0.928 | 100% | 100% | 0.78032 | ||
| 45 | IBM-MIT-Purdue | 5001 | 2019-11-25 | Purdue | 0.68547 | 0.099 | 100% | 98% | 0.76606 | ||
| 46 | IBM-MIT-Purdue | 11188 | 2020-01-25 | Purdue | 0.69501 | 0.108 | 100% | 98% | 0.75741 | ||
| 47 | UCF | 5596 | 2019-12-06 | UCF-P | 0.69848 | 0.304 | 86% | 100% | 0.73165 | ||
| 48 | IBM-MIT-Purdue | 6443 | 2019-12-17 | Purdue | 0.70117 | 0.82690 | 1.072 | 100% | 100% | 0.77819 | 0.87029 |
| 49 | IBM-MIT-Purdue | 4876 | 2019-11-19 | Purdue | 0.70382 | 0.088 | 97% | 98% | 0.76211 | ||
| 50 | Team_Vision | 6865 | 2019-12-20 | STARK | 0.71736 | 0.793 | 97% | 100% | 0.77673 | ||
| 51 | Team_Vision | 4434 | 2019-11-05 | STARK | 0.71911 | 0.774 | 97% | 100% | 0.77884 | ||
| 52 | UCF | 5439 | 2019-12-04 | UCF-P | 0.72830 | 0.304 | 86% | 100% | 0.75572 | ||
| 53 | IBM-MIT-Purdue | 8154 | 2020-01-09 | Purdue | 0.74090 | 0.380 | 100% | 100% | 0.82467 | ||
| 54 | IBM-MIT-Purdue | 9978 | 2020-01-18 | Purdue | 0.74091 | 0.145 | 100% | 100% | 0.82468 | ||
| 55 | UCF | 5077 | 2019-11-27 | UCF-P | 0.74243 | 0.312 | 86% | 100% | 0.77197 | ||
| 56 | IBM-MIT-Purdue | 4228 | 2019-10-31 | Purdue | 0.75410 | 0.093 | 97% | 98% | 0.82355 | ||
| 57 | IBM-MIT-Purdue | 4520 | 2019-11-08 | Purdue | 0.75509 | 0.087 | 100% | 98% | 0.82359 | ||
| 58 | IBM-MIT-Purdue | 10037 | 2020-01-19 | Purdue | 0.79701 | 0.151 | 86% | 100% | 0.84131 | ||
| 59 | IBM-MIT-Purdue | 4179 | 2019-10-30 | Purdue | 0.81573 | 0.093 | 97% | 98% | 0.86015 | ||
| 60 | IBM-MIT-Purdue | 10041 | 2020-01-19 | Purdue | 0.82020 | 0.143 | 100% | 100% | 0.88273 | ||
| 61 | IBM-MIT-Purdue | 3830 | 2019-10-21 | Purdue | 0.82025 | 0.246 | 100% | 97% | 0.86425 | ||
| 62 | Team_Vision | 4071 | 2019-10-29 | STARK | 0.82121 | 0.757 | 75% | 100% | 0.85712 | ||
| 63 | Team_Vision | 8506 | 2020-01-12 | STARK | 0.82442 | 0.739 | 94% | 58% | 0.85486 | ||
| 64 | Team_Vision | 3557 | 2019-10-08 | STARK | 0.83722 | 0.723 | 78% | 100% | 0.88187 | ||
| 65 | IBM-MIT-Purdue | 10038 | 2020-01-19 | Purdue | 0.84098 | 0.144 | 81% | 100% | 0.87514 | ||
| 66 | Team_Vision | 3382 | 2019-09-23 | STARK | 0.84258 | 0.726 | 81% | 100% | 0.88996 | ||
| 67 | UCF | 3472 | 2019-09-26 | UCF-P | 0.84269 | 0.292 | 100% | 100% | 0.89750 | ||
| 68 | INF | 8367 | 2020-01-11 | set3 | 0.84990 | 0.88917 | 1.332 | 89% | 85% | 0.89547 | 0.92076 |
| 69 | IBM-MIT-Purdue | 8290 | 2020-01-10 | Purdue | 0.85131 | 0.301 | 100% | 100% | 0.88623 | ||
| 70 | INF | 7212 | 2019-12-29 | INF_MEVA1 | 0.85864 | 0.922 | 89% | 100% | 0.87950 | ||
| 71 | INF | 6967 | 2019-12-23 | set3 | 0.86506 | 0.911 | 89% | 100% | 0.88947 | ||
| 72 | IBM-MIT-Purdue | 10973 | 2020-01-24 | Purdue | 0.86613 | 0.151 | 72% | 100% | 0.88674 | ||
| 73 | IBM-MIT-Purdue | 10972 | 2020-01-24 | Purdue | 0.86613 | 0.152 | 72% | 100% | 0.88674 | ||
| 74 | IBM-MIT-Purdue | 10896 | 2020-01-23 | Purdue | 0.86614 | 0.143 | 72% | 100% | 0.88673 | ||
| 75 | IBM-MIT-Purdue | 9866 | 2020-01-17 | Purdue | 0.86887 | 0.142 | 72% | 100% | 0.88733 | ||
| 76 | UCF | 3104 | 2019-09-10 | UCF-P | 0.87068 | 0.291 | 97% | 100% | 0.91260 | ||
| 77 | INF | 6965 | 2019-12-23 | INF_MEVA1 | 0.87216 | 0.906 | 89% | 100% | 0.89303 | ||
| 78 | IBM-MIT-Purdue | 10040 | 2020-01-19 | Purdue | 0.87422 | 0.147 | 70% | 100% | 0.89421 | ||
| 79 | Team_Vision | 2694 | 2019-08-27 | STARK | 0.87562 | 0.91641 | 1.795 | 86% | 80% | 0.90833 | 0.92705 |
| 80 | IBM-MIT-Purdue | 9873 | 2020-01-17 | Purdue | 0.87606 | 0.139 | 70% | 100% | 0.89165 | ||
| 81 | UCF | 4510 | 2019-11-06 | UCF-P | 0.87657 | 0.293 | 97% | 100% | 0.91490 | ||
| 82 | INF | 7152 | 2019-12-26 | speed1x | 0.88292 | 0.989 | 89% | 86% | 0.89853 | ||
| 83 | Team_Vision | 12208 | 2020-02-11 | STARK | 0.89021 | 0.783 | 94% | 36% | 0.89837 | ||
| 84 | UCF | 2014 | 2019-08-15 | UCF-P | 0.89321 | 0.236 | 91% | 81% | 0.92557 | ||
| 85 | Team_Vision | 12319 | 2020-02-12 | STARK | 0.89465 | 0.841 | 97% | 26% | 0.89748 | ||
| 86 | UCF | 4060 | 2019-10-25 | UCF-P | 0.89623 | 0.273 | 97% | 100% | 0.92032 | ||
| 87 | Edge-Intelligence | 7427 | 2020-01-02 | Edge_Intelligence | 0.92170 | 0.736 | 13% | 96% | 0.95539 | ||
| 88 | INF | 3276 | 2019-09-13 | INF_MEVA_IOD | 0.93009 | 0.588 | 94% | 91% | 0.94479 | ||
| 89 | INF | 8273 | 2020-01-10 | INF_MEVA1 | 0.93512 | 0.95304 | 1.054 | 59% | 100% | 0.93680 | 0.95338 |
| 90 | INF | 2928 | 2019-08-29 | INF_MEVA1 | 0.97344 | 0.788 | 89% | 98% | 0.98175 | ||
| 91 | IBM-MIT-Purdue | 8278 | 2020-01-10 | Purdue | 0.97569 | 0.361 | 29% | 100% | 0.98028 | ||
| 92 | INF | 7151 | 2019-12-26 | meva_inf2 | 0.97762 | 0.931 | 83% | 100% | 0.97655 | ||
| 93 | IBM-MIT-Purdue | 8287 | 2020-01-10 | Purdue | 0.97778 | 0.361 | 24% | 100% | 0.98142 | ||
| 94 | IBM-MIT-Purdue | 8284 | 2020-01-10 | Purdue | 0.97778 | 0.374 | 24% | 100% | 0.98142 | ||
| 95 | IBM-MIT-Purdue | 8288 | 2020-01-10 | Purdue | 0.97778 | 0.359 | 24% | 100% | 0.98142 | ||
| 96 | IBM-MIT-Purdue | 8286 | 2020-01-10 | Purdue | 0.97778 | 0.357 | 24% | 100% | 0.98142 | ||
| 97 | IBM-MIT-Purdue | 8289 | 2020-01-10 | Purdue | 0.97778 | 0.370 | 24% | 100% | 0.98142 | ||
| 98 | INF | 4959 | 2019-11-21 | meva_inf2 | 0.98157 | 0.98830 | 1.703 | 83% | 89% | 0.98886 | 0.99250 |
| 99 | DIVA TE Baseline | 1786 | 2019-08-02 | RC3D | 0.98216 | 0.959 | 29% | 100% | 0.98722 | ||
| 100 | INF | 3163 | 2019-09-12 | speed1x | 0.98315 | 0.860 | 78% | 94% | 0.98496 | ||
| 101 | Team_Vision | 8299 | 2020-01-10 | STARK | 0.99165 | 0.061 | 83% | 86% | 0.99143 | ||
| 102 | Team_Vision | 8295 | 2020-01-10 | STARK | 0.99211 | 0.060 | 72% | 96% | 0.99190 | ||
| 103 | DIVA TE Baseline | 7911 | 2020-01-07 | RC3D | 0.99255 | 0.99552 | 2.791 | 27% | 75% | 0.99302 | 0.99521 |
| 104 | DIVA TE Baseline | 7928 | 2020-01-07 | RC3D-WHEEL | 0.99608 | 0.99608 | 2.715 | 27% | 37% | 0.99580 | 0.99580 |
| 105 | INF | 2682 | 2019-08-27 | INF_MEVA1 | 1.00000 | 0.000 | 0% | 100% | 1.00000 | ||
| 106 | Team_Vision | 8313 | 2020-01-10 | STARK | 1.00000 | 0.049 | 0% | 79% | 1.00000 | ||
| 107 | IBM-MIT-Purdue | 6056 | 2019-12-11 | Purdue | 1.00000 | 0.037 | 0% | 100% | 1.00000 | ||
| 108 | IBM-MIT-Purdue | 6055 | 2019-12-11 | Purdue | 1.00000 | 0.037 | 0% | 98% | 1.00000 | ||
| 109 | IBM-MIT-Purdue | 6000 | 2019-12-10 | Purdue | 1.00000 | 0.037 | 0% | 98% | 1.00000 | ||
| 110 | Jay Chou | 5904 | 2019-12-10 | Jack | 1.00000 | 0.179 | 0% | 0% | 1.00000 | ||
| 111 | IBM-MIT-Purdue | 5818 | 2019-12-09 | Purdue | 1.00000 | 0.037 | 0% | 98% | 1.00000 | ||
| 112 | WeiLai | 5817 | 2019-12-09 | Tryyitry | 1.00000 | 0.177 | 0% | 0% | 1.00000 | ||
| 113 | WeiLai | 5651 | 2019-12-07 | Tryyitry | 1.00000 | 0.178 | 0% | 0% | 1.00000 | ||
| 114 | WeiLai | 5576 | 2019-12-05 | Tryyitry | 1.00000 | 0.178 | 0% | 0% | 1.00000 | ||
| 115 | IBM-MIT-Purdue | 5506 | 2019-12-04 | Purdue | 1.00000 | 0.039 | 0% | 98% | 1.00000 | ||
| 116 | IBM-MIT-Purdue | 11642 | 2020-01-27 | Purdue | 1.00000 | 0.016 | 0% | 100% | 1.00000 | ||
| 117 | WeiLai | 5452 | 2019-12-04 | Tryyitry | 1.00000 | 0.178 | 0% | 0% | 1.00000 | ||
| 118 | IBM-MIT-Purdue | 5384 | 2019-12-02 | Purdue | 1.00000 | 0.039 | 0% | 98% | 1.00000 | ||
| 119 | INF | 4513 | 2019-11-08 | INF_MEVA1 | 1.00000 | 0.039 | 0% | 98% | 1.00000 |
SDL19-scoring-EO
Updated: 2020-06-24 08:38:25 -0400
SDL19-scoring-IR
Updated: 2023-09-18 16:41:11 -0400
| RANK | SCORING REQUEST NAME | TEAM NAME | SUBMISSION ID | SYSTEM NAME | SYSTEM ID | PARTIAL AUDC* | RELATIVE PROCESSING TIME | DETECTED ACTIVITY TYPES† | PROCESSED FILES‡ | MEAN-P_MISS@0.04TFA |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190903-171132-2439.sr-20190904-151050-8165 | UMD | 1907 | UMD | 281 | 0.82140 | 0.727 | 97% | 94% | 0.87521 |
| 2 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190905-120934-4541.sr-20190905-120934-7789 | UMD | 2074 | UMD | 281 | 0.84254 | 0.744 | 97% | 94% | 0.88553 |
| 3 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190913-094214-5138.sr-20190913-094214-7316 | UCF | 3104 | UCF-P | 270 | 0.96630 | 0.291 | 78% | 100% | 0.97125 |
| 4 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20190814-164723-7643.sr-20190814-164723-9641 | NIST-TEST | 1790 | NIST Test | 263 | 0.99139 | 0.930 | 27% | 100% | 0.99280 |
| 5 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20190919-133858-1097.sr-20190919-133858-3388 | DIVA TE Baseline | 1786 | RC3D | 276 | 0.99139 | 0.959 | 27% | 100% | 0.99280 |
| 6 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190930-141448-4379.sr-20190930-141451-2761 | Team_Vision | 3382 | STARK | 282 | 0.99268 | 0.726 | 45% | 100% | 0.99345 |
| 7 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190905-151917-3849.sr-20190905-151917-6400 | UCF | 2014 | UCF-P | 270 | 0.99346 | 0.236 | 56% | 100% | 0.99547 |
| 8 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190905-154602-1817.sr-20190905-154603-7118 | Team_Vision | 2048 | STARK | 282 | 0.99682 | 1.132 | 51% | 100% | 0.99673 |
| 9 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190905-165243-1769.sr-20190905-165244-9598 | Team_Vision | 2694 | STARK | 282 | 0.99804 | 1.795 | 48% | 94% | 0.99869 |
| 10 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190903-152553-6173.sr-20190903-152553-9672 | INF | 2928 | INF_MEVA1 | 283 | 1.00000 | 0.788 | 54% | 100% | 1.00000 |
| 11 | ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190828-173003-1496.sr-20190828-173003-3976 | INF | 2682 | INF_MEVA1 | 283 | 1.00000 | 0.000 | 0% | 100% | 1.00000 |
SDL19-scoring-IR
Updated: 2023-09-18 16:41:11 -0400
Activities for the ActEV Sequestered Data Leaderboard
We have updated the activity names for the SDL. We intend for these names to be used for the duration of the DIVA program. The list below shows the "ActEV 2020 SDL Activity Name" and "ActEV 2019 SDL Activity Name" of the 37 activities to be detected for the ActEV SDL evaluation. Detailed activity definitions are in the ActEV Annotation Definitions for MEVA Data document. Note that current scoring for the SDL is based exclusively on activity detection, and object detection is not considered. The link to the activity-name-mapping.csv .
ActEV SDL Activities
| ActEV 2020 SDL Activity Name | ActEV 2019 SDL Activity Name (DEPRECATED) |
| person_abandons_package | abandon_package |
| person_closes_facility_door | person_closes_facility_door |
| person_closes_trunk | Closing_Trunk |
| person_closes_vehicle_door | person_closes_vehicle_door |
| person_embraces_person | person_person_embrace |
| person_enters_scene_through_structure | person_enters_through_structure |
| person_enters_vehicle | person_enters_vehicle |
| person_exits_scene_through_structure | person_exits_through_structure |
| person_exits_vehicle | person_exits_vehicle |
| hand_interacts_with_person | hand_interaction |
| person_carries_heavy_object | Transport_HeavyCarry |
| person_interacts_with_laptop | person_laptop_interaction |
| person_loads_vehicle | person_loads_vehicle |
| person_transfers_object | object_transfer |
| person_opens_facility_door | person_opens_facility_door |
| person_opens_trunk | Open_Trunk |
| person_opens_vehicle_door | person_opens_vehicle_door |
| person_talks_to_person | Talking |
| person_picks_up_object | person_picks_up_object |
| person_purchases | person_purchasing |
| person_reads_document | person_reading_document |
| person_rides_bicycle | Riding |
| person_puts_down_object | person_sets_down_object |
| person_sits_down | person_sitting_down |
| person_stands_up | person_standing_up |
| person_talks_on_phone | specialized_talking_phone |
| person_texts_on_phone | specialized_texting_phone |
| person_steals_object | theft |
| person_unloads_vehicle | Unloading |
| vehicle_drops_off_person | vehicle_drops_off_person |
| vehicle_picks_up_person | vehicle_picks_up_person |
| vehicle_reverses | vehicle_reversing |
| vehicle_starts | vehicle_starting |
| vehicle_stops | vehicle_stopping |
| vehicle_turns_left | vehicle_turning_left |
| vehicle_turns_right | vehicle_turning_right |
| vehicle_makes_u_turn | vehicle_u_turn |
Task for the ActEV Sequestered Data Leaderboard
In the SDL evaluation, there is one Activity Detection (AD) task for detecting and temporally localizing activities.
Activity Detection (AD)
For the Activity Detection task, given a target activity, a system automatically detects and temporally
localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the
type of activity must be correct and temporally overlap the true activity for at least one second. Additional
details may be found in the
SDL Evaluation Plan.
Algorithm Delivery for the SDL participants
System delivery to the leaderboard must be in a form compatible with the ActEV Command Line Interface (ActEV CLI) and submitted to NIST for testing. The command line interface implementation that you will provide formalizes the entire process of evaluating a system, by providing the evaluation team a means to: (1) download and install your software via a single URL, (2) verify that the delivery works AND produces output that is “consistent” with output you produce, and (3) process a large collection of video in a fault-tolerant, parallelizable manner.
To complete this task you will need the following items described in detail below:
- FAQ - Validation Phase Processing
- CLI Description
- The Abstract CLI Git Repository
- The CLI Implementation Primer
- The Validation Data Set
- Example CLI-Compliant Implementation
- NIST Hardware and Initial Operating System Description
- SDL Submission Processing Pipeline
1. FAQ - Validation Phase Processing
The ActEV SDL - Validation Phase Processing
FAQ2. CLI Description
3. The Abstract CLI GIT Repository
The Abstract CLI GIT repository. The repo contains documentation.
4. The CLI Implementation Primer
There are 6 steps to adapt your code to the CLI. The ActEV Evaluation CLI Programming Primer describes the steps to clone the Abstract CLI and begin adapting the code to your implementation.
5. The Validation Data Set
As mentioned above, the CLI is used to verify the downloaded software is correctly installed and produces the same output at NIST as you produce locally. In order to do so, we have provided a small validation data set (ActEV-Eval-CLI-Validation-Set3) as part of the ActEV SDL Dataset that will be processed both at your site and at NIST. Please use this data in Step 3 of the Primer described above.
6. Example CLI-Compliant Implementation
The links below provide two example CLI implementations for the leaderboard baseline algorithm:
- Framework based RC3D baseline system with CLI implementation see for details.
7. NIST Independent Evaluation Infrastructure Specification
NIST will begin installing your system from a fresh Ubuntu 18.04 cloud image available from https://cloud-images.ubuntu.com/releases/18.04/release/ on the following hardware.
- Chassis: Asus ESC4000 G4S
- CPU: 2 x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (12 cores/CPU)
- Motherboard: Asus Intel® C621 PCH chipset
- HDD/SSD: 2x 1.92GB Intel SSD DC S4500
- RAM: 12x 16GB DDR4-2400 ECC RDIMM
- GPU: Four PNY RTX2080Ti blower style
- OS: Ubuntu 18.04
- Storage Volume- 1TB (variable)
- Supplied object store (read only) for source video
8. SDL Submission Processing Pipeline
There are three stages for the submission processing pipeline. They are:
- Validation: during this stage we run through each of the ActEV CLI commands to install your system, run the system on the validation set, compare the produced output to the validation set, and finally, take a snapshot of your system to re-use during execution.
- Execution: during this stage, we use the snapshot to process the sequestered data. Presently, we can divide the sequestered data into 1-hour sub-parts of the data set or process the whole dataset. Each sub-part is nominally 12, 5-minute files. We are only processing MEVA data through your system. Presently, we have a runtime limit per part and if a part fails to be processed, we retry it once.
- Scoring: After all the parts have been processed, the outputs are merged, and scored.
Datasets
- ActEV-supported data sets
- Multiview Extended Video with Activities (MEVA)
- VIRAT
- Data access from mevadata.org: Accessing and using MEVA and MEVA Download Instructions (This includes, 4.6 hr UAV video collected, 3D model of the facility)
- Data access : See actev-data-repo. Access credentials provided during signup
- Kinetics
- AVA
- Moments-in-Time
- ActivityNet
- NVIDIA's CityFlow dataset
Framework
The DIVA Framework is a software framework designed to provide an architecture and a set of software modules which will facilitate the development of activity recognition analytics. The Framework is developed as a fully open source project on GitHub. The following links will help you get started with the framework:- DIVA Framework Github Repository This is the main DIVA Framework site, all development of the framework happens here.
- DIVA Framework Issue Tracker Submit any bug reports or feature requests for the framework here.
- DIVA Framework Main Documentation PageThe source for the framework documentation is maintained in the Github repository using Sphinx. A built version is maintained on ReadTheDocs at this link. A good place to get started in the documentation, after reading the Introduction is the UseCase section which will walk you though a number of typical use cases with the framework.
- KWIVER Github Repository This is the main KWIVER site, all development of the framework happens here.
- KWIVER Issue Tracker Submit any bug reports or feature requests for the KWIVER here. If there's any question about whether your issues belongs in the KWIVER or DIVA framework issues tracker, submit to the DIVA tracker and we'll sort it out..
- KWIVER Main Documentation Page The source for the KWIVER documentation is maintained in the Github repository using Sphinx. A built version is maintained on ReadTheDocs at this link. A good place to get started in the documentation, after reading the Introduction are the Arrows and Sprokit sections, both of which are used by the KWIVER framework.
Baseline Algorithms
KITWARE has adapted two "baseline" activity recognition algorithms to work within the DIVA Framework:Visualization Tools
Annotation Tools
- Kitware annotation tool (the tool natively supports the DIVA format)
- The VGG Image Annotator
- Scalabel (used for annotation of Berkeley DeepDrive project)
- VATIC - Video Annotation Tool
- BeaverDam
- VoTT: Visual Object Tagging Tool
- Computer Vision Annotation Tool (CVAT)
- Efficient Annotation of Segmentation Datasets with Polygon-RNN++
Contact Us
For ActEV Evaluation information (data, evaluation code, etc.) please email: actev-nist@nist.gov
For ActEV Evaluation Discussion Please Visit our Google Group.