ActEV Sequestered Data Leaderboard


ActEV SDL Leaderboard


SDL19-scoring-EO

Updated: 2020-01-29 11:34:35 -0500
RANK SCORING_SUBMISSION_ID SCORING REQUEST NAME TEAM NAME SUBMISSION ID SYSTEM NAME SYSTEM ID <REALTIME PARTIAL AUDC* >REALTIME PARTIAL AUDC* RELATIVE PROCESSING TIME DETECTED ACTIVITY TYPES† PROCESSED FILES‡ <REALTIME MEAN-P_MISS@0.04TFA
1 11725 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200128-173618-8145.sr-20200128-173618-9987 UMD submission_description/11045|11045 UMD+UCF 326 0.41585 0.65517 1.154 100% 100% 0.49360
2 10820 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200122-112821-7686.sr-20200122-112821-9188 UCF submission_description/10640|10640 UCF-P 270 0.43810 0.362 100% 100% 0.52356
3 10182 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200120-124912-8942.sr-20200120-124913-0413 UCF submission_description/9105|9105 UCF-P 270 0.44056 0.424 97% 100% 0.52513
4 8978 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200114-094657-1186.sr-20200114-094657-2665 UCF submission_description/8328|8328 UCF-P 270 0.44360 0.403 100% 100% 0.52972
5 10894 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200123-034832-1571.sr-20200123-034832-3198 UCF submission_description/10821|10821 UCF-P 270 0.44473 0.407 100% 100% 0.54228
6 5813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191207-183303-2020.sr-20191207-183303-3485 UMD submission_description/5595|5595 UMD 281 0.47596 0.725 97% 100% 0.54479
7 8429 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-220215-1118.sr-20200111-220215-2680 UMD submission_description/8222|8222 UMD 281 0.47668 0.68909 1.260 100% 100% 0.54492
8 8357 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-033656-2617.sr-20200111-033656-4084 UMD submission_description/8198|8198 UMD 281 0.48037 0.68601 1.134 97% 100% 0.54825
9 8004 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200108-080622-3945.sr-20200108-080622-5226 UCF submission_description/7728|7728 UCF-P 270 0.48290 0.438 94% 100% 0.51235
10 10819 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200122-110041-0660.sr-20200122-110041-2755 UMD submission_description/10000|10000 UMD 281 0.49061 0.803 100% 100% 0.56996
11 7902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200107-084859-7555.sr-20200107-084859-9065 UMD submission_description/7472|7472 UMD 281 0.49689 0.61793 1.042 97% 93% 0.55515
12 7628 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200103-201849-4904.sr-20200103-201849-6284 UCF submission_description/7235|7235 UCF-P 270 0.51676 0.347 97% 100% 0.55164
13 6813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191220-054840-0488.sr-20191220-054840-1911 UCF submission_description/6499|6499 UCF-P 270 0.52068 0.333 97% 100% 0.57885
14 4897 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191119-140314-0793.sr-20191119-140314-4363 UMD submission_description/4809|4809 UMD 281 0.53144 0.965 100% 98% 0.60011
15 3974 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191023-115220-9973.sr-20191023-115221-2327 UMD submission_description/3666|3666 UMD 281 0.53147 0.970 100% 98% 0.60101
16 6396 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191213-112124-8334.sr-20191213-112124-9953 UMD submission_description/6285|6285 UMD 281 0.53459 0.723 100% 98% 0.59596
17 5999 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191210-174641-6248.sr-20191210-174641-7467 UCF submission_description/5851|5851 UCF-P 270 0.60431 0.293 97% 100% 0.64157
18 8240 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00316_Edge-Intelligence_20200110-031108-3165.sr-20200110-031108-4757 Edge-Intelligence submission_description/8003|8003 Edge-Intelligence 316 0.62842 0.939 97% 100% 0.75488
19 6814 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-115038-7999.sr-20191220-115038-9539 IBM-MIT-Purdue submission_description/6513|6513 Purdue 312 0.64182 0.272 100% 100% 0.73321
20 11636 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-040601-4391.sr-20200127-040601-5932 IBM-MIT-Purdue submission_description/11346|11346 Purdue 312 0.64182 0.128 100% 100% 0.73320
21 11590 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-004703-9394.sr-20200127-004704-1030 IBM-MIT-Purdue submission_description/11345|11345 Purdue 312 0.64182 0.125 100% 100% 0.73320
22 8005 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200108-081958-6808.sr-20200108-081958-8298 Edge-Intelligence submission_description/7845|7845 Edge_Intelligence 320 0.64737 0.887 94% 100% 0.76268
23 3078 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190905-120934-4541.sr-20190905-120934-6642 UMD submission_description/2074|2074 UMD 281 0.65325 0.744 97% 98% 0.73860
24 6902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-235401-7601.sr-20191220-235401-9202 IBM-MIT-Purdue submission_description/6815|6815 Purdue 312 0.65351 0.315 100% 100% 0.73730
25 5379 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-021929-1722.sr-20191129-021929-3345 IBM-MIT-Purdue submission_description/5004|5004 Purdue 312 0.65503 0.124 100% 98% 0.73745
26 5380 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-022020-9039.sr-20191129-022021-0562 IBM-MIT-Purdue submission_description/5005|5005 Purdue 312 0.65913 0.124 100% 98% 0.73971
27 6812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191219-234702-7711.sr-20191219-234703-5380 IBM-MIT-Purdue submission_description/6501|6501 Purdue 312 0.66055 0.442 100% 100% 0.74490
28 11439 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-180453-0069.sr-20200126-180453-1583 IBM-MIT-Purdue submission_description/11343|11343 Purdue 312 0.66394 0.038 100% 98% 0.74397
29 11527 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-212645-2748.sr-20200126-212645-4572 IBM-MIT-Purdue submission_description/11344|11344 Purdue 312 0.66394 0.035 100% 98% 0.74397
30 11342 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-060218-3152.sr-20200126-060218-4619 IBM-MIT-Purdue submission_description/11268|11268 Purdue 312 0.66496 0.124 100% 100% 0.74147
31 6432 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191213-172239-8743.sr-20191213-172240-0161 IBM-MIT-Purdue submission_description/6303|6303 Purdue 312 0.66900 0.096 100% 100% 0.75098
32 6855 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-175217-7070.sr-20191220-175217-8617 IBM-MIT-Purdue submission_description/6547|6547 Purdue 312 0.67114 0.099 100% 100% 0.75100
33 3058 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190903-171132-2439.sr-20190903-171132-4143 UMD submission_description/1907|1907 UMD 281 0.67831 0.727 0.74917
34 5120 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-141345-4986.sr-20191127-141345-6913 IBM-MIT-Purdue submission_description/4999|4999 Purdue 312 0.67912 0.098 100% 98% 0.76349
35 5180 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-201530-0873.sr-20191127-201530-2638 IBM-MIT-Purdue submission_description/5001|5001 Purdue 312 0.68547 0.099 100% 98% 0.76606
36 11269 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-013750-5311.sr-20200126-013750-6816 IBM-MIT-Purdue submission_description/11188|11188 Purdue 312 0.69501 0.108 100% 98% 0.75741
37 5815 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191208-003354-3071.sr-20191208-003354-4279 UCF submission_description/5596|5596 UCF-P 270 0.69848 0.304 86% 100% 0.73165
38 6816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-135245-0601.sr-20191220-135245-2185 IBM-MIT-Purdue submission_description/6443|6443 Purdue 312 0.70117 0.82690 1.072 100% 100% 0.77819
39 4955 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191120-020423-5382.sr-20191120-020423-6818 IBM-MIT-Purdue submission_description/4876|4876 Purdue 312 0.70382 0.088 97% 98% 0.76211
40 7907 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200107-092828-2753.sr-20200107-092831-5397 Team_Vision submission_description/6865|6865 STARK 282 0.71736 0.793 97% 100% 0.77673
41 5535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191204-142823-9331.sr-20191204-142824-0649 UCF submission_description/5439|5439 UCF-P 270 0.72830 0.304 86% 100% 0.75572
42 8275 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200110-091609-1531.sr-20200110-091609-3374 IBM-MIT-Purdue submission_description/8154|8154 Purdue 312 0.74090 0.380 100% 100% 0.82467
43 10394 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-000324-4735.sr-20200121-000324-6679 IBM-MIT-Purdue submission_description/9978|9978 Purdue 312 0.74091 0.145 100% 100% 0.82468
44 5378 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191128-201821-9045.sr-20191128-201822-0355 UCF submission_description/5077|5077 UCF-P 270 0.74243 0.312 86% 100% 0.77197
45 4507 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-114135-1903.sr-20191106-114135-9240 IBM-MIT-Purdue submission_description/4228|4228 Purdue 312 0.75410 0.093 97% 98% 0.82355
46 4659 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191109-113627-1003.sr-20191109-113627-7178 IBM-MIT-Purdue submission_description/4520|4520 Purdue 312 0.75509 0.087 100% 98% 0.82359
47 10535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-164242-1566.sr-20200121-164242-3473 IBM-MIT-Purdue submission_description/10037|10037 Purdue 312 0.79701 0.151 86% 100% 0.84131
48 4508 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-121322-4750.sr-20191106-121323-3067 IBM-MIT-Purdue submission_description/4179|4179 Purdue 312 0.81573 0.093 97% 98% 0.86015
49 10706 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-013602-6833.sr-20200122-013602-8836 IBM-MIT-Purdue submission_description/10041|10041 Purdue 312 0.82020 0.143 100% 100% 0.88273
50 4070 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191028-165446-2748.sr-20191028-165446-8950 IBM-MIT-Purdue submission_description/3830|3830 Purdue 312 0.82025 0.246 100% 97% 0.86425
51 4428 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191101-144530-2898.sr-20191101-144532-0236 Team_Vision submission_description/4071|4071 STARK 282 0.82121 0.757 75% 100% 0.85712
52 10104 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200120-055837-4585.sr-20200120-055837-5853 Team_Vision submission_description/8506|8506 STARK 282 0.82442 0.739 94% 58% 0.85486
53 3639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191010-112752-8152.sr-20191010-112755-4284 Team_Vision submission_description/3557|3557 STARK 282 0.83722 0.723 78% 100% 0.88187
54 10762 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-045506-2717.sr-20200122-045506-4324 IBM-MIT-Purdue submission_description/10038|10038 Purdue 312 0.84098 0.144 81% 100% 0.87514
55 3500 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190930-141448-4379.sr-20190930-141451-1874 Team_Vision submission_description/3382|3382 STARK 282 0.84258 0.726 81% 100% 0.88996
56 3973 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191023-114820-4426.sr-20191023-114820-7137 UCF submission_description/3472|3472 UCF-P 270 0.84269 0.292 100% 100% 0.89750
57 9969 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200118-173929-1848.sr-20200118-173929-3375 INF submission_description/8367|8367 set3 290 0.84990 0.88917 1.332 89% 85% 0.89547
58 9496 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-132317-0783.sr-20200116-132317-2298 IBM-MIT-Purdue submission_description/8290|8290 Purdue 312 0.85131 0.301 100% 100% 0.88623
59 7547 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200103-081512-4482.sr-20200103-081512-5627 INF submission_description/7212|7212 INF_MEVA1 283 0.85864 0.922 89% 100% 0.87950
60 7139 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20191225-070143-9529.sr-20191225-070144-0759 INF submission_description/6967|6967 set3 290 0.86506 0.911 89% 100% 0.88947
61 11121 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-221522-0054.sr-20200124-221522-1316 IBM-MIT-Purdue submission_description/10973|10973 Purdue 312 0.86613 0.151 72% 100% 0.88674
62 11062 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-175059-7650.sr-20200124-175059-8813 IBM-MIT-Purdue submission_description/10972|10972 Purdue 312 0.86613 0.152 72% 100% 0.88674
63 10970 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200123-211058-0522.sr-20200123-211058-1721 IBM-MIT-Purdue submission_description/10896|10896 Purdue 312 0.86614 0.143 72% 100% 0.88673
64 10251 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-161145-9437.sr-20200120-161146-0616 IBM-MIT-Purdue submission_description/9866|9866 Purdue 312 0.86887 0.142 72% 100% 0.88733
65 3257 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190913-094214-5138.sr-20190913-094214-6400 UCF submission_description/3104|3104 UCF-P 270 0.87068 0.291 97% 100% 0.91260
66 7061 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191224-125830-3199.sr-20191224-125830-4603 INF submission_description/6965|6965 INF_MEVA1 283 0.87216 0.906 89% 100% 0.89303
67 10634 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-221418-2880.sr-20200121-221418-4184 IBM-MIT-Purdue submission_description/10040|10040 Purdue 312 0.87422 0.147 70% 100% 0.89421
68 3083 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190905-165243-1769.sr-20190905-165244-8608 Team_Vision submission_description/2694|2694 STARK 282 0.87562 0.91641 1.795 86% 80% 0.90833
69 10320 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-193353-6571.sr-20200120-193353-7807 IBM-MIT-Purdue submission_description/9873|9873 Purdue 312 0.87606 0.139 70% 100% 0.89165
70 4596 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191108-232219-1501.sr-20191108-232219-2801 UCF submission_description/4510|4510 UCF-P 270 0.87657 0.293 97% 100% 0.91490
71 7264 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191231-014617-6841.sr-20191231-014617-8065 INF submission_description/7152|7152 speed1x 298 0.88292 0.989 89% 86% 0.89853
72 4427 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191101-142638-9265.sr-20191101-142639-0480 UCF submission_description/4060|4060 UCF-P 270 0.89623 0.273 97% 100% 0.92032
73 7682 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200104-082151-3830.sr-20200104-082151-5082 Edge-Intelligence submission_description/7427|7427 Edge_Intelligence 320 0.92170 0.736 13% 96% 0.95539
74 4069 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20191028-164751-7980.sr-20191028-164752-0112 INF submission_description/3276|3276 INF_MEVA_IOD 307 0.93009 0.588 94% 91% 0.94479
75 8751 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200113-110217-3652.sr-20200113-110217-4800 INF submission_description/8273|8273 INF_MEVA1 283 0.93512 0.95304 1.054 59% 100% 0.93680
76 3055 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190903-152553-6173.sr-20190903-152553-8801 INF submission_description/2928|2928 INF_MEVA1 283 0.97344 0.788 89% 98% 0.98175
77 8924 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200114-025702-0518.sr-20200114-025702-1708 IBM-MIT-Purdue submission_description/8278|8278 Purdue 312 0.97569 0.361 29% 100% 0.98028
78 7336 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191231-195249-0336.sr-20191231-195249-1460 INF submission_description/7151|7151 meva_inf2 288 0.97762 0.931 83% 100% 0.97655
79 8858 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200113-211626-3700.sr-20200113-211626-4873 IBM-MIT-Purdue submission_description/8284|8284 Purdue 312 0.97778 0.374 24% 100% 0.98142
80 9843 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200117-024347-1627.sr-20200117-024347-2974 IBM-MIT-Purdue submission_description/8286|8286 Purdue 312 0.97778 0.357 24% 100% 0.98142
81 9712 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-200717-3029.sr-20200116-200717-4191 IBM-MIT-Purdue submission_description/8288|8288 Purdue 312 0.97778 0.359 24% 100% 0.98142
82 9639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-164611-8699.sr-20200116-164612-0130 IBM-MIT-Purdue submission_description/8289|8289 Purdue 312 0.97778 0.370 24% 100% 0.98142
83 9786 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-232636-3806.sr-20200116-232636-5045 IBM-MIT-Purdue submission_description/8287|8287 Purdue 312 0.97778 0.361 24% 100% 0.98142
84 5557 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191205-142328-8277.sr-20191205-142329-1674 INF submission_description/4959|4959 meva_inf2 288 0.98157 0.98830 1.703 83% 89% 0.98886
85 1991 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20190814-164723-7643.sr-20190814-164723-8758 NIST-TEST submission_description/1790|1790 NIST Test 263 0.98216 0.930 29% 100% 0.98722
86 3330 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20190919-133858-1097.sr-20190919-133858-2351 DIVA TE Baseline submission_description/1786|1786 RC3D 276 0.98216 0.959 29% 100% 0.98722
87 4068 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191028-164450-6506.sr-20191028-164450-9167 INF submission_description/3163|3163 speed1x 298 0.98315 0.860 78% 94% 0.98496
88 8816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-173052-2474.sr-20200113-173052-3647 Team_Vision submission_description/8299|8299 STARK 282 0.99165 0.061 83% 86% 0.99143
89 8812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-172127-0711.sr-20200113-172127-2341 Team_Vision submission_description/8295|8295 STARK 282 0.99211 0.060 72% 96% 0.99190
90 2892 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190828-173003-1496.sr-20190828-173003-2992 INF submission_description/2682|2682 INF_MEVA1 283 1.00000 0.000 0% 100% 1.00000
91 6281 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111546-3723.sr-20191212-111546-4849 IBM-MIT-Purdue submission_description/6055|6055 Purdue 312 1.00000 0.037 0% 98% 1.00000
92 5446 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191204-022616-0789.sr-20191204-022616-2066 IBM-MIT-Purdue submission_description/5384|5384 Purdue 312 1.00000 0.039 0% 98% 1.00000
93 6280 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-104321-2720.sr-20191212-104321-4127 IBM-MIT-Purdue submission_description/6000|6000 Purdue 312 1.00000 0.037 0% 98% 1.00000
94 5737 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191207-123158-7735.sr-20191207-123158-8833 IBM-MIT-Purdue submission_description/5506|5506 Purdue 312 1.00000 0.039 0% 98% 1.00000
95 4808 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191114-140029-1145.sr-20191114-140029-2873 INF submission_description/4513|4513 INF_MEVA1 283 1.00000 0.039 0% 98% 1.00000
96 5997 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191210-114411-9709.sr-20191210-114412-1397 IBM-MIT-Purdue submission_description/5818|5818 Purdue 312 1.00000 0.037 0% 98% 1.00000
97 6201 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153847-1658.sr-20191211-153847-2839 WeiLai submission_description/5817|5817 Tryyitry 314 1.00000 0.177 0% 0% 1.00000
98 6202 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153859-9306.sr-20191211-153900-0465 WeiLai submission_description/5651|5651 Tryyitry 314 1.00000 0.178 0% 0% 1.00000
99 6203 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153906-9288.sr-20191211-153907-0663 WeiLai submission_description/5576|5576 Tryyitry 314 1.00000 0.178 0% 0% 1.00000
100 6204 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153918-0443.sr-20191211-153918-1804 WeiLai submission_description/5452|5452 Tryyitry 314 1.00000 0.178 0% 0% 1.00000
101 6200 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00315_Jay-Chou_20191211-153836-2972.sr-20191211-153836-4343 Jay Chou submission_description/5904|5904 Jack 315 1.00000 0.179 0% 0% 1.00000
102 11721 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200128-125907-5613.sr-20200128-125907-6835 IBM-MIT-Purdue submission_description/11642|11642 Purdue 312 1.00000 0.016 0% 100% 1.00000
103 6282 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111731-1693.sr-20191212-111731-2887 IBM-MIT-Purdue submission_description/6056|6056 Purdue 312 1.00000 0.037 0% 100% 1.00000
104 8822 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-174128-0858.sr-20200113-174128-2010 Team_Vision submission_description/8313|8313 STARK 282 1.00000 0.049 0% 79% 1.00000

SDL19-scoring-EO

Updated: 2020-01-29 11:34:36 -0500

SDL19-scoring-IR

Updated: 2020-01-29 11:34:39 -0500
RANK SCORING_SUBMISSION_ID SCORING REQUEST NAME TEAM NAME SUBMISSION ID SYSTEM NAME SYSTEM ID <REALTIME PARTIAL AUDC* >REALTIME PARTIAL AUDC* RELATIVE PROCESSING TIME DETECTED ACTIVITY TYPES† PROCESSED FILES‡ <REALTIME MEAN-P_MISS@0.04TFA
1 10819 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200122-110041-0660.sr-20200122-110041-3688 UMD submission_description/10000|10000 UMD 281 0.75688 0.803 94% 100% 0.80264
2 8429 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-220215-1118.sr-20200111-220215-3627 UMD submission_description/8222|8222 UMD 281 0.76307 0.76307 1.260 94% 100% 0.81421
3 7902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200107-084859-7555.sr-20200107-084859-9955 UMD submission_description/7472|7472 UMD 281 0.76875 0.76875 1.042 91% 100% 0.81331
4 8357 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-033656-2617.sr-20200111-033656-4961 UMD submission_description/8198|8198 UMD 281 0.76993 0.76993 1.134 94% 100% 0.81324
5 5813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191207-183303-2020.sr-20191207-183303-4371 UMD submission_description/5595|5595 UMD 281 0.77010 0.725 89% 100% 0.80940
6 6396 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191213-112124-8334.sr-20191213-112125-1131 UMD submission_description/6285|6285 UMD 281 0.77140 0.723 97% 100% 0.81635
7 3974 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191023-115220-9973.sr-20191023-115221-4006 UMD submission_description/3666|3666 UMD 281 0.77746 0.970 89% 100% 0.81079
8 4897 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191119-140314-0793.sr-20191119-140314-5348 UMD submission_description/4809|4809 UMD 281 0.77749 0.965 89% 100% 0.81079
9 11725 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200128-173618-8145.sr-20200128-173619-0892 UMD submission_description/11045|11045 UMD+UCF 326 0.78784 0.78784 1.154 97% 100% 0.80344
10 10894 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200123-034832-1571.sr-20200123-034832-4329 UCF submission_description/10821|10821 UCF-P 270 0.81208 0.407 100% 100% 0.83407
11 3058 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190903-171132-2439.sr-20190904-151050-8165 UMD submission_description/1907|1907 UMD 281 0.82140 0.727 97% 94% 0.87521
12 6432 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191213-172239-8743.sr-20191213-172240-1096 IBM-MIT-Purdue submission_description/6303|6303 Purdue 312 0.83397 0.096 100% 100% 0.90113
13 5120 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-141345-4986.sr-20191127-141345-7995 IBM-MIT-Purdue submission_description/4999|4999 Purdue 312 0.83541 0.098 100% 100% 0.90568
14 5180 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-201530-0873.sr-20191127-201530-3571 IBM-MIT-Purdue submission_description/5001|5001 Purdue 312 0.83550 0.099 100% 100% 0.90567
15 8978 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200114-094657-1186.sr-20200114-094657-3580 UCF submission_description/8328|8328 UCF-P 270 0.84127 0.403 97% 100% 0.84625
16 3078 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190905-120934-4541.sr-20190905-120934-7789 UMD submission_description/2074|2074 UMD 281 0.84236 0.744 97% 94% 0.88549
17 5380 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-022020-9039.sr-20191129-022021-1734 IBM-MIT-Purdue submission_description/5005|5005 Purdue 312 0.84811 0.124 100% 100% 0.91150
18 5379 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-021929-1722.sr-20191129-021929-4337 IBM-MIT-Purdue submission_description/5004|5004 Purdue 312 0.84827 0.124 100% 100% 0.91231
19 11636 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-040601-4391.sr-20200127-040601-6831 IBM-MIT-Purdue submission_description/11346|11346 Purdue 312 0.85233 0.128 100% 100% 0.91307
20 11590 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-004703-9394.sr-20200127-004704-1921 IBM-MIT-Purdue submission_description/11345|11345 Purdue 312 0.85233 0.125 100% 100% 0.91307
21 6814 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-115038-7999.sr-20191220-115039-0506 IBM-MIT-Purdue submission_description/6513|6513 Purdue 312 0.85233 0.272 100% 100% 0.91307
22 11527 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-212645-2748.sr-20200126-212645-5772 IBM-MIT-Purdue submission_description/11344|11344 Purdue 312 0.85478 0.035 100% 100% 0.91614
23 11439 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-180453-0069.sr-20200126-180453-2498 IBM-MIT-Purdue submission_description/11343|11343 Purdue 312 0.85478 0.038 100% 100% 0.91614
24 6816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-135245-0601.sr-20191220-135245-3689 IBM-MIT-Purdue submission_description/6443|6443 Purdue 312 0.85625 0.85625 1.072 100% 100% 0.91761
25 6812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191219-234702-7711.sr-20191219-234703-6560 IBM-MIT-Purdue submission_description/6501|6501 Purdue 312 0.85759 0.442 100% 100% 0.92499
26 4732 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191112-144852-1012.sr-20191112-144852-3707 IBM-MIT-Purdue submission_description/4660|4660 Purdue 312 0.85825 0.087 100% 100% 0.92276
27 10182 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200120-124912-8942.sr-20200120-124913-1637 UCF submission_description/9105|9105 UCF-P 270 0.85863 0.424 100% 100% 0.86120
28 4659 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191109-113627-1003.sr-20191109-113627-8252 IBM-MIT-Purdue submission_description/4520|4520 Purdue 312 0.85961 0.087 100% 100% 0.93935
29 10820 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200122-112821-7686.sr-20200122-112822-0383 UCF submission_description/10640|10640 UCF-P 270 0.87051 0.362 100% 100% 0.87354
30 6902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-235401-7601.sr-20191220-235402-0107 IBM-MIT-Purdue submission_description/6815|6815 Purdue 312 0.87326 0.315 100% 100% 0.90380
31 4070 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191028-165446-2748.sr-20191028-165447-0013 IBM-MIT-Purdue submission_description/3830|3830 Purdue 312 0.87802 0.246 100% 100% 0.95373
32 6855 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-175217-7070.sr-20191220-175217-9512 IBM-MIT-Purdue submission_description/6547|6547 Purdue 312 0.87829 0.099 100% 100% 0.90638
33 4507 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-114135-1903.sr-20191106-114136-0201 IBM-MIT-Purdue submission_description/4228|4228 Purdue 312 0.88140 0.093 97% 100% 0.94599
34 4508 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-121322-4750.sr-20191106-121323-4002 IBM-MIT-Purdue submission_description/4179|4179 Purdue 312 0.88180 0.093 97% 100% 0.95227
35 4955 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191120-020423-5382.sr-20191120-020423-7702 IBM-MIT-Purdue submission_description/4876|4876 Purdue 312 0.88330 0.088 83% 100% 0.92867
36 11342 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-060218-3152.sr-20200126-060218-5498 IBM-MIT-Purdue submission_description/11268|11268 Purdue 312 0.88815 0.124 100% 100% 0.90887
37 10706 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-013602-6833.sr-20200122-013602-9771 IBM-MIT-Purdue submission_description/10041|10041 Purdue 312 0.89151 0.143 100% 100% 0.92110
38 8275 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200110-091609-1531.sr-20200110-091609-4167 IBM-MIT-Purdue submission_description/8154|8154 Purdue 312 0.89563 0.380 100% 100% 0.93596
39 10394 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-000324-4735.sr-20200121-000324-7531 IBM-MIT-Purdue submission_description/9978|9978 Purdue 312 0.89563 0.145 100% 100% 0.93596
40 9496 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-132317-0783.sr-20200116-132317-3212 IBM-MIT-Purdue submission_description/8290|8290 Purdue 312 0.89563 0.301 100% 100% 0.93596
41 11269 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-013750-5311.sr-20200126-013750-7710 IBM-MIT-Purdue submission_description/11188|11188 Purdue 312 0.90481 0.108 100% 100% 0.93280
42 8240 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00316_Edge-Intelligence_20200110-031108-3165.sr-20200110-031108-5673 Edge-Intelligence submission_description/8003|8003 Edge-Intelligence 316 0.90795 0.939 97% 100% 0.90508
43 8004 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200108-080622-3945.sr-20200108-080622-6125 UCF submission_description/7728|7728 UCF-P 270 0.92333 0.438 86% 100% 0.92110
44 8005 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200108-081958-6808.sr-20200108-081958-9170 Edge-Intelligence submission_description/7845|7845 Edge_Intelligence 320 0.92361 0.887 94% 94% 0.92075
45 6813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191220-054840-0488.sr-20191220-054840-2775 UCF submission_description/6499|6499 UCF-P 270 0.93293 0.333 91% 82% 0.93052
46 10535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-164242-1566.sr-20200121-164242-4398 IBM-MIT-Purdue submission_description/10037|10037 Purdue 312 0.94086 0.151 72% 100% 0.96263
47 10762 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-045506-2717.sr-20200122-045506-5220 IBM-MIT-Purdue submission_description/10038|10038 Purdue 312 0.94500 0.144 62% 100% 0.96100
48 7628 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200103-201849-4904.sr-20200103-201849-7180 UCF submission_description/7235|7235 UCF-P 270 0.94610 0.347 86% 82% 0.94443
49 3973 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191023-114820-4426.sr-20191023-114820-8834 UCF submission_description/3472|3472 UCF-P 270 0.95301 0.292 83% 100% 0.97139
50 10251 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-161145-9437.sr-20200120-161146-1778 IBM-MIT-Purdue submission_description/9866|9866 Purdue 312 0.96021 0.142 70% 100% 0.95921
51 10320 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-193353-6571.sr-20200120-193353-8725 IBM-MIT-Purdue submission_description/9873|9873 Purdue 312 0.96254 0.139 64% 100% 0.96125
52 10970 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200123-211058-0522.sr-20200123-211058-2592 IBM-MIT-Purdue submission_description/10896|10896 Purdue 312 0.96408 0.143 70% 100% 0.96800
53 11121 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-221522-0054.sr-20200124-221522-2197 IBM-MIT-Purdue submission_description/10973|10973 Purdue 312 0.96409 0.151 70% 100% 0.96802
54 11062 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-175059-7650.sr-20200124-175059-9697 IBM-MIT-Purdue submission_description/10972|10972 Purdue 312 0.96409 0.152 70% 100% 0.96802
55 9969 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200118-173929-1848.sr-20200118-173929-4320 INF submission_description/8367|8367 set3 290 0.96491 0.96491 1.332 62% 100% 0.97584
56 10634 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-221418-2880.sr-20200121-221418-5157 IBM-MIT-Purdue submission_description/10040|10040 Purdue 312 0.96604 0.147 64% 100% 0.96953
57 3257 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190913-094214-5138.sr-20190913-094214-7316 UCF submission_description/3104|3104 UCF-P 270 0.96630 0.291 78% 100% 0.97125
58 7682 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200104-082151-3830.sr-20200104-082151-6002 Edge-Intelligence submission_description/7427|7427 Edge_Intelligence 320 0.96665 0.736 13% 100% 0.96350
59 9639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-164611-8699.sr-20200116-164612-1412 IBM-MIT-Purdue submission_description/8289|8289 Purdue 312 0.96736 0.370 59% 100% 0.97474
60 6200 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00315_Jay-Chou_20191211-153836-2972.sr-20191211-153836-5308 Jay Chou submission_description/5904|5904 Jack 315 0.96884 0.179 51% 100% 0.98214
61 7139 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20191225-070143-9529.sr-20191225-070144-1669 INF submission_description/6967|6967 set3 290 0.97426 0.911 59% 100% 0.97768
62 4069 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20191028-164751-7980.sr-20191028-164752-1135 INF submission_description/3276|3276 INF_MEVA_IOD 307 0.97430 0.588 83% 94% 0.97774
63 4804 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191113-152522-8003.sr-20191113-152526-3773 Team_Vision submission_description/4434|4434 STARK 282 0.97668 0.774 89% 100% 0.97965
64 7547 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200103-081512-4482.sr-20200103-081512-6526 INF submission_description/7212|7212 INF_MEVA1 283 0.97918 0.922 62% 94% 0.98104
65 7061 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191224-125830-3199.sr-20191224-125830-5590 INF submission_description/6965|6965 INF_MEVA1 283 0.98156 0.906 64% 100% 0.98402
66 7264 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191231-014617-6841.sr-20191231-014617-8935 INF submission_description/7152|7152 speed1x 298 0.98244 0.989 64% 82% 0.98438
67 7907 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200107-092828-2753.sr-20200107-092831-6310 Team_Vision submission_description/6865|6865 STARK 282 0.98773 0.793 86% 83% 0.98900
68 4596 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191108-232219-1501.sr-20191108-232219-3782 UCF submission_description/4510|4510 UCF-P 270 0.98791 0.293 56% 94% 0.98864
69 10104 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200120-055837-4585.sr-20200120-055837-6726 Team_Vision submission_description/8506|8506 STARK 282 0.98832 0.739 78% 54% 0.99005
70 4427 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191101-142638-9265.sr-20191101-142639-1652 UCF submission_description/4060|4060 UCF-P 270 0.99044 0.273 54% 100% 0.99141
71 5999 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191210-174641-6248.sr-20191210-174641-9327 UCF submission_description/5851|5851 UCF-P 270 0.99055 0.293 75% 100% 0.99031
72 3330 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20190919-133858-1097.sr-20190919-133858-3388 DIVA TE Baseline submission_description/1786|1786 RC3D 276 0.99139 0.959 27% 100% 0.99280
73 6201 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153847-1658.sr-20191211-153847-3737 WeiLai submission_description/5817|5817 Tryyitry 314 0.99139 0.177 27% 100% 0.99280
74 6202 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153859-9306.sr-20191211-153900-1566 WeiLai submission_description/5651|5651 Tryyitry 314 0.99139 0.178 27% 100% 0.99280
75 6204 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153918-0443.sr-20191211-153918-2704 WeiLai submission_description/5452|5452 Tryyitry 314 0.99139 0.178 27% 100% 0.99280
76 6203 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153906-9288.sr-20191211-153907-1515 WeiLai submission_description/5576|5576 Tryyitry 314 0.99139 0.178 27% 100% 0.99280
77 8751 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200113-110217-3652.sr-20200113-110217-5648 INF submission_description/8273|8273 INF_MEVA1 283 0.99176 0.99176 1.054 35% 100% 0.99142
78 3639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191010-112752-8152.sr-20191010-112755-5280 Team_Vision submission_description/3557|3557 STARK 282 0.99258 0.723 45% 100% 0.99295
79 3500 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190930-141448-4379.sr-20190930-141451-2761 Team_Vision submission_description/3382|3382 STARK 282 0.99268 0.726 45% 100% 0.99345
80 8924 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200114-025702-0518.sr-20200114-025702-2624 IBM-MIT-Purdue submission_description/8278|8278 Purdue 312 0.99294 0.361 24% 100% 0.99399
81 5378 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191128-201821-9045.sr-20191128-201822-1378 UCF submission_description/5077|5077 UCF-P 270 0.99302 0.312 40% 94% 0.99278
82 5557 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191205-142328-8277.sr-20191205-142329-2540 INF submission_description/4959|4959 meva_inf2 288 0.99335 0.99335 1.703 75% 64% 0.99325
83 4428 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191101-144530-2898.sr-20191101-144532-2436 Team_Vision submission_description/4071|4071 STARK 282 0.99338 0.757 43% 82% 0.99333
84 3080 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190905-135645-9484.sr-20190905-135646-1912 UCF submission_description/2014|2014 UCF-P 270 0.99346 0.236 56% 100% 0.99547
85 9786 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-232636-3806.sr-20200116-232636-6034 IBM-MIT-Purdue submission_description/8287|8287 Purdue 312 0.99421 0.361 13% 100% 0.99530
86 9712 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-200717-3029.sr-20200116-200717-5095 IBM-MIT-Purdue submission_description/8288|8288 Purdue 312 0.99421 0.359 13% 100% 0.99530
87 9843 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200117-024347-1627.sr-20200117-024347-3789 IBM-MIT-Purdue submission_description/8286|8286 Purdue 312 0.99421 0.357 13% 100% 0.99530
88 8858 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200113-211626-3700.sr-20200113-211626-5742 IBM-MIT-Purdue submission_description/8284|8284 Purdue 312 0.99421 0.374 13% 94% 0.99530
89 5815 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191208-003354-3071.sr-20191208-003354-5163 UCF submission_description/5596|5596 UCF-P 270 0.99584 0.304 45% 100% 0.99573
90 5535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191204-142823-9331.sr-20191204-142824-1746 UCF submission_description/5439|5439 UCF-P 270 0.99636 0.304 43% 100% 0.99626
91 3083 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190905-165243-1769.sr-20190905-165244-9598 Team_Vision submission_description/2694|2694 STARK 282 0.99804 0.99901 1.795 48% 94% 0.99869
92 8812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-172127-0711.sr-20200113-172127-3465 Team_Vision submission_description/8295|8295 STARK 282 0.99899 0.060 40% 82% 0.99896
93 7336 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191231-195249-0336.sr-20191231-195249-2332 INF submission_description/7151|7151 meva_inf2 288 0.99914 0.931 62% 100% 0.99912
94 11721 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200128-125907-5613.sr-20200128-125907-7721 IBM-MIT-Purdue submission_description/11642|11642 Purdue 312 1.00000 0.016 0% 0% 1.00000
95 3055 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190903-152553-6173.sr-20190903-152553-9672 INF submission_description/2928|2928 INF_MEVA1 283 1.00000 0.788 54% 100% 1.00000
96 5446 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191204-022616-0789.sr-20191204-022616-3057 IBM-MIT-Purdue submission_description/5384|5384 Purdue 312 1.00000 0.039 0% 0% 1.00000
97 5737 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191207-123158-7735.sr-20191207-123158-9723 IBM-MIT-Purdue submission_description/5506|5506 Purdue 312 1.00000 0.039 0% 0% 1.00000
98 2892 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190828-173003-1496.sr-20190828-173003-3976 INF submission_description/2682|2682 INF_MEVA1 283 1.00000 0.000 0% 100% 1.00000
99 5997 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191210-114411-9709.sr-20191210-114412-2408 IBM-MIT-Purdue submission_description/5818|5818 Purdue 312 1.00000 0.037 0% 0% 1.00000
100 6280 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-104321-2720.sr-20191212-104321-5060 IBM-MIT-Purdue submission_description/6000|6000 Purdue 312 1.00000 0.037 0% 0% 1.00000
101 6281 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111546-3723.sr-20191212-111546-5775 IBM-MIT-Purdue submission_description/6055|6055 Purdue 312 1.00000 0.037 0% 0% 1.00000
102 6282 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111731-1693.sr-20191212-111731-3834 IBM-MIT-Purdue submission_description/6056|6056 Purdue 312 1.00000 0.037 0% 0% 1.00000
103 8822 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-174128-0858.sr-20200113-174128-2929 Team_Vision submission_description/8313|8313 STARK 282 1.00000 0.049 0% 82% 1.00000
104 8816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-173052-2474.sr-20200113-173052-4549 Team_Vision submission_description/8299|8299 STARK 282 1.00000 0.061 29% 76% 1.00000
105 4068 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191028-164450-6506.sr-20191028-164451-1116 INF submission_description/3163|3163 speed1x 298 1.00000 0.860 48% 100% 1.00000
106 4808 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191114-140029-1145.sr-20191114-140029-3974 INF submission_description/4513|4513 INF_MEVA1 283 1.00000 0.039 0% 100% 1.00000

SDL19-scoring-IR

Updated: 2020-01-29 11:34:40 -0500

*Partial AUDC is the area under the DET curve between a Time-based False Alarm rate of 0 and 0.2. Value of a perfect system is 0.

†The fraction of activity types with detected instances in the processed data set.

‡The fraction of videos processed and scored.

Algorithm Delivery for the DIVA teams

System delivery to the leaderboard must be in a form compatible with the ActEV Command Line Interface (ActEV CLI) and submitted to NIST for testing. The command line interface implementation that you will provide formalizes the entire process of evaluating a system, by providing the evaluation team a means to: (1) download and install your software via a single URL, (2) verify that the delivery works AND produces output that is “consistent” with output you produce, and (3) process a large collection of video in a fault-tolerant, parallelizable manner.

To complete this task you will need the following items described in detail below:

  1. FAQ - Validation Phase Processing
  2. CLI Description
  3. The Abstract CLI Git Repository
  4. The CLI Implementation Primer
  5. The Validation Data Set
  6. Example CLI-Compliant Implementation
  7. NIST Hardware and Initial Operating System Description

FAQ - Validation Phase Processing

The ActEV SDL - Validation Phase Processing FAQ

CLI Description

The ActEV CLI description

The Abstract CLI GIT Repository

The Abstract CLI GIT repository. The repo contains documentation.

The CLI Implementation Primer

There are 6 steps to adapt your code to the CLI. The ActEV Evaluation CLI Programming Primer describes the steps to clone the Abstract CLI and begin adapting the code to your implementation.

The Validation Data Set

As mentioned above, the CLI is used to verify the downloaded software is correctly installed and produces the same output at NIST as you produce locally. In order to do so, we have provided a small validation data set (ActEV-Eval-CLI-Validation-Set3) as part of the ActEV SDL Dataset that will be processed both at your site and at NIST. Please use this data in Step 3 of the Primer described above.

Example CLI-Compliant Implementation

The link below provides an example CLI implementation for the leaderboard baseline algorithm:

NIST Independent Evaluation Infrastructure Specification

NIST will begin installing your system from a fresh Ubuntu 18.04 cloud image available from https://cloud-images.ubuntu.com/releases/18.04/release/ on the following hardware.

  • Chassis: Asus ESC4000 G4S
  • CPU: 2 x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (12 cores/CPU)
  • Motherboard: Asus Intel® C621 PCH chipset
  • HDD/SSD: 2x 1.92GB Intel SSD DC S4500
  • RAM: 12x 16GB DDR4-2400 ECC RDIMM
  • GPU: Four PNY RTX2080Ti blower style
  • OS: Ubuntu 18.04
  • Storage Volume- 1TB (variable)
  • Supplied object store (read only) for source video


Contact Us

For ActEV Evaluation information (data, evaluation code, etc.) please email: actev-nist@nist.gov

For ActEV Evaluation Discussion Please Visit our Google Group. https://groups.google.com/a/list.nist.gov/forum/#!forum/trecvid.actev

Activities for the ActEV Sequestered Data Leaderboard

The list below specifies the activities to be detected for the ActEV SDL evaluation. Detailed activity definitions are in the ActEV Annotation Definitions for MEVA Data document. Note that current scoring for the SDL is based exclusively on activity detection, and object detection is not considered.

ActEV SDL Activities


person_closes_facility_door
person_closes_vehicle_door
Closing_Trunk
person_enters_through_structure
person_enters_vehicle
person_exits_through_structure
person_exits_vehicle
person_loads_vehicle
Open_Trunk
person_opens_facility_door
person_opens_vehicle_door
Transport_HeavyCarry
person_unloads_vehicle
vehicle_turning_left
vehicle_turning_right
vehicle_u_turn
Riding
Talking
specialized_talking_phone
specialized_texting_phone
person_sitting_down
person_standing_up
person_reading_document
object_transfer
person_picks_up_object
person_sets_down_object
hand_interaction
person_person_embrace
person_purchasing
person_laptop_interaction
vehicle_stopping
vehicle_starting
vehicle_reversing
vehicle_picks_up_person
vehicle_drops_off_person
abandon_package
theft
Task for the ActEV Sequestered Data Leaderboard
In the SDL evaluation, there is one Activity Detection (AD) task for detecting and temporally localizing activities

Activity Detection (AD)
For the Activity Detection task, given a target activity, a system automatically detects and temporally localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the type of activity must be correct and temporally overlap the true activity for at least one second. Additional details may be found in the SDL Evaluation Plan .
ActEV Sequestered Data Leaderboard (SDL) Evaluation

Updates
  • 12/17/19: The ActEV Scorer has been updated (eval. plan: pp. 14-15)
  • 12/17/19: The updated ActEV scorer will be deployed after the Jan 10th, 2020 submissions are scored
  • 10/15/19: The top two teams on the SDL leaderboard will be invited to give oral presentations at the HADCV'20 workshop based on the CLI submission deadline of Jan 10th, 2020 at 12:00 noon EST
Summary
The ActEV (Activities in Extended Video) Sequestered Data Leaderboard is an ongoing ranking of software systems that watch lengthy videos and detect activities of interest. Anyone can submit their system to NIST, which will then run the system on sequestered data, score the results and post the score to the leaderboard. The sequestered data is from the MEVA dataset, which contains hours of videos, including indoor and outdoor scenes, night and day, crowds and individuals, and videos are from both EO (Electro-Optical) and IR (Infrared) sensors. Hours can go by with no activities, but then multiple activities happen simultaneously. The data is multi-camera, in that multiple cameras may be pointed at the same scene at the same time. There are also two separate leaderboards for EO and IR videos.
What
Build and submit a software system to watch videos and detect if and when an activity of interest occurs. System runtime must be 1 times the video length on the designated evaluation hardware.
Who
Everyone. Anyone who registers can submit to the evaluation server.
How
Register here and then follow the instructions on the Algorithm Submission tab above. Systems must follow a NIST-defined command line interface and automatically run on NIST’s servers, both of which are described in the instructions.
Evaluation Task
Detect if and when an activity occurs. Given a target activity type and a set of videos, submitted systems must automatically detect all instances of the activity in the videos. While different activities instances have different durations, a submitted system will be considered to have detected an activity if it correctly identifies at least 1 second of the activity.
Data
The data is from the Multiview Extended Video with Activities (MEVA) dataset and the videos are from both EO (Electro-Optical) and IR (Infrared) sensors. The data used for evaluation is sequestered, but the public MEVA dataset includes hundreds of hours of data from the same cameras at the same facility, which can be used for training. If you register for ActEV you can download the MEVA dataset for free; info on how to download the data is on the data tab. ActEV does not provide annotations, but instructions on how to make and share activity annotations are at mevadata.org.
Metrics
Submitted activity detection systems must give a confidence score for each activity they detect. Detected activities are then thresholded based on this confidence score. Varying the threshold makes a trade-off between being sensitive enough to identify true activity instances (low threshold) vs. not making false alarms when no activity is present (high threshold). Submitted systems are scored on both of these, measured by Probability of Missed Detection (Pmiss) and Time-based False Alarm (TFA). Pmiss is the portion of activities where the system did not detect the activity for at least 1 second. TFA is the portion of time that the system detected an activity when in fact there was none. Submitted systems are scored for Pmiss and TFA at multiple thresholds, creating a detect error tradeoff (DET) curve. The leaderboard ranking of a system is based on a summary of its DET curve: the average value of 1-Pmiss across the TFA range between 0% to 20%. Higher numbers are better, as they reflect fewer errors. See details in the SDL evaluation plan or check out the ActEV Scoring Software GitHub repo.
Evaluation Plan
Task coordinator
ActEV NIST team (ActEV-nist@nist.gov)
News
13June
ActEV SDL starts

ActEV SDL Evaluation Schedule
June 13, 2019: ActEV SDL opens

August 15, 2019 11 AM EST : Deadline for submissions to be included in ActEV SDL Evaluation Standings that will be reported at TRECVID 2019 workshop (November 12-13, 2019)

Leaderboard Remains Open: Leaderboard will continue to remain open to allow participants to show continued progress on this challenging problem

January 10th, 2020 at 12:00 noon EST: We will invite the top two teams on the SDL leaderboard to give oral presentations at the WACV HADCV'20 workshop based on the CLI submission deadline


Datasets

Framework

The DIVA Framework is a software framework designed to provide an architecture and a set of software modules which will facilitate the development of activity recognition analytics. The Framework is developed as a fully open source project on GitHub. The following links will help you get started with the framework: The DIVA Framework is based on KWIVER, an open source framework designed for building complex computer vision systems. The following links will help you learn more about KWIVER:
  • KWIVER Github Repository This is the main KWIVER site, all development of the framework happens here.
  • KWIVER Issue Tracker Submit any bug reports or feature requests for the KWIVER here. If there's any question about whether your issues belongs in the KWIVER or DIVA framework issues tracker, submit to the DIVA tracker and we'll sort it out..
  • KWIVER Main Documentation PageThe source for the KWIVER documentation is maintained in the Github repository using Sphinx. A built version is maintained on ReadTheDocs at this link. A good place to get started in the documentation, after reading the Introduction are the Arrows and Sprokit sections, both of which are used by the KWIVER framework.

Baseline Algorithms

KITWARE has adapted two "baseline" activity recognition algorithms to work within the DIVA Framework:

Annotation Tools

ActEV SDL Dataset

The ActEV Data GIT Repo is the data distribution mechanism for the ActEV evaluations. The repo presently consists of a collection of corpora and partition definition files to be used for evaluations. Future additions will include source annotations and community provided data/annotations. The repo contains textual data but not the large-sized corpora (videos, etc.). For these large corpora, the repo contains the necessary information to download a given corpus giving the user control of the downloaded content. Access credentials are provided during ActEV registration.

To sign up, simply create a login account by registering for ActEV SDL, and you will receive credentials for the video data download. If there is any issue please email us at actev-nist@nist.gov



If you have any question, please email to actev-nist@nist.gov