ActEV Sequestered Data Leaderboard


ActEV SDL Leaderboard


SDL19-scoring-EO

Updated: 2020-02-22 03:08:37 -0500
RANK SCORING_SUBMISSION_ID SCORING REQUEST NAME TEAM NAME SUBMISSION ID SUBMISSION DATE SYSTEM NAME SYSTEM ID PARTIAL AUDC* TIME LIMITED PARTIAL AUDC* RELATIVE PROCESSING TIME DETECTED ACTIVITY TYPES† PROCESSED FILES‡ MEAN-P_MISS@0.04TFA TIME LIMITED MEAN-P_MISS@0.04TFA
1 11725 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200128-173618-8145.sr-20200128-173618-9987 UMD submission_description/11045|11045 2020-01-24 UMD+UCF 326 0.41585 0.65517 1.154 100% 100% 0.49360 0.67902
2 10820 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200122-112821-7686.sr-20200122-112821-9188 UCF submission_description/10640|10640 2020-01-21 UCF-P 270 0.43810 0.362 100% 100% 0.52356
3 10182 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200120-124912-8942.sr-20200120-124913-0413 UCF submission_description/9105|9105 2020-01-15 UCF-P 270 0.44056 0.424 97% 100% 0.52513
4 8978 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200114-094657-1186.sr-20200114-094657-2665 UCF submission_description/8328|8328 2020-01-10 UCF-P 270 0.44360 0.403 100% 100% 0.52972
5 10894 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200123-034832-1571.sr-20200123-034832-3198 UCF submission_description/10821|10821 2020-01-22 UCF-P 270 0.44473 0.407 100% 100% 0.54228
6 5813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191207-183303-2020.sr-20191207-183303-3485 UMD submission_description/5595|5595 2019-12-06 UMD 281 0.47596 0.725 97% 100% 0.54479
7 8429 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-220215-1118.sr-20200111-220215-2680 UMD submission_description/8222|8222 2020-01-10 UMD 281 0.47668 0.68909 1.260 100% 100% 0.54492 0.71689
8 8357 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-033656-2617.sr-20200111-033656-4084 UMD submission_description/8198|8198 2020-01-09 UMD 281 0.48037 0.68601 1.134 97% 100% 0.54825 0.71585
9 8004 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200108-080622-3945.sr-20200108-080622-5226 UCF submission_description/7728|7728 2020-01-04 UCF-P 270 0.48290 0.438 94% 100% 0.51235
10 8132 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200109-135041-5138.sr-20200109-135041-6745 UMD submission_description/7992|7992 2020-01-08 UMD 281 0.48333 0.68298 1.098 97% 100% 0.55028 0.71410
11 10819 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200122-110041-0660.sr-20200122-110041-2755 UMD submission_description/10000|10000 2020-01-19 UMD 281 0.49061 0.803 100% 100% 0.56996
12 12034 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200207-053436-7072.sr-20200207-053436-9714 INF submission_description/11911|11911 2020-02-06 set3 290 0.49220 0.642 97% 98% 0.56143
13 7902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200107-084859-7555.sr-20200107-084859-9065 UMD submission_description/7472|7472 2020-01-02 UMD 281 0.49689 0.61793 1.042 97% 93% 0.55515 0.65471
14 7628 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200103-201849-4904.sr-20200103-201849-6284 UCF submission_description/7235|7235 2019-12-30 UCF-P 270 0.51676 0.347 97% 100% 0.55164
15 6813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191220-054840-0488.sr-20191220-054840-1911 UCF submission_description/6499|6499 2019-12-18 UCF-P 270 0.52068 0.333 97% 100% 0.57885
16 4897 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191119-140314-0793.sr-20191119-140314-4363 UMD submission_description/4809|4809 2019-11-15 UMD 281 0.53144 0.965 100% 98% 0.60011
17 3974 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191023-115220-9973.sr-20191023-115221-2327 UMD submission_description/3666|3666 2019-10-10 UMD 281 0.53147 0.970 100% 98% 0.60101
18 12332 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200212-201017-1346.sr-20200212-201017-2816 UCF submission_description/12222|12222 2020-02-11 UCF-P 270 0.53408 0.396 100% 94% 0.59012
19 6396 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191213-112124-8334.sr-20191213-112124-9953 UMD submission_description/6285|6285 2019-12-12 UMD 281 0.53459 0.723 100% 98% 0.59596
20 12137 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20200209-183511-9480.sr-20200209-183512-1614 INF submission_description/12068|12068 2020-02-09 speed1x 298 0.53654 0.642 97% 94% 0.59701
21 11975 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200206-171924-8432.sr-20200206-171925-0294 INF submission_description/11850|11850 2020-02-03 meva_inf2 288 0.59044 0.960 94% 97% 0.64252
22 5999 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191210-174641-6248.sr-20191210-174641-7467 UCF submission_description/5851|5851 2019-12-09 UCF-P 270 0.60431 0.293 97% 100% 0.64157
23 8240 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00316_Edge-Intelligence_20200110-031108-3165.sr-20200110-031108-4757 Edge-Intelligence submission_description/8003|8003 2020-01-08 Edge-Intelligence 316 0.62842 0.939 97% 100% 0.75488
24 6814 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-115038-7999.sr-20191220-115038-9539 IBM-MIT-Purdue submission_description/6513|6513 2019-12-18 Purdue 312 0.64182 0.272 100% 100% 0.73321
25 11636 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-040601-4391.sr-20200127-040601-5932 IBM-MIT-Purdue submission_description/11346|11346 2020-01-26 Purdue 312 0.64182 0.128 100% 100% 0.73320
26 11590 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-004703-9394.sr-20200127-004704-1030 IBM-MIT-Purdue submission_description/11345|11345 2020-01-26 Purdue 312 0.64182 0.125 100% 100% 0.73320
27 3078 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190905-120934-4541.sr-20190905-120934-6642 UMD submission_description/2074|2074 2019-08-16 UMD 281 0.65325 0.744 97% 98% 0.73860
28 6902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-235401-7601.sr-20191220-235401-9202 IBM-MIT-Purdue submission_description/6815|6815 2019-12-20 Purdue 312 0.65351 0.315 100% 100% 0.73730
29 5379 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-021929-1722.sr-20191129-021929-3345 IBM-MIT-Purdue submission_description/5004|5004 2019-11-25 Purdue 312 0.65503 0.124 100% 98% 0.73745
30 5380 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-022020-9039.sr-20191129-022021-0562 IBM-MIT-Purdue submission_description/5005|5005 2019-11-25 Purdue 312 0.65913 0.124 100% 98% 0.73971
31 6812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191219-234702-7711.sr-20191219-234703-5380 IBM-MIT-Purdue submission_description/6501|6501 2019-12-18 Purdue 312 0.66055 0.442 100% 100% 0.74490
32 4732 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191112-144852-1012.sr-20191112-144852-2735 IBM-MIT-Purdue submission_description/4660|4660 2019-11-11 Purdue 312 0.66383 0.087 100% 98% 0.74381
33 11439 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-180453-0069.sr-20200126-180453-1583 IBM-MIT-Purdue submission_description/11343|11343 2020-01-26 Purdue 312 0.66394 0.038 100% 98% 0.74397
34 11527 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-212645-2748.sr-20200126-212645-4572 IBM-MIT-Purdue submission_description/11344|11344 2020-01-26 Purdue 312 0.66394 0.035 100% 98% 0.74397
35 11342 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-060218-3152.sr-20200126-060218-4619 IBM-MIT-Purdue submission_description/11268|11268 2020-01-26 Purdue 312 0.66496 0.124 100% 100% 0.74147
36 6432 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191213-172239-8743.sr-20191213-172240-0161 IBM-MIT-Purdue submission_description/6303|6303 2019-12-12 Purdue 312 0.66900 0.096 100% 100% 0.75098
37 6855 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-175217-7070.sr-20191220-175217-8617 IBM-MIT-Purdue submission_description/6547|6547 2019-12-18 Purdue 312 0.67114 0.099 100% 100% 0.75100
38 3058 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190903-171132-2439.sr-20190903-171132-4143 UMD submission_description/1907|1907 2019-08-13 UMD 281 0.67831 0.727 0.74917
39 5120 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-141345-4986.sr-20191127-141345-6913 IBM-MIT-Purdue submission_description/4999|4999 2019-11-25 Purdue 312 0.67912 0.098 100% 98% 0.76349
40 11819 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200130-120346-5169.sr-20200130-120346-7359 INF submission_description/11670|11670 2020-01-28 INF_MEVA1 283 0.67983 0.928 100% 100% 0.78032
41 5180 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-201530-0873.sr-20191127-201530-2638 IBM-MIT-Purdue submission_description/5001|5001 2019-11-25 Purdue 312 0.68547 0.099 100% 98% 0.76606
42 11269 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-013750-5311.sr-20200126-013750-6816 IBM-MIT-Purdue submission_description/11188|11188 2020-01-25 Purdue 312 0.69501 0.108 100% 98% 0.75741
43 5815 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191208-003354-3071.sr-20191208-003354-4279 UCF submission_description/5596|5596 2019-12-06 UCF-P 270 0.69848 0.304 86% 100% 0.73165
44 6816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-135245-0601.sr-20191220-135245-2185 IBM-MIT-Purdue submission_description/6443|6443 2019-12-17 Purdue 312 0.70117 0.82690 1.072 100% 100% 0.77819 0.87029
45 4955 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191120-020423-5382.sr-20191120-020423-6818 IBM-MIT-Purdue submission_description/4876|4876 2019-11-19 Purdue 312 0.70382 0.088 97% 98% 0.76211
46 7907 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200107-092828-2753.sr-20200107-092831-5397 Team_Vision submission_description/6865|6865 2019-12-20 STARK 282 0.71736 0.793 97% 100% 0.77673
47 5535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191204-142823-9331.sr-20191204-142824-0649 UCF submission_description/5439|5439 2019-12-04 UCF-P 270 0.72830 0.304 86% 100% 0.75572
48 8275 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200110-091609-1531.sr-20200110-091609-3374 IBM-MIT-Purdue submission_description/8154|8154 2020-01-09 Purdue 312 0.74090 0.380 100% 100% 0.82467
49 10394 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-000324-4735.sr-20200121-000324-6679 IBM-MIT-Purdue submission_description/9978|9978 2020-01-18 Purdue 312 0.74091 0.145 100% 100% 0.82468
50 5378 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191128-201821-9045.sr-20191128-201822-0355 UCF submission_description/5077|5077 2019-11-27 UCF-P 270 0.74243 0.312 86% 100% 0.77197
51 4507 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-114135-1903.sr-20191106-114135-9240 IBM-MIT-Purdue submission_description/4228|4228 2019-10-31 Purdue 312 0.75410 0.093 97% 98% 0.82355
52 4659 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191109-113627-1003.sr-20191109-113627-7178 IBM-MIT-Purdue submission_description/4520|4520 2019-11-08 Purdue 312 0.75509 0.087 100% 98% 0.82359
53 10535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-164242-1566.sr-20200121-164242-3473 IBM-MIT-Purdue submission_description/10037|10037 2020-01-19 Purdue 312 0.79701 0.151 86% 100% 0.84131
54 4508 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-121322-4750.sr-20191106-121323-3067 IBM-MIT-Purdue submission_description/4179|4179 2019-10-30 Purdue 312 0.81573 0.093 97% 98% 0.86015
55 10706 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-013602-6833.sr-20200122-013602-8836 IBM-MIT-Purdue submission_description/10041|10041 2020-01-19 Purdue 312 0.82020 0.143 100% 100% 0.88273
56 4070 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191028-165446-2748.sr-20191028-165446-8950 IBM-MIT-Purdue submission_description/3830|3830 2019-10-21 Purdue 312 0.82025 0.246 100% 97% 0.86425
57 4428 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191101-144530-2898.sr-20191101-144532-0236 Team_Vision submission_description/4071|4071 2019-10-29 STARK 282 0.82121 0.757 75% 100% 0.85712
58 10104 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200120-055837-4585.sr-20200120-055837-5853 Team_Vision submission_description/8506|8506 2020-01-12 STARK 282 0.82442 0.739 94% 58% 0.85486
59 11947 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200206-111142-9816.sr-20200206-111143-1243 UCF submission_description/11849|11849 2020-02-02 UCF-P 270 0.82611 0.344 100% 14% 0.82180
60 3639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191010-112752-8152.sr-20191010-112755-4284 Team_Vision submission_description/3557|3557 2019-10-08 STARK 282 0.83722 0.723 78% 100% 0.88187
61 10762 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-045506-2717.sr-20200122-045506-4324 IBM-MIT-Purdue submission_description/10038|10038 2020-01-19 Purdue 312 0.84098 0.144 81% 100% 0.87514
62 3500 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190930-141448-4379.sr-20190930-141451-1874 Team_Vision submission_description/3382|3382 2019-09-23 STARK 282 0.84258 0.726 81% 100% 0.88996
63 3973 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191023-114820-4426.sr-20191023-114820-7137 UCF submission_description/3472|3472 2019-09-26 UCF-P 270 0.84269 0.292 100% 100% 0.89750
64 9969 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200118-173929-1848.sr-20200118-173929-3375 INF submission_description/8367|8367 2020-01-11 set3 290 0.84990 0.88917 1.332 89% 85% 0.89547 0.92076
65 9496 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-132317-0783.sr-20200116-132317-2298 IBM-MIT-Purdue submission_description/8290|8290 2020-01-10 Purdue 312 0.85131 0.301 100% 100% 0.88623
66 7547 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200103-081512-4482.sr-20200103-081512-5627 INF submission_description/7212|7212 2019-12-29 INF_MEVA1 283 0.85864 0.922 89% 100% 0.87950
67 7139 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20191225-070143-9529.sr-20191225-070144-0759 INF submission_description/6967|6967 2019-12-23 set3 290 0.86506 0.911 89% 100% 0.88947
68 11121 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-221522-0054.sr-20200124-221522-1316 IBM-MIT-Purdue submission_description/10973|10973 2020-01-24 Purdue 312 0.86613 0.151 72% 100% 0.88674
69 11062 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-175059-7650.sr-20200124-175059-8813 IBM-MIT-Purdue submission_description/10972|10972 2020-01-24 Purdue 312 0.86613 0.152 72% 100% 0.88674
70 10970 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200123-211058-0522.sr-20200123-211058-1721 IBM-MIT-Purdue submission_description/10896|10896 2020-01-23 Purdue 312 0.86614 0.143 72% 100% 0.88673
71 10251 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-161145-9437.sr-20200120-161146-0616 IBM-MIT-Purdue submission_description/9866|9866 2020-01-17 Purdue 312 0.86887 0.142 72% 100% 0.88733
72 3257 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190913-094214-5138.sr-20190913-094214-6400 UCF submission_description/3104|3104 2019-09-10 UCF-P 270 0.87068 0.291 97% 100% 0.91260
73 7061 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191224-125830-3199.sr-20191224-125830-4603 INF submission_description/6965|6965 2019-12-23 INF_MEVA1 283 0.87216 0.906 89% 100% 0.89303
74 10634 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-221418-2880.sr-20200121-221418-4184 IBM-MIT-Purdue submission_description/10040|10040 2020-01-19 Purdue 312 0.87422 0.147 70% 100% 0.89421
75 3083 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190905-165243-1769.sr-20190905-165244-8608 Team_Vision submission_description/2694|2694 2019-08-27 STARK 282 0.87562 0.91641 1.795 86% 80% 0.90833 0.92705
76 10320 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-193353-6571.sr-20200120-193353-7807 IBM-MIT-Purdue submission_description/9873|9873 2020-01-17 Purdue 312 0.87606 0.139 70% 100% 0.89165
77 4596 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191108-232219-1501.sr-20191108-232219-2801 UCF submission_description/4510|4510 2019-11-06 UCF-P 270 0.87657 0.293 97% 100% 0.91490
78 7264 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191231-014617-6841.sr-20191231-014617-8065 INF submission_description/7152|7152 2019-12-26 speed1x 298 0.88292 0.989 89% 86% 0.89853
79 12283 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200212-075455-9720.sr-20200212-075456-1184 Team_Vision submission_description/12208|12208 2020-02-11 STARK 282 0.89021 0.783 94% 36% 0.89837
80 12350 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200213-084119-5212.sr-20200213-084120-7284 Team_Vision submission_description/12319|12319 2020-02-12 STARK 282 0.89465 0.841 97% 26% 0.89748
81 4427 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191101-142638-9265.sr-20191101-142639-0480 UCF submission_description/4060|4060 2019-10-25 UCF-P 270 0.89623 0.273 97% 100% 0.92032
82 7682 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200104-082151-3830.sr-20200104-082151-5082 Edge-Intelligence submission_description/7427|7427 2020-01-02 Edge_Intelligence 320 0.92170 0.736 13% 96% 0.95539
83 4069 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20191028-164751-7980.sr-20191028-164752-0112 INF submission_description/3276|3276 2019-09-13 INF_MEVA_IOD 307 0.93009 0.588 94% 91% 0.94479
84 8751 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200113-110217-3652.sr-20200113-110217-4800 INF submission_description/8273|8273 2020-01-10 INF_MEVA1 283 0.93512 0.95304 1.054 59% 100% 0.93680 0.95338
85 3055 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190903-152553-6173.sr-20190903-152553-8801 INF submission_description/2928|2928 2019-08-29 INF_MEVA1 283 0.97344 0.788 89% 98% 0.98175
86 8924 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200114-025702-0518.sr-20200114-025702-1708 IBM-MIT-Purdue submission_description/8278|8278 2020-01-10 Purdue 312 0.97569 0.361 29% 100% 0.98028
87 7336 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191231-195249-0336.sr-20191231-195249-1460 INF submission_description/7151|7151 2019-12-26 meva_inf2 288 0.97762 0.931 83% 100% 0.97655
88 9639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-164611-8699.sr-20200116-164612-0130 IBM-MIT-Purdue submission_description/8289|8289 2020-01-10 Purdue 312 0.97778 0.370 24% 100% 0.98142
89 8858 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200113-211626-3700.sr-20200113-211626-4873 IBM-MIT-Purdue submission_description/8284|8284 2020-01-10 Purdue 312 0.97778 0.374 24% 100% 0.98142
90 9843 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200117-024347-1627.sr-20200117-024347-2974 IBM-MIT-Purdue submission_description/8286|8286 2020-01-10 Purdue 312 0.97778 0.357 24% 100% 0.98142
91 9786 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-232636-3806.sr-20200116-232636-5045 IBM-MIT-Purdue submission_description/8287|8287 2020-01-10 Purdue 312 0.97778 0.361 24% 100% 0.98142
92 9712 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-200717-3029.sr-20200116-200717-4191 IBM-MIT-Purdue submission_description/8288|8288 2020-01-10 Purdue 312 0.97778 0.359 24% 100% 0.98142
93 5557 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191205-142328-8277.sr-20191205-142329-1674 INF submission_description/4959|4959 2019-11-21 meva_inf2 288 0.98157 0.98830 1.703 83% 89% 0.98886 0.99250
94 1991 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00263_NIST-TEST_20190814-164723-7643.sr-20190814-164723-8758 NIST-TEST submission_description/1790|1790 2019-08-07 NIST Test 263 0.98216 0.930 29% 100% 0.98722
95 3330 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20190919-133858-1097.sr-20190919-133858-2351 DIVA TE Baseline submission_description/1786|1786 2019-08-02 RC3D 276 0.98216 0.959 29% 100% 0.98722
96 4068 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191028-164450-6506.sr-20191028-164450-9167 INF submission_description/3163|3163 2019-09-12 speed1x 298 0.98315 0.860 78% 94% 0.98496
97 8816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-173052-2474.sr-20200113-173052-3647 Team_Vision submission_description/8299|8299 2020-01-10 STARK 282 0.99165 0.061 83% 86% 0.99143
98 8812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-172127-0711.sr-20200113-172127-2341 Team_Vision submission_description/8295|8295 2020-01-10 STARK 282 0.99211 0.060 72% 96% 0.99190
99 11818 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20200130-113642-8277.sr-20200130-113642-9365 DIVA TE Baseline submission_description/7911|7911 2020-01-07 RC3D 276 0.99255 0.99552 2.791 27% 75% 0.99302 0.99521
100 11851 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00324_DIVA-TE-Baseline_20200204-101445-5317.sr-20200204-101445-6546 DIVA TE Baseline submission_description/7928|7928 2020-01-07 RC3D-WHEEL 324 0.99608 0.99608 2.715 27% 37% 0.99580 0.99580
101 6203 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153906-9288.sr-20191211-153907-0663 WeiLai submission_description/5576|5576 2019-12-05 Tryyitry 314 1.00000 0.178 0% 0% 1.00000
102 8822 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-174128-0858.sr-20200113-174128-2010 Team_Vision submission_description/8313|8313 2020-01-10 STARK 282 1.00000 0.049 0% 79% 1.00000
103 6202 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153859-9306.sr-20191211-153900-0465 WeiLai submission_description/5651|5651 2019-12-07 Tryyitry 314 1.00000 0.178 0% 0% 1.00000
104 2892 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190828-173003-1496.sr-20190828-173003-2992 INF submission_description/2682|2682 2019-08-27 INF_MEVA1 283 1.00000 0.000 0% 100% 1.00000
105 4808 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191114-140029-1145.sr-20191114-140029-2873 INF submission_description/4513|4513 2019-11-08 INF_MEVA1 283 1.00000 0.039 0% 98% 1.00000
106 11721 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200128-125907-5613.sr-20200128-125907-6835 IBM-MIT-Purdue submission_description/11642|11642 2020-01-27 Purdue 312 1.00000 0.016 0% 100% 1.00000
107 5446 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191204-022616-0789.sr-20191204-022616-2066 IBM-MIT-Purdue submission_description/5384|5384 2019-12-02 Purdue 312 1.00000 0.039 0% 98% 1.00000
108 5737 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191207-123158-7735.sr-20191207-123158-8833 IBM-MIT-Purdue submission_description/5506|5506 2019-12-04 Purdue 312 1.00000 0.039 0% 98% 1.00000
109 5997 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191210-114411-9709.sr-20191210-114412-1397 IBM-MIT-Purdue submission_description/5818|5818 2019-12-09 Purdue 312 1.00000 0.037 0% 98% 1.00000
110 6201 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153847-1658.sr-20191211-153847-2839 WeiLai submission_description/5817|5817 2019-12-09 Tryyitry 314 1.00000 0.177 0% 0% 1.00000
111 6282 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111731-1693.sr-20191212-111731-2887 IBM-MIT-Purdue submission_description/6056|6056 2019-12-11 Purdue 312 1.00000 0.037 0% 100% 1.00000
112 6281 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111546-3723.sr-20191212-111546-4849 IBM-MIT-Purdue submission_description/6055|6055 2019-12-11 Purdue 312 1.00000 0.037 0% 98% 1.00000
113 6200 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00315_Jay-Chou_20191211-153836-2972.sr-20191211-153836-4343 Jay Chou submission_description/5904|5904 2019-12-10 Jack 315 1.00000 0.179 0% 0% 1.00000
114 6204 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153918-0443.sr-20191211-153918-1804 WeiLai submission_description/5452|5452 2019-12-04 Tryyitry 314 1.00000 0.178 0% 0% 1.00000
115 6280 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-104321-2720.sr-20191212-104321-4127 IBM-MIT-Purdue submission_description/6000|6000 2019-12-10 Purdue 312 1.00000 0.037 0% 98% 1.00000

SDL19-scoring-EO

Updated: 2020-02-22 03:08:39 -0500

SDL19-scoring-IR

Updated: 2020-02-22 03:08:37 -0500
RANK SCORING_SUBMISSION_ID SCORING REQUEST NAME TEAM NAME SUBMISSION ID SUBMISSION DATE SYSTEM NAME SYSTEM ID PARTIAL AUDC* TIME LIMITED PARTIAL AUDC* RELATIVE PROCESSING TIME DETECTED ACTIVITY TYPES† PROCESSED FILES‡ MEAN-P_MISS@0.04TFA TIME LIMITED MEAN-P_MISS@0.04TFA
1 10819 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200122-110041-0660.sr-20200122-110041-3688 UMD submission_description/10000|10000 2020-01-19 UMD 281 0.75688 0.803 94% 100% 0.80264
2 8429 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-220215-1118.sr-20200111-220215-3627 UMD submission_description/8222|8222 2020-01-10 UMD 281 0.76307 0.76307 1.260 94% 100% 0.81421 0.81421
3 7902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200107-084859-7555.sr-20200107-084859-9955 UMD submission_description/7472|7472 2020-01-02 UMD 281 0.76875 0.76875 1.042 91% 100% 0.81331 0.81331
4 8357 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20200111-033656-2617.sr-20200111-033656-4961 UMD submission_description/8198|8198 2020-01-09 UMD 281 0.76993 0.76993 1.134 94% 100% 0.81324 0.81324
5 5813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191207-183303-2020.sr-20191207-183303-4371 UMD submission_description/5595|5595 2019-12-06 UMD 281 0.77010 0.725 89% 100% 0.80940
6 6396 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191213-112124-8334.sr-20191213-112125-1131 UMD submission_description/6285|6285 2019-12-12 UMD 281 0.77140 0.723 97% 100% 0.81635
7 3974 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191023-115220-9973.sr-20191023-115221-4006 UMD submission_description/3666|3666 2019-10-10 UMD 281 0.77746 0.970 89% 100% 0.81079
8 4897 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20191119-140314-0793.sr-20191119-140314-5348 UMD submission_description/4809|4809 2019-11-15 UMD 281 0.77749 0.965 89% 100% 0.81079
9 11725 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00326_UMD_20200128-173618-8145.sr-20200128-173619-0892 UMD submission_description/11045|11045 2020-01-24 UMD+UCF 326 0.78784 0.78784 1.154 97% 100% 0.80344 0.80344
10 10894 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200123-034832-1571.sr-20200123-034832-4329 UCF submission_description/10821|10821 2020-01-22 UCF-P 270 0.81208 0.407 100% 100% 0.83407
11 3058 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190903-171132-2439.sr-20190903-171132-5139 UMD submission_description/1907|1907 2019-08-13 UMD 281 0.82140 0.727 97% 94% 0.87521
12 12034 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200207-053436-7072.sr-20200207-053437-0564 INF submission_description/11911|11911 2020-02-06 set3 290 0.83021 0.642 94% 100% 0.85994
13 12137 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20200209-183511-9480.sr-20200209-183512-2495 INF submission_description/12068|12068 2020-02-09 speed1x 298 0.83023 0.642 91% 100% 0.85996
14 6432 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191213-172239-8743.sr-20191213-172240-1096 IBM-MIT-Purdue submission_description/6303|6303 2019-12-12 Purdue 312 0.83397 0.096 100% 100% 0.90113
15 5120 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-141345-4986.sr-20191127-141345-7995 IBM-MIT-Purdue submission_description/4999|4999 2019-11-25 Purdue 312 0.83541 0.098 100% 100% 0.90568
16 5180 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191127-201530-0873.sr-20191127-201530-3571 IBM-MIT-Purdue submission_description/5001|5001 2019-11-25 Purdue 312 0.83550 0.099 100% 100% 0.90567
17 8978 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200114-094657-1186.sr-20200114-094657-3580 UCF submission_description/8328|8328 2020-01-10 UCF-P 270 0.84127 0.403 97% 100% 0.84625
18 3078 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00281_UMD_20190905-120934-4541.sr-20190905-120934-7789 UMD submission_description/2074|2074 2019-08-16 UMD 281 0.84236 0.744 97% 94% 0.88549
19 5380 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-022020-9039.sr-20191129-022021-1734 IBM-MIT-Purdue submission_description/5005|5005 2019-11-25 Purdue 312 0.84811 0.124 100% 100% 0.91150
20 5379 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191129-021929-1722.sr-20191129-021929-4337 IBM-MIT-Purdue submission_description/5004|5004 2019-11-25 Purdue 312 0.84827 0.124 100% 100% 0.91231
21 11590 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-004703-9394.sr-20200127-004704-1921 IBM-MIT-Purdue submission_description/11345|11345 2020-01-26 Purdue 312 0.85233 0.125 100% 100% 0.91307
22 11636 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200127-040601-4391.sr-20200127-040601-6831 IBM-MIT-Purdue submission_description/11346|11346 2020-01-26 Purdue 312 0.85233 0.128 100% 100% 0.91307
23 6814 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-115038-7999.sr-20191220-115039-0506 IBM-MIT-Purdue submission_description/6513|6513 2019-12-18 Purdue 312 0.85233 0.272 100% 100% 0.91307
24 11439 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-180453-0069.sr-20200126-180453-2498 IBM-MIT-Purdue submission_description/11343|11343 2020-01-26 Purdue 312 0.85478 0.038 100% 100% 0.91614
25 11527 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-212645-2748.sr-20200126-212645-5772 IBM-MIT-Purdue submission_description/11344|11344 2020-01-26 Purdue 312 0.85478 0.035 100% 100% 0.91614
26 6816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-135245-0601.sr-20191220-135245-3689 IBM-MIT-Purdue submission_description/6443|6443 2019-12-17 Purdue 312 0.85625 0.85625 1.072 100% 100% 0.91761 0.91761
27 6812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191219-234702-7711.sr-20191219-234703-6560 IBM-MIT-Purdue submission_description/6501|6501 2019-12-18 Purdue 312 0.85759 0.442 100% 100% 0.92499
28 4732 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191112-144852-1012.sr-20191112-144852-3707 IBM-MIT-Purdue submission_description/4660|4660 2019-11-11 Purdue 312 0.85825 0.087 100% 100% 0.92276
29 10182 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200120-124912-8942.sr-20200120-124913-1637 UCF submission_description/9105|9105 2020-01-15 UCF-P 270 0.85863 0.424 100% 100% 0.86120
30 4659 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191109-113627-1003.sr-20191109-113627-8252 IBM-MIT-Purdue submission_description/4520|4520 2019-11-08 Purdue 312 0.85961 0.087 100% 100% 0.93935
31 10820 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200122-112821-7686.sr-20200122-112822-0383 UCF submission_description/10640|10640 2020-01-21 UCF-P 270 0.87051 0.362 100% 100% 0.87354
32 6902 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-235401-7601.sr-20191220-235402-0107 IBM-MIT-Purdue submission_description/6815|6815 2019-12-20 Purdue 312 0.87326 0.315 100% 100% 0.90380
33 4070 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191028-165446-2748.sr-20191028-165447-0013 IBM-MIT-Purdue submission_description/3830|3830 2019-10-21 Purdue 312 0.87802 0.246 100% 100% 0.95373
34 6855 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191220-175217-7070.sr-20191220-175217-9512 IBM-MIT-Purdue submission_description/6547|6547 2019-12-18 Purdue 312 0.87829 0.099 100% 100% 0.90638
35 4507 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-114135-1903.sr-20191106-114136-0201 IBM-MIT-Purdue submission_description/4228|4228 2019-10-31 Purdue 312 0.88140 0.093 97% 100% 0.94599
36 11947 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200206-111142-9816.sr-20200206-111143-2146 UCF submission_description/11849|11849 2020-02-02 UCF-P 270 0.88172 0.344 100% 100% 0.88404
37 4508 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191106-121322-4750.sr-20191106-121323-4002 IBM-MIT-Purdue submission_description/4179|4179 2019-10-30 Purdue 312 0.88180 0.093 97% 100% 0.95227
38 4955 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191120-020423-5382.sr-20191120-020423-7702 IBM-MIT-Purdue submission_description/4876|4876 2019-11-19 Purdue 312 0.88330 0.088 83% 100% 0.92867
39 12332 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200212-201017-1346.sr-20200212-201017-3748 UCF submission_description/12222|12222 2020-02-11 UCF-P 270 0.88522 0.396 100% 89% 0.88176
40 11342 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-060218-3152.sr-20200126-060218-5498 IBM-MIT-Purdue submission_description/11268|11268 2020-01-26 Purdue 312 0.88815 0.124 100% 100% 0.90887
41 10706 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-013602-6833.sr-20200122-013602-9771 IBM-MIT-Purdue submission_description/10041|10041 2020-01-19 Purdue 312 0.89151 0.143 100% 100% 0.92110
42 11975 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20200206-171924-8432.sr-20200206-171925-1408 INF submission_description/11850|11850 2020-02-03 meva_inf2 288 0.89471 0.960 94% 88% 0.90400
43 9496 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-132317-0783.sr-20200116-132317-3212 IBM-MIT-Purdue submission_description/8290|8290 2020-01-10 Purdue 312 0.89563 0.301 100% 100% 0.93596
44 8275 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200110-091609-1531.sr-20200110-091609-4167 IBM-MIT-Purdue submission_description/8154|8154 2020-01-09 Purdue 312 0.89563 0.380 100% 100% 0.93596
45 10394 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-000324-4735.sr-20200121-000324-7531 IBM-MIT-Purdue submission_description/9978|9978 2020-01-18 Purdue 312 0.89563 0.145 100% 100% 0.93596
46 11819 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200130-120346-5169.sr-20200130-120346-8243 INF submission_description/11670|11670 2020-01-28 INF_MEVA1 283 0.90249 0.928 100% 100% 0.93185
47 11269 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200126-013750-5311.sr-20200126-013750-7710 IBM-MIT-Purdue submission_description/11188|11188 2020-01-25 Purdue 312 0.90481 0.108 100% 100% 0.93280
48 8240 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00316_Edge-Intelligence_20200110-031108-3165.sr-20200110-031108-5673 Edge-Intelligence submission_description/8003|8003 2020-01-08 Edge-Intelligence 316 0.90795 0.939 97% 100% 0.90508
49 8004 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200108-080622-3945.sr-20200108-080622-6125 UCF submission_description/7728|7728 2020-01-04 UCF-P 270 0.92333 0.438 86% 100% 0.92110
50 8007 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200108-084806-2597.sr-20200108-084806-5080 Edge-Intelligence submission_description/7845|7845 2020-01-06 Edge_Intelligence 320 0.92361 0.887 94% 94% 0.92075
51 6813 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191220-054840-0488.sr-20191220-054840-2775 UCF submission_description/6499|6499 2019-12-18 UCF-P 270 0.93293 0.333 91% 82% 0.93052
52 10535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-164242-1566.sr-20200121-164242-4398 IBM-MIT-Purdue submission_description/10037|10037 2020-01-19 Purdue 312 0.94086 0.151 72% 100% 0.96263
53 10762 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200122-045506-2717.sr-20200122-045506-5220 IBM-MIT-Purdue submission_description/10038|10038 2020-01-19 Purdue 312 0.94500 0.144 62% 100% 0.96100
54 7628 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20200103-201849-4904.sr-20200103-201849-7180 UCF submission_description/7235|7235 2019-12-30 UCF-P 270 0.94610 0.347 86% 82% 0.94443
55 3973 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191023-114820-4426.sr-20191023-114820-8834 UCF submission_description/3472|3472 2019-09-26 UCF-P 270 0.95301 0.292 83% 100% 0.97139
56 10251 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-161145-9437.sr-20200120-161146-1778 IBM-MIT-Purdue submission_description/9866|9866 2020-01-17 Purdue 312 0.96021 0.142 70% 100% 0.95921
57 10320 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200120-193353-6571.sr-20200120-193353-8725 IBM-MIT-Purdue submission_description/9873|9873 2020-01-17 Purdue 312 0.96254 0.139 64% 100% 0.96125
58 10970 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200123-211058-0522.sr-20200123-211058-2592 IBM-MIT-Purdue submission_description/10896|10896 2020-01-23 Purdue 312 0.96408 0.143 70% 100% 0.96800
59 11062 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-175059-7650.sr-20200124-175059-9697 IBM-MIT-Purdue submission_description/10972|10972 2020-01-24 Purdue 312 0.96409 0.152 70% 100% 0.96802
60 11121 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200124-221522-0054.sr-20200124-221522-2197 IBM-MIT-Purdue submission_description/10973|10973 2020-01-24 Purdue 312 0.96409 0.151 70% 100% 0.96802
61 9969 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20200118-173929-1848.sr-20200118-173929-4320 INF submission_description/8367|8367 2020-01-11 set3 290 0.96491 0.96491 1.332 62% 100% 0.97584 0.97584
62 10634 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200121-221418-2880.sr-20200121-221418-5157 IBM-MIT-Purdue submission_description/10040|10040 2020-01-19 Purdue 312 0.96604 0.147 64% 100% 0.96953
63 3257 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20190913-094214-5138.sr-20190913-094214-7316 UCF submission_description/3104|3104 2019-09-10 UCF-P 270 0.96630 0.291 78% 100% 0.97125
64 7682 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00320_Edge-Intelligence_20200104-082151-3830.sr-20200104-082151-6002 Edge-Intelligence submission_description/7427|7427 2020-01-02 Edge_Intelligence 320 0.96665 0.736 13% 100% 0.96350
65 9639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-164611-8699.sr-20200116-164612-1412 IBM-MIT-Purdue submission_description/8289|8289 2020-01-10 Purdue 312 0.96736 0.370 59% 100% 0.97474
66 6200 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00315_Jay-Chou_20191211-153836-2972.sr-20191211-153836-5308 Jay Chou submission_description/5904|5904 2019-12-10 Jack 315 0.96884 0.179 51% 100% 0.98214
67 7139 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00290_INF_20191225-070143-9529.sr-20191225-070144-1669 INF submission_description/6967|6967 2019-12-23 set3 290 0.97426 0.911 59% 100% 0.97768
68 4069 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00307_INF_20191028-164751-7980.sr-20191028-164752-1135 INF submission_description/3276|3276 2019-09-13 INF_MEVA_IOD 307 0.97430 0.588 83% 94% 0.97774
69 4804 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191113-152522-8003.sr-20191113-152526-3773 Team_Vision submission_description/4434|4434 2019-11-05 STARK 282 0.97668 0.774 89% 100% 0.97965
70 7547 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200103-081512-4482.sr-20200103-081512-6526 INF submission_description/7212|7212 2019-12-29 INF_MEVA1 283 0.97918 0.922 62% 94% 0.98104
71 7061 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191224-125830-3199.sr-20191224-125830-5590 INF submission_description/6965|6965 2019-12-23 INF_MEVA1 283 0.98156 0.906 64% 100% 0.98402
72 7264 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191231-014617-6841.sr-20191231-014617-8935 INF submission_description/7152|7152 2019-12-26 speed1x 298 0.98244 0.989 64% 82% 0.98438
73 7907 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200107-092828-2753.sr-20200107-092831-6310 Team_Vision submission_description/6865|6865 2019-12-20 STARK 282 0.98773 0.793 86% 83% 0.98900
74 4596 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191108-232219-1501.sr-20191108-232219-3782 UCF submission_description/4510|4510 2019-11-06 UCF-P 270 0.98791 0.293 56% 94% 0.98864
75 10104 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200120-055837-4585.sr-20200120-055837-6726 Team_Vision submission_description/8506|8506 2020-01-12 STARK 282 0.98832 0.739 78% 54% 0.99005
76 4427 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191101-142638-9265.sr-20191101-142639-1652 UCF submission_description/4060|4060 2019-10-25 UCF-P 270 0.99044 0.273 54% 100% 0.99141
77 5999 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191210-174641-6248.sr-20191210-174641-9327 UCF submission_description/5851|5851 2019-12-09 UCF-P 270 0.99055 0.293 75% 100% 0.99031
78 6202 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153859-9306.sr-20191211-153900-1566 WeiLai submission_description/5651|5651 2019-12-07 Tryyitry 314 0.99139 0.178 27% 100% 0.99280
79 3330 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20190919-133858-1097.sr-20190919-133858-3388 DIVA TE Baseline submission_description/1786|1786 2019-08-02 RC3D 276 0.99139 0.959 27% 100% 0.99280
80 6201 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153847-1658.sr-20191211-153847-3737 WeiLai submission_description/5817|5817 2019-12-09 Tryyitry 314 0.99139 0.177 27% 100% 0.99280
81 6203 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153906-9288.sr-20191211-153907-1515 WeiLai submission_description/5576|5576 2019-12-05 Tryyitry 314 0.99139 0.178 27% 100% 0.99280
82 6204 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00314_WeiLai_20191211-153918-0443.sr-20191211-153918-2704 WeiLai submission_description/5452|5452 2019-12-04 Tryyitry 314 0.99139 0.178 27% 100% 0.99280
83 8751 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20200113-110217-3652.sr-20200113-110217-5648 INF submission_description/8273|8273 2020-01-10 INF_MEVA1 283 0.99176 0.99176 1.054 35% 100% 0.99142 0.99142
84 3639 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191010-112752-8152.sr-20191010-112755-5280 Team_Vision submission_description/3557|3557 2019-10-08 STARK 282 0.99258 0.723 45% 100% 0.99295
85 3500 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190930-141448-4379.sr-20190930-141451-2761 Team_Vision submission_description/3382|3382 2019-09-23 STARK 282 0.99268 0.726 45% 100% 0.99345
86 8924 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200114-025702-0518.sr-20200114-025702-2624 IBM-MIT-Purdue submission_description/8278|8278 2020-01-10 Purdue 312 0.99294 0.361 24% 100% 0.99399
87 5378 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191128-201821-9045.sr-20191128-201822-1378 UCF submission_description/5077|5077 2019-11-27 UCF-P 270 0.99302 0.312 40% 94% 0.99278
88 5557 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191205-142328-8277.sr-20191205-142329-2540 INF submission_description/4959|4959 2019-11-21 meva_inf2 288 0.99335 0.99335 1.703 75% 64% 0.99325 0.99325
89 4428 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20191101-144530-2898.sr-20191101-144532-2436 Team_Vision submission_description/4071|4071 2019-10-29 STARK 282 0.99338 0.757 43% 82% 0.99333
90 9843 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200117-024347-1627.sr-20200117-024347-3789 IBM-MIT-Purdue submission_description/8286|8286 2020-01-10 Purdue 312 0.99421 0.357 13% 100% 0.99530
91 9712 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-200717-3029.sr-20200116-200717-5095 IBM-MIT-Purdue submission_description/8288|8288 2020-01-10 Purdue 312 0.99421 0.359 13% 100% 0.99530
92 9786 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200116-232636-3806.sr-20200116-232636-6034 IBM-MIT-Purdue submission_description/8287|8287 2020-01-10 Purdue 312 0.99421 0.361 13% 100% 0.99530
93 8858 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200113-211626-3700.sr-20200113-211626-5742 IBM-MIT-Purdue submission_description/8284|8284 2020-01-10 Purdue 312 0.99421 0.374 13% 94% 0.99530
94 5815 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191208-003354-3071.sr-20191208-003354-5163 UCF submission_description/5596|5596 2019-12-06 UCF-P 270 0.99584 0.304 45% 100% 0.99573
95 5535 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00270_UCF_20191204-142823-9331.sr-20191204-142824-1746 UCF submission_description/5439|5439 2019-12-04 UCF-P 270 0.99636 0.304 43% 100% 0.99626
96 11818 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00276_DIVA-TE-Baseline_20200130-113642-8277.sr-20200130-113643-0211 DIVA TE Baseline submission_description/7911|7911 2020-01-07 RC3D 276 0.99677 0.99777 2.791 27% 94% 0.99694 0.99771
97 11851 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00324_DIVA-TE-Baseline_20200204-101445-5317.sr-20200204-101445-7608 DIVA TE Baseline submission_description/7928|7928 2020-01-07 RC3D-WHEEL 324 0.99740 0.99805 2.715 27% 70% 0.99762 0.99797
98 3083 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20190905-165243-1769.sr-20190905-165244-9598 Team_Vision submission_description/2694|2694 2019-08-27 STARK 282 0.99804 0.99901 1.795 48% 94% 0.99869 0.99969
99 8812 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-172127-0711.sr-20200113-172127-3465 Team_Vision submission_description/8295|8295 2020-01-10 STARK 282 0.99899 0.060 40% 82% 0.99896
100 12350 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200213-084119-5212.sr-20200213-084120-8280 Team_Vision submission_description/12319|12319 2020-02-12 STARK 282 0.99904 0.841 59% 11% 0.99901
101 7336 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00288_INF_20191231-195249-0336.sr-20191231-195249-2332 INF submission_description/7151|7151 2019-12-26 meva_inf2 288 0.99914 0.931 62% 100% 0.99912
102 12283 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200212-075455-9720.sr-20200212-075456-2119 Team_Vision submission_description/12208|12208 2020-02-11 STARK 282 0.99935 0.783 40% 18% 0.99932
103 4068 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00298_INF_20191028-164450-6506.sr-20191028-164451-1116 INF submission_description/3163|3163 2019-09-12 speed1x 298 1.00000 0.860 48% 100% 1.00000
104 8822 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-174128-0858.sr-20200113-174128-2929 Team_Vision submission_description/8313|8313 2020-01-10 STARK 282 1.00000 0.049 0% 82% 1.00000
105 4808 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20191114-140029-1145.sr-20191114-140029-3974 INF submission_description/4513|4513 2019-11-08 INF_MEVA1 283 1.00000 0.039 0% 100% 1.00000
106 5446 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191204-022616-0789.sr-20191204-022616-3057 IBM-MIT-Purdue submission_description/5384|5384 2019-12-02 Purdue 312 1.00000 0.039 0% 0% 1.00000
107 3055 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190903-152553-6173.sr-20190903-152553-9672 INF submission_description/2928|2928 2019-08-29 INF_MEVA1 283 1.00000 0.788 54% 100% 1.00000
108 5737 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191207-123158-7735.sr-20191207-123158-9723 IBM-MIT-Purdue submission_description/5506|5506 2019-12-04 Purdue 312 1.00000 0.039 0% 0% 1.00000
109 5997 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191210-114411-9709.sr-20191210-114412-2408 IBM-MIT-Purdue submission_description/5818|5818 2019-12-09 Purdue 312 1.00000 0.037 0% 0% 1.00000
110 6282 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111731-1693.sr-20191212-111731-3834 IBM-MIT-Purdue submission_description/6056|6056 2019-12-11 Purdue 312 1.00000 0.037 0% 0% 1.00000
111 11721 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20200128-125907-5613.sr-20200128-125907-7721 IBM-MIT-Purdue submission_description/11642|11642 2020-01-27 Purdue 312 1.00000 0.016 0% 0% 1.00000
112 6280 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-104321-2720.sr-20191212-104321-5060 IBM-MIT-Purdue submission_description/6000|6000 2019-12-10 Purdue 312 1.00000 0.037 0% 0% 1.00000
113 2892 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00283_INF_20190828-173003-1496.sr-20190828-173003-3976 INF submission_description/2682|2682 2019-08-27 INF_MEVA1 283 1.00000 0.000 0% 100% 1.00000
114 6281 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00312_IBM-MIT-Purdue_20191212-111546-3723.sr-20191212-111546-5775 IBM-MIT-Purdue submission_description/6055|6055 2019-12-11 Purdue 312 1.00000 0.037 0% 0% 1.00000
115 8816 ActEV-2018_AD_ActEV19-SDL-Scoring_SYS-00282_Team-Vision_20200113-173052-2474.sr-20200113-173052-4549 Team_Vision submission_description/8299|8299 2020-01-10 STARK 282 1.00000 0.061 29% 76% 1.00000

SDL19-scoring-IR

Updated: 2020-02-22 03:08:38 -0500

*Partial AUDC is the area under the DET curve between a Time-based False Alarm rate of 0 and 0.2. Value of a perfect system is 0.

†The fraction of activity types with detected instances in the processed data set.

‡The fraction of videos processed and scored.

Algorithm Delivery for the DIVA teams

System delivery to the leaderboard must be in a form compatible with the ActEV Command Line Interface (ActEV CLI) and submitted to NIST for testing. The command line interface implementation that you will provide formalizes the entire process of evaluating a system, by providing the evaluation team a means to: (1) download and install your software via a single URL, (2) verify that the delivery works AND produces output that is “consistent” with output you produce, and (3) process a large collection of video in a fault-tolerant, parallelizable manner.

To complete this task you will need the following items described in detail below:

  1. FAQ - Validation Phase Processing
  2. CLI Description
  3. The Abstract CLI Git Repository
  4. The CLI Implementation Primer
  5. The Validation Data Set
  6. Example CLI-Compliant Implementation
  7. NIST Hardware and Initial Operating System Description

FAQ - Validation Phase Processing

The ActEV SDL - Validation Phase Processing FAQ

CLI Description

The ActEV CLI description

The Abstract CLI GIT Repository

The Abstract CLI GIT repository. The repo contains documentation.

The CLI Implementation Primer

There are 6 steps to adapt your code to the CLI. The ActEV Evaluation CLI Programming Primer describes the steps to clone the Abstract CLI and begin adapting the code to your implementation.

The Validation Data Set

As mentioned above, the CLI is used to verify the downloaded software is correctly installed and produces the same output at NIST as you produce locally. In order to do so, we have provided a small validation data set (ActEV-Eval-CLI-Validation-Set3) as part of the ActEV SDL Dataset that will be processed both at your site and at NIST. Please use this data in Step 3 of the Primer described above.

Example CLI-Compliant Implementation

The link below provides an example CLI implementation for the leaderboard baseline algorithm:

NIST Independent Evaluation Infrastructure Specification

NIST will begin installing your system from a fresh Ubuntu 18.04 cloud image available from https://cloud-images.ubuntu.com/releases/18.04/release/ on the following hardware.

  • Chassis: Asus ESC4000 G4S
  • CPU: 2 x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (12 cores/CPU)
  • Motherboard: Asus Intel® C621 PCH chipset
  • HDD/SSD: 2x 1.92GB Intel SSD DC S4500
  • RAM: 12x 16GB DDR4-2400 ECC RDIMM
  • GPU: Four PNY RTX2080Ti blower style
  • OS: Ubuntu 18.04
  • Storage Volume- 1TB (variable)
  • Supplied object store (read only) for source video


Contact Us

For ActEV Evaluation information (data, evaluation code, etc.) please email: actev-nist@nist.gov

For ActEV Evaluation Discussion Please Visit our Google Group. https://groups.google.com/a/list.nist.gov/forum/#!forum/trecvid.actev

Activities for the ActEV Sequestered Data Leaderboard

The list below specifies the activities to be detected for the ActEV SDL evaluation. Detailed activity definitions are in the ActEV Annotation Definitions for MEVA Data document. Note that current scoring for the SDL is based exclusively on activity detection, and object detection is not considered.

ActEV SDL Activities


person_closes_facility_door
person_closes_vehicle_door
Closing_Trunk
person_enters_through_structure
person_enters_vehicle
person_exits_through_structure
person_exits_vehicle
person_loads_vehicle
Open_Trunk
person_opens_facility_door
person_opens_vehicle_door
Transport_HeavyCarry
person_unloads_vehicle
vehicle_turning_left
vehicle_turning_right
vehicle_u_turn
Riding
Talking
specialized_talking_phone
specialized_texting_phone
person_sitting_down
person_standing_up
person_reading_document
object_transfer
person_picks_up_object
person_sets_down_object
hand_interaction
person_person_embrace
person_purchasing
person_laptop_interaction
vehicle_stopping
vehicle_starting
vehicle_reversing
vehicle_picks_up_person
vehicle_drops_off_person
abandon_package
theft
Task for the ActEV Sequestered Data Leaderboard
In the SDL evaluation, there is one Activity Detection (AD) task for detecting and temporally localizing activities

Activity Detection (AD)
For the Activity Detection task, given a target activity, a system automatically detects and temporally localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the type of activity must be correct and temporally overlap the true activity for at least one second. Additional details may be found in the SDL Evaluation Plan .
ActEV Sequestered Data Leaderboard (SDL) Evaluation

Updates
  • 12/17/19: The ActEV Scorer has been updated (eval. plan: pp. 14-15)
  • 12/17/19: The updated ActEV scorer will be deployed after the Jan 10th, 2020 submissions are scored
  • 10/15/19: The top two teams on the SDL leaderboard will be invited to give oral presentations at the HADCV'20 workshop based on the CLI submission deadline of Jan 10th, 2020 at 12:00 noon EST
Summary
The ActEV (Activities in Extended Video) Sequestered Data Leaderboard is an ongoing ranking of software systems that watch lengthy videos and detect activities of interest. Anyone can submit their system to NIST, which will then run the system on sequestered data, score the results and post the score to the leaderboard. The sequestered data is from the MEVA dataset, which contains hours of videos, including indoor and outdoor scenes, night and day, crowds and individuals, and videos are from both EO (Electro-Optical) and IR (Infrared) sensors. Hours can go by with no activities, but then multiple activities happen simultaneously. The data is multi-camera, in that multiple cameras may be pointed at the same scene at the same time. There are also two separate leaderboards for EO and IR videos.
What
Build and submit a software system to watch videos and detect if and when an activity of interest occurs. System runtime must be 1 times the video length on the designated evaluation hardware.
Who
Everyone. Anyone who registers can submit to the evaluation server.
How
Register here and then follow the instructions on the Algorithm Submission tab above. Systems must follow a NIST-defined command line interface and automatically run on NIST’s servers, both of which are described in the instructions.
Evaluation Task
Detect if and when an activity occurs. Given a target activity type and a set of videos, submitted systems must automatically detect all instances of the activity in the videos. While different activities instances have different durations, a submitted system will be considered to have detected an activity if it correctly identifies at least 1 second of the activity.
Data
The data is from the Multiview Extended Video with Activities (MEVA) dataset and the videos are from both EO (Electro-Optical) and IR (Infrared) sensors. The data used for evaluation is sequestered, but the public MEVA dataset includes hundreds of hours of data from the same cameras at the same facility, which can be used for training. If you register for ActEV you can download the MEVA dataset for free; info on how to download the data is on the data tab. ActEV does not provide annotations, but instructions on how to make and share activity annotations are at mevadata.org.
Metrics
Submitted activity detection systems must give a confidence score for each activity they detect. Detected activities are then thresholded based on this confidence score. Varying the threshold makes a trade-off between being sensitive enough to identify true activity instances (low threshold) vs. not making false alarms when no activity is present (high threshold). Submitted systems are scored on both of these, measured by Probability of Missed Detection (Pmiss) and Time-based False Alarm (TFA). Pmiss is the portion of activities where the system did not detect the activity for at least 1 second. TFA is the portion of time that the system detected an activity when in fact there was none. Submitted systems are scored for Pmiss and TFA at multiple thresholds, creating a detect error tradeoff (DET) curve. The leaderboard ranking of a system is based on a summary of its DET curve: the average value of 1-Pmiss across the TFA range between 0% to 20%. Higher numbers are better, as they reflect fewer errors. See details in the SDL evaluation plan or check out the ActEV Scoring Software GitHub repo.
Evaluation Plan
Task coordinator
ActEV NIST team (ActEV-nist@nist.gov)
News
13June
ActEV SDL starts

ActEV SDL Evaluation Schedule
June 13, 2019: ActEV SDL opens

August 15, 2019 11 AM EST : Deadline for submissions to be included in ActEV SDL Evaluation Standings that will be reported at TRECVID 2019 workshop (November 12-13, 2019)

Leaderboard Remains Open: Leaderboard will continue to remain open to allow participants to show continued progress on this challenging problem

January 10th, 2020 at 12:00 noon EST: We will invite the top two teams on the SDL leaderboard to give oral presentations at the WACV HADCV'20 workshop based on the CLI submission deadline


Datasets

Framework

The DIVA Framework is a software framework designed to provide an architecture and a set of software modules which will facilitate the development of activity recognition analytics. The Framework is developed as a fully open source project on GitHub. The following links will help you get started with the framework: The DIVA Framework is based on KWIVER, an open source framework designed for building complex computer vision systems. The following links will help you learn more about KWIVER:
  • KWIVER Github Repository This is the main KWIVER site, all development of the framework happens here.
  • KWIVER Issue Tracker Submit any bug reports or feature requests for the KWIVER here. If there's any question about whether your issues belongs in the KWIVER or DIVA framework issues tracker, submit to the DIVA tracker and we'll sort it out..
  • KWIVER Main Documentation PageThe source for the KWIVER documentation is maintained in the Github repository using Sphinx. A built version is maintained on ReadTheDocs at this link. A good place to get started in the documentation, after reading the Introduction are the Arrows and Sprokit sections, both of which are used by the KWIVER framework.

Baseline Algorithms

KITWARE has adapted two "baseline" activity recognition algorithms to work within the DIVA Framework:

Annotation Tools

ActEV SDL Dataset

The ActEV Data GIT Repo is the data distribution mechanism for the ActEV evaluations. The repo presently consists of a collection of corpora and partition definition files to be used for evaluations. Future additions will include source annotations and community provided data/annotations. The repo contains textual data but not the large-sized corpora (videos, etc.). For these large corpora, the repo contains the necessary information to download a given corpus giving the user control of the downloaded content. Access credentials are provided during ActEV registration.

To sign up, simply create a login account by registering for ActEV SDL, and you will receive credentials for the video data download. If there is any issue please email us at actev-nist@nist.gov



If you have any question, please email to actev-nist@nist.gov