ActEV: Activities in Extended Video
Datasets
- ActEV-supported data sets
-
Multiview Extended Video with Activities (MEVA) See
the License and the README for
context.
- NEW: Annotations from the T&E team are available with additional annotations posted weekly.
- VIRAT
- Data access from mevadata.org: Accessing and using MEVA and MEVA Download Instructions
- Data access from git and NIST : See actev-data-repo. Access credentials provided during signup
-
Multiview Extended Video with Activities (MEVA) See
the License and the README for
context.
- Kinetics
- AVA
- Moments-in-Time
- ActivityNet
- NVIDIA's CityFlow dataset
- Live Datasets for Visual AI (Visym
Collector)
- People in Public - 175k: 184,379 video clips of the ActEV activity classes for training in the unknown facility use-case.
Framework
The DIVA Framework is a software framework designed to provide an architecture and a set of software modules which will facilitate the development of activity recognition analytics. The Framework is developed as a fully open source project on GitHub. The following links will help you get started with the framework:- DIVA Framework Github Repository This is the main DIVA Framework site, all development of the framework happens here.
- DIVA Framework Issue Tracker Submit any bug reports or feature requests for the framework here.
- DIVA Framework Main Documentation PageThe source for the framework documentation is maintained in the Github repository using Sphinx. A built version is maintained on ReadTheDocs at this link. A good place to get started in the documentation, after reading the Introduction is the UseCase section which will walk you though a number of typical use cases with the framework.
- KWIVER Github Repository This is the main KWIVER site, all development of the framework happens here.
- KWIVER Issue Tracker Submit any bug reports or feature requests for the KWIVER here. If there's any question about whether your issues belongs in the KWIVER or DIVA framework issues tracker, submit to the DIVA tracker and we'll sort it out..
- KWIVER Main Documentation PageThe source for the KWIVER documentation is maintained in the Github repository using Sphinx. A built version is maintained on ReadTheDocs at this link. A good place to get started in the documentation, after reading the Introduction are the Arrows and Sprokit sections, both of which are used by the KWIVER framework.
Baseline Algorithms
KITWARE has adapted two "baseline" activity recognition algorithms to work within the DIVA Framework:Visualization Tools
Annotation Tools
- Kitware annotation tool (the tool natively supports the DIVA format)
- The VGG Image Annotator
- Scalabel (used for annotation of Berkeley DeepDrive project)
- VATIC - Video Annotation Tool
- BeaverDam
- VoTT: Visual Object Tagging Tool
- Computer Vision Annotation Tool (CVAT)
- Efficient Annotation of Segmentation Datasets with Polygon-RNN++
For information on data, evaluation code, etc., please email: actev-nist@nist.gov
For ActEV evaluation discussion, please visit our Google Group: https://groups.google.com/a/list.nist.gov/forum/#!forum/trecvid.actev
An ActEV
- Activity Name - A mnemonic handle for the activity
- Activity Description - Textual description of the activity
- Begin time rule definition - The specification of what determines the beginning time of the activity
- End time rule definition - The specification of what determines the ending time of the activity
- Required object type list - The list of objects systems are expected to identify for the activity. Note: this aspect of an activity not addressed by ActEV-PC.
For example:
- Description: A person closing the door to a vehicle.
- Start: The event begins 1 s before the door starts to move.
- End: The event ends after the door stops moving. People in cars who close the car door from within is a closing event if you can still see the person within the car. If the person is not visible once they are in the car, then the closing should not be annotated as an event.
- Objects associated with the activity : Person; and Door or Vehicle
- Description: A vehicle turning left or right is determined from the POV of the driver of the vehicle. The vehicle may not stop for more than 10 s during the turn.
- Start: Annotation begins 1 s before vehicle has noticeably changed direction.
- End: Annotation ends 1 s after the vehicle is no longer changing direction and linear motion has resumed. Note: This event is determined after a reasonable interpretation of the video.
- Objects associated with the activity : Vehicle
- Description: An object moving from person to vehicle.
- Start: The event begins 2 s before the cargo to be loaded is extended toward the vehicle (i.e., before a person’s posture changes from one of “carrying” to one of “loading”).
- End: The event ends after the cargo is placed into the vehicle and the person-cargo contact is lost. In the event of occlusion, it ends when the loss of contact is visible.
- Objects associated with the activity: Person; and Vehicle
The names of the 37 Known Activities for ActEV’21 SDL
person_abandons_package | person_loads_vehicle | person_stands_up |
person_closes_facility_door | person_transfers_object | person_talks_on_phone |
person_closes_trunk | person_opens_facility_door | person_texts_on_phone |
person_closes_vehicle_door | person_opens_trunk | person_steals_object |
person_embraces_person | person_opens_vehicle_door | person_unloads_vehicle |
person_enters_scene_through_structure | person_talks_to_person | vehicle_drops_off_person |
person_enters_vehicle | person_picks_up_object | vehicle_picks_up_person |
person_exits_scene_through_structure | person_purchases | vehicle_reverses |
person_exits_vehicle | person_reads_document | vehicle_starts |
hand_interacts_with_person | person_rides_bicycle | vehicle_stops |
person_carries_heavy_object | person_puts_down_object | vehicle_turns_left |
person_interacts_with_laptop | person_sits_down | vehicle_turns_right |
vehicle_makes_u_turn |
- TRECVID'23 ActEV SRL Challenge starts from June 01, 2023.
- TRECVID'23 ActEV SRL Challenge results submission deadline : October 02, 2023: 4:00 PM EST
- Primary Metric is Activity and Object Detection (AOD) and is based on Pmiss@0.1RFA.
- The TRECVID'23 ActEV SRL test dataset is the same as for CVPR'22 ActivityNet challenge and the TRECVID'22 ActEV evaluation
NIST maintains the ActEV Scoring Software on the Scoring software for the Activities in Extended Video (ActEV) evaluation GitHub repo.
Location | View 1 | View 2 |
---|---|---|
Indoor | ||
Outdoor |
For the Activity Detection task, given a target activity, a system automatically detects and temporally localizes all instances of the activity. For a system-identified activity instance to be evaluated as correct, the type of activity must be correct and the temporal overlap must fall within a minimal requirement.
For the Activity and Object Detection task, given a target activity, a system detects and temporally localizes all instances of the activity and spatially detects/localizes the people and/or objects associated with the target activity. For a system-identified instance to be scored as correct, it must meet the temporal overlap criteria for the AD task and in addition meet the spatial overlap of the identified objects during the activity instance.
For the Activity Object Detection and Tracking task, given a target activity, a system detects and temporally localizes all instances of the activity, spatio-temporally detects/localizes the people and/or objects associated with the target activity, and properly assigns IDs the objects play in the activity. For a system-identified instance to be scored as correct, it must meet the temporal overlap criteria and spatio-temporal overlap of the objects for the AOD task and correctly assign the IDs to the objects as described in the activity definition.