A snapshot of video search

Automated Video Analysis for Military Operations (AVAMO)

STTR Award: W31P4Q-08-C-0464, Program Officer: M. Desai

AVAMO is an advanced system for automated analysis of video data collected from a variety of platforms. Avamo is an end-to-end system, from extraction of relevant information from low-resolution full motion video, to visual search, abnormality detection, and activity recognition.

Objective

The goal of the STTR effort is to automate analysis of video.

  • Significantly progress state of art in automated video analysis
  • Incorporate research into working systems
  • Ease burden on video analysts
  • Increase useful information extracted from video

Approach

  • Two-pronged approach to activity descriptors
    • Trajectory based
    • Spatiotemporal descriptors, don’t require tracking
  • Build upon descriptors to find similar events (search), discover abnormal events, and build models for complex activities.

The goal of Phase I effort

  1. Build framework for extracting information from video
  2. Develop methods for search by user defined example
  3. Abnormality detection
  4. Begin pre-determined activity detection
Schemetic diagram of avamo phase 1 effort

Current progress in Phase I effort

We developed trajectory and spatiotemporal descriptors, improved general tracking, and performed recognition tests for set of queries.

We successfully tested advanced proprietary video stabilization to perform tracking from moving platform video.

  • Essentially reduces moving camera problem to fixed camera.
  • Tracking output is in relation to fixed ground plane, not frame boundaries.
  • Had planned for future development, using results today.

We performed activity search for both track based and local activity activities.

Screenshot of activity search
Screenshot of activity search

Relationship with other work at Mayachitra

We have much related work in automated image and video analysis , visual indexing/search, image registration, and video stabilization:

  • Query refinement (DARPA SBIR Phase I)
  • Indexing and search of large scale video descriptor databases (VIRAT)
  • Image classification, labeling, scene understanding (ONR Grant, SBIR Phase II)
  • Advanced methods for visual search of large databases of geospatial imagery (Geo*)
  • State of the art solutions for image/video registration, mosaicking, blending and stabilization (Geo*)
  • Mayachitra imago, microscopic image management and analysis (NIH STTR Phase II)