Introduction

This article describes the various components of Vision Pack and how they are packaged into various configurations.  Each component is packaged stand-alone as .tar.gz archives and also as Docker runtime containers.

The overarching goal of Vision Pack is to provide an optimized vision pipeline to get video from a camera to the inference engine and visualizing the results.  The libraries offer low-level C API as well as high-level GStreamer plug-ins which provides an API for numerous languages, including C.

Components

The core components of Vision Pack begin with Au-Zone's highly optimized Deep View RT inference engine providing best in class inference performance and support of the latest state of the art models.  Vision Pack extends the standard Deep View RT engine through VAAL, the Vision Acceleration and Abstraction Library, to enable platform optimized vision processing and visualization.  Finally, VSL, the Video Stream Library, provides cross-process and cross-container sharing of video frames while maintaining full DMA support.  The VSL plug-ins also allow GStreamer pipelines to be distributed across processes and containers.  These components enables developers to improve the modularity of their vision pipelines while maintain the best possible performance.

Programming API

Native C

The Vision Pack components are all provided with low-level C API with consistent and stable ABI interfaces allowing easy integration into existing applications and well established methods of interfacing into alternative programming languages.  Just about every programming language offers a well documented method of calling into C.

GStreamer

The GStreamer plugins provides a rich multimedia framework and Vision Pack leverages this to provide integration of image and video data into the Vision ML pipeline.  The GStreamer plugins extend the various Vision Pack API with high-level GObject interfaces allowing interoperability to various programming languages. Details of GStreamer and GObject introspection for cross-language API are well documented in the links below.

• GStreamer Application Development Manual
• GObject Introspection
• GObject API Reference