Virtual Video C Library Function Reference

This page describes the use of the Virtual Video C Library to communicate with the Camera Server and PTZ Server in the Virtual Video mod. Before reading the following documentation, users should review the architecture of the Virtual Video Tool described here, and may also wish to view the documentation for the underlying network protocol for a better understanding of how the library is implemented.

The C Library is based on the Windows Sockets 2 API, and is distributed as a single static lib file VVClient.lib and header file VVClient.h. To use the library, your application should specify #include "VVClient.h" and link to both VVClient.lib and ws2_32.lib (the Windows Sockets 2 static library). The Virtual Video C library can be downloaded here. Included in this package is the source code (in the samples directory) for a simple Camera/PTZ client that you can use as a basis for building your own clients.

The C Library functions are divided into three major groups: initialization and uninitialization of the library (functions prefixed with VV), communication with the Camera Server (functions prefixed with Cam) and communication with the PTZ Server (functions prefixed with PTZ). Each client connection to the Camera Server is represented by a VVCamera structure. All Cam* functions take a pointer to the VVCamera structure representing the camera to which the command is applied. Similarly, each client connection to the PTZ Server is represented by a VVPTZStatus structure. All PTZ* functions take a pointer to the VVPTZStatus structure representing the controller to which the command is applied.

To implement an actively controlled PTZ camera, call CamSetType on a connected camera, passing VV_TRUE to the ptzControl parameter and a unique integer identifier to the ptzID parameter. Then, call PTZSetID on a controller client, passing the same identifier in the ptzID parameter. Subsequent motion commands sent by the PTZ client will be applied to the camera with the matching identifier. Note that applications involving static cameras are not required to call any of the PTZ* library functions.

A minimal client application would call the following sequence of functions:

  • Initialize the Windows Sockets 2 library with VVInitialize.
  • Instantiate a new VVCamera structure and connect to the default camera (camID = 0) on the Camera Server by calling CamConnect. This function also fills out the VVCamera structure with the default camera settings retrieved from the server. If implementing active control, instantiate a new VVPTZStatus structure and connect to the default controller (ptzID = 0) on the PTZ Server by calling PTZConnect.
  • Create a new camera by calling CamSetID and passing a unique non-zero identifier to the camID parameter. CamSetID fills out the VVCamera structure with the new camera settings retrieved from the server. Note that this step is not necessary if you only intend to use one camera, ie. the default camera (camID = 0).
  • Change the appropriate default settings by calling CamSetType, CamSetIntrinsicParameters and CamSetExtrinsicParameters, passing the VVCamera structure instantiated above. If active control is required, the call to CamSetType should specify VV_TRUE for ptzControl and a ptzID corresponding to an existing controller (eg. the default controller, ptzID = 0).
  • In the main processing loop, retrieve new frames by calling CamGetLatestFrame, and send PTZ commands to the camera by calling VVPTZSetStatus. The latest frame can be read from the image field of the VVCamera structure.
  • Before exiting the application, disconnect from the servers by calling CamDisconnect and PTZDisconnect, passing the VVCamera and VVPTZStatus structures instantiated above.
  • Uninitialize the Windows Sockets 2 library with VVUninitialize.

The following C code excerpt implements the above minimal behaviour:

VVCamera camera;
VVPTZStatus ptzStatus;
const VVCAMID camID = 1;
const VVPTZID ptzID = 0;

VVInitialize();
CamConnect(&camera, "hostname");
PTZConnect(&ptzStatus, "hostname");

CamSetID(&camera, camID);
CamSetType(&camera, VV_STDCAM, VV_TRUE, ptzID, NULL);
CamSetIntrinsicParameters(&camera, ...);
CamSetExtrinsicParameters(&camera, ...);

while (...)
{
   CamGetLatestFrame(&camera, VV_TRUE);
   ... do some processing on camera.image
   PTZSetStatus(&ptzStatus, ...);
}

PTZDisconnect(&ptzStatus);
CamDisconnect(&camera);
VVUninitialize();

The following sections detail the structures and functions implemented by the C Library.

Socket Wrapper Functions:

Camera Server Structures and Functions:

PTZ Server Structures and Functions:

Socket Wrapper Functions
 
VVInitialize/VVUninitialize
VVINFO VVInitialize(void);
VVINFO VVUninitialize(void);
 
Description:
VVInitialize initializes the Windows Sockets 2 library and must be called once per application before any other library functions. VVUninitialize closes the Windows Sockets 2 library and must be called after all cameras have disconnected and no further library calls are made. VVInitialize is a wrapper for WSAStartup and VVUninitialize is a wrapper for WSACleanup (see Windows Sockets 2 documentation).

Return Value:
Both functions return VV_OK on a successful connection/disconnection. A return value of VV_SOCKERR indicates that a Windows Sockets error has occurred.
Camera Server Structures and Functions
 
VVCamera Structure
typedef struct vv_camera_t
{

SOCKET socket;
VV_BOOL connected;

VVCAMID camID;
VVCAMTYPE type;
VVBOOL ptzControl;
VVPTZID ptzID;
char saveDir[VVMAXPATH];

int width, height;
int framerate;
float FOV;

float x, y, z;
float rx, ry, rz;

int frameNum;
float elapsedTime;
VVBOOL compressed;
int imagesz;
unsigned char *image;

float gtHorizFOV;
float gtX, gtY, gtZ;
float gtRx, gtRy, gtRz;
int gtNumTargets;
VVGtTargetPtr gtTargets;
int gtLabelMapSz;
unsigned int *gtLabelMap;
} VVCamera;
 
Parameters:
socket
Socket handle for connection to Camera Server
connected
Read-only flag indicating whether camera is connected to server
camID
Unique camera identifier
type
Camera type, valid values are VV_STDCAM or VV_OMNICAM
ptzControl
Read/write flag indicating whether PTZ control is enabled
ptzID
Identifies the camera to a PTZ controller
saveDir
Null-terminated string specifying path on server to save rendered frames.
width, height
Frame dimensions in pixels
framerate
Frame rate in frames/sec
FOV
Horizontal field of view in degrees at unity zoom
x, y, z
Commanded camera translation in inches relative to inital player position
rx, ry, rz
Commanded camera orientation represented as Euler angles in degrees
frameNum
Current frame number, increases by 1 per rendered frame
elapsedTime
Time this frame was rendered in seconds since game started
compressed
Flag indicating whether data in image buffer is JPEG compressed or 24-bit RGB
imagesz
Length in bytes of data in image buffer
image
Latest rendered image
gtHorizFOV
Current (ground truth) horizontal field of view (function of zoom)
gtX, gtY, gtZ
Current camera translation inches relative to the map origin
gtRx, gtRy, gtRz
Current camera orientation as Euler angles in degrees
gtNumTargets
Number of targets with ground truth (requires ground truth enabled)
gtTargets
Array of VVGtTarget structures providing ground truth for each visible target
gtLabelMapSz
Number of elements in the gtLabelMap array
gtLabelMap
Ground truth foreground label map

Description:
Each simulated camera is associated with a VVCamera structure, which stores connection information and the camera state. A pointer to the VVCamera structure is passed as the first parameter in all function calls that communicate with the Camera Server about the associated camera. All CamGet* functions (including CamGetLatestFrame) communicate the camera state returned from the server through this structure. Note that the memory for the image buffer is allocated internally and released on the call to CamDisconnect.
 
VVGtCent/VVGtBbox/VVGtTarget Structure
typedef struct vv_gt_cent_t
{

float x, y, z;
} VVGtCent;
typedef struct vv_gt_bbox_t
{

unsigned int top, bottom, left, right;
unsigned int pix_count;
} VVGtBbox;
typedef struct vv_target_t
{

unsigned int label;
VVGtCent centWorld;
VVGtBbox occBbox;
VVGtBbox fullBbox;
} VVGtTarget;
 
Parameters:
x, y, z
3D coordinates of the target centroid in inches relative to map origin
top, bottom, left, right
Coordinates of target bounding box edges in pixels from top-left of frame
pix_count
Number of foreground target pixels in the bounding box
label
Unique target label to identify target pixels in foreground label map
centWorld
3D centroid of target in world frame
occBbox
Bounding box around visible target pixels in frame
fullBbox
Bounding box around all target pixels (visible or not) in frame

Description:
The VVGtCent, VVGtBbox and VVGtTarget structures together describe the ground truth data for a foreground target. Ground truth for all targets in the latest frame are stored in the gtTargets array in the VVCamera structure. Note that ground truth generation must be enabled for this array to contain valid data (see the groundtruth console command). Further information on the generation and use of ground truth can be found on the Ground Truth page.
 
CamConnect/CamDisconnect
VVINFO CamConnect(
VVCameraPtr pCamera,
char *hostname
);
VVINFO CamDisconnect(
VVCameraPtr pCamera
);
 
Parameters:
pCamera
Pointer to VVCamera structure
hostname
Null-terminated string specifying hostname of PC running game server

Description:
CamConnect establishes a new connection to the camera server running on hostname and fills the VVCamera structure pointed to by pCamera with the default camera settings. Future operations on this camera should pass this structure to the pCamera parameter of the appropriate Cam* function call. CamDisconnect disconnects the camera associated with pCamera from the camera server. All camera settings are lost after disconnection and must be set again if the camera is reconnected. Applications can determine whether a camera is currently connected to the server by inspecting the connected field of the associated VVCamera structure.

Return Value:
Both functions return VV_OK on a successful connection/disconnection. A return value of VV_SOCKERR indicates that a Windows Sockets error has occurred and the operation could not be completed.
 
CamGetID/CamSetID
VVINFO CamGetID(
VVCameraPtr pCamera
);
VVINFO CamSetID(
VVCameraPtr pCamera,
VVCAMID camID
);
 
Parameters:
pCamera
Pointer to VVCamera structure for a connected camera
camID
Unique camera identifier

Description:
CamGetID fills pCamera->camID with the identifier of the current camera. CamSetID connects the client to the camera identified by camID. If a camera with the corresponding ID already exists, the client is connected to that camera without modifying any existing camera settings. If a camera with the corresponding ID does not exist, a new camera is created with the requested ID and default camera settings. The VVCamera structure pointed to by pCamera is updated with the settings for this camera.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
 
CamGetType/CamSetType
VVINFO CamGetType(
VVCameraPtr pCamera
);
VVINFO CamSetType(
VVCameraPtr pCamera,
VVCAMTYPE type,
VVBOOL ptzControl,
VVPTZID ptzID,
char *saveDir
);
 
Parameters:
pCamera
Pointer to VVCamera structure for a connected camera
type
Type of camera (see comments)
ptzControl
Flag indicating whether camera responds to PTZ commands
ptzID
Identifier linking camera to a PTZ controller
saveDir
Path where rendered frames are saved to disk on the server.

Description:
CamGetType fills pCamera with the current camera settings returned by the Camera Server for the connection represented by pCamera. CamSetType sends the specified camera type settings to the Camera Server for this connection. The VVCamera structure pointed to by pCamera is also updated with the latest settings. The currently supported values for type are:
  • VV_STDCAM: pin-hole projective camera
  • VV_OMNICAM: spherically projected omnicam with 360 horizontal and +/-70 degree vertical FOV
Setting ptzControl to VV_TRUE and passing a unique value to ptzID indicates that the camera will accept PTZ commands with a matching ptzID from the PTZ Server (see PTZSetStatus). PTZ control is disabled if ptzControl is VV_FALSE or type is VV_OMNICAM. If saveDir is not NULL, all rendered frames (including those not retrieved by the client) are written to disk by the server at the path given by saveDir. The maximum length of the path is 100 (MAXPATH) characters, and the server must have write access to the directory. Frames are named in the format clientCCC_frmNNNNNN.jpg where CCC is the client number (increases for each new connection) and NNNNNN is the frame number. If saveDir is NULL, frames are not saved to disk. Note: saving frames to disk introduces a small but noticable performance hit on the game engine.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid. CamSetType returns VV_ERROR if the saveDir string exceeds 100 (MAXPATH) characters.
 
CamGetIntrinsicParameters/CamSetIntrinsicParameters
VVINFO CamGetIntrinsicParameters(
VVCameraPtr pCamera
);
VVINFO CamSetIntrinsicParameters(
VVCameraPtr pCamera,
int width,
int height,
int framerate,
float FOV
);
 
Parameters:
pCamera
Pointer to VVCamera structure for a connected camera
width, height
Dimensions of frame in pixels
framerate
Maximum framerate in frames/sec
FOV
Horizontal field of view in degrees

Description:
CamSetIntrinsicParameters sends the specified intrinisic camera parameters to the Camera Server for the connection represented by pCamera. The VVCamera structure pointed to by pCamera is also updated with these latest settings. CamGetIntrinsicParameters fills pCamera with the current intrinsic parameters read from the Camera Server for this connection.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
 
CamGetExtrinsicParameters/CamSetExtrinsicParameters
VVINFO CamGetExtrinsicParameters(
VVCameraPtr pCamera
);
VVINFO CamSetExtrinsicParameters(
VVCameraPtr pCamera,
float x,
float y,
float z,
float rx,
float ry,
float rz
);
 
Parameters:
pCamera
Pointer to VVCamera structure for a connected camera
x, y, z
Position of camera in inches relative to initial player position
rx, ry, rz
Orientation of camera as Euler angles in degrees

Description:
CamSetExtrinsicParameters sends the specified extrinisic camera parameters to the Camera Server for the connection represented by pCamera. The VVCamera structure pointed to by pCamera is also updated with these latest settings. CamGetIntrinsicParameters fills pCamera with the current extrinsic parameters read from the Camera Server for this connection. Note that position coordinates are specified in inches, which are the same units used in the map.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
 
CamGetLatestFrame
VVINFO CamGetLatestFrame(
VVCameraPtr pCamera,
VVBOOL decompress
);
 
Parameters:
pCamera
Pointer to VVCamera structure for a connected camera
decompress
Flag indicating whether JPEG/label map data should be return raw or decompressed

Description:
This function reads the latest rendered frame and target ground truth from the Camera Server for the connection represented by pCamera. The rendered frame is copied to the image buffer of the VVCamera structure pointed to by pCamera, and the frameNum and elapsedTime fields are updated with the current frame information. The camera settings for the captured frame are copied to the gt* fields of pCamera. If ground truth generation is enabled (see the groundtruth console command) the target ground truth is copied to the gtTargets array and the foreground label map is copied to gtLabelMap. If decompress is VV_TRUE, image is filled with uncompressed, interleaved 24-bit RGB data and gtLabelMap is an uncompressed 32-bit integer buffer with a foreground label for each pixel in image. If decompress is VV_FALSE, image contains the original compressed JPEG stream transmitted from the Camera Server, and gtLabelMap contains the RLE compressed label map transmitted from the server. The length of data (in bytes) in the image buffer is returned in the imagesz field of pCamera and the number of array elements in gtLabelMap are returned in gtLabelMapSz. Note that frames are rendered at a fixed rate that is independent of calls to CamGetLatestFrame, such that successive calls to this function without sufficient delay may return the same frame. Callers can detect this condition by inspecting the frameNum field. Callers are not responsible for allocating/deallocating the data arrays in the pCamera.

Return Value:
Returns VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
PTZ Server Structures and Functions
 
VVPTZStatus Structure
typedef struct vv_ptzstatus_t
{

SOCKET socket;
VVBOOL connected;

VVPTZID ptzID;

float pan, panVel;
float tilt, tiltVel;
float zoom, zoomVel;

VVAXISMODE pan_mode;
VVAXISMODE tilt_mode;
VVAXISMODE zoom_mode;

float scanPanMin, scanPanMax;
float scanPanSpeed;
float scanTilt, scanZoom;
VVBOOL scanning;

} VVPTZStatus;
 
Parameters:
socket
Socket handle for connection to PTZ Server
connected
Read-only flag indicating whether controller is connected to server
ptzID
Identifies cameras that will receive commands from this controller
pan
Target pan angle in degrees
panVel
Maximum pan velocity in degrees/sec to reach target pan angle
tilt
Target tilt angle in degrees
tiltVel
Maximum tilt velocity in degrees/sec to reach target tilt angle
zoom
Target zoom factor (focal length multiplier)
zoomVel
Maximum zoom velocity in magnification/sec to reach target zoom
pan_mode, tilt_mode, zoom_mode
PTZ control mode for the pan, tilt and zoom axes (see PTZSetStatus)
scanPanMin, scanPanMax
Lower and upper pan boundaries in automatic scanning mode
scanPanSpeed
Automatic scanning speed in degrees/sec
scanTilt, scanZoom
The tilt in degrees and zoom factor in automatic scanning mode
scanning
Read/write flag indicating whether automatic scanning mode is enabled

Description:
Each simulated PTZ controller is associated with a VVPTZStatus structure, which stores connection information and the controller state. A pointer to the VVPTZStatus structure is passed as the first parameter in all function calls that communicate with the PTZ Server about the associated controller.
 
PTZConnect/PTZDisconnect
VVINFO PTZConnect(
VVPTZStatusPtr pPtz,
char *hostname
);
VVINFO PTZDisconnect(
VVPTZStatusPtr pPtz
);
 
Parameters:
pPtz
Pointer to VVPTZStatus structure
hostname
Null-terminated string specifying hostname of PC running game server

Description:
PTZConnect establishes a new connection to the PTZ server running on hostname and fills the VVPTZStatus structure pointed to by pPtz with the default values. Future operations on this PTZ controller should pass this structure to the pPtz parameter of the associated PTZ* function call. PTZDisconnect disconnects the controller associated with pPtz from the PTZ server. All PTZ settings are lost after disconnection and must be set again if the controller is reconnected. Applications can determine whether a controller is currently connected to the server by inspecting the connected field of the associated VVPTZStatus structure.

Return Value:
Both functions return VV_OK on a successful connection/disconnection. A return value of VV_SOCKERR indicates that a Windows Sockets error has occurred and the operation could not be completed.
 
PTZGetID/PTZSetID
VVINFO PTZGetID(
VVPTZStatusPtr pPtz
);
VVINFO PTZSetID(
VVPTZStatusPtr pPtz,
VVPTZID ptzID
);
 
Parameters:
pPtz
Pointer to VVPTZStatus structure for a connected controller client
ptzID
Unique controller identifier

Description:
PTZGetID fills pPtz->ptzID with the identifier of the current controller. PTZSetID connects the client to the controller identified by ptzID. If a controller with the corresponding ID already exists, the client is connected to that controller without modifying any existing settings. If a controller with the corresponding ID does not exist, a new controller is created with the requested ID and default controller settings. The VVPTZStatus structure pointed to by pPtz is updated with the settings for this controller.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
 
PTZGetStatus/PTZSetStatus
VVINFO PTZGetStatus(
VVPTZStatusPtr pPtz
);
VVINFO PTZSetStatus(
VVPTZStatusPtr pPtz,
VVAXISMODE axisMode,
VVBOOL updatePan, float pan, float panVel,
VVBOOL updateTilt, float tilt, float tiltVel,
VVBOOL updateZoom, float zoom, float zoomVel
);
 
Parameters:
pPtz
Pointer to VVPTZStatus structure for a connected controller
axisMode
One of the VV_AXISMODE_* values indicating the control mode of the next motion
updatePan
Flag indicating whether to update target pan/panVel in this call
pan
Absolute target pan angle in degrees
panVel
Maximum pan velocity in degrees/sec to reach target pan
updateTilt
Flag indicating whether to update target tilt/tiltVel in this call
tilt
Absolute target tilt angle in degrees
tiltVel
Maximum tilt velocity in degrees/sec to reach target tilt
updateZoom
Flag indicating whether to update target zoom/zoomVel in this call
zoom
Zoom as factor of focal length
zoomVel
Maximum zoom speed in magnification/sec to reach target zoom

Description:
PTZSetStatus sends the specified controller ID, control mode, joint angles and velocities to the PTZ Server for the connection represented by pPtz, and also fills the VVPTZStatus structure pointed to by pPtz with these latest settings. The interpretation of the specified joint angles and velocities depends on the value of axisMode, as described below:

VVAXISMODE Value Description
VV_AXISMODE_ABSPOS Move axis to absolute position at specified velocity (default behaviour)
VV_AXISMODE_RELPOS Move axis by relative offset from current position at specified velocity
VV_AXISMODE_VEL Move axis continuously at specified velocity (position parameter ignored)

The update* flags indicate which axes should be updated with the specified parameters in this call. This allows the axes to be controlled independently and in different control modes. The specified positions/velocities become the new target state for cameras connected to this controller. Note that the camera may only reach the new target state after a short time depending on the specified velocity. PTZGetStatus fills pPtz with the last zoom, joint angles and velocities as returned by the PTZ Server for this connection. The values returned by PTZGetStatus are the target parameters, and do not reflect the current position or motion of the camera (the actual camera location/orientation for each frame are stored in the gt* fields of the VVCamera structure returned by CamGetLatestFrame). Note that PTZGetStatus does not return the current axis control mode, and always returns absolute target positions regardless of the current control mode.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
 
PTZGetAutoScan/PTZSetAutoScan
VVINFO PTZGetAutoScan(
VVPTZStatusPtr pPtz
);
VVINFO PTZSetAutoScan(
VVPTZStatusPtr pPtz,
float panMin,
float panMax,
float panSpeed,
float tilt,
float zoom,
VVBOOL scanning
);
 
Parameters:
pPtz
Pointer to VVPTZStatus structure for a connected controller
panMin, panMax
Pan angle boundaries in degrees
panSpeed
Pan speed in degrees/sec
tilt
Tilt angle in degrees
zoom
Zoom as a factor of focal length
scanning
Flag indicating whether camera is in automatic scanning mode

Description:
PTZSetAutoScan sends the specified scanning parameters to the PTZ Server for the connection represented by pPtz. The VVPTZStatus structure pointed to by pPtz is also updated with these latest settings. If scanning is VV_TRUE, the the controller enters automatic scanning mode and continuously pans between panMin and panMax at a fixed tilt, zoom and panSpeed. To halt the camera, exit from scanning mode by calling PTZSetAutoScan again with scanning set to VV_FALSE. Note that the pan range, speed, tilt and zoom are still updated with the specified parameters on any call with scanning set to VV_FALSE. PTZGetAutoScan fills pPtz with the current scanning parameters read from the PTZ Server for this connection.

Return Value:
Both functions return VV_OK if the command was successful. A return value of VV_SOCKERR indicates that a Windows Sockets error occurred and you should not assume that either the command or response was successful. A return value of VV_INVALIDRESPONSE indicates that the command was successfully sent, but either no response was received or the response was invalid.
© Copyright ObjectVideo, 2006. All rights reserved.