Collector extracts region channel information from video frames for server signal processing. More...

#include "dfx/Collector.h"

Public Member Functions
virtual	~Collector ()
	Collector destructor.

virtual std::shared_ptr< dfx::Frame >	createFrame (const dfx::VideoFrame &videoFrame)=0
	Creates a Frame from a user supplied video frame. More...

virtual CollectorState	startCollection ()=0
	When the client is ready to start a measurement they will notify the collector to start collecting data. More...

virtual bool	prepareMeasurement (const std::vector< uint8_t > &createMeasurementPayload)=0
	When the client creates a new Measurement with the API, the Collector should be prepared with the API Measurement CreateResponse payload. More...

virtual CollectorState	getCollectorState ()=0
	The current internal state of this measurement collector. More...

virtual std::string	getMode () const =0
	Returns the operating mode of the Collector. More...

virtual bool	setTargetFPS (float targetFPS)=0
	Identifies what the anticipated FPS is suppose to be. More...

virtual bool	setChunkDurationSeconds (float chunkDurationSeconds)=0
	Sets the desired collected chunk duration in seconds. More...

virtual float	getChunkDurationSeconds ()=0
	Returns the desired collected chunk duration in seconds. More...

virtual bool	setNumberChunks (uint32_t numberChunks)=0
	Sets number of chunks before the measurement is complete. More...

virtual uint32_t	getNumberChunks ()=0
	Returns the total number of chunks the collector will process for this measurement. More...

virtual bool	enableConstraint (const std::string &constraintID)=0
	Enable constraint handling for the identified constraint. More...

virtual bool	disableConstraint (const std::string &constraintID)=0
	Disable a constraint for the identified constraint. More...

virtual std::vector< std::string >	getEnabledConstraints ()=0
	Returns the currently enabled constraints. More...

virtual std::vector< std::string >	getAvailableConstraints () const =0
	Obtain all available constraint IDs. More...

virtual ConstraintResult	checkConstraints (std::shared_ptr< dfx::Frame > &frame, std::vector< std::pair< std::string, dfx::ConstraintResult >> &results)=0
	Check for constraint violations. More...

virtual std::string	getConstraintErrorMessage (const std::string &violationID)=0
	Obtain a message which represents the violation ID. More...

virtual bool	setConstraintsConfig (const std::string &key, const std::string &propertyValue)=0
	Configures the constraint system properties. More...

virtual std::string	getConstraintsConfig (const std::string &key)=0
	Obtains a constraint system property. More...

virtual std::vector< std::string >	getRequiredPosePointIDs () const =0
	Return the set of pose points for the configured study. More...

virtual CollectorState	defineRegions (std::shared_ptr< dfx::Frame > &frame)=0
	Define the regions of interest based upon the MPEG-4 Facial Data Points. More...

virtual CollectorState	extractChannels (std::shared_ptr< dfx::Frame > &frame)=0
	Extracts the channels of interest from the video frame and regions of interest. More...

virtual bool	isChunkReady () const =0
	Identifies when a measurement has collected sufficient information to make a server request. More...

virtual int32_t	numberFramesNeeded () const =0
	The approximate number of additional video frames required before sufficient information has been collected to make the server request. More...

virtual std::shared_ptr< dfx::ChunkData >	getChunkData ()=0
	The measurement chunk data which has been collected up for the current chunk until this point. More...

virtual std::shared_ptr< dfx::MeasurementResult >	decodeMeasurementResult (const std::vector< uint8_t > &measurementResponse)=0
	Decodes the binary measurement response payload from the server into a MeasurementResult. More...

virtual bool	setProperty (const std::string &key, const std::string &value)=0
	Client applications wishing to associate their own metadata properties can associate key=value strings to the Collector. More...

virtual std::string	getProperty (const std::string &key)=0
	Returns a metadata property by key name if it exists. More...

virtual void	removeProperty (const std::string &key)=0
	Removes a metadata property by key name if it exists.

virtual std::map< std::string, std::string >	getProperties ()=0
	Obtains the Chunk metadata for this measurement request. More...

virtual std::string	getLastErrorMessage ()=0
	If there was an error getLastErrorMessage may contain more information about why the error occurred. More...

virtual void	cancelCollection ()=0
	Cancels the current collection. More...

virtual void	stopCollection ()=0
	Stop the current collection. More...

virtual void	forceComplete ()=0
	Forces the completion of a collection. More...

virtual void	resetCollection ()=0
	Resets the measurement state to construct a new chunk. More...

virtual bool	setFaceAttribute (const std::string &faceID, FaceAttribute key, FaceAttributeValue value)=0
	Add user data information to a payload using FaceAttribute key and values. More...

virtual bool	setFaceAttribute (const std::string &faceID, FaceAttribute key, double value)=0
	Add user data information to a payload using FaceAttribute key and numeric/boolean value. More...

virtual bool	setCloudResultsFeedback (const std::string &jsonCloudResults)=0
	Provide the cloud results to the Collector to potentially improve the result. More...

virtual bool	initializeModel (int8_t modelType, const std::vector< unsigned char > &data)=0
	Configures a model within the Collector. More...

Detailed Description

Collector extracts region channel information from video frames for server signal processing.

A Collector involves significant processing on video frames using located faces to identify regions of interest, extract and summarize the data for server signal processing.

Member Function Documentation

◆ cancelCollection()

virtual void dfx::Collector::cancelCollection ( )

pure virtual

Cancels the current collection.

When a collection violates as constraint or otherwise needs to stop this method can be called placing the collector back into a WAIT state until the client calls startCollection again.

◆ checkConstraints()

virtual ConstraintResult dfx::Collector::checkConstraints	(	std::shared_ptr< dfx::Frame > &	frame,
		std::vector< std::pair< std::string, dfx::ConstraintResult >> &	results
	)

pure virtual

Check for constraint violations.

When some constraints are enabled and a constraint is detected as being violated that constraint will be added to the violations list.

Parameters

frame	the video frame currently being operated on
results	is set of constraints and associated state

Returns: result of the constraint check

◆ createFrame()

virtual std::shared_ptr<dfx::Frame> dfx::Collector::createFrame ( const dfx::VideoFrame & videoFrame )

pure virtual

Creates a Frame from a user supplied video frame.

Parameters

videoFrame is the video Frame to wrap.

Returns: a reference to a Frame.

◆ decodeMeasurementResult()

virtual std::shared_ptr<dfx::MeasurementResult> dfx::Collector::decodeMeasurementResult ( const std::vector< uint8_t > & measurementResponse )

pure virtual

Decodes the binary measurement response payload from the server into a MeasurementResult.

Parameters

measurementResponse from the server to decode

Returns: the MeasurementResult object

◆ defineRegions()

virtual CollectorState dfx::Collector::defineRegions ( std::shared_ptr< dfx::Frame > & frame )

pure virtual

Define the regions of interest based upon the MPEG-4 Facial Data Points.

A Collector leverages many regions of interest which are areas of a detected face to locate the required signals. This defineRegions call will analyze and build the candidate region set which the channels will be constructed from.

Parameters

frame the video frame currently being operated on

Returns: the Collector state after define regions

◆ disableConstraint()

virtual bool dfx::Collector::disableConstraint ( const std::string & constraintID )

pure virtual

Disable a constraint for the identified constraint.

If the constraint was currently enabled, it will be removed from the set of currently active constraints. If it was not active or does not exist, the request is ignored.

Parameters

constraint to disable

Returns: true if the constraintID was changeable, false otherwise.

◆ enableConstraint()

virtual bool dfx::Collector::enableConstraint ( const std::string & constraintID )

pure virtual

Enable constraint handling for the identified constraint.

Some constraint validation can be done on the frames provided by the client to ensure they meet sufficient criteria to satisfy server data quality.

This may include standard checks for things like face presence and frame rate along with movement, lighting, etc.

Parameters

constraint to enable

Returns: true if the constraintID was changeable, false otherwise.

◆ extractChannels()

virtual CollectorState dfx::Collector::extractChannels ( std::shared_ptr< dfx::Frame > & frame )

pure virtual

Extracts the channels of interest from the video frame and regions of interest.

A Collector will collapse the candidate regions of interest down to a set of channels of interest which the server will further refine into a set of signals.

Parameters

frame the video frame currently being operated on

Returns: the Collector state after extract channels

◆ forceComplete()

virtual void dfx::Collector::forceComplete ( )

pure virtual

Forces the completion of a collection.

Collections will trigger a ChunkAvailable when enough data has been collected. When processing a video file, the final chunk may be insufficient in length and never trigger the ChunkAvailable state. forceComplete() will force the collector into a measurement Complete state and create a final payload with any remaining state.

◆ getAvailableConstraints()

virtual std::vector<std::string> dfx::Collector::getAvailableConstraints ( ) const

pure virtual

Obtain all available constraint IDs.

The set of available known constraint IDs which can be enabled by calling setEnableConstraints.

Returns: the vector of available constraint IDs.

◆ getChunkData()

virtual std::shared_ptr<dfx::ChunkData> dfx::Collector::getChunkData ( )

pure virtual

The measurement chunk data which has been collected up for the current chunk until this point.

The collected measurement chunk data is offloaded to the ChunkData and resets the internal collector state to continue processing more frames in the next chunk.

Returns: ChunkData collected to this point or null if the data is was not valid to create a chunk. ie. chunk duration elapsed but the number of valid frames was insufficient to form a useful payload.

◆ getChunkDurationSeconds()

virtual float dfx::Collector::getChunkDurationSeconds ( )

pure virtual

Returns the desired collected chunk duration in seconds.

Returns: the chunk durations in seconds.

◆ getCollectorState()

virtual CollectorState dfx::Collector::getCollectorState ( )

pure virtual

The current internal state of this measurement collector.

The internal state of this measurement collector. This is the same state returned by defineRegions or extractChannels.

Once the measurement is in error state, it will remain in ERROR until reset which will place it back in WAITING.

Returns: the current internal state.

◆ getConstraintErrorMessage()

virtual std::string dfx::Collector::getConstraintErrorMessage ( const std::string & violationID )

pure virtual

Obtain a message which represents the violation ID.

When presenting on a GUI, the returned constaint message is a more meaningful representation of the constraint violation ID.

Parameters

violationID the constraint ID to obtain the error message for

Returns: error message of the corresponding violationID.

◆ getConstraintsConfig()

virtual std::string dfx::Collector::getConstraintsConfig ( const std::string & key )

pure virtual

Obtains a constraint system property.

Returns the constraint value associated with the key.

Parameters

key	the configuration key to work on, currently "json"

Returns: the value of the configuration key or an empty string if key did not exist.

◆ getEnabledConstraints()

virtual std::vector<std::string> dfx::Collector::getEnabledConstraints ( )

pure virtual

Returns the currently enabled constraints.

Returns: a list of the currently enabled constraint IDs.

◆ getLastErrorMessage()

virtual std::string dfx::Collector::getLastErrorMessage ( )

pure virtual

If there was an error getLastErrorMessage may contain more information about why the error occurred.

Returns: the last known error

◆ getMode()

virtual std::string dfx::Collector::getMode ( ) const

pure virtual

Returns the operating mode of the Collector.

The mode is configured in the Factory prior to Collector creation.

Returns: the operating mode for current collector.

◆ getNumberChunks()

virtual uint32_t dfx::Collector::getNumberChunks ( )

pure virtual

Returns the total number of chunks the collector will process for this measurement.

Returns: the total number of chunks.

◆ getProperties()

virtual std::map<std::string, std::string> dfx::Collector::getProperties ( )

pure virtual

Obtains the Chunk metadata for this measurement request.

The returned chunk metadata is a superset of the metadata provided by the client application. Additional measurement details are augmented into the chunk metadata.

Returns: a map containing the metadata key/value strings

◆ getProperty()

virtual std::string dfx::Collector::getProperty ( const std::string & key )

pure virtual

Returns a metadata property by key name if it exists.

The returned string will be the value of the current chunk metadata property.

Returns: a string containing the property value, or an empty string if the key does not exist.

◆ getRequiredPosePointIDs()

virtual std::vector<std::string> dfx::Collector::getRequiredPosePointIDs ( ) const

pure virtual

Return the set of pose points for the configured study.

Once a study has been initialized this method will return the list of pose points required for each found face.

Returns: the set of MPEG-4 Facial Data Points required for each face.

◆ initializeModel()

virtual bool dfx::Collector::initializeModel	(	int8_t	modelType,
		const std::vector< unsigned char > &	data
	)

pure virtual

Configures a model within the Collector.

Parameters

modelType	A numeric identifier associated with the model implementation
data	Configuration data associated with the model

Returns: true if the model was configured, false if it was not (invalid model, models not supported)

◆ isChunkReady()

virtual bool dfx::Collector::isChunkReady ( ) const

pure virtual

Identifies when a measurement has collected sufficient information to make a server request.

Returns: true if the chunk is ready to be sent, false otherwise

◆ numberFramesNeeded()

virtual int32_t dfx::Collector::numberFramesNeeded ( ) const

pure virtual

The approximate number of additional video frames required before sufficient information has been collected to make the server request.

The number frames needed is a rough guide for the client application to know how many more data points will be required for a request to be sent. It is not a guarantee as variance will occur for a number of reasons: variable frame rate, participant movement, along with environment and camera impacts. When the measurement has collected enough data this will be zero and isChunkReady will return true. The number of frames is not necessarily monotomically decreasing.

Returns: the anticipated number of frames remaining or zero if sufficient information has been collected. If there is an error or inability to calculate, -1 is returned.

◆ prepareMeasurement()

virtual bool dfx::Collector::prepareMeasurement ( const std::vector< uint8_t > & createMeasurementPayload )

pure virtual

When the client creates a new Measurement with the API, the Collector should be prepared with the API Measurement CreateResponse payload.

Parameters

createMeasurementPayload the payload from DFX server

Returns: true if collector was happy, false otherwise

◆ resetCollection()

virtual void dfx::Collector::resetCollection ( )

pure virtual

Resets the measurement state to construct a new chunk.

The Collector maintains internal buffers and state when constructing a chunk which needs to be reset when a new chunk collection starts. A client application can discard a collection and all accumulated state by calling reset().

◆ setChunkDurationSeconds()

virtual bool dfx::Collector::setChunkDurationSeconds ( float chunkDurationSeconds )

pure virtual

Sets the desired collected chunk duration in seconds.

Identifies to the collector what the desired chunk duration should be in terms of seconds. This will typically ensure that a chunk is ready for collecting whenever the duration has been reached.

Parameters

chunkDurationSeconds the chunk duration in seconds.

Returns: true if the chunk durations was successfully changed, false otherwise.

◆ setCloudResultsFeedback()

virtual bool dfx::Collector::setCloudResultsFeedback ( const std::string & jsonCloudResults )

pure virtual

Provide the cloud results to the Collector to potentially improve the result.

If the results obtained from the cloud are available to the Collector, it can make additional optimizations on future payloads and provide more informed feedback.

Parameters

jsonCloudResults the JSON results obtained from the Cloud API

Returns: true if result data was valid, false if result data was not valid.

◆ setConstraintsConfig()

virtual bool dfx::Collector::setConstraintsConfig	(	const std::string &	key,
		const std::string &	propertyValue
	)

pure virtual

Configures the constraint system properties.

The constraint system configuration accepts a key "json" and the corresponding JSON payload to configure the system.

Parameters

key	the configuration key to work on, currently "json"
property	the value to set this property to

Returns: true if the property was successfully set, false on error

◆ setFaceAttribute() [1/2]

virtual bool dfx::Collector::setFaceAttribute	(	const std::string &	faceID,
		FaceAttribute	key,
		double	value
	)

pure virtual

Add user data information to a payload using FaceAttribute key and numeric/boolean value.

Parameters

faceID	the face identifier to use, typically "1"
key	the attribute key identifying the value being provided
value	the attribute value associated with key

Returns: true if value was applied, false if invalid value or unable to apply.

◆ setFaceAttribute() [2/2]

virtual bool dfx::Collector::setFaceAttribute	(	const std::string &	faceID,
		FaceAttribute	key,
		FaceAttributeValue	value
	)

pure virtual

Add user data information to a payload using FaceAttribute key and values.

Parameters

faceID	the face identifier to use, typically "1"
key	the attribute key identifying the value being provided
value	the attribute value associated with key

Returns: true if value was applied, false if invalid value or unable to apply.

◆ setNumberChunks()

virtual bool dfx::Collector::setNumberChunks ( uint32_t numberChunks )

pure virtual

Sets number of chunks before the measurement is complete.

Parameters

numberChunks the number of chunks.

Returns: true if the number of chunks was successfully changed, false otherwise.

◆ setProperty()

virtual bool dfx::Collector::setProperty	(	const std::string &	key,
		const std::string &	value
	)

pure virtual

Client applications wishing to associate their own metadata properties can associate key=value strings to the Collector.

If there is metadata that a client applications wishes to tag against a Collector request, ie. barcode or participant name they should invoke setProperty() with the appropriate value to ensure that it is properly encoded into the Chunk Payload.

Parameters

key	a key identifier for the metadata. If the key already exists it will be replaced
value	the value to associate with the key

Returns: true if property was set, false otherwise

◆ setTargetFPS()

virtual bool dfx::Collector::setTargetFPS ( float targetFPS )

pure virtual

Identifies what the anticipated FPS is suppose to be.

The anticipated target FPS defaults to 30.0 FPS, but if a video frame rate of 60.0 FPS or higher is required it can be customized. While the actual FPS is established based on the individual VideoFrame timestamps, the anticipated FPS assists with model calibration.

Post-processing a pre-recorded video file will generally have a more consistent FPS and yield less noisy results then a live extraction which needs to share CPU processing power for video frame handling with video frame processing.

If the target or actual FPS falls outside model tolerances either too high or too low the collection will be rejected.

Parameters

targetFPS the anticipated target FPS

Returns: true if the targetFPS was successfully changed, false otherwise.

◆ startCollection()

virtual CollectorState dfx::Collector::startCollection ( )

pure virtual

When the client is ready to start a measurement they will notify the collector to start collecting data.

Prior to starting, a measurement should be in the WAITING state. A call to resetCollector will always ensure the measurement is in WAITING state.

While in a WAITING state, regions can be obtained and constraints analyzed for violations prior to a collection starting.

Returns: the current internal state, which should be COLLECTING unless an ERROR occurred.

◆ stopCollection()

virtual void dfx::Collector::stopCollection ( )

pure virtual

Stop the current collection.

When a collection violates as constraint or otherwise needs to stop this method can be called placing the collector back into a WAIT state until the client calls startCollection again.

Deprecated:: replaced by cancelCollection which more accurately describes the activity since the measurement can not be resumed before being reset. Please use cancelCollection as this method will eventually be removed as it simply calls cancelCollection.

The documentation for this class was generated from the following file:

dfx/Collector.h

Public Member Functions

Detailed Description

Member Function Documentation

◆ cancelCollection()

◆ checkConstraints()

◆ createFrame()

◆ decodeMeasurementResult()

◆ defineRegions()

◆ disableConstraint()

◆ enableConstraint()

◆ extractChannels()

◆ forceComplete()

◆ getAvailableConstraints()

◆ getChunkData()

◆ getChunkDurationSeconds()

◆ getCollectorState()

◆ getConstraintErrorMessage()

◆ getConstraintsConfig()

◆ getEnabledConstraints()

◆ getLastErrorMessage()

◆ getMode()

◆ getNumberChunks()

◆ getProperties()

◆ getProperty()

◆ getRequiredPosePointIDs()

◆ initializeModel()

◆ isChunkReady()

◆ numberFramesNeeded()

◆ prepareMeasurement()

◆ resetCollection()

◆ setChunkDurationSeconds()

◆ setCloudResultsFeedback()

◆ setConstraintsConfig()

◆ setFaceAttribute() [1/2]

◆ setFaceAttribute() [2/2]

◆ setNumberChunks()

◆ setProperty()

◆ setTargetFPS()

◆ startCollection()

◆ stopCollection()