The DeepAffex Cloud will occasionally fail to return a result successfully. This usually occurs when the Signal-to-Noise Ratio (SNR) of the extracted blood-flow signal isn't high enough or when there are other issues with it.
While the user cannot do much about this for blood-flow extracted from videos, if a measurement is being made live using a camera, your application can provide actionable feedback to the user based on the Constraints system in the Extraction Library which may increase the chances of a successful result. You can also cancel a failing measurement early without waiting for a round trip over the internet.
dfxdemo, introduced earlier, has the ability to
make measurements using your computer's webcam and provide actionable feedback
using the Constraints system while you do so. Assuming you have completed all
the prerequisite steps needed to make measurements (i.e. registered your
license, logged in as a user and selected a study,) you can run the following
dfxdemo measure make_camera
Note: If your computer has multiple webcams and you want to select a different
one, you can use the
--camera <index> argument. You can pass
--help to get a
full list of options.
Once the demo starts, you may see multiple feedback messages in red, asking you,
for example, to look straight at the camera or to hold still. Once all the
constraints are met, you can press
s to start the measurement. You may also
see warnings during the measurement and if you violate constraints too much
during the measurement, it may fail.
The flowchart below shows the steps needed to make a measurement from a webcam with constraints enabled.
Programmatically, you need to call
checkConstraints on every
typically, before starting a collection.
dfx_frame = collector.createFrame(dfx_video_frame) for dfx_face in dfx_faces: dfx_frame.addFace(face) result, details = collector.checkConstraints(dfx_frame)
result return value is the aggregate result of the constraint check and
can be either
GOODmeans that the constraints are all met and collection can proceed. If it hasn't started yet you could choose to automatically start it or allow the user to start it using an input. (Recall that a measurement is started by calling
WARNmeans that constraints are currently violated but the user has still has a chance to change conditions and change things back to
GOODand perhaps still have a successful measurement. Feedback should be provided to the user.
ERRORmeans that the collection has failed because of too many constraint violations. Obviously,
ERRORcan only be returned after the collection has started. On an
ERROR, the application is expected to call
cancelCollectionto terminate the ongoing measurement and
resetCollectionto reset the internal collector state.
On a result of
details return value contains a map of
reason strings and
result values. The reason strings correspond to each
possible constraint that is violated with a
The possible reason strings and their meanings are listed below:
|Reason String||Meaning||Possible user feedback|
|FaceNone||No faces detected in the frame||Move face into frame|
|FaceOffTarget||User's face is not in the constraints target region||Move face into target region|
|FaceDirection||User is not looking at camera||Look straight at the camera|
|FaceFar||User's face is too far from camera||Move closer to the camera|
|FaceMovement||User is moving too much||Hold still|
|ImageBright||Image is too bright||Make the image darker|
|ImageDark||Image is too dark||Make the image brighter|
|ImageBackLit||User's face is backlit||Remove illumination from behind the face|
|LowFps||Frame rate is too low||Improve frame rate|
Clearly, feedback to the user can and should be customized for your specific
LowFps reasons in particular can be dealt with
using application specific camera settings and controls. In addition to the
above, you may get
ImageEmpty (an empty image was passed) and
(capture device was moving during capture) which will not be discussed in this
The constraints that will be checked by the DFX
Collector are configurable by
the developer. You can query the currently set constraints by calling
collector.getConstraintsConfig("json") and modify the values by calling
collector.setConstraintsConfig("json", json_string) - the return value and
the modified values are expected to be JSON strings which will contain a simple
JSON dictionary. Typically, you would get the existing constraints, modify what
you need to and set them again.
constraints_cfg_str = collector.getConstraintsConfig("json") # Parse the json, modify the values as needed modified_constraints_cfg_str = parse_and_modify(constraints_cfg_str) collector.setConstraintsConfig("json", modified_constraints_cfg_str)
The following constraints can be adjusted:
The capture frame rate in frames/second has to exceed
27). This check can be disabled using
The face has to be present within a target rectangle within the frame. (The
rectangle is specified as a percent of the frame size using
boxHeight_pct) (default: 50, 50, 50, 99).
These defaults are optimized for the landscape orientation. This check can be
The face has to be close enough to the camera. Minimum inter-pupillary distance
in pixels (
minInterPupilDist_px) is used a proxy for distance (default: 60.0).
This check can be disable using
checkDistance. (This check assumes that you
haven't disabled the face rotation constraint described below.)
The rectangle containing the face has to be brighter than
(default: 48) and darker than
threshBright. (default: 225). This check can be
If the region around the face, created by growing the face rectangle by
backLightSearchMult (default: 2.0) and cropping at the bottom of the face
rectangle, has more than
backLightMaxPixels_pct percentage (default: 30%) of
its pixels brighter than
backLightThresh (default: 240) then the backlight
constraint is violated. This check can be disabled using
The user has to look at the camera. The maximum allowed left-right face
maxFaceRotLR_deg degrees (default: 12.5) and max allowed up-down
face rotation is
maxFaceRotUD_deg degrees. (default: 25). This check can be
The nose position is tracked in time (for the last
milliseconds) (default: 2000) and if it exceeds
(default: 10), then a warning is issued. This check can be disabled using
The perpendicular distance from the eyebrow midpoint to the line joining the
two lateral canthuses is calculated in mm for both eyebrows and tracked. If the
range of the larger tracked value exceeds
(default: 3), then a warning is issued. This check can be disabled using
NOTE: The face movement and eyebrow movement constraints are only checked if a measurement is active.
We recommend using your judgement to modify and/or disable constraints to get the best results for your particular use case. Testing with a wide variety of users and in a wide range of environments is very helpful!