Anura™ Core SDK for Android

Anura™ Core SDK for Android provides all the essential components for integrating NuraLogix™ Affective AI and DeepAffex™ into your Android application. It includes DeepAffex™ Extraction Library, a DeepAffex™ Cloud API client, a camera control module, a face tracker, and other components to allow your application to take measurements using DeepAffex™. This is the same SDK used to build Anura Lite by NuraLogix™.

Anura™ Core SDK for Android has a pipeline architecture (Source, Pipe and Sink modules). A typical measurement pipeline is shown below:

Android Video Pipeline

Requirements

The latest version of the Anura™ Core SDK for Android is 2.4. It requires Android 7.1 (API Level 25) or higher. For a detailed list of requirements, please refer to the platform requirements chapter.

Getting Started

Accessing Anura™ Core SDK and Sample App

Please contact NuraLogix™ to get access to the private Git repository.

Dependencies

Anura™ Core SDK for Android AARs are included in the app/libs folder of the Sample App. You can set up project dependencies in your build.gradle as shown below:

    // Anura™ Core SDK
    implementation(name: 'anura-core-sdk-2.4.0.5264', ext: 'aar')
    implementation(name: 'anura-opencv-4.5.1', ext: 'aar')

    // Anura™ Core SDK Dependencies
    implementation "com.google.mediapipe:solution-core:$mediaPipeVersion"
    implementation "com.google.mediapipe:facemesh:$mediaPipeVersion"
    implementation "org.jetbrains.kotlinx:kotlinx-serialization-json:$kotlinXJsonSerialization"
    implementation "com.google.code.gson:gson:$gsonVersion"
    implementation "org.java-websocket:Java-WebSocket:$websocketVersion"

DeepAffex™ License Key and Study ID

Before your application can connect to DeepAffex™ Cloud API, you will need a valid DeepAffex™ license key and Study ID, which can be found on DeepAffex Dashboard.

The included Sample App sets the DeepAffex™ license key and Study ID in app/server.properties file, which would get included in the application at build time.

Below is the included starting template app/server.properties.template:

# Caution: Do not use quotation marks when filling the values below.

# Your DeepAffex™ License Key and Study ID can be found on DeepAffex™ Dashboard
# https://dashboard.deepaffex.ai

DFX_LICENSE_KEY=
DFX_STUDY_ID=

# These URLs are for the International DeepAffex™ service.
# If the app is operating in Mainland China, replace "ai" with "cn"
# See more regional clusters: https://docs.deepaffex.ai/guide/cloud/2-1_regions.html

DFX_REST_URL=https://api.deepaffex.ai
DFX_WS_URL=wss://api.deepaffex.ai

You can then remove the .template suffix and you should be able to run sample app and take a measurement.

Measurement Pipeline

Anura™ Core SDK for Android provides a MeasurementPipeline interface that connects the various components of the SDK, from fetching camera video frames all the way through rendering the video in the UI. It also provides listener callback methods to observe the measurement state, and to receive and parse results from DeepAffex™ Cloud.

The included Sample App demonstrates how to set up a MeasurementPipeline and manage its lifecycle. The example provided in AnuraExampleMeasurementActivity showcases how to use and setup MeasurementPipeline.

Starting a Measurement

The MeasurementPipeline interface provides a simple function call to start a measurement. The SDK will automatically handle making the API calls to DeepAffex™ Cloud API over WebSocket:

  • Create Measurement: Creates a measurement on DeepAffex™ Cloud and generates a Measurement ID.
  • Subscribe to Results: Get results in real-time during a measurement, and when a measurement is complete.
  • Add Data: Send facial blood-flow data collected by DeepAffex™ Extraction Library to DeepAffex™ Cloud for processing

Measurement Questionnaire and Partner ID

Your application can include user demographic information and medical history when taking a measurement: 

 val exampleDemographics = mapOf(
    "gender:1" to "female",
    "age:1" to "23",
    "height:1" to "175",
    "weight:1" to "60"
    "smoking:1" to "0",
    "diabetes:1" to "type2",
    "bloodpressuremedication:1" to "1"
)

measurementPipeline.startMeasurement(
    exampleDemographics,
    BuildConfig.DFX_STUDY_ID,
    "",
    "example-partner-id-1"
)

Partner ID can hold a unique-per-user identifier, or any other value which could be used to link your application's end users with their measurements taken on DeepAffex™ Cloud. This is because your application's end users are considered anonymous users on DeepAffex™ Cloud.

For more information, please refer to:

Handling Measurement State

Your application can observe MeasurementState of MeasurementPipeline through this MeasurementPipelineListener callback method:

fun onMeasurementStateLiveData(measurementState: LiveData<MeasurementState>?) { ... }

This allows your application to respond to events from MeasurementPipeline and update the UI accordingly. Please refer to AnuraExampleMeasurementActivity for an example on how to best handle MeasurementState changes.

Measurement Results

MeasurementPipelineListener provides 2 callback methods for getting measurement results during and after a measurement is complete:

/**
* Called during a measurement
*/
fun onMeasurementPartialResult(payload: ChunkPayload?, results: MeasurementResults) { ... }

/**
* Called when a measurement is complete
*/
fun onMeasurementDone(payload: ChunkPayload?, results: MeasurementResults) { ... }

These callback methods pass a MeasurementResults instance that contains the results of the current measurement so far, which can be accessed through 2 methods:

/**
* Returns all the current results for each signal ID in a `[String]: [SignalResult]` hash map
*/
val allResults : Map<String, SignalResult>

/**
* Get the current result for a [signalID]
* @return A Double value of the result. If the provided signal ID doesn't have a result,
* it returns `Double.NaN`
*/
fun result(signalID: String): Double

For a list of all the available results from DeepAffex™ and signal IDs, please refer to DeepAffex™ Points Reference.

Measurement Constraints

Anura™ Core SDK utilizes the Constraints system of DeepAffex™ Extraction Library to check the user's face position and movement, and provide actionable feedback to the user to increase the chances of a successful measurement. MeasurementPipelineListener provides a callback method to check for constraint violations:

fun onMeasurementConstraint(
    isMeasuring: Boolean,
    status: ConstraintResult.ConstraintStatus,
    constraints: MutableMap<String, ConstraintResult.ConstraintStatus>
) { 
    ...
}

Please refer to AnuraExampleMeasurementActivity for an example on how to best handle constraint violations and display feedback to the user.

Signal-to-Noise Ratio and Early Measurement Cancellation

DeepAffex™ Cloud will occasionally fail to return a measurement result. This usually occurs when the signal-to-noise ratio (SNR) of the extracted blood-flow signal isn't good enough due to insufficient lighting or too much user or camera movement.

In such a scenario, your application might prefer to cancel the measurement early and provide feedback to the user to improve the measurement conditions.

MeasurementPipeline interface provides a method to check the SNR of the measurement and determine if the measurement should be cancelled:

fun shouldCancelMeasurement(SNR: Double, chunkOrder: Int): Boolean

The SNR of the measurement so far is available from MeasurementResults. Below is an example of how to cancel a measurement early:

fun onMeasurementPartialResult(payload: ChunkPayload?, results: MeasurementResults) {
    ...
    if (measurementPipeline.shouldCancelMeasurement(results.snr, results.chunkOrder)) {
        stopMeasurement()
        // Show feedback to the user that the measurement was cancelled early
        return
    }
    ...
}

Measurement UI Customization

The measurement UI can be customized through MeasurementUIConfiguration. By default, MeasurementUIConfiguration will provide a measurement UI similar to the one in Anura Lite by NuraLogix™.

Below is an example of the parameters that can be adjusted by MeasurementUIConfiguration:

val customMeasurementUIConfig = MeasurementUIConfiguration().apply {
    measurementOutlineStyle = MeasurementUIConfiguration.MeasurementOutlineStyle.OVAL
    showHistograms = true
    overlayBackgroundColor = Color.WHITE
    measurementOutlineActiveColor = Color.BLACK
    measurementOutlineInactiveColor = Color.WHITE
    histogramActiveColor = Color.YELLOW
    histogramInactiveColor = Color.GREEN
    statusMessagesTextColor = Color.CYAN
    timerTextColor = Color.BLUE
}

measurementView.setMeasurementUIConfiguration(customMeasurementUIConfig)

The MeasurementView class also provides some helper methods to modify the icon at the top, the methods are;

  with(measurementView) {
    showAnuraIcon(boolean show)
    setIconImageResource(int resourceId)
    setIconHeightFromTop(int marginTop)
  }

A screenshot is shown below: