Nuance Mix Runtime Event Logs

The Mix ASR, NLU, TTS, and Dialog runtime services generate event logs that can be used by Nuance customers to troubleshoot deployments, monitor operational data, produce reports, generate billing records, track usage, and so on.

Mix offers two REST APIs to help customers access and use this data:

• The Nuance Kafka Consumer API provides access to the event logs generated by the services. Customers can use this API to pull the data that they need and then manipulate and process this data for their use cases; for example, for reporting or troubleshooting.
• The Nuance Audio File Storage Service (AFSS) API provides access to the audio files.

Note: Before you can use the Mix logging APIs, Nuance must enable event logs in your application. Please contact your Nuance representative for more information.

Kafka API essentials

The Nuance Kafka Consumer API uses the standard Kafka concepts and terminology. This section assumes that you have a basic understanding of Kafka.

In particular, the Kakfa Consumer API uses the following concepts.

Messages/events

A Kafka message corresponds to an event that occurred in Mix. There are two main types of Mix events:

• Runtime API events: These events are related to the ASRaaS, NLUaaS, DLGaaS, and TTSaaS API transactions (for example, a DLGaaS StartRequest). As such, these messages are organized around the service requests and responses.
• Dialog log events: These events are related to the behavior of the dialog application (for example, the dialog was started, input was received, a transition occurred, and so on) and are used for reporting. These messages are organized around the dialog events.

Each message has a key that is composed of the following two fields:

Field	Description
service	Name of the runtime service that generated this message. Valid values are: TTSaaS, ASRaaS, NLUaaS, DLGaaS.
id	Randomly-generated UUID that uniquely identifies the message.

Topics

Example of topic in Mix

"topic": "DEMO-OMNICHANNEL-APP-DEV"

Kafka messages are organized into topics. In Mix, topic names correspond to app IDs. Therefore, a Mix topic stores messages for a specific app ID.

Partitions

Example of Get partitions endpoint:

{
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "partitions": [
        0
    ]
}

Kafka topics are divided into partitions.

Topics and partitions are created by Nuance. To get the list of partitions for a topic, use the Get Partitions endpoint.

This endpoint returns the topic name and partition numbers that correspond to the client credentials used when creating the access token.

Consumers

Consumers pull the messages from topics. You create your own consumers using the Create Consumer endpoint.

In Mix, consumer names follow this nomenclature:

consumer-UUIDv4

where UUIDv4 is a Version 4 UUID. For example:

consumer-9872ec64-50d9-4dcd-9f4f-e8b5099a7f4f

Consumer groups

A consumer group is a set of consumers that cooperate to consume data from topics. Partitions are assigned to a consumer group. Each consumer in a consumer group receives messages from a different subset of the assigned partitions.

You create your own consumer groups using the Create Consumer endpoint.

In Mix, consumer group names follow this nomenclature:

appID-appID/topic-clientName-clientName-two-digit number

Where:

• appID/topic: Is the name of the topic; in Mix, this is the app ID name.
• clientName: Is the name of one of the clients defined for the app ID; this information is available from the Mix.dashboard Manage tab. Nuance recommends that you create one client per organization that needs to access event logs for an APP ID. See Recommendations.
• two-digit number: A two-digit number, starting at 00. See note below.

For example:

appID-DEMO-OMNICHANNEL-APP-DEV-clientName-default-00

Consuming messages

Consumers pull the messages from topics. A consumer must be assigned to one or more partitions to pull messages. There are two ways to assign partitions:

• By manually assigning the consumer to a specific partition using the Assign Partitions endpoint
• By subscribing to a topic using the Subscribe endpoint; in this case, consumers are assigned dynamically to the partitions of a topic

Note: Use only one of the above methods.

The following diagram summarizes some of the concepts covered so far. Topic A includes two partitions, and each partition stores messages as they are generated. Consumers are part of consumer groups and read the messages.

To determine which messages to pull, Kafka uses offsets.

Using offsets

Offsets specify a position in a partition and determine where a consumer should start pulling messages.

Each consumer tracks its position (that is, offset) in a partition. By default, when a consumer is created, it is set to start reading the newest message in a partition. For example, in the diagram above, a new consumer assigned to partition 0 will start reading the next message coming at offset 9.

Kafka uses the following terms when talking about offsets:

• The earliest offset, also called beginning offset, points to the first available offset in a partition.
• The latest offset, also called end offset, points to the last available offset in a partition.
• The current offset points to the last message that Kafka has sent to a consumer. The current offset ensures that the same records are not sent again to the same consumer.
• The committed offset points to the last record that a consumer has successfully processed. This can be set manually with the Commit Offsets endpoint or automatically by setting the auto.commit.enable parameter to true when creating consumers.

Offsets are attached to a consumer group, so consumers should use the same consumer group to maintain the offsets.

Authorization

Nuance Mix uses the OAuth 2.0 protocol for authorization. All client applications must provide an access token to access event logs and audio files for an app ID.

The access token is created using client credentials for an app ID. Specify a scope of log to get access to the event logs and audio files, as shown in the following example:

$ export CLIENT_ID="appID%3ADEMO-OMNICHANNEL-APP-DEV%3Ageo%3Aqa%3AclientName%3AScriptUser"
$ export SECRET="riAbk888CC2B.97D7eUklVe6pD"
$ export TOKEN="`curl -s -u "$CLIENT_ID:$SECRET" "https://auth.crt.nuance.co.uk/
oauth2/token" -d 'grant_type=client_credentials' -d 'scope=log' | python -m 
json.tool |  python -c 'import sys, json; print(json.load(sys.stdin)["access_token"])'`"

The access token returned will provide access to event logs for the app ID linked to the client credentials specified. For example, the credentials in the sample code above will provide access to the event logs for the DEMO-OMNICHANNEL-APP-DEV app ID:

This access token must be included in every API request. Specify the access token as a bearer token in the request header.

For more information about the authorization process, see Authorization in Mix.

You must reuse the same access token across the different consumer processes and not acquire a new one at each API call. See Reusing access tokens for details.

Quick start

This section takes you through the process of getting event logs for a Mix application using Postman.

It assumes that:

• Event logs have been enabled for your Mix application. Please contact your Nuance representative for more information.
• You have deployed Mix resources in the Mix runtime environment and have accessed them using the ASRaaS, NLUaaS, DLGaaS, or TTSaaS runtime service. You can use the Quick Start coffee app, which uses DLGaaS, for this purpose.

Import the Nuance-Kafka-REST-API collection into Postman

You can get up and running quickly by importing the Nuance-Kafka-REST-API collection into Postman.

This documentation provides the Nuance-Kafka-REST-API_Postman_collection.zip file, which includes the following files:

• Nuance-Kafka-REST-API.postman_collection.json, which includes the pre-built requests for the Nuance Kafka endpoints
• Nuance-Kafka-REST-API.postman_environment_variables.json, which includes a set of variables used by the collection endpoints

1. Download and unzip the Nuance-Kafka-REST-API_Postman_collection.zip file.
2. In Postman, select File->Import.
3. Click the Folder tab.
4. Click Choose folder from your computer, navigate to the directory that contains the Nuance-Kafka-REST-API_Postman_collection folder, select it, and click Select folder.
5. Click Import.

The Kafka Postman collection and environment variables are imported.

Configure the consumer name and configure group name variables

Example of NUANCE-KAFKA environment variables
(Click the image for a close-up view)

Most requests require a valid consumer name and consumer group name.

The Nuance-Kafka-REST-API collection uses variables for these values. To configure the variables for the endpoints:

1. In Postman, click Environments (left side of the Postman window).
   The list of environments appears.
2. Click NUANCE-KAFKA.
   The variables for the NUANCE-KAFKA environment are displayed.
3. Set a valid consumer name in the Current value field for the consumer_name variable.
4. Set a valid consumer group name in the Current value field for the consumer_group variable.
5. Leave all other fields as is.
6. Click Save to save the consumer variable values.

Make sure to click the Save button, otherwise your updates will not be saved.

Select the NUANCE-KAFKA environment

Selecting the NUANCE-KAFKA environment

The Nuance-Kafka-REST-API collection requires variables that are set in the NUANCE-KAFKA environment. Always make sure that the NUANCE-KAFKA environment is selected when you run endpoints from the Nuance-Kafka-REST-API collection:

1. In Postman, click Collections (left side of the Postman window) and expand the Nuance-Kafka-REST-API collection.
2. In the top-right corner of the Postman window, to the left of the eye icon , select NUANCE-KAFKA.

You can now run the endpoints.

Authorize your application

Authorize your application: Postman Authorization tab

To access the Mix Kafka API, you need a valid access token. This procedure describes how to create a valid access token using the Authorization endpoint.

The credentials that you use to get the access token will determine the event logs that you have access to. For more information, see Authorization.

1. In the Nuance-Kafka-REST-API collection, select the Authorization request.
2. Click the Authorization tab and specify the following information:
   • In Username, enter the client ID for your Mix application. See Create a client ID for details. Make sure to escape the colon in the client ID, as described in Specifying the client ID in your application.
   • In Password, enter the client secret for this client ID. See Generate the client secret for details.
3. Click Send.

Response

The access token is returned in the response body; for example:

{
    "access_token": "eyJhbGciOiJSUzI1NiIsImtpZ...YOE",
    "expires_in": 899,
    "scope": "log",
    "token_type": "bearer"
}

This token is saved in the logaccesstoken NUANCE-KAFKA environment variable and will be used automatically for the following requests in this environment.

Note that the access token is valid for 15 minutes. You will need to get a new one after it expires.

Get the topic name and partition numbers

First get the topic name and partition numbers with the Get Partitions endpoint:

To get the topic and partitions:

1. In the Nuance-Kafka-REST-API collection, expand the Partitions folder and select Get Partitions.
2. Click Send.

Response

The topic name and partition numbers are returned in the response body; for example:

{
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "partitions": [
        0
    ]
}

Create a consumer in a consumer group

Create a consumer: Postman Body tab

You create a consumer in a consumer group with the Create Consumer endpoint.

By default, when you create a consumer, it is set to start reading the newest message in a partition. In this quick start, we want to retrieve the event logs that have already been created, so we will set the auto.offset.reset parameter to earliest. This will ensure that the new consumer instance will start reading logs from the beginning of the partition.

Note: If the consumer group name already exists, the consumer is added to this group. Otherwise, the group is created.

To create a consumer in a consumer group:

1. Expand the Consumers and Groups folder and select Create Consumer.
2. Click the Body tab and set the auto.offset.reset parameter to earliest.
3. Click Send.

Response

A 204 No Content code is returned, indicating that the consumer and consumer group were created successfully.

Subscribe the consumer to a partition

Assign partitions dynamically: Postman Body tab

The next step is to subscribe the consumer to one or more partitions to get messages. There are two ways to subscribe to a partition:

• By assigning the consumer to a specific partition
• By assigning partitions dynamically

In this quick start, we'll assign the partition dynamically, using the Subscribe endpoint. This endpoint dynamically assigns the consumer to the partitions of the specified topic.

To assign the consumer to a specific partition:

1. Expand the Subscriptions folder and select Subscribe.
2. Click the Body tab and specify the topic name obtained in Get the topic and partitions.
3. Click Send.

Response

A 204 No Content code is returned, indicating that the partition was assigned successfully.

Get event log records

You fetch event log records for a consumer with the Get Records endpoint.

To get event log records:

1. Expand the Records folder and select Get Records.
2. Click Send.

Response

Event logs are returned; for example:

[
    {
        "topic": "DEMO-OMNICHANNEL-APP-DEV",
        "key": {
            "service": "DLGaaS",
            "id": "45206957-3770-4883-87c8-3b8350e39611"
        },
        "value": {
            "specversion": "1.0",
            "service": "DLGaaS",
            "source": "NIIEventLogger",
            "type": "NIIEventLog",
            "id": "45206957-3770-4883-87c8-3b8350e39611",
            "timestamp": "2021-02-08T16:48:12.330Z",
            ...

Delete the consumer

For a graceful shutdown of your application, delete the consumer with the Delete Consumer endpoint.

To delete the consumer:

1. Expand the Consumers and Groups folder and select Delete Consumer.
2. Click Send.

Response

A 204 No Content code is returned, indicating that the consumer was deleted successfully.

Consumer lifecycle

Consumer lifecycle for a consumer instance

The Quick Start section provides a very simple use case to introduce the main concepts and steps required when using the Kafka API.

This section describes recommended practices for using the API in a real-life scenario.

The example in this use case assumes:

• Two partitions: 0 and 1
• Two consumers: c1 and c2
• One consumer group: g1

1. Create a consumer instance and specify:
   • A new consumer instance (c1)
   • A consumer group (g1)
   • Parameter auto.offset.reset: earliest
     This will ensure that the new consumer instance will start reading logs from the beginning of the partition. Note that this parameter is ignored if the group already exists and an offset was committed.
2. Get the list of available partitions.
   This will determine the number of consumer instances that need to be created. Nuance recommends that the number of consumer instances that you create should be less than or equal to the number of available partitions in a topic.
3. Call the Assign Partitions endpoint to manually assign partition 0 to consumer instance c1.
4. Call the Get Records endpoint in a loop to fetch the records for consumer c1, but do not process them at this point to avoid consumer performance issues.
5. Call the Get Assigned Partitions endpoint to get the list of partitions assigned to consumer c1.
6. Call the Commit Offsets endpoint to commit the offset for the last fetched record.
7. Create a consumer instance as described in step 1, using c2 as the consumer instance.
   Creating and managing consumer instance c2 can be done in parallel with creating and managing consumer instance c1.
8. Call the Assign Partitions endpoint to manually assign partition 1 to consumer instance c2.
9. Call the Get Records endpoint in a loop to fetch the records for consumer c2, as described in step 4.
10. When completed, do a graceful shutdown of the consumers:
    1. Delete consumer c1.
    2. Delete consumer c2.

Retarting consumers

To restart consumers, repeat all the steps above. The consumers will read new records from the last commit offsets.

Recommendations

Creating clients

Nuance recommends that you create a client that is dedicated to collecting logs. In addition, if multiple organizations need to access event logs for the same app ID, Nuance recommends that you create one client per organization.

For example, let's say that two different groups in your company (research and professional services) need to access event logs. Create two different clients in the Manage Tab, one for each group, as follows:

For more information on creating clients, see Create a client ID .

Creating customer groups

If the same client must be shared between multiple organizations, Nuance recommends that you create different client groups, one per organization. Offsets are attached to a consumer group, so creating different groups will ensure that committing offsets will not impact other users.

Reusing access tokens

You must reuse the same access token across the different consumer processes. Do not acquire a new token at each API call. The same token can be used between the Kafka and AFSS APIs.

For example, if you are running 20 consumer processes for a specific app ID, generate a single access token and share it across the 20 consumers.

Nuance also recommends that you use the token until it expires. The expected token expiration validity period is returned in the token response.

Please note that the application may be limited to a certain maximum number of access token requests over a period of time.

Number of consumer instances

The number of consumer instances that you create should be less than or equal to the number of available partitions in a topic.

Nuance recommends that, for production applications, you use one consumer instance per partition.

Retention period

Messages

The retention period for messages is 28 days. To avoid any loss of data, consumers should make sure that the data is consumed within the retention period.

Consumers should use the same consumer group to maintain the offsets (all offsets are attached to the consumer group).

Audio files

The retention period for audio files is 14 days.

Graceful shutdown of consumers

Consumers should call the Delete Consumer and Unsubscribe endpoints for a graceful shutdown of the consumer.

Handling incoming event logs

Event logs are generated as your customers use your application; as such, they are often generated in waves and usually consist of a combination of peaks and slower periods.

Nuance recommends that you write your client so that it pulls data continuously; however, keep in mind that the client may not always be able to consume logs at the same rate as they are being produced. For example, a client may keep up with data collection during off-peak hours, fall behind in peak hours, and then catch up again in off-peak hours.

If your peak periods are so busy that your client cannot keep up, Nuance does not recommend that you use the log API for real-time monitoring.

Troubleshooting

This section describes some of the most common issues when working with the Kafka API.

117: Invalid consumer name

Error message

{
    "error_code": 117,
    "message": "Invalid consumer name"
}

The consumer name specified is not valid.

See Consumers for more information.

121: Invalid client name

Error message

{
    "error_code": 121,
    "message": "Invalid client name."
}

The client name specified in the consumer group name is not valid.

See Consumer groups for more information.

401: Unauthorized

Error message

{
  "error":
  {
    "code": 401,
    "status": "Unauthorized",
    "message": "The request could not be authorized"
  }
}

The credentials specified are incorrect or expired.

Make sure that you have generated the access token and that you are providing the credentials as described in Authorization. Note that the token needs to be regenerated regularly. See Access token lifetime for details.

40403: Consumer instance not found

Error message

{
    "error_code": 40403,
    "message": "Consumer instance not found."
}

The default inactivity time for a consumer instance is 2 minutes. If the consumer is not active for more than 2 minutes, the consumer is deleted. This ensures that customer instances that are not used do not take server resources.

To solve this issue:

• Keep the consumer instance alive by using the Get Records endpoint at regular intervals (using a loop is recommended, as described in Consumer lifecycle)
• Recreate the consumer

40903: Illegal state: No current assignment for partition

Error message

{
  "error_code": 40903,
  "message": "Illegal state: No current assignment for partition ..."
}

This error indicates that the consumer is trying to access a partition that is not assigned to it.

To solve the issue, call the Get Assigned Partitions endpoint to get the list of partitions assigned to the consumer instance.

40903: Illegal state: Subscription to topics, partitions and pattern are mutually exclusive

Error message

{
  "error_code": 40903,
  "message": "Illegal state: Subscription to topics, partitions and pattern are mutually exclusive"
}

This error indicates that both the Subscribe endpoint and the Assign Partitions endpoint were used to assign partitions for the consumer.

Use either the Subscribe endpoint (to assign partitions automatically) or the Assign Partitions endpoint (to manually assign partitions).

Consumer gets stuck or times out

This may occur if the consumer instance is not deleted during shutdown. A consumer instance remains alive in the server until the consumer times out. In this scenario, the orphan consumer instance keeps a partition blocked until the timeout period is reached and the consumer is deleted. Any new consumer created during this period may get stuck since the partition is blocked by the orphan consumer.

Make sure to manage the lifecycle of consumer instances as recommended in Consumer lifecycle.

About payloads

Each event log message has the same top-level structure, which is based on standard Kafka messages.

The top-level value field contains high-level information about the event that generated the event log and follows the CloudEvents - Version 1.0 specification.

In particular, it includes the data field, which provides the actual contents of the payloads. This field includes a set of standard fields, which are available for all payloads, as well as a set of service-specific fields.

This section first describes the top-level structure of the event logs. The remainder of this document describes the specifics of each service payloads.

Top-level Kafka structure

Top-level Kafka structure

[{
        "topic": "DEMO-OMNICHANNEL-APP-DEV",
        "key": {
            "service": "DLGaaS",
            "id": "ca6e4281-47c1-4cc8-a3e2-48d0bcda29de"
        },
        "value": {
          // Contents of message. See Value field.
        },
        "partition": 0,
        "offset": 1057
    }
]

All Kafka messages have the following top-level structure:

Field	Description
topic	Name of the topic. In Mix, this is the app ID.
key	Key for this message. Messages with the same key go to the same partition in the topic. The key includes two fields: service: Name of the runtime service that generated this message. Valid values are: TTSaaS, ASRaaS, NLUaaS, DLGaaS. id: Randomly-generated UUID that uniquely identifies the message.
value	Contents of the Kafka message. See Value field.
partition	Partition number for this topic. See Partitions.
offset	Integer number representing the last record that Kafka has sent to a consumer. See Using offsets.

Value field

Sample value field structure

...
"value": {
    "specversion": "1.0",
    "service": "DLGaaS",
    "source": "nuance.dlg.v1.DialogService/Start",
    "type": "Start",
    "id": "892e1679-065d-4bdd-8112-ff12f64e70cf",
    "timestamp": "2021-03-15T14:39:00.393Z",
    "appid": "DEMO-OMNICHANNEL-APP-DEV",
    "datacontenttype": "application/json",
    "data": {
        ...
    }
},
...

The top-level value field contains high-level information about the event that generated the event log. It follows the CloudEvents - Version 1.0 specification and has the following structure:

Field	Description
specversion	Version of the specification.
service	Name of the runtime service that generated this message. Valid values are: TTSaaS, ASRaaS, NLUaaS, DLGaaS.
source	Context (endpoint) in which the event was generated. Possible values are service-specific. See Source and type fields.
type	Type of the event. Possible values depend on the source. See Source and type fields.
id	Randomly-generated UUID that identifies the event. The combination of the source and id fields is unique for each distinct event.
timestamp	Timestamp of when the event occurred.
appid	App ID for this event.
datacontenttype	Content type of data field. Always application/json.
data	Contents of the payload. See Data field.

Source and type fields

The source field specifies the context in which the event was generated.

Service	source field	type field
ASRaaS	nuance.asr.v1.Recognizer/Recognize	Recognize
NLUaaS	nuance.nlu.v1.Runtime/Interpret	Interpret
DLGaaS	nuance.dlg.v1.DialogService/Start	Start
DLGaaS	nuance.dlg.v1.DialogService/Status	Status
DLGaaS	nuance.dlg.v1.DialogService/Update	Update
DLGaaS	nuance.dlg.v1.DialogService/Execute	Execute
DLGaaS	nuance.dlg.v1.DialogService/ExecuteStream	ExecuteStream
DLGaaS	nuance.dlg.v1.DialogService/Stop	Stop
DLGaaS	NIIEventLogger	NIIEventLog
TTSaaS	nuance.tts.v1.Synthesizer/Synthesize	Synthesize
TTSaaS	nuance.tts.v1.Synthesizer/UnarySynthesize	Synthesize

Data field

"data": {
  "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
  "traceid": "547d2679e93eb9631fca96e1adf36dd1",
  "requestid": "f30758a8-536f-9239-ad31-8cce730eeeb0",
  "userid": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "locale": "en-US",
  "clientData": { // for TTS and NII logs
    "company": "Aardvark",
    "location": "Montreal"
  },
  "processingTime": {
    "startTime": "2021-03-15T14:40:59.196Z",
    "durationMs": 70
  },
  "request": {
    // Service-specific fields
    "clientData": { // for ASR, DLG, and NLU logs
      "company": "Aardvark",
      "location": "Montreal"
    }
  },
  "response": {
    "status": {
      "code": 200,
      "message": "OK",
      "details": ""
    }
    // Service-specific fields
  }
}

The data field specifies the actual content of the payload. It includes a set of standard fields, which are available for all payloads, as well as a set of service-specific fields, which depend on the value of the dataContentType field.

The following table lists the standard fields:

Field	Description
dataContentType	Defines the type of content that will be available in the data field. See dataContentType field.
traceid	Currently not used.
requestid	Nuance-generated UUID that identifies an entire API request through all the services. This field can be used for debugging and to correlate all the event logs that relate to a specific interaction. See Correlate logs for a specific API interaction.
clientRequestid	Client-provided ID that identifies an entire API request through all the services. See Client-provided request ID.
userid	A combination of the user_id field provided in the API and the appid field. (For example, for DLGaaS, the user_id is provided as part of the StartRequest.) To generate the userid value, the appid and user_id fields are concatenated and separated by a colon, as follows: appid:user_id. This value is then hashed using SHA256 and provided as a 64-character hexadecimal string. See User ID.
locale	Locale used for this interaction.
processingTime	Included in the final response payload, when duration can be calculated. Includes the following fields: startTime: Time that the request is received. firstAudioBufferTime: Time that the first audio buffer is received. Applies to TTSaaS only. durationMs: Duration of the request in milliseconds. The duration is defined as the number of milliseconds between the time that the request has arrived and the time that the response it sent. It includes the time required by the other invoked services, such as when the dialog engine invokes NLU.
clientData	Contents of the client_data map provided in the API. See Client data. For TTSaaS and NII logs, this is provided at the top level of the data field. For DLGaaS, NLUaaS, and ASRaaS, the client_data field is provided in the request object.
request	Contents of the request, if applicable. Value is service-specific.
response	Contents of the response, if applicable. Can contain a status code, message, and details. Value is service-specific.

dataContentType field

The dataContentType field specifies the content type of the data field. In particular, it determines which additional service-specific fields are included in the payload.

The dataContentType value has the following structure:

application/<i>contenttype</i>.<i>version</i>+json

Where:

• contenttype defines the type of content that will be available in the event log
• version specifies the version of the specification for this data content type; see About payload versions

The following table links to the sections that document each dataContentType payload:

Service	Field	See
ASRaaS	application/x-nuance-asr-callsummary.v2+json	asr-callsummary payload
	application/x-nuance-asr-finalresultresponse.v2+json	asr-finalresultresponse payload
	application/x-nuance-asr-finalstatusresponse.v2+json	asr-finalstatusresponse payload
	application/x-nuance-asr-partialresultresponse.v2+json	asr-partialresultresponse payload
	application/x-nuance-asr-recognitioninitmessage.v2+json	asr-recognitioninitmessage payload
	application/x-nuance-asr-startspeechresponse.v2+json	asr-startspeechresponse payload
	application/x-nuance-asr-statusresponse.v2+json	asr-statusresponse payload
NLUaaS	application/x-nuance-nluaas-interpretation.v1+json	nluaas-interpretation payload
DLGaaS	application/x-nuance-dlg-interaction-summary.v1+json	dlg-interaction-summary payload
	application/x-nuance-dlg-nii-logs.v1+json	dlg-nii-logs payload
TTSaaS	application/x-nuance-tts-callsummary.v3+json	tts-callsummary

About payload versions

The version number (for example, v1 in application/x-nuance-dlg-nii-logs.v1+json) specifies the version of the specification for this data content type. Non-breaking changes may be applied to the specification without updating the version number. Non-breaking changes include updates such as adding fields, adding new events, adding new values, and so on.

The client application that consumes the payloads should take this into account and should not assume that the structure and content are fixed.

Should a breaking change occur (for example, an event or a field is no longer available), then the version number will be increased to indicate this change.

ASR payloads

A typical ASR interaction includes a subset or all of the following events:

• An application/x-nuance-asr-recognitioninitmessage event is first sent and includes the content of the RecognitionInitMessage, which provides the parameters and resources for recognition.
• The application/x-nuance-asr-statusresponse event is usually sent after the recognitionInitMessage event is received. It provides the content of the 100 Continue status message that is sent back to the client to indicate that recognition parameters and resources were accepted and successfully configured.
• The application/x-nuance-asr-startspeechresponse event indicates the beginning of audio as detected by Krypton.
• If partial results were requested, a series of application/x-nuance-asr-partialresultresponse events are sent, one for each partial recognition result.
• The application/x-nuance-asr-finalresultresponse event provides the final recognition result.
• The application/x-nuance-asr-finalstatusresponse event is usually sent after the finalresultresponse event is received. It provides the content of the 200 Success status message that is sent back to the client to indicate that audio was processed, recognition completed, and a result returned with at least one hypothesis.
• Finally, the application/x-nuance-asr-callsummary event is sent. It provides a summary of the ASR interaction (full recognition turn), including request parameters, results, statistics, and internal events.

The code samples in this section provide all the event logs generated for recognizing the following utterance:

"My credit card number is 3141592653589793"

asr-recognitioninitmessage payload

Sample payload

"data": {
  "request": {
    "recognitionInitMessage": {
      "parameters": {
        "language": "eng-USA",
        "topic": "NTEFAST",
        "audioFormat": {
          "pcm": {
            "sampleRateHz": 16000
          }
        },
        "recognitionFlags": {
          "autoPunctuate": false,
          "filterProfanity": false,
          "suppressCallRecording": false
        }
      },
      "clientData": {
        "applicationName": "appID:test_user_nuance_com_test3:geo:us:clientName:default"
      }
    }
  },
  "asrSessionid": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
  "dataContentType": "application/x-nuance-asr-recognitioninitmessage.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212",
  "audioUrn": "urn:nuance-mix:log:service:audio/asraas/31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c"
}

Current version: v2

The application/x-nuance-asr-recognitioninitmessage payloads provide the content of the RecognitionInitMessage sent in the ASRaaS RecognitionRequest. This message specifies the parameters and resources for recognition.

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-recognitioninitmessage dataContentType include the following fields:

Field	Description
request	Contains the RecognitionInitMessage payload
asrSessionid	Identifier of the current ASR session
audioUrn	URN of the ASR resource. This field is helpful when downloading audio files of the utterances that were sent for recognition to ASRaaS. See AFSS API for details.

asr-statusresponse payload

Sample payload

"data": {
  "response": {
    "status": {
      "code": 100,
      "message": "Continue",
      "details": "recognition started on audio/l16;rate=16000 stream"
    }
  },
  "asrSessionid": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
  "dataContentType": "application/x-nuance-asr-statusresponse.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Current version: v2

The application/x-nuance-asr-statusresponse payloads provide the content of the 100 Continue status message that is sent back to the client to indicate that recognition parameters and resources were accepted and successfully configured.

This event is usually sent after the recognitionInitMessage event is received and also indicates that recognition has started.

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-statusresponse dataContentType include the following fields:

Field	Description
response	Contains the 100 Continue status code. See Status messages and codes in the ASRaaS documentation for details.
asrSessionid	Identifier of the current ASR session

asr-startspeechresponse payload

Sample payload

"data": {
  "response": {
    "startOfSpeech": {
      "firstAudioToStartOfSpeechMs": 360
    }
  },
  "asrSessionid": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
  "dataContentType": "application/x-nuance-asr-startspeechresponse.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Current version: v2

The application/x-nuance-asr-startspeechresponse payloads indicate the beginning of audio as detected by Krypton. This event is triggered only when the recognition parameter enableUtteranceDetection is true (the default).

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-startspeechresponse dataContentType include the following fields:

Field	Description
response	Contains the StartOfSpeech payload
asrSessionid	Identifier of the current ASR session

asr-partialresultresponse payload

Sample payload

"data":
{
  "response":
  {
    "result":
    {
      "resultType": 1,
      "absStartMs": 0,
      "absEndMs": 1290,
      "dataPack":
      {
        "language": "ara-XWW",
        "topic": "GEN",
        "version": "4.2.0"
      },
      "hypotheses": [
        {
          "rejected": false,
          "formattedText": "كل سنه وانت طيب",
          "minimallyFormattedText": "كل سنه وانت طيب",
          "words": [
            {
              "text": "كل",
              "startMs": 80,
              "endMs": 240,
              "silenceAfterWordMs": 0
            },
            {
              "text": "سنه",
              "startMs": 240,
              "endMs": 640,
              "silenceAfterWordMs": 0
            },
            {
              "text": "وانت",
              "startMs": 640,
              "endMs": 860,
              "silenceAfterWordMs": 0
            },
            {
              "text": "طيب",
              "startMs": 860,
              "endMs": 1200,
              "silenceAfterWordMs": 0
            }
          ]
        }
      ]
    }
  },
  "asrSessionid": "6cdfce4b-76c6-41c1-9dad-76500323ffcb",
  "dataContentType": "application/x-nuance-asr-partialresultresponse.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Current version: v2

The application/x-nuance-asr-partialresultresponse payloads provide the partial recognition results.

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-partialresultresponse dataContentType include the following fields:

Field	Description
response	Contains the Result payload for the partial result
redactedReason	If content was redacted, provides the reason. For example, the "suppressCallRecording" value indicates that the suppress_call_recording RecognitionFlags field was set to true. See ASR values possibly masked for details.
asrSessionid	Identifier of the current ASR session

asr-finalresultresponse payload

Sample payload: Example 1

"data": {
  "response": {
    "result": {
      "resultType": 0,
      "absStartMs": 360,
      "absEndMs": 9820,
      "utteranceInfo": {
        "durationMs": 9460,
        "clippingDurationMs": 0,
        "droppedSpeechPackets": 0,
        "dsp": {
          "snrEstimate": 27,
          "level": 16897,
          "stereo": false,
          "initialSilenceMs": 380,
          "initialEnergy": -70.5528,
          "finalEnergy": -63.327,
          "meanEnergy": 151.079
        }
      },
      "hypotheses": [{
          "rejected": false,
          "formattedText": "My credit card number is 3141592653589793",
          "minimallyFormattedText": "My credit card number is three one four one five nine two six five three five eight nine seven nine three",
          "averageConfidence": 489,
          "words": [{
              "text": "My",
              "confidence": 500,
              "startMs": 380,
              "endMs": 500,
              "silenceAfterWordMs": 0
            }, {
              "text": "credit",
              "confidence": 500,
              "startMs": 500,
              "endMs": 960,
              "silenceAfterWordMs": 0
            }, {
              "text": "card",
              "confidence": 500,
              "startMs": 960,
              "endMs": 1200,
              "silenceAfterWordMs": 0
            }, {
              "text": "number",
              "confidence": 500,
              "startMs": 1200,
              "endMs": 1520,
              "silenceAfterWordMs": 0
            }, {
              "text": "is",
              "confidence": 500,
              "startMs": 1520,
              "endMs": 1740,
              "silenceAfterWordMs": 600
            }, {
              "text": "3141592653589793",
              "confidence": 367,
              "startMs": 2340,
              "endMs": 8820,
              "silenceAfterWordMs": 0
            }
          ]
        }
      ]
    }
  },
  "asrSessionid": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
  "dataContentType": "application/x-nuance-asr-finalresultresponse.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Sample payload: Example 2-Call recording suppressed

"data":
{
  "response":
  {
    "result":
    {
      "resultType": 0,
      "absStartMs": 980,
      "absEndMs": 8760,
      "utteranceInfo":
      {
        "durationMs": 7780,
        "clippingDurationMs": 0,
        "droppedSpeechPackets": 0,
        "dsp":
        {
          "snrEstimateDb": 20,
          "level": 20481,
          "numChannels": 1,
          "initialSilenceMs": 220,
          "initialEnergy": -60.0424,
          "finalEnergy": -51.0569,
          "meanEnergy": 118.718
        }
      },
      "dataPack":
      {
        "language": "eng-GBR",
        "topic": "GEN",
        "version": "4.1.0",
        "id": "GMT20200227064218"
      },
      "hypotheses": [],
      "redactedReason": "suppressCallRecording"
    }
  },
  "asrSessionid": "a3134271-8d79-450d-9b2e-9bd53e196056",
  "dataContentType": "application/x-nuance-asr-finalresultresponse.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Current version: v2

The application/x-nuance-asr-finalresultresponse payloads provide the final recognition result.

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-finalresultresponse dataContentType include the following fields:

Field	Description
response	Contains the Result payload for the final result
redactedReason	If content was redacted, provides the reason. For example, the "suppressCallRecording" value indicates that the suppress_call_recording RecognitionFlags field was set to true. See ASR values possibly masked for details.
asrSessionid	Identifier of the current ASR session

asr-finalstatusresponse payload

Sample payload

"data": {
  "response": {
    "status": {
      "code": 200,
      "message": "Success"
    }
  },
  "processingTime": {
    "startTime": "2020-12-05T13:15:56.176Z",
    "durationMs": 10101
  },
  "asrSessionid": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
  "audioDurationMs": 9940,
  "dataContentType": "application/x-nuance-asr-finalstatusresponse.v2+json",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Current version: v2

The application/x-nuance-asr-finalstatusresponse payloads provide the content of the 200 Success status message that is sent back to the client to indicate that audio was processed, recognition completed, and a result returned with at least one hypothesis.

This event is usually sent after the finalresultresponse event is received.

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-finalstatusresponse dataContentType include the following fields:

Field	Description
response	Contains the 200 Success status code. See Status messages and codes for details.
asrSessionid	Identifier of the current ASR session
audioDurationMs	Audio duration in milliseconds

asr-callsummary payload

Sample payload

"data": {
  "dataContentType": "application/x-nuance-asr-callsummary.v2+json",
  "callsummary": {
    "sessionId": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
    "clientData": {
      "applicationName": "appID:test_user_nuance_com_test3:geo:us:clientName:default"
    },
    "serverData": {
      "hostname": "ntefast-eng-usa-410-krypton-bdd85d4ff-6nnm8",
      "platform": "linux",
      "arch": "x64",
      "versions": {
        "Krypton": "4.6.0-202011042154",
        "S3": "12.24.100.04509",
        "MRec": "1.42.100.19403",
        "TextProc": "15.07.100.02109",
        "S3Config": "4.12.0"
      }
    },
    "absStartTime": 1607174156199,
    "dataPack": {
      "language": "eng-USA",
      "topic": "NTEFAST",
      "version": "4.1.0",
      "id": "ndp40-ntefast-eng-USA-r1"
    },
    "recognitionObjects": [],
    "Recognize": [{
        "absStartTime": 1607174156203,
        "recognitionParameters": {
          "autoPunctuation": false,
          "enableNBestOnRejection": true,
          "enableSpeakerProfileUpdate": true,
          "enableUtteranceDetection": true,
          "audioFormat": "audio/l16;rate=16000",
          "autoEnd": true,
          "enablePartialResults": false,
          "immutablePartialResults": false,
          "startRecognitionTimers": true,
          "includeTokenization": false,
          "enableProfanityFilter": false,
          "suppressCallRecording": false,
          "confidenceFloatRepresentation": false,
          "suppressInitialCapitalization": false,
          "allowZeroBaseLmWeight": false
        },
        "audioPacketStats": {
          "packetsReceived": 497,
          "bytesReceived": 318080,
          "minimumSize": 640,
          "maximumSize": 640,
          "firstPacketTime": "2020-12-05T13:15:56.279Z",
          "lastPacketTime": "2020-12-05T13:16:06.208Z",
          "audioDurationMs": 9940
        },
        "recognitionObjects": [],
        "utterances": [{
            "nBest": [{
                "formattedText": "My credit card number is 3141592653589793",
                "minimallyFormattedText": "My credit card number is three one four one five nine two six five three five eight nine seven nine three",
                "IP!tokens": "yZS...bA=",
                "IP!prepostprocessing_tokens": "29uzyWVLBE4VrANT7g....HAw=="
              }
            ],
            "info": {
              "duration": 9460,
              "stereo": false,
              "level": 16897,
              "clippingDuration": 0,
              "snrEstimate": 27,
              "droppedSpeechPackets": 0,
              "droppedNonSpeechPackets": 0,
              "initialSilence": 380,
              "initialEnergy": -70.5528,
              "finalEnergy": -63.327,
              "meanEnergy": 151.079
            },
            "relStartTime": 360,
            "relEndTime": 9820,
            "completionCause": "SUCCESS"
          }
        ],
        "IP!S3Configuration": "QPh...OgSbjw==",
        "metrics": {
          "firstAudioToStartOfSpeech": 743,
          "firstAudioToResult": 9931
        },
        "absEndTime": 1607174166228,
        "IP!S3Events": "L4sFrmz..so+bzPlqw==",
        "audioDump": "http://minio-prod.fabric-prod.svc.cluster.local:9000/afss/da268859842ef28cd22f7b277e9c755538e9d70e45f2b8d785128afecd26affa/asraas/31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
        "audioUrn": "urn:nuance-mix:log:service:audio/asraas/31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c"
      }
    ],
    "IPKey": "tk.....KHLNp0hFA==",
    "absEndTime": 1607174166275,
    "IP!S3Events": "L4s...jZs="
  },
  "processingTime": {
    "startTime": "2020-12-05T13:15:56.176Z",
    "durationMs": 10101
  },
  "asrSessionid": "31bd3cb7-aaa0-4b22-a8ee-6a5993dbd50c",
  "requestid": "3136437b-f95b-4a37-8c18-5c6ee836e212"
}

Current version: v2

The application/x-nuance-asr-callsummary payloads provide summary information about the ASR interaction (full recognition turn), including request parameters, results, statistics, and internal events.

In addition to the standard fields described in Data field, messages with the application/x-nuance-asr-callsummary dataContentType include the following fields:

Field	Description
serverData	Details about the Krypton server. See serverData for details.
absStartTime	Audio stream start time
dataPack	Information about the current data pack. See DataPack in the ASRaaS documentation for details.
recognitionObjects	List of recognition objects (wordsets and domain LMs) used during the session. See recognitionObjects for details.
Recognize	Information about the Recognize command. See Recognize for details.
IPKey	RSA-encrypted key used to encrypt IP data (S3Configuration, S3Events, and tokens)
absEndTime	Audio stream end time
IP!S3Events	S3 events received during the session. Encrypted.

serverData

Details about the Krypton server.

Field	Description
hostname	Name of the machine running the Krypton server
platform	Software platform of the Krypton server
arch	Software architecture of the Krypton server
versions	Versions of the different components:
	Krypton	Krypton software version
	S3	S3 library version
	MRec	MRec library version
	TextProc	TextProc library version
	S3Config	S3Config library version

RecognitionObjects

List of recognition objects (wordsets and domain LMs) used during the session.

Field	Description
id	ID of the recognition object, as provided by the client
S3Id	ID correlating the related S3 event
type	Type of recognition object (DLM/wordset)
weight	Weight value provided by the client application
url	URL of the object

Recognize

Information about the Recognize command.

Field	Description
absStartTime	Audio stream start time
recognitionParameters	Recognition parameters and values used for the session. Some of these values can be configured in the ASRaaS RecognitionInitMessage message. The other values are server defaults.
	autoPunctuation	Whether to enable auto punctuation
	enableSpeakerProfileUpdate	If speaker profiles are used in the session, whether updated speaker adaptation data is stored after the session
	enableNBestOnRejection	Whether N-best results are returned (true) or discarded (false) when recognition ends with a rejection (also known as “no-match”)
	enableUtteranceDetection	Whether to activate the internal utterance detection mechanism
	audioFormat	Incoming audio mime type
	autoEnd	Whether the recognition turn ends when Krypton detects the end of an utterance
	enablePartialResults	Whether to enable partial results
	immutablePartialResults	When partial results are enabled (see enablePartialResults above), whether to return stable partial results
	startRecognitionTimers	Whether timers are activated when a recognition turn starts
	includeTokenization	Whether to include tokenized results
	enableProfanityFilter	Whether to filter profanities and other unacceptable language from transcriptions
	suppressCallRecording	Whether call recording is disabled for the recognition turn
	confidenceFloatRepresentation	Nuance internal
	utteranceEndSilence	Minimum amount of silence (time designation) that causes an utterance to end
AudioPacketStats	Provides statistics about the audio packets received
	packetsReceived	Number of audio packet received by ASRaaS
	bytesReceived	Total audio bytes received
	minimumSize	Size (in bytes) of the smallest audio packet received
	maximumSize	Size (in bytes) of the largest audio packet received
	firstPacketTime	Date and time the first audio packet was received
	lastPacketTime	Date and time the last audio packet was received
	audioDurationMs	The duration of the audio received minus begin and end silence periods as detected by ASRaaS
RecognitionObjects	List of recognition objects (wordsets and domain LMs) used for recognition. See RecognitionObjects for a description of the fields.
nBest	List of n-best results
info	Information about the recognized sentence in the transcription result. See UtteranceInfo for a description of each field.
relStartTime	Start time, relative to Recognize.absStartTime, computed on audio
relEndTime	End time, relative to relative to Recognize.absStartTime, computed on audio
completionCause	In response to Recognize, the reason for the response: SUCCESS: The result was correctly produced. ERROR: An error prevented the result from being computed. NO-INPUT-TIMEOUT: The noInputTimeout interval has elapsed: no result is delivered. RECOGNITION-TIMEOUT: The recognitionTimeout interval elapsed: the result may or may not be delivered. END-OF-STREAM: An EndOfInput command terminated the Recognize turn: the result may or may not be delivered. NO-MATCH: The result has been produced, but is marked as rejected. This completion cause is returned for extremely noisy and/or unintelligible audio, or when the audio is not compatible with the declared audio format.
IP!S3Configuration	S3 parameters provided by krypton to S3. Encrypted with an IPKey.
metrics	Metrics for the call; available if enabled in configuration
absEndTime	Audio stream end time
IP!S3Events	S3 events received during the call. Encrypted.
audioDump	URL of the audio recording
audioUrn	URN of the audio file for the utterance. This field can be used to download the audio file with the AFSS API. See AFSS API for details.

nBest

List of n-best results.

Field	Description
confidence	The confidence score for the entire transcription, 0 to 1
PCI!formattedText	Formatted text of the result, for example, $500. Encrypted with a PCIKey.
PCI!minimallyFormattedText	Slightly formatted text of the result, for example, Five hundred dollars. Encrypted with a PCIKey.
IP!tokens	Nuance-internal representation of the recognition result. Encrypted with an IPKey.

metrics

Metrics for the call; available if enabled in configuration.

Field	Description
firstAudioToStartOfSpeech	The number of milliseconds from the arrival of the first audio message/binary packet after Recognize to transmission of the StartOfSpeech event
firstAudioToResult	The number of milliseconds from the arrival of the first audio message/binary packet after Recognize to the current time

NLU payload

The x-nuance-nluaas-interpretation payload provides an overview of the interactions that occur in an NLUaaS interpretation session.

nluaas-interpretation payload

Sample payload: Single-intent interpretation

"data": {
  "dataContentType": "application/x-nuance-nluaas-interpretation.v1+json",
  "locale": "en-US",
  "traceid": "441249de36ff9f682f69a63064e0b125",
  "request": {
    "resources": [],
    "clientData": {
      "x-nuance-dialog-session-id": "8baf36f2-886d-468d-9e4c-057fdbde66e4",
      "x-nuance-dialog-seqid": "15"
    },
    "parameters": {
      "postProcessingScriptParameters": {},
      "interpretationResultType": "SINGLE_INTENT",
      "interpretationInputLoggingMode": "PLAINTEXT",
      "maxInterpretations": 0
    },
    "model": {
      "type": "SEMANTIC_MODEL",
      "uri": "urn:nuance:mix/eng-USA/TestBotInterface/mix.nlu",
      "requestTimeoutMs": 0
    },
    "userId": "",
    "input": {
      "text": "I want a double espresso",
      "inputUnion": "text"
    }
  },
  "processingTime": {
    "startTime": "2021-03-22T15:13:00.285Z",
    "durationMs": 127
  },
  "response": {
    "status": {
      "code": 200,
      "message": "OK",
      "details": ""
    },
    "result": {
      "interpretations": [{
          "singleIntentInterpretation": {
            "entities": {
              "COFFEE_TYPE": {
                "entities": [{
                    "entities": {},
                    "textRange": {
                      "startIndex": 16,
                      "endIndex": 24
                    },
                    "confidence": 1,
                    "origin": "GRAMMAR",
                    "literal": "espresso",
                    "sensitive": false,
                    "audioRange": null,
                    "stringValue": "espresso",
                    "valueUnion": "stringValue"
                  }
                ]
              },
              "COFFEE_SIZE": {
                "entities": [{
                    "entities": {},
                    "textRange": {
                      "startIndex": 9,
                      "endIndex": 15
                    },
                    "confidence": 1,
                    "origin": "GRAMMAR",
                    "literal": "double",
                    "sensitive": false,
                    "audioRange": null,
                    "stringValue": "lg",
                    "valueUnion": "stringValue"
                  }
                ]
              }
            },
            "intent": "ORDER_COFFEE",
            "confidence": 1,
            "origin": "GRAMMAR"
          },
          "interpretationUnion": "singleIntentInterpretation"
        }
      ],
      "literal": "I want a double espresso",
      "sensitive": false
    }
  }
}

Current version: v1

Messages with the application/x-nuance-nluaas-interpretation.v1+json dataContentType contain no additional service-specific fields beyond those described in Data field. Note, however, that for NLUaaS and ASRaaS the clientData field is part of the request object.

NLUaaS request and response objects

The request object describes the input, as well as the model, used for the interpretation. Parameters, extra resources, client data, and user ID may also be included to customize the interpretation. See InterpretRequest for details.

The input to interpret is either:

• Plain text (inputUnion: "text"), as shown in the example at the right
• Result from ASRaaS ("inputUnion": "asrResult") as shown below, with the value encrypted (truncated in the example for brevity).
  

"input":
    {
      "asrResult":
      {
        "type_url": "type.googleapis.com/nuance.asr.v1.Result",
        "value":
        {
          "type": "Buffer",
          "data": [16, 210, ..., 46, 48]
        }
      },
      "inputUnion": "asrResult"
    }

The response object, in addition to providing a status message, describes the interpretation result, both the input (literal) and the candidate interpretations of the input. For more example payloads, see InterpretResult.

Notice that the first child under interpretations specifies the result type (type of interpretation). For example:

"result": {
   "interpretations": [{
       "singleIntentInterpretation": {
         ...
       }
     },
     ...

There are two interpretation result types:

• Single-intent interpretation results include one or more alternative intents, complete with entities if they occur in the text. Each intent is shown with a confidence score and indicates whether the match was done from a grammar file (GRAMMAR) or a SSM or statistical semantic model (STATISTICAL) file. See SingleIntentInterpretation.

• Multi-intent interpretation results contain a tree of nodes representing the detected operators, intents, and entities and their associations. Multi-intent interpretation requires a semantic model that is enabled for multi-intent (not currently supported). Although multi-intent interpretation may be requested without error, the results contain the same information as a single-intent interpretation. You will notice some formatting differences, for example, the root of the multi-intent interpretation contains the intent. See MultiIntentInterpretation.

Dialog payloads

There are two types of dialog event logs:

• Runtime API transaction logs: These event logs provide information about the DLGaaS transactions. As such, the event logs are organized around the service requests and responses and are identified with the [application/x-nuance-dlg-interaction-summary](#dlg-interaction-summary-payload) dataContentType.
• Dialog event logs: These events are related to the behavior of the dialog application (for example, the dialog was started, input was received, a transition occurred, and so on) and are used for reporting. These messages are organized around the dialog events and are identified with the [application/x-nuance-dlg-nii-logs](#dlg-nii-logs-payload) dataContentType.

A complete dialog session contains a combination of these two types of event logs.

dlg-interaction-summary payload

Sample payload: DLGaaS-Start-End event

"data":
{
  "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
  "traceid": "1f16b4bd07b2f1f9c548acaedc487c56",
  "requestid": "01b73d0f-bfd6-9921-a160-ad037d2e6036",
  "sessionid": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
  "channel": "default",
  "sessionTimeoutSec": 900,
  "locale": "en-US",
  "processingTime":
  {
    "startTime": "2021-03-15T14:41:46.313Z",
    "durationMs": 110
  },
  "events": [
    {
      "event": "DLGaaS-Start-End",
      "time": "2021-03-15T14:41:46.424Z"
    }
  ],
  "request":
  {
    "selector":
    {
      "channel": "default",
      "language": "en-US",
      "library": "default"
    },
    "payload":
    {
      "modelRef":
      {
        "uri": "urn:nuance-mix:tag:model/A1880_C1/mix.dialog"
      },
      "data": {}
    },
    "sessionTimeoutSec": 900
  },
  "response":
  {
    "status":
    {
      "code": 200,
      "message": "OK",
      "detail": ""
    },
    "payload":
    {
      "sessionId": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13"
    }
  }
}

Sample payload: DLGaaS-ExecuteStream-End event

"data":
{
  "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
  "traceid": "362ce333dfa6d3009c716ca40ca4cfd5",
  "requestid": "d69cb4e2-15c5-9b2b-9827-45bc6ad073e8",
  "sessionid": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
  "processingTime":
  {
    "startTime": "2021-03-15T14:41:52.607Z",
    "durationMs": 6815
  },
  "events": [
    {
      "event": "DLGaaS-ExecuteStream-End",
      "time": "2021-03-15T14:41:59.423Z"
    }
  ],
  "request":
  {
    "sessionId": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
    "selector":
    {
      "channel": "default",
      "language": "en-US",
      "library": "default"
    },
    "payload": {}
  },
  "response":
  {
    "status":
    {
      "code": 200,
      "message": "OK",
      "detail": ""
    },
    "payload":
    {
      "messages": [
        {
          "visual": [
            {
              "text": "Perfect, a double espresso coming right up!"
            }
          ],
          "view": {}
        }
      ],
      "endAction":
      {
        "data": {},
        "id": "End dialog"
      }
    }
  }
}

Sample payload: DLGaaS-Stop-End event

"data":
{
  "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
  "traceid": "c04aca32c263b1a1cbf467b7182e6863",
  "requestid": "d7823d68-f9bb-9a8e-a819-66499a3df1ac",
  "sessionid": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
  "processingTime":
  {
    "startTime": "2021-03-15T14:41:59.550Z",
    "durationMs": 8
  },
  "events": [
    {
      "event": "DLGaaS-Stop-End",
      "time": "2021-03-15T14:41:59.558Z"
    }
  ],
  "request":
  {
    "sessionId": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13"
  },
  "response":
  {
    "status":
    {
      "code": 404,
      "message": "NOT_FOUND",
      "detail": "Could not find session for [c475ff51-d60c-4ad0-b1df-7fb4aae43b13]"
    }
  }
}

Current version: v1

The application/x-nuance-dlg-interaction-summary payloads provide an overview of the Runtime API interactions that occur in a DLGaaS session.

In addition to the standard fields described in Data field, messages with the application/x-nuance-dlg-interaction-summary dataContentType include the following fields:

Field	Description
sessionid	ID of the session. It is always included for all dialog events, except for DLGaaS-Start-Start. For DLGaaS-Start-Start, the session ID is included only when it is passed in the StartRequest.
events	Array of event and time values for this transaction. See table below for details.
channel	Channel used for the interaction
sessionTimeoutSec	Session timeout value (in seconds), after which the session is terminated
request	Contents of the request. See table below for details.
response	Contents of the response. See table below for details.

Note: The DLGaaS-Start-Start event will only include the sessionid when it is passed in on the Start request. If the session is not passed in, the dialog service will create one and return it in the DLGaaS-Start-End event. All other requests will include it.

Request and response objects

The request and response objects available in the event logs depend on the dialog events that generated the logs.

The following table describes the contents of the request and response objects according to the event that generated the event log:

API Method	Possible events	Contents of request	Contents of response1
Start	DLGaaS-Start-Start for StartRequest	N/A	N/A
	DLGaaS-Start-End for StartResponse	Content provided in StartRequest	StartResponse payload
Status	DLGaaS-Status-Start for StatusRequest	N/A	N/A
	DLGaaS-Status-End for StatusResponse	Content provided in StatusRequest	StatusResponse payload
Update	DLGaaS-Update-Start for UpdateRequest	N/A	N/A
	DLGaaS-Update-End for UpdateResponse	Content provided in UpdateRequest payload	N/A
Execute	DLGaaS-Execute-Start for ExecuteRequest	N/A	N/A
	DLGaaS-Execute-End for ExecuteResponse	Content provided in ExecuteRequest	ExecuteResponse payload
Execute Stream	DLGaaS-ExecuteStream-Start for ExecuteRequest in StreamInput	N/A	N/A
	DLGaaS-ExecuteStream-End for ExecuteResponse in StreamOutput	Content provided in the ExecuteRequest	ExecuteResponse payload
	ASRaaS-reco-Start when ASRaaS is invoked	N/A	N/A
	ASRaaS-reco-End when the result is received from ASRaaS	N/A	N/A
	NLUaaS-interpret-Start when NLUaaS is invoked	N/A	N/A
	NLUaaS-interpret-End when the result is received from NLUaaS	N/A	N/A
	TTSaaS-synthesize-Start when TTSaaS is invoked	N/A	N/A
	TTSaaS-synthesize-End when the result is received from TTSaaS	N/A	N/A
Stop	DLGaaS-Stop-Start for StopRequest	N/A	N/A
	DLGaaS-Stop-End for StopResponse	Content provided in StopRequest	N/A

^{1. In addition to default response content described in Data field.}

The following diagram shows at which point in a dialog interaction each event gets generated:

(Click the image for a close-up view)

dlg-nii-logs payload

Sample payload: dlg-nii-logs

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key": {
    "service": "DLGaaS",
    "id": "977f87ae-5923-4281-8e91-1f1968baea7b"
  },
  "value": {
    "specversion": "1.0",
    "service": "DLGaaS",
    "source": "NIIEventLogger",
    "type": "NIIEventLog",
    "id": "977f87ae-5923-4281-8e91-1f1968baea7b",
    "timestamp": "2021-01-19T19:06:07.113Z",
    "appid": "DEMO-OMNICHANNEL-APP-DEV",
    "datacontenttype": "application/json",
    "data": {
      "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
      "traceid": "8bbedbb2353cdc1d",
      "requestid": "2c06142d-71b7-4077-bf4b-e3892811b3d8",
      "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
      "locale": "en-US",
      "clientData": {},
      "seqid": "1",
      "events": [{
          "name": "session-start",
          "value": {
            "runtime": {
              "api": "v1"
            },
            "project": {
              "name": "TestMixClient",
              "namespace": "alex.smith@company.com",
              "deployed": "2020-12-10T19:52:25.224Z",
              "contextTag": "A1880_C1",
              "id": "8655"
            },
            "selector": {
              "channel": "Digital VA text",
              "language": "en-US"
            },
            "version": {
              "asr": {
                "en_US": "2"
              },
              "nlu": {
                "en_US": "2"
              },
              "dlg": "2"
            },
            "user": {
              "systemID": "4561 9219 9923",
              "userChannelID": "163.128.3.254",
              "userAuxiliaryID": "7319434000843499",
              "userGlobalID": "123123123",
              "location": {
                "latitude": "21.161908",
                "longitude": "-86.8515279"
              }
            },
            "timeout": 900,
            "reporting": {
              "attributes": [{
                  "name": "preferred_coffee",
                  "type": "string"
                }
              ],
              "metrics": [{
                  "name": "quantity",
                  "type": "integer"
                }
              ],
              "dimensions": [{
                  "name": "coffee_price",
                  "type": "string"
                }
              ]
            }
          }
        }
      ]
    }
  },
  "partition": 0,
  "offset": 312
}

Current version: v1

The application/x-nuance-dlg-nii-logs payloads provide detailed information about events related to the behavior of the dialog application (for example, the dialog was started, input was received, a transition occurred, and so on).

In addition to the standard fields described in Data field, messages with the application/x-nuance-dlg-nii-logs.v1+json dataContentType include the following fields:

Field	Description
sessionid	ID of the dialog session
seqid	Sequence ID of the NII log; starts at 1 and increments for each NII message in a dialog session. The seqid ensures that you can determine the sequence of the NII logs in a dialog session, since these logs may not be received in the order in which they are created and sent.
events	Global and custom events in the project; see NII log events for details. A single NII message may contain one or more events.

NII log events

NII logs can include the following global and custom events. Each event has two fields:

• name: Name of the event
• value: Contents of this event; the value field varies depending on the type of event

Event name	Description
session-start	Indicates the start of an interaction between a user and a dialog application.
selector	Provides the channel and language combination used for an interaction.
decision	Provides the next node in the dialog flow after a decision. Relates to a decision node.
transition	Indicates that the dialog flow encountered a transition.
message	Provides the contents of the message payload sent to the client application. Relates to a message node.
qa-config	Indicates that the dialog flow encountered a question and answer node.
input-required	Indicates that input is required from the user.
input-received	Indicates that input was received from the user.
intent	Indicates that an intent was set.
input-processed	Indicates that the input received from the user has been processed.
question-router	Indicates that the dialog flow encountered a question router node.
session-update	Provides session variables that were updated.
data-required	Indicates that the dialog flow encountered a data access node and requires data.
data-received	Indicates that data was received.
transfer-initiated	Indicates that an external actions node of type "Transfer" was encountered.
transfer-completed	Indicates that a transfer action is completed.
component-invoked	Indicates that a component was invoked.
component-returned	Indicates that the dialog flow has returned after having invoked a component.
event	Indicates that client application returned an event.
error	Indicates that an error occurred in the dialog flow.
reporting-vars	Indicates that a variable marked with a reporting property was set or updated.
application-ended	Indicates that the application has ended. Relates to an external actions node of type "End".
session-stopped	Indicates the client explicitly stopped an active session by sending the StopRequest message.

session-start

Sample payload: "session-start" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "8bbedbb2353cdc1d",
  "requestid": "2c06142d-71b7-4077-bf4b-e3892811b3d8",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "1",
  "events": [{
      "name": "session-start",
      "value": {
        "runtime": {
          "api": "v1"
        },
        "project": {
          "name": "TestMixClient",
          "namespace": "alex.smith@company.com",
          "deployed": "2020-12-10T19:52:25.224Z",
          "contextTag": "A1880_C1",
          "id": "8655"
        },
        "selector": {
          "channel": "Digital VA text",
          "language": "en-US"
        },
        "version": {
          "asr": {
            "en_US": "2"
          },
          "nlu": {
            "en_US": "2"
          },
          "dlg": "2"
        },
        "user": {
          "systemID": "4561 9219 9923",
          "userChannelID": "163.128.3.254",
          "userAuxiliaryID": "7319434000843499",
          "userGlobalID": "123123123",
          "location": {
            "latitude": "21.161908",
            "longitude": "-86.8515279"
          }
        },
        "timeout": 900,
        "reporting": {
          "attributes": [{
              "name": "preferred_coffee",
              "type": "string"
            }
          ],
          "metrics": [{
              "name": "quantity",
              "type": "integer"
            }
          ],
          "dimensions": [{
              "name": "coffee_price",
              "type": "string"
            }
          ]
        }
      }
    }
  ]
}

This event indicates the start of an interaction between a user and a dialog application.

Field	Description
runtime	Provides information about the runtime environment:
	api	Version of the DLGaaS API protocol
project	Provides details about the project:
	name	Name of project
	namespace	Organization for this project
	deployed	Date and time that the project was deployed
	contextTag	Context tag of the application configuration used for this session
	id	ID of project
selector	Channel and language used for this interaction:
	channel	Channel name
	language	Language for this interaction, identified by a language code
version	Versions of the ASR, NLU, and dialog builds used for this session:
	asr	Language and version of the ASR build. This is the version of the build deployed in the Mix environment.
	nlu	Language and version of the NLU build. This is the version of the build deployed in the Mix environment.
	dlg	Version of the dialog build. This is the version of the build deployed in the Mix environment.
user	userData values configured for the user; see userData schema in the Mix.dialog documentation for details
timeout	The session timeout used. This can be either the default timeout value or a value specified in the StartRequest.
reporting	List of variables that are marked with a reporting property in Mix.dialog. See Reporting properties in the Mix.dialog documentation for more information.
	attributes	Array of variables marked with a reporting property of type attribute. Values provided include name and type.
	metrics	Array of variables marked with a reporting property of type metric. Values provided include name and type.
	dimensions	Array of variables marked with a reporting property of type dimension. Values provided include name and type.

selector

Sample payload: "selector" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "752e56c6ded0f281",
  "requestid": "059f0fdc-bf24-4314-a00d-687ff65dacdd",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "3",
  "events": [{
      "name": "selector",
      "value": {
        "channel": "Digital VA text",
        "language": "en-US"
      }
    }
  ]
}

This event provides the selector used for this interaction.

Field	Description
channel	Channel used for the interaction
language	Language used in this interaction, identified by language code

decision

Sample payload: "decision" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "752e56c6ded0f281",
  "requestid": "059f0fdc-bf24-4314-a00d-687ff65dacdd",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "4",
  "events": [{
      "name": "decision",
      "value": {
        "next": "WelcomeMessage",
        "label": "default"
      }
    }
  ]
}

This event indicates that a decision was executed and specifies the next node in the dialog flow. This event relates to a decision node.

Field	Description
next	Name of the next node in the dialog flow
label	Descriptive label for the decision

transition

Sample payload: "transition" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "97acfc8f98bc0de",
  "requestid": "cf26af47-a23c-4337-92fc-c51c19f21996",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "16",
  "events": [{
      "name": "transition",
      "value": {
        "from": {
          "component": "ORDER_COFFEE",
          "name": "Confirmation",
          "type": "playprompt",
          "uuid": "6cbb037d-65a2-48a0-a4b8-1a82573f1d90"
        },
        "to": {
          "component": "Main",
          "name": "Intent Mapper",
          "type": "intentmapper",
          "uuid": "24c31003-99b5-4c91-ba3d-6b6231a249a6"
        }
      }
    }
  ]
}

This event represents a transition from a source node to a target node in the dialog flow.

Field	Description
from	Source node for this transition:
	component	Component for the source node
	name	Name of the source node
	type	Type of node; values include decision, playprompt, dataaccess, dialog, intentmapper, message, questionanswer, questionrouter, transfer, end
	uuid	UUID of the node
to	Target node for this transition:
	component	Component for the target node
	name	Name of the target node
	type	Type of node; values include decision, playprompt, dataaccess, dialog, intentmapper, message, questionanswer, questionrouter, transfer, end
	uuid	UUID of the node

message

Sample payload: "message" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "2ef850949bb32b9b25744cc7d2f79091",
  "requestid": "a8f1df04-f804-9155-9ef9-5f1376c8b4d2",
  "sessionid": "e19ae4f8-f4c2-4f4a-b8db-401e30cbda69",
  "locale": "en-US",
  "clientData": {},
  "seqid": "6",
  "events": [{
      "name": "message",
      "value": {
        "nlg": [{
            "text": "Welcome to the Mix Bank Demo."
          }
        ],
        "visual": [{
            "text": "Welcome to the Mix Bank Demo.  Rich Text VA."
          }
        ],
        "audio": [{
            "text": "Welcome to the Mix Bank Demo. ",
            "audio": "en-US/prompts/default/default/welcome_to_mix_bank_demo_.wav?version=1.0_1614834167198"
          }
        ]
      }
    }
  ]
}

This event provides the contents of the message field returned to the client application.

See Message in the DLGaaS documentation for details.

qa-config

Sample payload: "qa-config" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "362ce333dfa6d3009c716ca40ca4cfd5",
  "requestid": "d69cb4e2-15c5-9b2b-9827-45bc6ad073e8",
  "sessionid": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
  "locale": "en-US",
  "clientData": {},
  "seqid": "20",
  "events": [{
      "name": "qa-config",
      "value": {
        "confirmationMode": "NEVER",
        "role": "validation",
        "confidence": {
          "low": "0.150",
          "high": "0.550"
        },
        "name": "COFFEE_SIZE",
        "entity": {
          "confirmationMode": "NEVER",
          "confidence": "1.000",
          "name": "COFFEE_SIZE",
          "value": "lg",
          "required": "false",
          "literal": "large",
          "status": "validated"
        }
      }
    }
  ]
}

This event indicates that the dialog flow encountered a question and answer node.

Field	Description
confirmationMode	Confirmation strategy for the question and answer node. Possible values are 'ALWAYS', 'IF_NECESSARY', or 'NEVER'.
role	Specifies the current role of the question and answer node. Possible values are: collection: No values are filled so the node needs to collect them confirmation: A yes/no confirm question was asked (currently not supported) validation: Input is currently being validated
confidence	Low and high confidence thresholds used
name	Name of the question and answer node
entity	If the entity is collected, the following information is provided: confirmationMode: Confirmation strategy used for the entity. Possible values are 'ALWAYS', 'IF_NECESSARY', or 'NEVER'. confidence: Confidence level of the interpretation. name: Name of the entity. value: Value of the entity. required: Indicates whether this entity must be collected. literal: Exact words used by the user. status: Indicates the status of collecting the entity. Possible values are empty, prefilled, collected, confirmed, validated. Otherwise, the field is null. See Status field for details.

Status field

The following diagram shows how the status field value is determined:

• The empty status indicates that the slot has no value.
• The prefilled status is used when the client application has filled the entity value or the slot has been filled previously.
• The collected status indicates that the user provided input and the entity value has been collected.
• The confirmed indicates that the entity value has been confirmed; this is driven either by the dialog service or upon processing the high confidence level.
• The validated status indicates that the entity value has been confirmed; this is driven by the dialog service.

input-required

Sample payload: Simple "input-required" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "a9f419174dfe1e13d0fb70943e099892",
  "requestid": "be824d02-938e-946f-ad8f-6134f60c729a",
  "sessionid": "b15302ac-be35-4d9c-b593-3789b091e83f",
  "locale": "en-US",
  "clientData": {},
  "seqid": "11",
  "events": [{
      "name": "input-required",
      "value": {
        "data": {}
      }
    }
  ]
}

Sample payload: "input-required" event with data and interactive elements

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "752e56c6ded0f281",
  "requestid": "059f0fdc-bf24-4314-a00d-687ff65dacdd",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "11",
  "events": [{
      "name": "input-required",
      "value": {
        "data": {
          "date": "now"
        },
        "selectables": [{
            "description": "",
            "display_text": "Discover",
            "display_image_uri": "",
            "value": {
              "value": "discover",
              "id": "payee"
            }
          }, {
            "description": "",
            "display_text": "American Express",
            "display_image_uri": "",
            "value": {
              "value": "amex",
              "id": "payee"
            }
          }, {
            "description": "",
            "display_text": "Visa",
            "display_image_uri": "",
            "value": {
              "value": "visa",
              "id": "payee"
            }
          }
        ]
      }
    }
  ]
}

This event indicates that input is required from the user. This event will also include data and interactive elements configured in the question and answer node, if any.

Field	Description
data	Data sent to the client application. See Sending data in the DLGaaS documentation for details.
selectables	Interactive elements configured in Mix.dialog for this question and answer node. See Interactive elements in the DLGaaS documentation for details.

input-received

Sample payload: "input-received" event with interpretation

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "data": {
    "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
    "traceid": "97acfc8f98bc0de",
    "requestid": "cf26af47-a23c-4337-92fc-c51c19f21996",
    "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
    "locale": "en-US",
    "clientData": {},
    "seqid": "13",
    "events": [{
        "name": "input-received",
        "value": {
          "userText": "I want an espresso",
          "interpretation": {
            "isSuccess": "true",
            "confidence": "1.0",
            "inputmode": "text",
            "utterance": "I want an espresso",
            "data": {
              "COFFEE_TYPE": "espresso",
              "INTENT": "ORDER_COFFEE"
            },
            "slot_literals": {
              "COFFEE_TYPE": "espresso",
              "INTENT": "I want an espresso"
            },
            "slot_confidences": {
              "COFFEE_TYPE": "0.9515329",
              "INTENT": "1.0"
            }
          }
        }
      }
    ]
  }
}  

Sample payload: "input-received" event with selected item

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "89eba3e69ec15c0c12ab90b1901149c0",
  "requestid": "efc93c71-5046-95e0-85a9-75592a4d23f4",
  "sessionid": "e19ae4f8-f4c2-4f4a-b8db-401e30cbda69",
  "locale": "en-US",
  "clientData": {},
  "seqid": "24",
  "events": [{
      "name": "input-received",
      "value": {
        "selectedItem": {
          "id": "payee",
          "value": "discover"
        }
      }
    }
  ]
}

This event indicates that input was received from the user. This can either be an interpretation or a selected interactive element:

Field	Description
userText	Text collected from end user, if applicable. Only applies for interpretations.
interpretation	An interpretation of the input; see Interpretresult in the DLGaaS documentation for a description of the fields.
selectedItem	The item selected by the user. See Interactive elements in the DLGaaS documentation for details.

intent

Sample payload: "intent" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "97acfc8f98bc0de",
  "requestid": "cf26af47-a23c-4337-92fc-c51c19f21996",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "14",
  "events": [{
      "name": "intent",
      "value": {
        "value": "ORDER_COFFEE"
      }
    }
  ]
}

This event indicates that an intent was set.

Field	Description
value	Name of the intent that was set

input-processed

Sample payload: "input-processed" event, with result "collected"

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "97acfc8f98bc0de",
  "requestid": "cf26af47-a23c-4337-92fc-c51c19f21996",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "15",
  "events": [{
      "name": "input-processed",
      "value": {
        "result": "collected",
        "counters": {
          "nomatch": 0,
          "help": 0,
          "repeat": 0,
          "invalidRecoOption": 0,
          "noToConfirm": 0,
          "try": 0,
          "turn": 1,
          "noinput": 0
        }
      }
    }
  ]
}

Sample payload: "input-processed" event, with result "maxnoinputs"

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "c044de1e2c3ff3f58f5d4b43bdd7d077",
  "requestid": "b0c37992-8b68-9d65-9c07-c3b7d1267a88",
  "sessionid": "cad5b7e8-41fd-488e-a169-0cdd2017cad2",
  "locale": "en-US",
  "clientData": {},
  "seqid": "20",
  "events": [{
      "name": "input-processed",
      "value": {
        "result": "maxnoinputs",
        "counters": {
          "nomatch": 0,
          "help": 0,
          "repeat": 0,
          "invalidRecoOption": 0,
          "noToConfirm": 0,
          "try": 0,
          "turn": 2,
          "noinput": 2
        }
      }
    }
  ]
}

This event indicates that the input received from the user has been processed.

Field	Description
result	Indicate the results of the processing operation. Possible values are collected, maxnomatches, maxnoinputs, maxretries, maxturns, nomatch, noinput
counters	Provides the number of times the following events occurred:
	nomatch	Number of times the user's input could not be matched
	help	Number of times the user asked for help in an interaction
	repeat	Number of times the user asked the dialog to repeat the question
	invalidRecoOption	Number of times the response received was not what was expected (for example, a response was received for an entity called DATE but the response was not a date)
	noToConfirm	Number of times the user said "no" in the confirmation phase
	try	Number of recognition steps (recognition request and result) to get a piece of information
	turn	Number of turns (that is, number of recognition and confirmation steps) in the interaction
	noinput	Number of times the user did not provide an input

question-router

Sample payload: "question-router" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "97acfc8f98bc0de",
  "requestid": "cf26af47-a23c-4337-92fc-c51c19f21996",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "18",
  "events": [{
      "name": "question-router",
      "value": {
        "belief": [{
            "confirmationMode": null,
            "inputMode": null,
            "confidence": "0.952",
            "name": "COFFEE_TYPE",
            "value": "espresso",
            "required": "false",
            "literal": "espresso",
            "status": "filled"
          }, {
            "confirmationMode": null,
            "inputMode": null,
            "confidence": null,
            "name": "COFFEE_SIZE",
            "value": null,
            "required": null,
            "literal": null,
            "status": null
          }
        ]
      }
    }
  ]
}

This event indicates that the dialog flow encountered a question router node and provides the information that has been accumulated so far about the entities to collect.

Field	Description
belief	For every entity that needs to be collected, this field provides the information that has been accumulated so far
	confirmationMode	Confirmation strategy used. Possible values are 'ALWAYS', 'IF_NECESSARY', or 'NEVER'.
	inputMode	Input mode; for example: voice, select, dtmf, text
	confidence	Confidence level of the interpretation
	name	Name of the entity
	value	Value of the entity
	required	Indicates whether this entity must be collected
	literal	Exact words used by the user
	status	Indicates the status of collecting the entity. Possible values are empty, prefilled, collected, confirmed, validated. See Status field for details.

session-update

Sample payload: "session-update" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "db3f1849cc4471cc11d8f509df07b715",
  "requestid": "e4e5563b-ca9a-933e-a997-a7a50805fbe3",
  "sessionid": "16d80b2e-8e18-4f1b-a146-90b02c3f7272",
  "locale": "en-US",
  "clientData": {},
  "seqid": "16",
  "events": [{
      "name": "session-update",
      "value": {
        "data": {
          "user_name": "****"
        }
      }
    }
  ]
}

This event provides session variables that were updated with the UpdateRequest method.

Field	Description
data	Session variables that were updated

data-required

Sample payload: "data-required" event, client-side integration

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "b43ba10286fed5486e875612ee55db29",
  "requestid": "64d3d58e-9052-9ddd-ba69-8588c040983c",
  "sessionid": "8a7c5d63-6983-4c5c-835a-0ffcf6b27e33",
  "locale": "en-US",
  "clientData": {},
  "seqid": "6",
  "events": [{
      "name": "data-required",
      "value": {
        "id": "GetUserDetails",
        "type": "external",
        "data": {
          "userID": "18470293"
        }
      }
    }
  ]
}

Sample payload: "data-required" event, server-side integration

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "b8fb1468bb4aa1be997dca245c4d1881",
  "requestid": "4894fcdf-8ebf-9750-a96b-5c29099f2a8f",
  "sessionid": "734339bb-c1a3-4fee-a873-b6e3122b2ec7",
  "locale": "en-US",
  "clientData": {},
  "seqid": "35",
  "events": [{
      "name": "data-required",
      "value": {
        "id": "GetCoffeePrice",
        "type": "internal",
        "data": {
          "size": "large",
          "type": "latte"
        },
        "timeout": {
          "connect": 0,
          "request": 30000
        },
        "endpoint": {
          "type": "REST",
          "method": "POST",
          "uri": "https://coffee.app.com:443/dataaccess-examples/coffee/price",
          "headers": ["API_KEY"]
        }
      }
    }
  ]
}

This event indicates that the dialog flow encountered a data access node and requires data.

The contents of the event depend on the integration type:

• For client-side integrations, the event provides the data access node name as well as any data that was sent
• For server-side integrations, the event provides the data access node name, any data that was sent, as well as details about the endpoint and the timeout settings

Field	Description
id	Name of the data access node that requested the data
type	Integration type: external: Client-side integration internal: Server-side integration
data	Data to be sent as configured in the data access node
timeout	Timeout values specified in the data access node; applies to server-side integrations only
	connect	connect timeout value, in milliseconds
	request	request timeout value, in milliseconds
endpoint	Applies to server-side integrations only
	type	Always REST
	method	GET or POST
	uri	The final URI called
	headers	List of header names sent, if any

data-received

Sample payload: "data-received" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "ee544a6130c8be935c0e461fb0d51394",
  "requestid": "ff19f364-77c0-94a0-aa27-ea03f1991acf",
  "sessionid": "8e539261-9d69-49f6-9c80-777b53e5fe95",
  "locale": "en-US",
  "clientData": {},
  "seqid": "8",
  "events": [{
      "name": "data-received",
      "value": {
        "id": "SessionData",
        "data": {
          "returnCode": "0",
          "preferred_coffee": "espresso ",
          "user_name": "****"
        },
        "returnCode": "0"
      }
    }
  ]
}

This event indicates that the requested data has been received.

Field	Description
id	Name of the data access node that requested the data
data	Data requested
returnCode	Variable indicating whether the query was successful. A returnCode of "0" indicates that the transaction was successful.
returnMessage	Return message (optional)
duration	Duration of the request in milliseconds (applies to server-side integrations only)

transfer-initiated

Sample payload: "transfer-initiated" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "e130302b942d312fdae6344756bedc62",
  "requestid": "f9681763-99dc-92e7-91cd-4289371c6eda",
  "sessionid": "cc69a575-b4f1-4384-a143-98ed7136af81",
  "locale": "en-US",
  "clientData": {},
  "seqid": "31",
  "events": [{
      "name": "transfer-initiated",
      "value": {
        "id": "TransferToAgent",
        "data": {
          "user_name": "****",
          "userID": "****"
        }
      }
    }
  ]
}

This event indicates that an external actions node of type "Transfer" was encountered. See Transfer actions in the DLGaaS documentation for details.

Field	Description
id	Name of the external actions node of type "Transfer"
data	Data configured in the transfer node and sent to the client application

transfer-completed

Sample payload: "transfer-completed" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "a28a6b7fceeb088241c1abb08f21c076",
  "requestid": "8e04269a-bf84-9829-bbe5-9249ebc49507",
  "sessionid": "cc69a575-b4f1-4384-a143-98ed7136af81",
  "locale": "en-US",
  "clientData": {},
  "seqid": "33",
  "events": [{
      "name": "transfer-completed",
      "value": {
        "returnCode": "0",
        "data": {
          "status": "Call successfully completed"
        }
      }
    }
  ]
}

This event indicates that a transfer action is completed. See Transfer actions in the DLGaaS documentation for details.

Field	Description
data	Any data provided in the requested_data field of the ExecuteRequestPayload
returnCode	Return code sent by the client application to indicate the status of the transfer
returnMessage	Return message sent by the client application (optional)

component-invoked

Sample payload: "component-invoked" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "5cd7a032b6fbcce60561d3ba1359c4f",
  "requestid": "a4d34efa-3884-95e2-ad85-c2f708a11318",
  "sessionid": "e19ae4f8-f4c2-4f4a-b8db-401e30cbda69",
  "locale": "en-US",
  "clientData": {},
  "seqid": "64",
  "events": [{
      "name": "component-invoked",
      "value": {
        "name": "Goodbye"
      }
    }
  ]
}

This event indicates that a component call node was invoked. See Set up a component call node in the Mix.dialog documentation for details.

Field	Description
value	Name of the component that was invoked

component-returned

Sample payload: "component-returned" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "5cd7a032b6fbcce60561d3ba1359c4f",
  "requestid": "a4d34efa-3884-95e2-ad85-c2f708a11318",
  "sessionid": "e19ae4f8-f4c2-4f4a-b8db-401e30cbda69",
  "locale": "en-US",
  "clientData": {},
  "seqid": "71",
  "events": [{
      "name": "component-returned",
      "value": {}
    }
  ]
}

This event indicates that the dialog flow has returned after having invoked a component.

Field	Description
value	Always empty

event

Sample payload: "event" event, sent by the client application

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "441249de36ff9f682f69a63064e0b125",
  "requestid": "64f78612-552a-9351-bbd9-b89d98c87118",
  "sessionid": "8baf36f2-886d-468d-9e4c-057fdbde66e4",
  "locale": "en-US",
  "clientData": {},
  "seqid": "17",
  "events": [{
      "name": "event",
      "value": {
        "name": "NO_INPUT",
        "message": ""
      }
    }
  ]
}

Sample payload: "event" event, sent by the client application

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "c044de1e2c3ff3f58f5d4b43bdd7d077",
  "requestid": "b0c37992-8b68-9d65-9c07-c3b7d1267a88",
  "sessionid": "cad5b7e8-41fd-488e-a169-0cdd2017cad2",
  "locale": "en-US",
  "clientData": {},
  "seqid": "19",
  "events": [{
      "name": "event",
      "value": {
        "name": "maxnoinputs",
        "message": ""
      }
    }
  ]
}

This event indicates that an event was received. This can be an event sent by the client application or an event that occurred during the dialog flow.

Field	Description
name	Event type. For events sent by the client application, this can be, for example: 'NO_INPUT', 'NO_MATCH', 'HANGUP', 'CUSTOM', 'SUCCESS'. See Dialog event in the DLGaaS documentation for details.
message	Optional message associated with the event

error

Sample payload: "error" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "c044de1e2c3ff3f58f5d4b43bdd7d077",
  "requestid": "b0c37992-8b68-9d65-9c07-c3b7d1267a88",
  "sessionid": "cad5b7e8-41fd-488e-a169-0cdd2017cad2",
  "locale": "en-US",
  "clientData": {},
  "seqid": "23",
  "events": [{
      "name": "error",
      "value": {
        "name": "error.stateengine",
        "message": "Make sure an Event Handler has been configured. No transition defined for event[event.nuance.dialog.odm.maxnoinputs] in node[event]"
      }
    }
  ]
}

This event indicates that an error occurred in the dialog flow.

Field	Description
name	Name of the error
message	Message associated with the error

reporting-vars

Sample payload: "reporting-vars" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "c044de1e2c3ff3f58f5d4b43bdd7d077",
  "requestid": "b0c37992-8b68-9d65-9c07-c3b7d1267a88",
  "sessionid": "cad5b7e8-41fd-488e-a169-0cdd2017cad2",
  "locale": "en-US",
  "clientData": {},
  "seqid": "3",
  "events": [{
      "name": "reporting-vars",
      "value": {
        "quantity": 5.0,
        "preferred_coffee": "espresso"
      }
    }
  ]
}

This event indicates that a variable marked with a reporting property in Mix.dialog was set or updated.

In particular, this event is logged if a variable is set or updated in the following cases:

• At session-start
• At application-end
• At StopRequest
• When a response asking for input is returned through the DLGaaS API (for example, qa_action, da_action, and so on)

See Reporting properties in the Mix.dialog documentation for more information.

Field	Description
value	Key-value pairs of the updated variables.

application-ended

Sample payload: "application-ended" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "5e49e59ead2572e7",
  "requestid": "fbc5586d-178b-465c-918c-aed78d4bb264",
  "sessionid": "0c234bb6-e2c4-4198-a7f7-53020b02d734",
  "locale": "en-US",
  "clientData": {},
  "seqid": "36",
  "events": [{
      "name": "application-ended",
      "value": {
        "id": "End dialog",
        "language": "en-US"
      }
    }
  ]
}

This event indicates that the dialog application flow has encountered an external actions node of type "end".

Note: No event is logged if the session times out or an unhandled error occurs. See Special note about dialog session termination for details.

Field	Description
id	Name of the end node
language	Language used in this interaction
data	Variables and entities to be sent to an external system, if applicable

session-stopped

Sample payload: "session-stopped" event

"data": {
  "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
  "traceid": "1e027862d6afcc01ec47e108f5f8e067",
  "requestid": "a226a66e-2024-93ec-8a5e-0560a46639ef",
  "sessionid": "b71189ad-97ef-46ba-863a-5d71ff8bbc49",
  "locale": "en-US",
  "clientData": {},
  "seqid": "12",
  "events": [{
      "name": "session-stopped"
    }
  ]
}

This event indicates that the client explicitly stopped an active session by sending the StopRequest message. This event is logged only when the session ID specified in the StopRequest corresponds to an active session. It is not logged if the session was already closed.

This event does not include additional fields.

Special note about dialog session termination

A dialog session is terminated when:

• The dialog application flow encounters an external actions node of type "end". In this case, an application-ended event is generated.
• The client application ends the session by calling StopRequest. In this case, a session-stopped event is generated.
• An unhandled error occurs. In this case, an error event is generated.
• The session times out. In this case, no event is generated, as there is no way for the dialog engine to capture when the session ends.

TTS payload

The x-nuance-tts-callsummary payload provides an overview of the interactions that occur in a TTSaaS synthesis session.

tts-callsummary payload

Sample payload: Synthesizer/Synthesize

"data":
{
  "dataContentType": "application/x-nuance-tts-callsummary.v3+json",
  "clientName": "default",
  "traceId": "7c5bcda51d0b9cc63eabcf8f662f82df",
  "sessionId": "8070e584-de4d-4daf-ac37-d9947de5eff1",
  "userId": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "processingTime":
  {
    "startTime": "2021-03-15T18:13:19.650Z",
    "firstAudioBufferTime": "2021-03-15T18:13:19.872Z",
    "durationMs": 363
  },
  "request":
  {
    "resources": []
  },
  "clientData":
  {
    "name": "Alex Smith",
    "x-nuance-dialog-session-id": "8070e584-de4d-4daf-ac37-d9947de5eff1"
  },
  "response":
  {
    "events": [
      // List of events
    ],
    "status":
    {
      "code": 200,
      "message": "OK",
      "details": ""
    }
  }
}

Current version: v3

Nuance Vocalizer for Cloud (NVC) offers two types of synthesis response. By default, the synthesized voice is streamed back to the application (Synthesizer/Synthesize), but a unary (non-streamed, single package) response may be requested instead (Synthesizer/UnarySynthesize). The same payload is returned with the exception of the source field. For more information on streamed versus unary responses, see the TTSaaS gRPC API documentation.

In addition to the standard fields described in Data field, messages with the application/x-nuance-tts-callsummary.v3+json dataContentType include service-specific fields as detailed below.

Request object

Sample request with synthesis resources: ActivePrompt database, ruleset, and user dictionary

"request": {
    "resources": [{
            "uri": "urn:nuance-mix:tag:model/MY_APDB/mix.tts?=language=en-US",
            "type": "application/x-vocalizer-activeprompt-db"
        }, {
            "uri": "urn:nuance-mix:tag:model/MY_RST/mix.tts?=language=en-US",
            "type": "application/x-vocalizer-rettt+text"
        }, {
            "uri": "urn:nuance-mix:tag:model/MY_DCB/mix.tts?=language=en-US",
            "type": "application/edct-bin-dictionary"
        }
    ]
}

The request object describes the synthesis resources used to tune the synthesized audio, as provided in the <../tts-grpc/v1/#input" target="_blank">Input message of a SynthesisRequest. Default is blank (empty array).

Resources may include:

• User dictionaries, which provide custom pronunciations
• Rulesets, which apply search-and-replace rules to input text
• ActivePrompt databases, which help tune synthesized audio under certain conditions, using Nuance Vocalizer Studio.

Response object

Sample response with NVOCcntv, NVOCcntg, NVOCinpt, and NVOCsynd events

"response":
{
  "events": [
    {
      "EVENT": "NVOCcntv",
      "VOICE_VOP": "Allison_full_vssq5f22",
      "CHARS": 50,
      "TTSTIME": "2021/03/15 14:39:01.660 UTC",
      "TTSAASTIME": "2021-03-15T14:39:01.660Z",
      "VOIC": "Allison",
      "VMDL": "full_vssq5f22",
      "LOCALE": "en-US",
      "DURS": 3347.39
    },
    {
      "EVENT": "NVOCcntg",
      "CHARS": 50,
      "TTSTIME": "2021/03/15 14:39:01.660 UTC",
      "TTSAASTIME": "2021-03-15T14:39:01.660Z"
    },
    {
      "EVENT": "NVOCinpt",
      "MIME": "text/plain;charset=utf-8",
      "TXSZ": 62,
      "TEXT": "Hello and welcome to the coffee app. What can I get you today?",
      "TTSTIME": "2021/03/15 14:39:01.660 UTC",
      "TTSAASTIME": "2021-03-15T14:39:01.660Z"
    },
    {
      "EVENT": "NVOCsynd",
      "INPT": 62,
      "DURS": 3347,
      "RSTT": "ok",
      "TTSTIME": "2021/03/15 14:39:01.660 UTC",
      "TTSAASTIME": "2021-03-15T14:39:01.660Z"
    }
  ],
  "status":
  {
    "code": 200,
    "message": "OK",
    "details": ""
  }
}

The response object, in addition to providing a status message, provides a list (array) of events. For more information about the synthesis response, see SynthesisResponse.

For a successful response, response.events will always contain:

• One or many NVOCcntv events (one per voice used in the request): Number of Unicode input characters for a voice and voice operating point combination processed in a single speak request.
• One NVOCcntg event (total character count): Number of Unicode input characters processed in a single speak request.
• One NVOCinpt event: Input text for the speak request.
• One NVOCsynd event: Logged at the end of synthesis.

Each event includes the following two fields:

• TTSTIME, the time the event was produced by the core engine.
• TTSAASTIME, the time NVC received the event from the core engine. May be used to diagnose slowness between the core engine and NVC.

NVOCcntv event

{
    "EVENT": "NVOCcntv",
    "VOICE_VOP": "Ava-Mls_xpremium",
    "CHARS": 36,
    "TTSTIME": "2020/09/15 10:10:20.504 UTC",
    "TTSAASTIME": "2020-09-15T10:10:20.504Z",
    "VOIC": "Ava-Mls",
    "VMDL": "xpremium",
    "LOCALE": "en-US",
    "DURS": 3490.2
}, {
    "EVENT": "NVOCcntv",
    "VOICE_VOP": "Paulina-Ml_xpremium-high",
    "CHARS": 18,
    "TTSTIME": "2020/09/15 10:10:20.504 UTC",
    "TTSAASTIME": "2020-09-15T10:10:20.504Z",
    "VOIC": "Paulina-Ml",
    "VMDL": "xpremium-high",
    "LOCALE": "es-MX",
    "DURS": 2069.93
}

The NVOCcntv event indicates the amount of Unicode input characters processed for a specific voice operating point in a single speak request. One event is logged per voice used in the request.

Field	Description
EVENT	NVOCcntv
VOICE_VOP	Name of the voice and its operating point
CHARS	Number of Unicode characters processed
VOIC	Voice name
VMDL	Voice model. For example, xpremium (standard) xpremium-high (enhanced). Other values will be returned for legacy voices (full_vssqf22 for the Allison voice, for example).
LOCALE	Locale ID
DURS	Output duration in milliseconds

NVOCcntg event

{
    "EVENT": "NVOCcntg",
    "CHARS": 54,
    "TTSTIME": "2020/09/15 10:10:20.505 UTC",
    "TTSAASTIME": "2020-09-15T10:10:20.505Z"
}

The NVOCcntg event indicates the total Unicode character count processed in a single speak request.

Field	Description
EVENT	NVOCcntg
CHARS	Number of Unicode characters processed

NVOCinpt event

{
    "EVENT": "NVOCinpt",
    "MIME": "application/synthesis+ssml",
    "TXSZ": 373,
    "TEXT": "<?xml version\\=\"1.0\"?><speak xmlns\\=\"http://www.w3.org/2001/10/synthesis\" 
    xml:lang\\=\"en-US\" version\\=\"1.0\" xmlns:xsi\\=\"http://www.w3.org/2001/XMLSchema-instance\"
    xsi:schemaLocation\\=\"http://www.w3.org/2001/10/synthesis http://www.w3.org/TR/speech-synthesis/synthesis.xsd\">English text here. 
    <voice xml:lang\\=\"es-MX\">Texto en español aquí</voice> and now back to English.</speak>",
    "TTSTIME": "2020/09/15 10:10:20.505 UTC",
    "TTSAASTIME": "2020-09-15T10:10:20.505Z"
}

The NVOCinpt event provides the input text for the speak request and is logged at the end of each speak request. If an error occurred or the synthesis request was stopped, NVOCinpt reports the information obtained prior to the error or interruption.

Field	Description
EVENT	NVOCinpt
MIME	MIME content type for the input text
TXSZ	Input size in bytes, the raw input prior to any processing
TEXT	Input text. Omitted when the suppress_input field is true in the SynthesisRequest (default is false).

NVOCsynd event

{
    "EVENT": "NVOCsynd",
    "INPT": 373,
    "DURS": 5560,
    "RSTT": "ok",
    "TTSTIME": "2020/09/15 10:10:20.506 UTC",
    "TTSAASTIME": "2020-09-15T10:10:20.506Z"
}

The NVOCsynd event is logged at the end of synthesis. If an error occurred or the synthesis request was stopped, NVOCsynd reports the information obtained prior to the error or interruption.

Field	Description
EVENT	NVOCsynd
INPT	Input size in bytes; the raw input prior to any processing
DURS	Output duration in milliseconds
RSTT	Result status: "error" if an error occurred "ok" if synthesis succeeded "stop" if the application stopped the synthesis operation

NVOCifst event

{
    "EVENT": "NVOCifst",
    "PROP": "cache.timeoutOpen=5000;inet.timeoutDownload=30000;inet.timeoutIO=30000;inet.timeoutOpen=30000",
    "TTSTIME": "2020/11/19 16:35:00.273 UTC",
    "URI": "urn:nuance-mix:tag:model/MY_APDB/mix.tts?=language=en-US",
    "TTSAASTIME": "2020-11-19T18:18:00.891Z"
}

The NVOCifst event indicates an Internet fetch has started.

Field	Description
EVENT	NVOCifst
PROP	Semicolon-separated list of Internet fetch properties, where each property is of the form property=value
URI	URI being fetched; can be a relative URI

NVOCifnd event

{
    "EVENT": "NVOCifnd",
    "FRST": "SUCCESS",
    "TTSTIME": "2020/11/19 16:35:00.327 UTC",
    "DSRC": "http",
    "SIZE": "12652",
    "MIME": "application/octet-stream",
    "URI": "urn:nuance-mix:tag:model/MY_APDB/mix.tts?=language=en-US",
    "TTSAASTIME": "2020-11-19T18:18:00.891Z"
}

The NVOCifnd indicates an Internet fetch has ended.

Field	Description
EVENT	NVOCifnd
FRST	Fetch result: SUCCESS if the fetch succeeded; otherwise a brief error description
DSRC	Data source: "cache" if it was loaded from a cache "file" if it was a local file "http" if it was fetched from a web server "validated" if it was loaded from a cache, but only after validating the cached copy with the web server empty string if the fetch failed
SIZE	Size in bytes. 0 if the fetch failed.
MIME	MIME content type. Empty string if the fetch failed.
URI	URI that was fetched. If the fetch was successful, this will be an absolute URI.

NVOCrlst event

The NVOCrlst event is written when a ruleset is used during synthesis.

Field	Description
EVENT	NVOCrlst
RULSET_NAME	URI that was fetched for the ruleset. Example: urn:nuance-mix:tag:model/ruleset/mix.tts?=language=en-US

NVOCdict event

The NVOCdict event is written when a user dictionary is used during synthesis.

Field	Description
EVENT	NVOCdict
URI	URI that was fetched for the user dictionary. Example: urn:nuance-mix:tag:model/user_dictionary/mix.tts?=language=en-US

NVOCactp event

The NVOCactp event is written when an ActivePrompt is used during synthesis. These events are written in the order in which the ActivePrompts are matched (highest to lowest precedence across the input text), not the order in which they appear in the output audio stream.

Field	Description
EVENT	NVOCactp
APID	ActivePrompt ID of the form domain::prompt

NVOCadon event

The NVOCadon event is written when a system-level dictionary is used during synthesis.

Field	Description
EVENT	NVOCadon
ADDON_NAME	URI that was fetched for the add-on. Example: urn:nuance-mix:tag:model/system_dictionary/mix.tts?=language=en-US

Managing sensitive information in an application

Mix lets you manage sensitive information in an application so that the information is redacted in the logs.

You can mark data as sensitive in the Mix tools as follows:

• In Mix.dialog, you can mark an entity, a variable, a question and answer node, or fields within a schema as sensitive.
• In Mix.nlu, you can mark an entity as sensitive.

There are two redaction options:

• Partial redaction: Only the information marked as sensitive is redacted in the dialog and NLU logs. Partial redaction is implemented by marking entities and variables as sensitive in the Mix tools.
• Complete redaction: All user data is redacted in the logs. Complete redaction is enabled by either:
  • Setting a question and answer node as sensitive. In this case, all user input collected at this node is fully redacted from the ASRaaS, NLUaaS, and DLGaaS logs.
  • Setting the suppress_log_user_data field to true in the DLGaaS StartRequest. In this case, all user input is fully redacted from the logs across all services for the entire session.
  • Setting a field in the NLUaaS, TTSaaS, or ASRaaS API. In this case, the logs are redacted for that specific service.

This section summarizes these options.

Partial redaction

When you mark an entity or a variable as sensitive in Mix.nlu and Mix.dialog and complete redaction is not enabled, then only the entity and variables marked as sensitive will be redacted in the dialog and NLU logs.

For details on the values masked with partial redaction, see:

• Dialog values possibly masked
• NLU values possibly masked

Complete redaction

ASRaaS, NLUaaS, TTSaaS, and DLGaaS all provide a field that disables logging of user data, as follows:

• ASRaaS: Set the suppress_call_recording RecognitionFlags field to True to disable call logging. See ASR values possibly masked for details.
• NLUaaS: Set the interpretation_input_logging_mode InterpretationParameters field to SUPPRESSED so that input is masked. See NLU values possibly masked for details.
• TTSaaS: Set the suppress_input EventParameters field to True to omit input text and URIs from log events. See TTS values possibly masked for details.
• DLGaaS: Set the suppress_log_user_data field in the StartRequestPayload to True to disable logging of user data for all the services when dialog orchestrates with these services. Otherwise, the client application must set the field when requesting a service. See Dialog values possibly masked for details.

You can also mark a question and answer node as sensitive. This enables complete redaction of all user input collected at this node (user text, utterance, intent and entity values and literals). This applies to NLU intent and entity collection, as well as to all events at that node: collection, recovery, confirmation, nomatch, noinput, max events, NO_INTENT, intent switching, and so on.

ASR values possibly masked

When the suppress_call_recording RecognitionFlags field is set to true, PCI information is suppressed from the event logs and the corresponding audio is removed. The hypothesis field is empty and the redactedReason field is provided, as shown in the second example in asr-finalresultresponse payload.

The minimally formatted text is still available in the final result returned to the application, but this information is not logged.

See the ASRaaS RecognitionFlags documentation for details on setting suppress_call_recording.

NLU values possibly masked

Example of partial redaction, COFFEE_TYPE entity is marked as sensitive

"data": {
  ...
  "request": {
    ...
    "input": "I want a ****redacted**** espresso"
  },
  "response": {
    "result": {
      "interpretations": [{
          "singleIntentInterpretation": {
            "entities": {
              "COFFEE_TYPE": {
                "entities": [{
                    "entities": {},
                    "metadata": {},
                    "textRange": {
                      "startIndex": 16,
                      "endIndex": 24
                    },
                    "confidence": 1,
                    "origin": "GRAMMAR",
                    "literal": "espresso",
                    "sensitive": false,
                    "formattedLiteral": "espresso",
                    "formattedTextRange": {
                      "startIndex": 16,
                      "endIndex": 24
                    },
                    "audioRange": null,
                    "stringValue": "espresso",
                    "valueUnion": "stringValue"
                  }
                ]
              },
              "COFFEE_SIZE": {
                "entities": [{
                    "entities": {},
                    "metadata": {},
                    "textRange": {
                      "startIndex": 9,
                      "endIndex": 15
                    },
                    "confidence": 1,
                    "origin": "GRAMMAR",
                    "literal": "****redacted****",
                    "sensitive": true,
                    "formattedLiteral": "****redacted****",
                    "formattedTextRange": {
                      "startIndex": 9,
                      "endIndex": 15
                    },
                    "audioRange": null,
                    "stringValue": "****redacted****",
                    "valueUnion": "stringValue"
                  }
                ]
              }
            },
            "metadata": {},
            "intent": "ORDER_COFFEE",
            "confidence": 1,
            "origin": "GRAMMAR"
          },
          "interpretationUnion": "singleIntentInterpretation"
        }
      ],
      "literal": "I want a ****redacted**** espresso",
      "sensitive": true,
      "formattedLiteral": "****redacted****"
    }
  }
}

Example of complete redaction:

"data": {
  ...
  "request": {
    ...
    "input": "****redacted****"
  },
  "processingTime": {
    "startTime": "2022-02-22T20:17:16.996Z",
    "durationMs": 74
  },
  "response": {
    ...
    "result": {
      "interpretations": "****redacted****",
      "literal": "****redacted****",
      "sensitive": true,
      "formattedLiteral": "****redacted****"
    }
  }
}

The NLU values masked depend on whether partial or complete redaction is enabled.

Partial redaction

When an entity is marked as sensitive (either in Mix.nlu or Mix.dialog) and complete redaction is not enabled, values in the data field are redacted as follows:

• request.input: Only the sensitive entity value is redacted from the input.
• response.result.interpretations: The literal, formatted literal, and string values are redacted for sensitive entities.
• response.result.literal: Only the sensitive entity value is redacted from the literal.
• response.result.formattedLiteral: The formatted literal value is redacted.

Complete redaction

When the interpretation_input_logging_mode field in the InterpretRequest is set to SUPPRESSED, values in the data field are redacted as follows:

• request.input: Fully redacted.
• response.result.interpretations: Fully redacted.
• response.result.literal: Fully redacted.
• response.result.formattedLiteral: Fully redacted.

See the NLUaaS InterpretationParameters documentation for details on setting interpretation_input_logging_mode.

Dialog values possibly masked

Example of message event, partial redaction of user_name variable

"events": [{
    "name": "message",
    "value": {
      "nlg": [],
      "visual": [{
          "text": "Hello "
        }, {
          "text": "****"
        }, {
          "text": "and welcome to the coffee app."
        }
      ],
      "audio": []
    }
  }
]

Example of message event, complete redaction

"events": [{
    "name": "message",
    "value": {
      "nlg": [],
      "visual": [{
          "text": "****"
        }, {
          "text": "****"
        }, {
          "text": "****"
        }
      ],
      "audio": []
    }
  }
]

Example of data field in session-update event, partial redaction of user_name variable

"events": [{
    "name": "session-update",
    "value": {
      "data": {
        "quantity": 7.0,
        "user_name": "****"
      }
    }
  }
]

Example of data field in session-update event, complete redaction

"events": [{
    "name": "session-update",
    "value": {
      "data": {
        "quantity": "****",
        "user_name": "****"
      }
    }
  }
]

Example of partial redaction in input-received event, COFFEE_SIZE entity is masked as sensitive

"events": [{
    "name": "input-received",
    "value": {
      "interpretation": {
        "isSuccess": "true",
        "confidence": "1.0",
        "inputmode": "text",
        "utterance": "I want a ****COFFEE_SIZE**** espresso",
        "data": {
          "COFFEE_TYPE": "espresso",
          "INTENT": "ORDER_COFFEE",
          "COFFEE_SIZE": "****"
        },
        "slot_literals": {
          "COFFEE_TYPE": "espresso",
          "INTENT": "****",
          "COFFEE_SIZE": "****"
        },
        "slot_confidences": {
          "COFFEE_TYPE": "1.0",
          "INTENT": "1.0",
          "COFFEE_SIZE": "1.0"
        }
      }
    }
  }
]

Example of complete redaction in input-received event

"events": [{
    "name": "input-received",
    "value": {
      "interpretation": {
        "isSuccess": "true",
        "confidence": "1.0",
        "inputmode": "text",
        "utterance": "I want a ****COFFEE_SIZE**** espresso",
        "data": {
          "COFFEE_TYPE": "****",
          "INTENT": "****",
          "COFFEE_SIZE": "****"
        },
        "slot_literals": {
          "COFFEE_TYPE": "****",
          "INTENT": "****",
          "COFFEE_SIZE": "****"
        },
        "slot_confidences": {
          "COFFEE_TYPE": "1.0",
          "INTENT": "1.0",
          "COFFEE_SIZE": "1.0"
        }
      }
    }
  }
]

The dialog values masked depend on whether partial or complete redaction is enabled.

Partial redaction

When an entity or a variable is marked as sensitive (either in Mix.nlu for entities or Mix.dialog for entities, variables, and question and answer nodes) and complete redaction is not enabled, values in Dialog application logs are redacted as follows:

• Messages: Only the sensitive variable or entity is redacted from messages.
• Data: The values of sensitive entities and variables are redacted from the data fields.
• Utterances: The values of sensitive entities are redacted from the utterance field.
• Entity values and literals: The values and literals of sensitive entities are redacted in the qa-config, question-router, and input-received event logs.
• Reporting variables: Only non-sensitive variables configured for reporting are included in the reporting-vars event log.

Complete redaction

When the suppress_log_user_data field in the StartRequest is set to true, values in Dialog application logs are redacted as follows:

• Messages: The messages are fully redacted.
• Data: All entity and variable values are redacted from the data fields.
• Utterances: The utterance fields are fully redacted.
• Entity values and literals: All entity values and literals are redacted in the qa-config, question-router, and input-received event logs.
• Reporting variables: Only non-sensitive variables configured for reporting are included in the reporting-vars event log.

See DLGaaS - Disabling logging documentation for details on setting suppress_log_user_data.

TTS values possibly masked

When the suppress_input field in the SynthesisRequest is set to true, the following fields will be masked:

• TEXT parameter of the NVOCinpt event is omitted
• URI parameter of the NVOCifst/NVOCifnd event and NVOCdict event is masked
• RULSET_NAME parameter of the NVOCrlst event is masked
• APID parameter of the NVOCactp event is masked
• ADDON_NAME parameter of the NVOCadon event is masked

See the TTSaaS gRPC API documentation for more information.

Correlating event logs

A single interaction can create multiple event logs. Event logs can be correlated in different ways:

• Correlate logs for a specific API interaction
• Correlate ASR, NLU, and TTS logs for a specific dialog session
• Correlate NII logs for a specific dialog session

Correlate logs for a specific API interaction

Example: Event logs in dialog start request interaction with requestid "50f04cf5-debd-9a6a-b1f0-75e408ce21d1"

// NII session-start

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "DLGaaS",
    "id": "ca6e4281-47c1-4cc8-a3e2-48d0bcda29de"
  },
  "value":
  {
    ...
    "data":
    {
      "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
      "traceid": "428c127f3c1adc1e39c90efee1c78331",
      "requestid": "50f04cf5-debd-9a6a-b1f0-75e408ce21d1",
      "sessionid": "b15302ac-be35-4d9c-b593-3789b091e83f",
      "locale": "en-US",
      "clientData": {},
      "seqid": "1",
      "events": [
        {
          "name": "session-start",
          "value":
          {
            ...
          }
        }
      ]
    }
  },
  "partition": 0,
  "offset": 1057
}

// NII selector
{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "DLGaaS",
    "id": "2d5afb58-928b-4b22-91eb-bd9cec61681e"
  },
  "value":
  {
    ...
    "data":
    {
      "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
      "traceid": "428c127f3c1adc1e39c90efee1c78331",
      "requestid": "50f04cf5-debd-9a6a-b1f0-75e408ce21d1",
      "sessionid": "b15302ac-be35-4d9c-b593-3789b091e83f",
      "locale": "en-US",
      "clientData": {},
      "seqid": "2",
      "events": [
        {
          "name": "selector",
          "value":
          {
            "channel": "default",
            "language": "en-US"
          }
        }
      ]
    }
  },
  "partition": 0,
  "offset": 1058
}

//DLGaaS-Start-Start
{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "DLGaaS",
    "id": "892e1679-065d-4bdd-8112-ff12f64e70cf"
  },
  "value":
  {
    ...
    "data":
    {
      "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
      "traceid": "428c127f3c1adc1e39c90efee1c78331",
      "requestid": "50f04cf5-debd-9a6a-b1f0-75e408ce21d1",
      "sessionid": "",
      "sessionTimeoutSec": 900,
      "locale": "en-US",
      "events": [
        {
          "event": "DLGaaS-Start-Start",
          "time": "2021-03-15T14:39:00.393Z"
        }
      ]
    }
  },
  "partition": 0,
  "offset": 1066
}

// DLGaaS-Start-End
{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "DLGaaS",
    "id": "f3ceed51-8a5b-4496-8fc1-6da596816164"
  },
  "value":
  {
    ...
    "data":
    {
      "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
      "traceid": "428c127f3c1adc1e39c90efee1c78331",
      "requestid": "50f04cf5-debd-9a6a-b1f0-75e408ce21d1",
      "sessionid": "b15302ac-be35-4d9c-b593-3789b091e83f",
      "channel": "default",
      "sessionTimeoutSec": 900,
      "locale": "en-US",
      "processingTime":
      {
        "startTime": "2021-03-15T14:39:00.393Z",
        "durationMs": 128
      },
      "events": [
        {
          "event": "DLGaaS-Start-End",
          "time": "2021-03-15T14:39:00.521Z"
        }
      ],
      "request":
      {
        ...
      },
      "response":
      {
        ...
      }
    }
  },
  "partition": 0,
  "offset": 1067
}

A single API request/response interaction will create multiple event logs across the different services and may also include dialog NII logs.

For example, consider a dialog Start request/response interaction; it might generate the following event logs:

• DLGaaS-Start-Start (dlg-interaction-summary payload)
• DLGaaS-Start-End (dlg-interaction-summary payload)
• NII session-start (dlg-nii-logs payload)
• NII selector (dlg-nii-logs payload)

To correlate all the event log records that apply to a specific request/response interaction, use the requestid field. All the logs for a specific customer interaction share the same requestid value.

The requestid field is particularly useful for interactions that span multiple services. For example, consider a dialog Execute request that requires NLU interpretation. In addition to the related NII logs, such an interaction might also include the following event logs:

• DLGaaS-ExecuteStream-Start dlg-interaction-summary payload)
• NLUaaS-interpret-Start dlg-interaction-summary payload)
• NLUaaS-interpret-End dlg-interaction-summary payload)
• DLGaaS-ExecuteStream-End dlg-interaction-summary payload)
• NLU Interpret request (nluaas-interpretation payload)

Correlate ASR, NLU, and TTS logs for a specific dialog session

Example: ASRaaS event log for dialog session "b15302ac-be35-4d9c-b593-3789b091e83f"

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "ASRaaS",
    "id": "6c4945f4-bc62-495d-9602-75015692bbbd"
  },
  "value":
  {
    "specversion": "1.0",
    "service": "ASRaaS",
    "source": "nuance.asr.v1.Recognizer/Recognize",
    "type": "Recognize",
    "id": "a77f74fc-0274-44e6-9e33-f2269784b6df",
    "timestamp": "2021-03-15T14:39:07.782Z",
    "datacontenttype": "application/json",
    "data":
    {
      "request":
      {
        "recognitionInitMessage":
        {
          ...
          "clientData":
          {
            "x-nuance-dialog-session-id": "b15302ac-be35-4d9c-b593-3789b091e83f",
            "x-nuance-dialog-seqid": "11"
          }
        }
      },
      "asrSessionid": "6c4945f4-bc62-495d-9602-75015692bbbd",
      "dataContentType": "application/x-nuance-asr-recognitioninitmessage.v2+json",
      "requestid": "516ef36d-5a12-9939-a232-ac6256110d0f",
      "audioUrn": "urn:nuance-mix:log:service:audio/asraas/6c4945f4-bc62-495d-9602-75015692bbbd"
    },
    "appid": "DEMO-OMNICHANNEL-APP-DEV"
  },
  "partition": 0,
  "offset": 1078
}

When DLGaaS calls ASRaaS, NLUaaS, or TTSaaS, it includes the following fields in the logs of the service, as part of the clientData of the request:

• x-nuance-dialog-session-id: ID of the dialog session.
• x-nuance-dialog-seqid: Sequence ID of the NII log message to which this event log applies.

These fields can be used to identify all the ASR, NLU, and TTS event logs that relate to a dialog session.

Correlate NII logs for a specific dialog session

Example: First message in NII sequence for dialog session "c475ff51-d60c-4ad0-b1df-7fb4aae43b13"

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "DLGaaS",
    "id": "03997a3c-c18d-419a-83a9-6ccc9e5cfd71"
  },
  "value":
  {
    ...
    "data":
    {
      "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
      "traceid": "1f16b4bd07b2f1f9c548acaedc487c56",
      "requestid": "01b73d0f-bfd6-9921-a160-ad037d2e6036",
      "sessionid": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
      "locale": "en-US",
      "clientData": {},
      "seqid": "1",
      "events": [
        {
          "name": "session-start",
          "value":
          {
            ...
          }
        }
      ]
    }
  },
  "partition": 0,
  "offset": 1091
}

Example: Second message in NII sequence for dialog session "c475ff51-d60c-4ad0-b1df-7fb4aae43b13"

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key":
  {
    "service": "DLGaaS",
    "id": "0e491cdd-37c3-4895-8657-5da5701c931b"
  },
  "value":
  {
    ...
    "data":
    {
      "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
      "traceid": "1f16b4bd07b2f1f9c548acaedc487c56",
      "requestid": "01b73d0f-bfd6-9921-a160-ad037d2e6036",
      "sessionid": "c475ff51-d60c-4ad0-b1df-7fb4aae43b13",
      "locale": "en-US",
      "clientData": {},
      "seqid": "2",
      "events": [
        {
          "name": "selector",
          "value":
          {
            "channel": "default",
            "language": "en-US"
          }
        }
      ]
    }
  },
  "partition": 0,
  "offset": 1092
}

To correlate the NII logs that correspond to a specific dialog session, use the following data fields:

• sessionid: Specifies the ID of the dialog session. All the NII logs for a specific dialog session share the same session ID. This is the session ID that is created with the DLGaaS StartRequest.
• seqid: Sequence ID of the NII log. Starts at 1 and increments for each NII message in a dialog session.

The seqid ensures that you can determine the sequence of the NII logs in a dialog session, since these logs may not be received in the order in which they are created and sent.

For a description of the NII log events, see dlg-nii-logs payload.

Inject content into the payloads

Your client application can inject the following content into the payloads:

• Client data
• User ID
• Client-provided request ID

Client Data

Sample client data field, in NII and DLGaaS logs

// DLGaaS log
{
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
        "service": "DLGaaS"
    },
    "value": {
        ...
        },
        "data": {
            ...
            "request": {
                ...
                "clientData": {
                    "field1": "default",
                    "field2": "optional"
                }
            },
            "response": {
            ...
            }
        }
    },
    "partition": 0,
    "offset": 890
}

// NII log
{
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
        "service": "DLGaaS"
    },
    "value": {
        ...
        },
        "data": {
            ...
            "clientData": {
                "field1": "default",
                "field2": "optional"
            },
            "seqid": "1",
            "events": [{
                    "name": "session-start",
                    "value": {
                        ...
                    }
                }
            ]
        }
    },
    "partition": 0,
    "offset": 893
}

You can inject a map of key-value pairs from the XaaS APIs, through the client_data field, as follows:

Service	API Message
ASRaaS	RecognitionInitMessage
NLUaaS	InterpretRequest
DLGaaS	StartRequest
TTSaaS	SynthesisRequest

This data is stored in the clientData field of the request.

If the client_data field is provided in the XaaS request, it will be included in all the event logs related to this request.

User ID

Sample user ID field

{
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
        "service": "DLGaaS"
    },
    "value": {
        ...
        },
        "data": {
            "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
            "traceid": "139d166b4b898ccb",
            "sessionid": "",
            "sessionTimeoutSec": 0,
            "locale": "en-US",
            "userid": "533663ad61ab5b49214f55129d6dd7fa44ebd13f5f4e6c42212ee286048b6c85",
            "events": [{
                    "event": "DLGaaS-Start-Start",
                    "time": "2020-11-23T22:05:49.385Z"
                }
            ]
        }
    },
    "partition": 0,
    "offset": 889
}

You can inject a user ID from the XaaS APIs, through the user_id field, as follows:

Service	API Message
ASRaaS	RecognitionInitMessage
NLUaaS	InterpretRequest
DLGaaS	StartRequest, ExecuteRequest, and StopRequest
TTSaaS	SynthesisRequest

The user ID will be included in the userid field under data. To generate the userid value, the appid and user_id fields are concatenated and separated by a colon, as follows: appid:user_id. This value is then hashed using SHA256 and provided as a 64-character hexadecimal string.

If the user_id field is provided in the XaaS request, it will be included in all the event logs related to this request.

Client-provided request ID

Sample metadata header in Python client for execute request

    metadata = [('x-client-request-id', 'sample_request_id')]
    execute_response, call = stub.Execute.with_call(execute_request, metadata=metadata)

Event logs for execute request in dlg-interaction-summary log

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key": {
    "service": "DLGaaS",
    "id": "dd1e5bd9-516a-4b0e-8d77-d7ffe3503792"
  },
  "value": {
    "specversion": "1.0",
    "service": "DLGaaS",
    "source": "nuance.dlg.v1.DialogService/Execute",
    "type": "Execute",
    ...
    "data": {
      "dataContentType": "application/x-nuance-dlg-interaction-summary.v1+json",
      "traceid": "8a99dc457aec7d9793aa549f2a2749f6",
      "requestid": "6a6f6188-7a19-9dc6-b7c8-b6f8caf7b9e4",
      "clientRequestid": "sample_request_id",
      ...

Event logs for execute request in dlg-nii-logs log

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "key": {
    "service": "DLGaaS",
    "id": "b45634f6-fc1a-4e1e-885c-a2d9630ebab1"
  },
  "value": {
    "specversion": "1.0",
    "service": "DLGaaS",
    "source": "NIIEventLogger",
    "type": "NIIEventLog",
    ...
    "data": {
      "dataContentType": "application/x-nuance-dlg-nii-logs.v1+json",
      "traceid": "8a99dc457aec7d9793aa549f2a2749f6",
      "requestid": "6a6f6188-7a19-9dc6-b7c8-b6f8caf7b9e4",
      "clientRequestid": "test1234",
      ...

You can inject your own request ID in the event logs by setting the x-client-request-id header in the gRPC metadata for a request in the client application. This field can be used for tracing purposes.

The header value will get logged in the clientRequestid field of the Data field for all the event logs generated for that request.

For example, the sample code shows how the x-client-request-id header is specified in the metadata for the Execute request in the client application. This header value then gets included in the clientRequestid field of all the event logs related to this Execute request.

Reference: Kafka consumer API

Version: v1

This document covers v1 of the Mix Kafka REST Consumer API.

URL

https://log.api.nuance.co.uk

Partitions

Get Partitions

Example responses

200 Response

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "partitions": [
    0
  ]
}

GET /partitions

Retrieve the list of available partitions for a topic.

Responses

Status	Meaning	Description	Schema
200	OK	List of available partitions	topicPartitions
404	Not Found	Topic does not exist	errorResponse

Get Partition Offsets

Example responses

200 Response

{
  "beginning_offset": 441,
  "end_offset": 641
}

GET /partitions/{partition}/offsets

Retrieve the first and last available offsets for a partition.

Parameters

Name	In	Type	Description
partition	path	number	Partition number

Responses

Status	Meaning	Description	Schema
200	OK	Partition offset details	partitionsOffsets

Consumers and Groups

Create Consumer

Example responses

400 Response

{
  "error_code": 401,
  "message": "The request could not be authorized"
}

POST /consumers

Create a new consumer instance in a consumer group.

The consumer group name must follow the pattern appID-{appId/topic}-clientName-{clientName}-{two-digit number}, for example, appID-DEMO-OMNICHANNEL-APP-DEV-clientName-default-01

The consumer name must follow the pattern consumer-{UUIDv4}, for example, consumer-9872ec64-50d9-4dcd-9f4f-e8b5099a7f4f

Body parameter

{
  "auto.offset.reset": "latest",
  "consumer.request.timeout.ms": "1000",
  "fetch.min.bytes": "-1",
  "auto.commit.enable": "false"
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	createConsumers	Object that provides the parameters used when creating a consumer

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer instance created successfully	None
400	Bad Request	Bad Request, topic does not exist	errorResponse

Delete Consumer

DELETE /consumers

Delete the specified consumer instance from a consumer group.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer instance deleted successfully	None

Subscriptions

Subscribe

POST /consumers/subscription

Subscribe a consumer to a topic to get dynamically assigned partitions. If a subscription already exists, it will be replaced.

Note: Use either the Subscribe endpoint (to assign partitions automatically) or Assign Partitions endpoint (to manually assign partitions).

Body parameter

{
  "topics": [
    "DEMO-OMNICHANNEL-APP-DEV"
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	subscription	Object that provides the topic to subscribe to

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer successfully subscribed to the topic specified	None

Unsubscribe

DELETE /consumers/subscription

Remove the existing topic subscription for a consumer.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer successfully unsubscribed from the topic specified	None

Get Subscription

Example responses

204 Response

{
  "topics": [
    "DEMO-OMNICHANNEL-APP-DEV"
  ]
}

GET /consumers/subscription

Retrieve the topic to which the consumer is currently subscribed.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}

Responses

Status	Meaning	Description	Schema
204	No Content	Current subscribed topic	subscription

Records

Get Records

Example responses

200 Response

[
  {
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
      "service": "DLGaaS",
      "id": "3a4b511c-c06b-497f-ba0a-24c1f95a7e60"
    },
    "value": {},
    "partition": 0,
    "offset": 210
  }
]

GET /consumers/records

Fetch data for the topic to which the specified consumer is subscribed. This request requires the accept: application/json header.

Notes:

• An inactivity timeout is configured for customer instances, so they will be deleted if not active for a certain period. Therefore, Nuance recommends that you call the Get Records endpoint at a regular interval to keep the consumer instance alive. See the documentation for more information.
• In some cases, this endpoint returns an empty array even if there are event logs available. For example, this can happen if an internal fetch timeout occurs. Nuance recommends that you try this endpoint again before determining that all event logs have been fetched.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
Accept	header	string	Required accept: application/json header
consumer-name	header	string	Consumer name consumer-{UUIDv4}

Responses

Status	Meaning	Description	Schema
200	OK	Records fetched by the consumer instance, in JSON format.	jsonConsumerRecords

Assignments

Assign Partitions

POST /consumers/assignments

Manually assign a list of partitions to this consumer. This request requires both the content-type: application/json header and the accept: application/json.

Note: Use either the Subscribe endpoint (to assign partitions automatically) or Assign Partitions endpoint (to manually assign partitions).

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
Accept	header	string	Required accept: application/json header
Content-Type	header	string	Required content-type: application/json header
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	partitionsAssignment	Object that provides the partitions to assign to a consumer

Responses

Status	Meaning	Description	Schema
204	No Content	Partition successfully assigned to the consumer	None

Get Assigned Partitions

Example responses

200 Response

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

GET /consumers/assignments

Retrieve the list of partitions currently manually assigned to this consumer. This request requires the accept: application/json header.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
Accept	header	string	Required accept: application/json header
consumer-name	header	string	Consumer name consumer-{UUIDv4}

Responses

Status	Meaning	Description	Schema
200	OK	List of partitions assigned to the specified consumer	partitionsAssignment

Offsets

Get Committed Offsets

Example responses

200 Response

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412,
      "metadata": "string"
    }
  ]
}

POST /consumers/committed/offsets

Retrieve the last committed offsets for the partitions specified.

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	partitions	Object that provides the partitions for which to retrieve offsets

Responses

Status	Meaning	Description	Schema
200	OK	Last committed offsets for the partitions specified	committedOffsets

Commit Offsets

POST /consumers/offsets

Commit a list of offsets for the consumer. When the POST body is empty, this endpoint commits all the records that have been fetched by the consumer instance.

Body parameter

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	offsets	Object that provides the offsets to commit

Responses

Status	Meaning	Description	Schema
204	No Content	Committed offsets for the partitions specified	None

Positions

Override Offsets

POST /consumers/positions

Overrides the offsets that the consumer will use when fetching the next set of records.

Body parameter

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	offsets	Object that provides the offsets to use when fetching the next set of records

Responses

Status	Meaning	Description	Schema
204	No Content	Committed offsets for the partitions specified.	None

Seek First Offset

POST /consumers/positions/beginning

Seek to the first offset for each of the partitions specified.

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	partitions	Object that specifies the partitions for which to seek the first offsets

Responses

Status	Meaning	Description	Schema
204	No Content	The current offset is set to the first available offset	None

Seek Last Offset

POST /consumers/positions/end

Seek to the last offset for each of the partitions specified.

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number}
consumer-name	header	string	Consumer name consumer-{UUIDv4}
body	body	partitions	Object that specifies the partitions for which to see the last offsets

Responses

Status	Meaning	Description	Schema
204	No Content	The current offset is set to the last available offset	None

Schemas

createConsumers

{
  "auto.offset.reset": "latest",
  "consumer.request.timeout.ms": "1000",
  "fetch.min.bytes": "-1",
  "auto.commit.enable": "false"
}

Object that provides the parameters used when creating a consumer

Properties

Name	Type	Description
auto.offset.reset	string	Offset reset policy for the consumer group. Determines the initial position to use when the group is created. Valid values are earliest (earliest available offset), latest (latest available offset), and none (current offset if the consumer group is present). If there is no current offset associated with the consumer group, the option none will throw an error. This property applies only if no offset is committed. When an offset is committed, auto.offset.reset is not considered. Default is latest.
consumer.request.timeout.ms	string	The maximum amount of time to wait for messages for a request if the maximum number of messages (as per fetch.min.bytes) has not yet been reached. Default is 1000.
fetch.min.bytes	string	Minimum number of bytes in message keys and values that must be returned by a single request before the timeout specified in consumer.request.timeout.ms expires. A value of -1 disables this functionality. Default is -1.
auto.commit.enable	string	If set to true, the committed offset will be automatically updated when the record is fetched using the Get Records endpoint. Default is false.

Enumerated Values

Property	Value
auto.offset.reset	earliest
auto.offset.reset	latest
auto.offset.reset	none
auto.commit.enable	true
auto.commit.enable	false

committedOffsets

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412,
      "metadata": "string"
    }
  ]
}

Last committed offsets for the partitions specified

Properties

Name	Type	Description
offsets	[any]	Array of committed offsets
» topic	string	Name of the topic. In Mix, this is the AppID.
» partition	number	Number of the partition
» offset	number	Committed offset
» metadata	string	Currently not used

offsets

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412
    }
  ]
}

Object that defines offsets

Properties

Name	Type	Description
offsets	[any]	Array of offsets
» topic	string	Name of the topic. In Mix, this is the AppID.
» partition	number	Number of the partition
» offset	number	Offset number

partitions

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Object that defines an array of partitions

Properties

Name	Type	Description
partitions	[any]	partitions
» topic	string	Name of the topic. In Mix, this is the AppID.
» partition	number	Number of the partition

topicPartitions

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "partitions": [
    0
  ]
}

Object that defines a topic and its available partitions

Properties

Name	Type	Description
topic	string	Name of the topic. In Mix, this is the AppID.
partitions	[number]	Array of partition numbers

errorResponse

{
  "error_code": 401,
  "message": "The request could not be authorized"
}

Object that defines an error response

Properties

Name	Type	Description
error_code	string	Error code
message	string	Error message

jsonConsumerRecords

[
  {
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
      "service": "DLGaaS",
      "id": "3a4b511c-c06b-497f-ba0a-24c1f95a7e60"
    },
    "value": {},
    "partition": 0,
    "offset": 210
  }
]

Records fetched by the consumer

Properties

Name	Type	Description
Records fetched by the consumer	[object]	Records fetched by the consumer
topic	string	Name of the topic. In Mix, this is the AppID.
key	object	Key for the message
» service	any	Name of the runtime service that generated this message
» id	any	Randomly-generated UUID that uniquely identifies the message
value	object	Contents of the Kafka message
partition	number	Partition number for this topic
offset	number	Offset for this record

subscription

{
  "topics": [
    "DEMO-OMNICHANNEL-APP-DEV"
  ]
}

Object that describes a topic subscription

Properties

Name	Type	Description
topics	[string]	Topic name. In Mix, this is the AppID.

partitionsAssignment

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Object that describes the assigned partitions

Properties

Name	Type	Description
partitions	[object]	Array of assigned partitions
» topic	string	Topic name. In Mix, this is the AppID.
» partition	number	Partition number

partitionsOffsets

{
  "beginning_offset": 441,
  "end_offset": 641
}

Object that provides the first and last available offsets in a partition

Properties

Name	Type	Description
beginning_offset	number	First available offset in the partition specified
end_offset	number	Last available offset in the partition specified

AFSS API

The Audio File Storage Service (AFSS) API lets you download audio files of the utterances that were sent for recognition to ASRaaS.

The API requires the URN of the resource to download, in the following format:

urn:nuance-mix:log:service:audio/{service}/{audioFileName}

For ASRaaS, this is obtained from the audioUrn field in the asr-callsummary payload event logs.

For example, consider the following event log:

...
"absEndTime": 1616532498670,
"IP!S3Events": "L4sFrmz...9MMxM0EC2s=",
"audioDump": "http://minio-prod.fabric-prod.svc.cluster.local:9000/afss/fd228157ac40497a6f9a51f43f476b8c40dc270de6d3f218810745104242b098/asraas/44620fec-4c48-4d41-82d9-68fd8732f84f",
"audioUrn": "urn:nuance-mix:log:service:audio/asraas/44620fec-4c48-4d41-82d9-68fd8732f84f"
...

To get the audio file for this URN, the REST endpoint would look something like this:

GET https://log.api.nuance.co.uk/api/v1/audio?urn=urn:nuance-mix:log:service:audio/asraas/44620fec-4c48-4d41-82d9-68fd8732f84f

Your client application must provide an access token to access the AFSS API. This access token can be shared between the AFSS and the Kafka APIs. See Authorization for details.

AFSS version: v1

This document covers v1 of the Nuance AFSS Rest API.

AFSS URL

https://log.api.nuance.co.uk

Audio resources

Get audio resources

GET /api/v1/audio

Retrieve the audio file that corresponds to the URN specified.

Parameters

Name	In	Type	Description
urn	query	string	URN of the resource to download. For ASRaaS, this is obtained with the audioUrn field.

Responses

Status	Meaning	Description	Schema
200	OK	Success	Inline

Change log

2022-07-06

• Updated application-ended section to specify that this event is not generated when the session times out.
• Provided additional details on dialog session termination events.

2022-04-07

• Updated the managing sensitive information in an application section to describe how question and answer nodes masked as sensitive are redacted in the logs.

2022-03-09

• Added a new section on managing sensitive information in an application.
• Documented the new partial redaction feature and updated the following sections accordingly:
  • Dialog values possibly masked
  • NLU values possibly masked

2022-02-09

Fixed incorrect description of userid under data.

2021-10-26

Added default values for createConsumers object.

2021-09-29

• Added DLGaaS Status request to the list of Source and type fields and Request and response objects.
• Added reporting field to session-start event.
• Added new reporting-vars event.
• Updated transition event to add new collected fields.
• Added session-stopped event.

2021-06-16

Added retention period for audio files.