You can use the extended streaming feature to stream audio content to
Dialogflow and stream human agent suggestions back.
Normally, you half-close or tell the Dialogflow API when to end the stream to generate the final transcript and Agent Assist suggestions. This happens at conversation turns, where the API receives the parameter is_final=true from the recognition result, StreamingAnalyzeContentResponse.recognition_result.
Extended streaming reduces the need for half-closing at conversation turns. It extends the connection timeout to three minutes, during which you can send audio streams without half-closing. The Dialogflow API automatically sends the final transcripts and Agent Assist suggestions back to the stream. You only restart the stream if it times out.
Streaming basics
The Agent Assist extended streaming feature is similar to audio
streaming for CCAI Transcription. Your system
streams audio data to the API, and Dialogflow streams back
StreamingAnalyzeContentResponse data. The returned data includes suggestions
for your human agents.
Extended Streaming only supports Agent Assist stage. See conversation
stage. To use this feature:
Call the streamingAnalyzeContent method and set the following fields:
StreamingAnalyzeContentRequest.audio_config.audio_encoding:
AUDIO_ENCODING_LINEAR_16 or AUDIO_ENCODING_MULAW
enable_extended_streaming: true.
The first streamingAnalyzeContent request prepares the stream and sets
your audio configuration.
In subsequent requests, you send audio bytes to the stream.
As long as you continue to send audio, you will keep receiving suggestions.
You don't need to manually close the stream. It will close automatically
once Agent Assist detects that utterances have stopped.
Restart the stream (which includes resending the initial audio
configuration) in the following cases:
The stream is broken (the stream stopped when it wasn't supposed to).
Your audio data is approaching the automatic timeout at 3 minutes.
You received a re-tryable error. You can retry up to three times.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-04-12 UTC."],[[["Extended Streaming allows streaming audio content to Dialogflow and receiving human agent suggestions in return, similar to audio streaming for CCAI Transcription."],["This feature, available \"as is\" and with potentially limited support under the Pre-GA Offerings Terms, is accessible via the `streamingAnalyzeContent` method in the RPC API and client libraries."],["To initiate Extended Streaming, users must set `enable_extended_streaming` to `true` and provide appropriate audio configurations (`AUDIO_ENCODING_LINEAR_16` or `AUDIO_ENCODING_MULAW`)."],["The stream remains active as long as audio data is sent, automatically closing when utterances stop, and it will automatically timeout after 3 minutes of activity."],["Users should restart the stream if it breaks unexpectedly, if approaching the three-minute timeout, or after receiving a re-tryable error (up to three retries are allowed)."]]],[]]
