Pipedream

In pursuit of that goal is Pipedream. Pipedream is a serverless, real-time API platform for programmatically transforming data from one state to another without a human hand. It provides multiple points of entry for Third Party Software as a Service to enable developers to connect APIs remarkably fast with triggers or actions in Node.js. With managed authentication for 300+ OAuth or key-based APIs, Pipedream makes environment variables accessible for credentials or references in any workflow. It is serverless so the platform does not require developers to deploy their automation engines to the cloud.

Since Pipedream works with any or all APIs, Pipedream’s platform automates the process of connecting Telnyx PSTN calls seamlessly. Symbl.ai is already enabled Pipedream. Pipedream therefore empowers developers to combine Symbl.ai’s Conversation Intelligence API platform with Telnyx to create powerful automations that save time, money, or effort throughout the entire process of a call.

In your automation you program Pipedream to enable call recording through the interpretation of a WebHook’s response data. After automating the recording, you enable Symbl.ai to process the recorded call upon completion. You literally daisy chain the three together, Telnyx, Pipedream, Symbl.ai.

Prerequisites

Pipedream Account
Telnyx Account
Symbl.ai Account
cURL

Telnyx

Telnyx is a real-time communications platform with full, feature-rich voice functionalities, making it quick and easy to set up and port numbers around the world, configure messaging, control VoIP and IP network functions, and define how and where communications can be used in real time. The Telnyx API can power a host of specialty applications, from call tracking to cloud-based PBX, dynamic security, and authentication use cases.

Telnyx’s Voice APIs provide you as a developer with access to recorded calls through a single API call.

Steps

To analyze a recorded call with Telnyx, you set up your Voice API configuration with Telnyx’s Call Control within the Telnyx Portal. Just follow these simple steps:

1. Sign up for a Telnyx account.
2. Create an application to configure how you connect your calls.
3. Buy or port a phone number to receive inbound calls, and assign this number to your new application.
4. Create an outbound voice profile to make outbound calls, and assign your application to this new profile.
5. Record the call
6. Generate a Public URL for recorded calls

These steps are summarized on Telnyx’s quickstart guide.

Portal Signup

Let’s begin with the easiest step of all. Head to the signup page to sign up for your free Telnyx account. It’ll give you access to our Mission Control Portal, where you can buy numbers, set up and manage connections and a lot more. The first step is to search and purchase a Telnyx Voice API number.

Search and Purchase

After clinking on numbers, you are provided with the ability to search and buy a number:

After selecting a number, checkout.

Editing your Call Control App

After selecting a number, you navigate to the Call Control App.

After clicking “Create Your First Application”, you edit your Call Control App.

There are three steps to take in editing the Call Control App:

1. Create an App Name.
2. Copy of the Call Control Application number as later the number is used in a cURL command as a uuid.
3. Paste the link you called your “Your unique URL.”

With those three steps out of the way, your Call Control App is ready to enable outbound calls through POST requests with cURL commands.

The First cURL Request

To initiate a Telnyx Voice API call for recording, you make the following POST request:

curl -X POST   --header "Content-Type: application/json"   --header "Accept: application/json"   --header "Authorization: Bearer YOUR_API_KEY"   --data '{"connection_id": "UUID", "to": "YOUR TELEPHONE NUMBER", "from": "YOUR TELNYX VOICE API NUMBER"}'   https://api.telnyx.com/v2/calls

As you can see above, you will need to change the variables for the following:

1. YOUR_API_KEY
2. YOUR_TELEPHONE_NUMBER
3. YOUR_TELNYX_VOICE_API_NUMBER
4. UUID

The UUID is the Call Control Application number from number 2 in the section above.
After the request successfully POSTs, the webhook server returns a POST request with return data like so:

{   "data": {     "event_type": "call.initiated",     "id": "3d4d3f3a-cfd6-437e-9592-7554a9996a45",     "occurred_at": "2021-09-14T18:52:40.155477Z",     "payload": {       "call_control_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",       "call_leg_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",       "call_session_id": "ef8ab3d8-158c-11ec-b771-02420a0d4168",       "client_state": null,       "connection_id": "1725531088727573914",       "direction": "outgoing",       "from": "+16182216200",       "state": "bridging",       "to": "+19168739475"     },     "record_type": "event"   },   "meta": {     "attempt": 1,     "delivered_to": "https://YOURPIPEDREAMWEBHOOK.pipedream.net"   } }

The Second cURL Request

To initiate the recording for Telnyx’s Voice API call, you execute the following POST request withcURL:

curl -X POST    --header "Content-Type: application/json"    --header "Accept: application/json"    --header "Authorization: Bearer YOUR_API_KEY"    --data '{"format": "mp3", "channels": "single"}'    https://api.telnyx.com/v2/calls/{call_control_id}/actions/record_start

As you can see above, you will need to change the variables for the following:

1. YOUR_API_KEY
2. call_control_id

The call_control_id comes directly from the return data from the last POST request.
After the request successfully POSTs, the terminal returns the following data:

{  "data": {   "record_type": "recording",   "recording_id": "c84048c4-e4cd-48f6-a94e-cf8be3fbd884",   "result": "ok"    }

The next step is to hang up the call. With the call hung up, the webhook server serves the public URL for accessing the recorded call.

Public URLs for Recorded Calls

After recording a call with Telnyx’s Voice API, you make a call with one of the call commands to expose the webhook for the public recording URL (e.g., public_recording_urls) in the call.recording.saved object.
After successfully making the call, the return data POSTs to the webhook server. Within the return data for the JSON, the recording_urls looks like the following before it is posted to the web hook:

{  "data": {   "event_type": "call.recording.saved",   "id": "0ccc7b54-4df3-4bca-a65a-3da1ecc777f0",   "occurred_at": "2018-02-02T22:25:27.521992Z",   "payload": {    "call_leg_id": "428c31b6-7af4-4bcb-b7f5-5013ef9657c1",    "call_session_id": "428c31b6-abf3-3bc1-b7f4-5013ef9657c1",    "channels": "single",    "client_state": "aGF2ZSBhIG5pY2UgZGF5ID1d",    "connection_id": "7267xxxxxxxxxxxxxx",    "public_recording_urls": {     "mp3": "http://example.com/recording.mp3",     "wav": "http://example.com/recording.wav"    },    "recording_ended_at": "2018-02-02T22:25:27.521992Z",    "recording_started_at": "2018-02-02T22:20:27.521992Z",    "recording_urls": {     "mp3": "http://example.com/recording.mp3",     "wav": "http://example.com/recording.wav"    }   },   "record_type": "event"  }    }

After posting to the webhook, the recording_urls looks like the following before it is posted to the web hook:

{  "data": {    "event_type": "call.recording.saved",    "id": "47ee6238-6c0d-4f49-bd46-b2b00ea42fbd",    "occurred_at": "2021-09-14T18:54:58.177200Z",    "payload": {   "call_control_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",   "call_leg_id": "ef8ffc44-158c-11ec-8de2-02420a0d4168",   "call_session_id": "ef8ab3d8-158c-11ec-b771-02420a0d4168",   "channels": "single",   "client_state": null,   "connection_id": "1725531088727573914",   "end_time": "2021-09-14T18:54:57.517467Z",   "format": "mp3",   "public_recording_urls": {},   "recording_ended_at": "2021-09-14T18:54:57.517467Z",   "recording_id": "c84048c4-e4cd-48f6-a94e-cf8be3fbd884",   "recording_started_at": "2021-09-14T18:54:38.008036Z",   "recording_urls": {     "mp3": "https://s3.amazonaws.com/telephony-recorder-prod/b880fcdb-d93b-46e9-9c62-2aa7c5b2eb06/2021-09-14/ef8ffc44-158c-11ec-8de2-02420a0d4168-1631645677.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASDC5E2MJDPKMH7UW%2F20210914%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210914T185458Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=b586b5ed26263edb7fd13c7b227c094e4ee9dd45403278d6f9335ee034c4d65d"   },   "start_time": "2021-09-14T18:54:38.008036Z"    },    "record_type": "event"  },  "meta": {    "attempt": 1,    "delivered_to": "https://YOURPIPEDREAMWEBHOOK.pipedream.net"  }   }

Symbl.ai requires nothing more than that URL to handle analyzing the Telnyx recorded call. The great thing about Telnyx is that Telnyx empowers developers to provide options with which to determine the file type of the recorded call hosted on the S3 bucket. In recording_urls above the file type is .mp3. Symbl.ai’s Conversation Intelligence API platform accepts .mp3 files right out of the box.

Pipedream

Pipedream is one of the most developer friendly platforms for automations. It greets developers with a special onboarding immediately upon login. In Pipedream’s special onboarding, Pipedream teaches you how to create an HTTP triggered workflow where you 1) trigger a workflow on an HTTP request, 2) retrieve data from an API using Node.js with npm, 3) deploy your changes for testing.

Automation Process for Pipedream with Telnyx

Pipedream initiates its automation after the enabled WebHook server returns the response data from the final POST request in the chain of POST requests made to create a recorded call. The chain is 1) v2/calls, 2) v2/calls/{call_control_id}/actions/record_start where the response data to the latter call is an .mp3 file hosted by AWS from the section on recording_urls. You access that data from the data pipeline in the following way: event.body.data.recording_urls. Accordingly, the process by which you automate the analysis of recorded call is a multi-step process so after you have automated the call recording, the next step is to create another automation for transferring the recorded call to Symbl.ai for processing.

Automating Telnyx Call Recording with Symbl.ai within Pipedream

After familiarizing yourself with Pipedream, the next step is to create a workflow. A workflow is a set of steps designed to respond to events over a webhook. In your workflow for automating call recording, you configure two steps, one for a trigger, another for processing.

Environment Variables

However, before creating a workflow, you need to configure your environment variables appropriately. Navigate to Settings > Environment Variables > New Environment Variable. For YOUR_ENV_VAR_NAME, set the name as TELNYX_API_KEY, add the value. After adding the value, the next step is to add your Symbl.ai appId and appSecret with their respective values for each.

The Event Flow

Pipedream is a reactive platform that reacts to events in real-time. Telnyx, in the same regard, is reactive, an event driven architecture where every action creates an opposite and equal reaction. In that reactive system, Telnyx creates events when a call is initiated (i.e., call.initiated), hung up (i.e., call.hangup), answered (i.e., call.answered), or recorded (i.e., call.recording.saved) for each of which Telnyx issues a POST request on its webhook. The request / response model, common among all APIs of the world, is a basic HTTP framework in motion.

Pipedream WebHook

Here is where both Pipedream as well as Telnyx are exceptionally developer friendly. Pipedream creates a link for a webhook whereas Telnyx receives the created links for webhooks. One fits into the other like as in the two go together like hand and glove. The next step is to take your Pipedream webhook so that you can add it to Telnyx.

1. Login into the Telnyx platform
2. Navigate from the Dashboard to Call Control / Applications
3. Select your number.
4. Navigate from Edit your Call Control App > Send a webhook to the URL.
5. Paste your Pipedream webhook url: “delivered_to”: “https://YOURPIPEDREAMWEBHOOK.pipedream.net“

After you have added your Pipedream WebHook url, close the Telnyx account. With the WebHook set up in the Telnyx platform, the next step is to create the workflow for responding to the events the Telnyx APIs POST to the Pipedream WebHook.

Your Pipedream Webhook in Telnyx’s Call Control App

Steps for Trigger

The first step is to set up a trigger. A trigger in the Pipedream platform is a step in the portal where the events posted to the webhook are handled. The workflow triggers the workflow on each request. The advantage that Pipedream provides is that the platform’s triggers not only handle the event, they handle the event’s JSON return data, parsing it entirely upon receipt. In terms of Telnyx, the webhook therefore parses the return data for the Telnyx events mentioned earlier: (i.e., call.initiated), hung up (i.e., call.hangup), answered (i.e., call.answered), or recorded (i.e., call.recording.saved).

Triggering

Pipedream’s own developer onborading teaches you how to properly configure an HTTP API as a trigger. Pipedream generates a unique URL to trigger :(i.e., your https://YOURPIPEDREAMWEBHOOK.pipedream.net). Since Pipedream automatically detects the JSON schema of the HTTP API’s return data, Pipedream provides developers with a readily made solution for enabling developers to gain access to the data contained in the step: steps.trigger.event. Consequently, you access the return data from that single instance (i.e., .event).

Processing with Node.js

In the next step, which is a sandbox for Node.js, Pipedream feeds the events or steps into a large async call (i.e., an invention of genius for its simplicity). In the async step, the Node.js app completes the calls to functions for handling the different types of functions with which to program automatic reactions. The async function call looks like the following:

async (event, steps) => {}

In the async call declare the following:

const axios = require('axios') const name = "Pipedream's Automation for Telnyx with Symbl.ai" const type = event.body.data.event_type

In the next step you need to program the Telnyx events mentioned earlier: (i.e., call.initiated), hung up (i.e., call.hangup), answered (i.e., call.answered), or recorded (i.e., call.recording.saved) in the following way:
After you have identified the async function call, the next step is to program a type to toggle, as you toggle from one type of event to another. In the following, you toggle from one type of Telnyx event to another:

const type = event.body.data.event_type if (type === "call.recording.saved") { console.log("The Call is Saved") } else if (type === "call.answered") { console.log("The Call is Answered") const callControlId = event.body.data.payload.call_control_id const recordingInitiated = await startRecording(callControlId) console.log("The recording is initiated:", recordingInitiated) } else if (type === "call.initiated" || type === "call.hangup") { $end() } else { $end() }

The code there is straight forward an if/else statement for toggling on different types of events. The next step is to program the reactions to be triggered in regards to different types of events. The first reaction is to trigger recording when the call is answered.

const startRecording = async (callControlId) => { const data = { format: 'mp3', channels: 'single' } try { const response = await axios({ method: 'POST', url: `https://api.telnyx.com/v2/calls/${callControlId}/actions/record_start`, headers: {Authorization: `Bearer ${process.env.TELNYX_API_KEY}`}, data: data }) return response.data } catch (error) { console.log(error) } }

After a call is answered, the callControlId is returned. With the callControlId returned, the next step is to make a POST request to Telnyx’s API endpoint for recording calls: /actions/record_start. After the function for enabling call recording is configured, the next step is designed to save the call for processing.

if (type === "call.recording.saved") { // Save the url from the event for recording const url = event.body.data.payload.recording_urls.mp3 //Submit Job const token = await symblToken() const job = await pushAudioUrl(url, token) console.log(job) }

The next step is to program the functions called to trigger both Symbl.ai’s authentication as well as its audio processing.

const symblToken = async () => { const data = { type: "application", appId: process.env.SYMBL_APPID, appSecret: process.env.SYMBL_SECRET } try { const response = await axios({ method: 'POST' , url: 'https://api.symbl.ai/oauth2/token:generate', headers: {'Content-Type': 'application/json'}, data: data }) return response.data.accessToken } catch (error) { console.log(error) } }

The function symblToken() is an HTTP POST request that passes the appId and appSecret over to Symbl.a’s authentication API, returning a token.
The next function is an HTTP POST request to Symbl.ai’s audio processing API. API sends the url from event.body.data.payload.recording_urls.mp3 over to Symbl.ai for processing.

const pushAudioUrl = async (url, token) => { const data = { name: name, url: url, webhookUrl: 'https://YOURPIPEDREAMWEBHOOK.pipedream.net' } try { const response = await axios({ method: 'POST', url: 'https://api.symbl.ai/v1/process/audio/url', headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${token}` }, data: data }) return response.data } catch (error) { console.log(error) } }

Upon Completion

After the automation is complete, you receive both a conversationId as well as a jobId with which to check the results you have achieved through the automation.

Pipedream’s Node.js with a conversationId & jobId

Community

Symbl.ai’s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai’s GitHub.

The post Create powerful automations on your Telnyx calls with Symbl.ai and Pipedream appeared first on Symbl.ai.

Symbl.ai’s WebSocket

Symbl.ai’s WebSocket is one of the most famous software products in the history of telecommunications, real-time communications, or real-time engagements, as it enables developers to transform a live conversation or its aspects into a nexus of connectivity with third party software as a service from APIs all over the world through the primary providers of real-time voice, video, message or broadcast SDKs such as Twilio, Vonage, or Agora.

Set up

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those, you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

$ curl -k -X POST "https://api.symbl.ai/oauth2/token:generate"
      -H "accept: application/json"
      -H "Content-Type: application/json"
      -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

Ideally a token server would handle authentication (with code that makes RESTful API call for generating token) so that neither the appSecret nor the appId were ever exposed. However, cURL sets you up immediately anyway. With thex-api-key handy you are now ready to establish a WebSocket endpoint for performing live transcription.

WebSocket’s Endpoint

To enable the WebSocket, you configure two values as query parameters that are fed directly into the WebSocket API’s endpoint. In turn you feed the WebSocket API’s endpoint directly into JavaScript’s own software for enabling real-time conversations. Here are the first two values:

const uniqueMeetingId = btoa('EMAIL@ADDRESS.COM'); 
const accessToken = '';

With these two values for the WebSocket’s API endpoint set, you feed these directly into the WebSocket API’s endpoint:

const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;

If you want to test your WebSocket before any further integration, you load that endpoint with the data for uniqueMeetingId together with the accessToken into Hoppscotch.io, a free, fast, sleek API request builder.

Creating an Instance of JavaScript’s native WebSocket API

The next step is to create an instance of JavaScript’s native WebSocket API:

const ws = new WebSocket(symblEndpoint);

On that instance, you call methods specific to handling live transcription.

// Fired when a message is received from the WebSocket server 
ws.onmessage = (event) => {
    console.log(event);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror = (err) => {
    console.error(err);
};
// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
    console.info('Connection to websocket closed');
};
// Fired when the connection succeeds.
ws.onopen = (event) => {
    ws.send(JSON.stringify({
        type: 'start_request',
        meetingTitle: 'Websockets How-to', // Conversation name    
        insightTypes: ['question', 'action_item'], // Will enable insight generation 
        config: {
            confidenceThreshold: 0.5,
            languageCode: 'en-US',
            speechRecognition: {
                encoding: 'LINEAR16',
                sampleRateHertz: 44100,
            }
        },
        speaker: {
            userId: 'example@symbl.ai',
            name: 'Example Sample',
        }
    }));
};

To set up the stream for accessing a user’s media devices such as their laptop’s microphone, program the brower’s navigator accordingly:

const stream = await navigator.mediaDevices.getUserMedia({
    audio: true,
    video: false
});

The code above asks the user for permission to access his or her device through a pop-up. In mobile applications permission for access to devices is hardcoded into the controlling directory of an application binary such as in the manifest for Android of the property list for iOS. JavaScript doesn’t compile so there is no need to ask for permission so that you can ask for permission! To set up the stream to handle events, program the following:

const handleSuccess = (stream) => {
    const AudioContext = window.AudioContext;
    const context = new AudioContext();
    const source = context.createMediaStreamSource(stream);
    const processor = context.createScriptProcessor(1024, 1, 1);
    const gainNode = context.createGain();
    source.connect(gainNode);
    gainNode.connect(processor);
    processor.connect(context.destination);
    processor.onaudioprocess = (e) => {
        // convert to 16-bit payload
        const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
        const targetBuffer = new Int16Array(inputData.length);
        for (let index = inputData.length; index > 0; index--) {
            targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
        }
        // Send to websocket
        if (ws.readyState === WebSocket.OPEN) {
            ws.send(targetBuffer.buffer);
        }
    };
};

Here is the complete code:

const uniqueMeetingId = btoa('email@address.com');
const accessToken = '';
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;
const ws = new WebSocket(symblEndpoint);
// Fired when a message is received from the WebSocket server 
ws.onmessage = (event) => {
    console.log(event);
};
// Fired when the WebSocket closes unexpectedly due to an error or lost connetion 
ws.onerror = (err) => {
    console.error(err);
};
// Fired when the WebSocket connection has been closed 
ws.onclose = (event) => {
    console.info('Connection to websocket closed');
};
// Fired when the connection succeeds. 
ws.onopen = (event) => {
    ws.send(JSON.stringify({
        type: 'start_request',
        meetingTitle: 'Websockets How-to', // Conversation name
        insightTypes: ['question', 'action_item'], // Will enable insight generation
        config: {
            confidenceThreshold: 0.5,
            languageCode: 'en-US',
            speechRecognition: {
                encoding: 'LINEAR16',
                sampleRateHertz: 44100,
            }
        },
        speaker: {
            userId: 'example@symbl.ai',
            name: 'Example Sample',
        }
    }));
};
const stream = await navigator.mediaDevices.getUserMedia({
    audio: true,
    video: false
});
const handleSuccess = (stream) => {
    const AudioContext = window.AudioContext;
    const context = new AudioContext();
    const source = context.createMediaStreamSource(stream);
    const processor = context.createScriptProcessor(1024, 1, 1);
    const gainNode = context.createGain();
    source.connect(gainNode);
    gainNode.connect(processor);
    processor.connect(context.destination);
    processor.onaudioprocess = (e) => { // convert to 16-bit payload    
        const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
        const targetBuffer = new Int16Array(inputData.length);
        for (let index = inputData.length; index > 0; index--) {
            targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
        }
        // Send to websocket
        if (ws.readyState === WebSocket.OPEN) {
            ws.send(targetBuffer.buffer);
        }
    };
};
handleSuccess(stream);

Twilio SDK for Real-Time Communication

The next step to enable real-time conversation intelligence in your Twilio SDK is to create an app with the Twilio Programmable JS SDK. Since the app is already live on Twilio’s GitHub (https://github.com/twilio/twilio-video-app-react), the only thing that you need to do is to tie Symbl.ai’s WebSocket to Twilio’s. H2: How to Add Symbl.ai’s WebSocket to Twilio There are a few prerequisite steps that you have to take to tie Symbl.ai’s WebSocket directly to Twilio’s. These are a couple of account specific steps:

1. Navigate to Symbl.ai’s GitHub where you find Twilio’s https://github.com/symblai/symbl-twilio-video-react. It is a multi-party video conferencing application that demonstrates Symbl’s Real-time APIs. This application is inspired by Twilio’s video app and is built using twilio-video.js and Create React App.
2. Sign up for a free Twilio account. After logging in, make sure to cache your Twilio appSecret and appId.

With those two steps out of the way, your next step is to add a Symbl-Twilio Connector to your application.

Twilio’s WebSocket

Since the entire app is already built for you, the next step for you is to tie Symbl.ai’s WebSocket to Twilio’s WebSocket. From Symbl.ai’s GitHub repository, navigation to the SymblWebSocketAPI.js file where you find nearly the exact same code for running the WebSocket in the browser as you do for running the WebSocket in an app like Twilio Programmable JS SDK‘s app. Below you find account specific details such as appId, appSecret, together with a base URL for the WebSocket API. After the account specific details, there is a class SymblWebSocketAPI. The class is defined with methods similar, as well as properties, similar, if not identical, to those mentioned earlier for Symbl.ai’s WebSocket in the browser. If you were to match the code for Symbl.ai’s WebSocket in the browser to the code for Symbl.ai’s WebSocket in Twilio’s Programmable Video JS SDK, there would be a one-to-one mapping of parts. The last but not least step you have to complete is to tie the two WebSockets together. To tie Twilio’s Programmable Video JS SDK’s WebSocket to Symbl.ai’s WebSocket, pass the targetBuffer.buffer into the Symbl.ai’s WebSocket’s ws.send() function.

Setup and Deploy

This application offers two options for authorizing your Symbl account, in the application, or via the included token server. Your Twilio account will be authorized via the token server. The default behavior is for your Symbl account to authorize in-app. A dialog box will be shown automatically if you’re opening the app for the first time. In the config.js file you will find enableInAppCredentials set to true. For this option you are not required to update the .env file with Symbl credentials.

Credentials

Demo

If you would like to test out our live demo, feel free to stage your own app on Heroku. If you would like to check out the code on our GitHub repo, please checkout the following link: https://github.com/symblai/symbl-twilio-video-react.

Twilio & Symbl.ai

Symbl.ai’s APIs are already running hundreds of thousands of open WebSocket connections simultaneously throughout the API World. Since Symbl.ai is a cross-domain, platform agnostic, general purpose Conversation Intelligence API platform for developers designed to enable voice, video, message or broadcast builders to augment their real-time engagements with Artificial Intelligence at the bleeding edge of sound, Symbl.ai’s APIs are designed to operate with any or all real-time communication applications or platforms. As Symbl.ai already runs on Agora.io, Dolby.io, Telnyx, or Twilio, Symbl.ai runs anywhere anytime. Sign up for your free transcription account today to discover what Symbl.ai can do for you.

Community

Symbl.ai‘s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai’s GitHub.  

The post Real-Time Conversation Intelligence with Twilio’s WebSockets in JavaScript appeared first on Symbl.ai.

With Symbl.ai developers are empowered to transform those channels digitally with the introduction of new layers of continuously deployed Artificial Intelligence models built to drive real-time engagement on Unix kernels of any type, including Ubuntu!

Python3 on a Local Programming Environment with Ubuntu 20.04

Automating speech recognition within an instance of Ubuntu on a local programming environment with Python3 requires a cloud such as Digital Ocean. To deploy an instance of Ubuntu on Digital Ocean, 1) sign up on the DigitalOcean new account registration page, 2) set up a droplet, choose the image entitled Ubuntu 20.04×64, 3) choose a plan, set up SSH, name the droplet, until finally clicking “Create Droplet” whereafter DO provides you with the IP address of your droplet. After copying the IP address, securely connect with the droplet through your terminal by executing the following command via SSH as a root user:

ssh root@your_IP_address

Setting up Python 3

Upon login to your new instance of Ubuntu on Digital Ocean with SSH, your next step is to configure Python3 to run on your instance of Ubuntu. The next few steps rely upon the command line. Open the command line Terminal by clicking on the search button in the upper-left hand corner of your screen or by pressing theCTRL, ALT, and Ttogether at the same time. Ubuntu 20.04 ships with Python3 pre-installed. To ensure your version of Python3 is updated, you run a command against the package manager with apt (i.e., Advanced Packaging Tool). If you are a macOS user who is familiar with Brew (i.e., “the missing package manager for macOS), then you are already familiar with the basic functionality of apt. The following few commands are designed to update your packages:

sudo apt update  sudo apt -y upgrade

Upon completion the next step is to check the version of Python3 your instance of Ubuntu has:

python3 -V

Ensure that you have the latest version of Python3. The next step is to add the pip command. The command is absolutely essential to installing or managing programming packages for Python.

sudo apt install -y python3-pip

With the pip command now installed into your version of Python3, you’re ready to add, remove or modify Python packages from your project as you see fit. You could run pip3 install package_name, for instance, to add packages like pandas by running pip3 install pandas, numpy by running pip3 install numpy, or matplotlib by running pip3 install matplotlib, where you’re able Last but not least is to install build essentials such as the GNU debugger, g++/GNU compiler collection, G++, dpkg-dev, or GCC (i.e., for transforming languages like Python3 into machine readable I/Os). In addition to build essentials are development tools such as OpenSSL project’s implementation of the SSL and TLS cryptographic protocols for secure communication over the Internet or the Foreign Function Interface library (i.e., for communications with the compiler). After installing packages specific to the compiler available on your instance of Ubuntu, you add a command for installing header files you need to build Python extensions.

sudo apt-get install build-essential libssl-dev libffi-dev python-dev

Setting up a Virtual Environment in Ubuntu’s Terminal

Since Python has its own unique way of downloading, storing, and resolving packages (or modules), its virtual environments are designed to isolate projects so that modules do not unintentionally cross pollinate the dependencies of one project with another.  You add the  venv module as part of the standard Python 3 library for creating virtual environments in the following way:

sudo apt-get install -y python3-venv

Without any fanfare or exaggeration, a project is merely a localized root directory with folders containing scripts so creating “environments” entails not much more than issuing a few commands from #!Shebang!

mkdir environments cd environments

After creating the directory where you will store your environment, the next step is to initiate the environment inside of the /environments directory with the following Python3 command:

python3 -m venv asr_env

With the asr_env set up to run Python3, the next step is to perform a sanity check with the #!Shebang ls command in the following way:

ls asr_env

If the asr_env’s output is the following, then the environment is properly set up:

bin include lib lib64 pyvenv.cfg share

Last but not least is the step to activate

source asr_env/bin/activate

Upon activation your environment displays a precursor in your environment’s namesake:

(asr_env) symblai@symblai: ~/environments$

With these steps in place, the next step is to create scripts in Nano with Symbl.ai’s Python SDK for automating speech recognition on Ubuntu.

Automating Speech Recognition on Ubuntu in Nano with Symbl.ai’s Python SDK

The ability to create instances of Linux distributions such as Ubuntu on droplets of Digital Ocean empowers developers with the ability to expand the number of operating systems with which to create brand new experiences for conversations, be they on macOS, Windows, or your favorite Linux distribution. Symbl.ai’s Conversation Intelligence API platform operates anywhere APIs operate, extending beyond merely automating the process of detecting speech events asynchronously or in real-time to granulized analysis of speech in general. Symbl.ai’s speech recognition APIs come as part of its overall Conversation Intelligence API platform where action-items, follow-ups, questions, topics, or sentiment analysis. Among the tools available for developers is Symbl.ai’s Python SDK.

How to Sign up for a Free Account with Symbl.ai

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

Nano

With the account credentials you need for running Symbl.ai’s Python SDK you create a script in Nano for analysing recorded calls.

Script

Create a new file called asr_recorded_call with the following command:

(asr_env) symblai@symblai: ~/environments$ nano asr_recorded_call.py

In your asr_recorded_call.py script add the following

import symbl recording_url = input("Enter your recording_url: ") print(recording_url) payload = { 'url':'recording_url', } credentials={app_id: <app_id>, app_secret: <app_secret>} conversation = symbl.Audio.process_url(payload=payload, credentials=credentials) response = conversation.get_messages(parameters={'sentiment': True}) timestamps, polarities = [], [] for message in response.messages:    timestamps.append(message.start_time)    polarities.append(message.sentiment.polarity.score) conversation_id = conversation.get_conversation_id() print("Your conversation ID is:", conversation_id)

After adding the above code to your script, exit nano by typing the CTRLand X keys while pressing y when prompted to save the file. To run the script, execute the following command.

(asr_env) symblai@symblai: ~/environments$ python  asr_recorded_call.py

Telnyx, Dolby.io, or Agora

In your script you add a url you received from a recorded call made on a PSTN number from Telnyx. With Symbl.ai, however, you are in a position as a developer to create an entire technology stack out of APIs for conversations augmented with artificial intelligence from Symbl.ai with any of the leading real-time communications, engagement, or CPaaS platforms such as  Agora.io, Dolby.io, or Telnyx.

Conclusion

With Symbl.ai’s Python SDK editors like Nano provide developers the opportunity to create automations with scripts in the terminal on instances of Linux distributions without much more than a few lines of #!Shebang! In your demo application you created an automation script for automating the process of analyzing a call with one of the leading CPaaS providers in the telecommunications space, Telnyx. Your script automatically recognizes speech.

What’s Next

Apart from learning how to code different things in Python3, there are many additional aspects of Symbl.ai’s Python SDK you may leverage for further automating speech recognition on instances of Linux distributions. In particular you are in a position to generate enhanced analytics for your audio or video conversations with Dolby.io or Agora.io, all of which may be automated entirely, or create time series analyses of speech events for an display of sentiment analysis with a graph. In addition to adding bells and whistles to your graphs like labels, you may consider how Symbl.ai’s Python SDK provides access to many of Symbl.ai’s APIs. In particular, you can enable Symbl.ai’s Python SDK on the real-time WebSockets API or any one of the Async APIs for voice, video, or messaging. A complete list is here:

• WebSocket API
• Telephony API
• Async Text API
• Async Audio (File) API
• Async Audio (URL) API
• Async Video (File) API
• Async Video (URL) API

In addition, Symbl.ai’s Python SDK contains links to enable Symbl.ai on its Telephony APIs with Python special methods. The methods for dialing into a call, work on the Session Initiation Protocol (SIP). With Symbl.ai’s Python SDK dialed into a call, your in a position to expand the experiences around calls in real-time. After connecting to a SIP call in real-time, you subscribe to Symbl.ai’s events such as events for contextual insights like follow-ups, questions, or action-items, or you create a request to deliver a post-meeting summary directly to your inbox.

Community

Symbl.ai‘s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai’s GitHub.  

The post Automate Speech Recognition on Ubuntu in the Nano editor with Symbl.ai’s Python SDK appeared first on Symbl.ai.

Explore Symbl.ai’s New Postman Public Workspace

At Symbl.ai, a conversation intelligence (CI) platform for developers, we recently published the Symbl.ai public workspace  on Postman for you to explore our APIs. Symbl.ai’s APIs, the vast majority of which operate with the leading internet protocols for real-time communications (RTC) or real-time engagement (RTE)—such as PSTN, SIP, WebSockets, or RTMP—are designed to develop new experiences around conversations through connections, transformations, or visualizations. Now, with our brand-new public workspace, developers can easily test out our APIs and field new ideas in building new conversation-driven experiences with AI. We created our public workspace to enable developers to try out our APIs for asynchronously uploading voice, video, or text messages for bulk offline processing. For instance, Symbl.ai’s new Management API provides you with the ability to create fully customized sets of vocabularies for tracking all words semantically related to those vocabularies lists on conversations. In addition, Symbl.ai’s Conversation APIs return analysis on conversation data for such things as topics, follow-ups, questions, action items, and more.

Symbl.ai’s public workspace Overview page in Postman

To check out these APIs, download the Symbl.ai API collection for free or explore the public workspace online in Postman. You will find a total of 35 requests in the following areas:

• Authentication
• Async APIs: Ingest and process video, audio, and text files
• Telephony API: Connect to VOIP and capture conversations in real time
• Conversation APIs: Extract contextual insights, and conversation analytics
• Management APIs: Create and track custom insights and intents

How to get started

Register for a free Symbl.ai account here. Grab both your `appId` and your `appSecret` to authenticate with Postman so that you receive your `x-api-key`. Below is an example of how to authenticate with the Symbl.ai public workspace.

Generating an authentication token

Symbl.ai and Postman: supporting API developers

Symbl.ai’s support for APIs is part of a broad initiative to help developers throughout their journeys. Our APIs are designed to facilitate the reduction of what is often described as the “Time to First API Call”—which is how long it takes a user to successfully make an API call with a response. Symbl.ai’s public workspace on Postman is meant to help you make that first API call right out of the gate. As Postman is the GitHub of the API world, another great advantage of Symbl.ai’s public workspace is its ability to provide developers with new opportunities to create cross-API playgrounds. With Postman public workspaces, you can fork a workspace. After forking you can add APIs to that workspace to create your own personalized workspace. Once you fork from Symbl.ai’s public workspace, you can pull updates from our public workspace to stay synchronized with our latest APIs. In addition to the new Symbl.ai public workspace, we partnered with Dolby.io to release a collaborative workspace. In the Dolby.io Community public workspace, Dolby.io has assembled a collection for transcribing meetings with Symbl.ai. Check out Dolby.io Community public workspace.

Postman Visualizations and Symbl.ai & Topic Previews

In addition to providing support for developer communities such as those around Dolby.io to create brand new real-time engagements with Symbl.ai’s publicly available Postman workspaces, Postman’s unique feature for visualizations enables you to create human readable forms of data. Topics, for instance, are important topics for artificial intelligence. In AI, topics are important as the primary focus or FOCII of a conversation. If two people are discussing which bags of coffee they like best, then coffee, for instance, is the focus of their conversation. Symbl.ai’s API for Topics, which is the topic of the following blog, enables developers to access AI insights from topics without anything more than a mere curl command or a configuration for hitting a RESTful API. With Postman’s own unique visualization tool Symbl.ai experiments with the creation of mindmaps for topics. These are really useful, if you would like to provide a visualization to users in ClubHouse-like voice-chat conversations a visually appealing display of what topics are being discussed in a room next to you that you aren’t occupying at the moment. Symbl.ai’s APIs enables you as a developer to add topics to such things as Twitter Spaces to create topic previews in real-time. In addition, Symbl.ai’s Conversation Intelligence platform’s AI driven topic previews’ feature is capable of being enabled on the backend for administrators to match users’ spoken interests or enable targeted ads campaigns.

Postman spotlights Symbl.ai as a startup to watch

Postman actively encourages developers to share ideas and build more—now with the help of public workspaces. We are excited to join Postman in this effort, and we are proud that Symbl.ai and our public workspace have been included in Postman’s Startups to Watch list spotlighting “promising, young startups who have the potential to disrupt their respective industries.” Please share your customized workspaces with Symbl.ai so that the team at Symbl.ai has a chance to spotlight your Postman development efforts on social media. Stuck? Free feel to ask us any questions on our Slack channel or send us an email at developer@symbl.ai.  

The post Git, Fork, or Clone Symbl.ai’s Postman Workspace appeared first on Symbl.ai.

In the following blog you refine an analysis of Symbl.ai recorded streams with Dolby.io’s Media Enhance API. You analyze your Symbl.ai recorded stream with Symbl.ai’s Python SDK to produce a time series analysis of sentiments. After your initial analysis does not provide a speech event, you refine your analysis by removing background noise with Dolby.io’s Media Enhance API. Upon being refined by Dolby.io’s Media Enhance API, Symbl.ai’s Python SDK provides you with a new analysis replete with new speech events for which an improved time series analysis arises.

Your journey traverses through an 1) introduction, 2) a Symbl.ai analysis of a Dolby.io recorded call with

Jayson DeLancey‘s Postman public workspace, 3) the refinement, 4) the improved analysis.

1) Introduction: Dolby.io’s Media Enhance API

One such strategy for improving the quality of a recorded stream for creating wildly connected calls is through the use of the Media Enhance API present on Dolby.io. The Media Enhance API works to remove the noise, isolate the spoken audio, and correct the volume and tone of the sample for a more modern representation of the speech.

The Dolby.io Media Enhance API is designed to improve the sound of your media. Our algorithms analyze your media and make corrections to quickly improve and produce professional quality audio. By using this API you can automate typically labor intensive and specialized media processing tasks to improve:

• Speech leveling
• Noise reduction
• Loudness correction
• Speech isolation
• Sibilance reduction
• Plosive reduction
• Dynamic equalization
• Tone shaping

With these features, developers may refine their analyses of wildly connected calls in a way that exposes a recorded stream to one of many enhancements for overall improved analyses.

2) How to Analyze a Call with Symbl.ai

Jayson DeLancey, the Developer Relations Manager at Dolby.io, produced a Postman Public Workspace as a guide for developers to embark upon the process of analyzing recorded streams with Symbl.ai. A public workspace empowers developers to collaborate when building a project with a convenient Graphical User Interface (GUI) for making RESTful API requests.

Called Transcribe Media with Symbl.ai, the public workspace is available here: https://www.postman.com/dolbyio/workspace/dolby-io-community/overview. To use the public workspace, you’ll need to:

1. Fork the collection into your own workspace
2. Fork the environment to add your API Keys for both Dolby.io and Symbl.ai accounts.

Within the Public Workspace, you navigate from folder to folder for what might be a typical series of steps for extending beyond mere automatic speech recognition to analyses.

To summarize the Dolby.io steps:

1. Authenticate with Dolby.io
2. Find a Meeting
3. Identify a Recording
4. Download a Recording

To summarize the Symbl.ai steps,

1. Authenticate
2. Submit an MP3 Recording
3. Get Transcript

With these steps you should be able to come up with a recorded call. If you have not, Dolby.io’s publically available onboarding provides you with access to a video, which may serve your purpose in lieu of a recorded call. Symbl.ai’s Conversation Intelligence API platform analyzes both video as well as audio so the switch from call to video does not affect the outcome at all. Here is the video file:

Analyzing the Call with Symbl.ai’s Python SDK

The next step is to analyze the audio or video with Symbl.ai’s Python SDK. You could use JavaScript or cURL, however, the Python SDK provides you with an end-to-end solution for both handling the processing as well as the post-processing analysis through the application of libraries such as Numpy, Matplotlib, or Pandas.

The first step is to import Symbl.ai’s Python SDK into your environment.

import symbl

The next step is to configure a payload for a call to Symbl.ai’s video processing API.

payload = {  'url':'',  }  credentials_dict = {'app_id': '', 'app_secret': ''}

The next step is to make the call to Symbl.ai’s video processing API.

conversation = symbl.Video.process_url(payload=payload, credentials=credentials_dict)

The next step is to save the call into an object through which sentiments may be fetched before being appended into empty initialized arrays:

response = conversation.get_messages(parameters={'sentiment': True}) timestamps, polarities = [], []     for message in response.messages:         timestamps.append(message.start_time)         polarities.append(message.sentiment.polarity.score)

The next step is to import Pandas, which brings with it both Numpy as well as Matplotlib, into your environment.

import pandas as pd

Last but not least is to configure your DataFrame for plotting.

df = pd.DataFrame({"timestamps": timestamps,"polarities": polarities})  df.plot(x="timestamps", y="polarities")

Here is the full code sample:

import symbl payload = {
          'url':'',
           }
 credentials_dict = {'app_id': '', 'app_secret': ''}
 conversation = symbl.Video.process_url(payload=payload, credentials=credentials_dict)
 response = conversation.get_messages(parameters={'sentiment': True})
 timestamps, polarities = [], [] for message in response.messages:
     timestamps.append(message.start_time)
     polarities.append(message.sentiment.polarity.score)
 import pandas as pd df = pd.DataFrame({"timestamps": timestamps,"polarities": polarities})
 df.plot(x="timestamps", y="polarities")

The result is that the video analyzed by Symbl.ai’s Python SDK is incapable of producing a proper time series analysis of speaker events, since the background noise clouds the speech events.

The resulting graph shows you that background noise affects your ability to provide improved analyses of calls. The next step is to remove the background noise with Dolby.io’s Media Enhance APIs.

3) Refining Symbl.ai’s Analysis of Streams with Dolby.io’s Media Enhance API

The first step to refining Symbl.ai’s analysis of streams is to sign up for an account at Dolby.io: https://dashboard.dolby.io/signup/ 

After you have signed up the next step is to configure a few API calls, which you will do with cURL to help speed up the process. The first API call is to transform your API key and secret into an access token. You can use token authentication or alternatively the x-api-key for authentication.

For x-api-key authentication, you pass only the api key in the x-api-key header. https://docs.dolby.io/media-apis/docs/authentication#api-key-authentication

For token authentication, you generate a token and then pass it in as an Authorization Bearer header. https://docs.dolby.io/media-apis/docs/authentication#oauth-bearer-token-authentication

1. Convert your combined API key and secret into a base 64 key:

export BASE64_KEY_SECRET=`echo -n "$DOLBYIO_API_KEY:$DOLBYIO_API_SECRET" | base64`

2. After exporting the keys, the next step is to obtain your JSON Web token (JWT).

curl -X POST 'https://api.dolby.com/media/oauth2/token'
        --header 'Authorization: Basic $BASE64_KEY_SECRET'
        --header 'Content-Type: application/x-www-form-urlencoded'
        --data 'grant_type=client_credentials'

3. With the JSON Web token returned from your API call, you transfer the video from earlier to Dolby.io for refinement in the following way:

curl -X POST https://api.dolby.com/media/enhance
       --header "Authentication: Bearer $DOLBYIO_ACCESS_TOKEN"
       --data '{
          "input": "s3://dolbyio/public/shelby/airplane.original.mp4",
          "output": "dlb://out/example-enhanced.mp4"
      }'

In our case, you will simply upload that same file to Google Cloud to keep it as a URL or use the URL our team has uploaded to the cloud.

4. Improved Analysis

The next step to demonstrating the improved analysis is to analyze the same audio/video file for speech events. Here is the code snippet:

import symbl
  payload = {  'url':'https://storage.googleapis.com/support-test-1/EnhancedMediaFile.mp4',  }
  credentials_dict = {'app_id': '', 'app_secret': ''}
  conversation = symbl.Video.process_url(payload=payload, credentials=credentials_dict)
  response = conversation.get_messages(parameters={'sentiment': True})
  timestamps, polarities = [],
 []
  for message in response.messages:
  timestamps.append(message.start_time)
  polarities.append(message.sentiment.polarity.score)
  import pandas as pd
  df = pd.DataFrame({"timestamps": timestamps,"polarities": polarities})
  df.plot(x="timestamps", y="polarities")

The result is a significant improvement over the previous video. As opposed to the previous video where no speech events were detected, the graph shows that speech events have now been detected.

Dolby.io’s Media Enhance API therefore provides you with the ability to improve Symbl.ai analyses of streams. Although there is still more work to do, anyone who becomes proficient in Dolby.io’s suite of Media Enhance APIs is set to transform a poor audio experience into a rich opportunity for analysis.

What’s Next

Contrasting your refined analysis of a timeseries with a waveform spectogram may be a great next step for you in attaining a deeper understanding of a wildly connected call. The points of the timeseries analysis may correlate with the points of the spectogram to provide insights into the context of event driven conversations.

Dolby.io’s Developer Relations Team under the the management of Jayson DeLancey provides a guide on how to create a spectogram analysis of a recorded stream. The guide covers how to import dependencies such as Numpy, Matploblib, or Scipy to derive the spectogram. With Python you refine the recorded stream just as is done in your current configuration. With the refined stream, you generate a spectrogram.

In addition to adding bells and whistles like a spectrogram to your analyses like labels, you may consider how Symbl.ai’s Python SDK provides access to many of Symbl.ai’s APIs. In particular, you can enable Symbl.ai’s Python SDK on the real-time Telephony API (i.e., Telephony API (in Real-time) or any one of the Async APIs for voice, video, or messaging. A complete list is here:

• Async Text API
• Async Audio (File) API
• Async Audio (URL) API
• Async Video (File) API
• Async Video (URL) API

In addition, Symbl.ai’s Python SDK contains links to enable Symbl.ai on its Telephony APIs with Python special methods. The methods for dialing into a call, work on the Session Initiation Protocol (SIP). With Symbl.ai’s Python SDK dialed into a call, your in a position to expand the experiences around calls in real-time. After connecting to a SIP call in real-time, you subscribe to Symbl.ai’s events such as events for contextual insights like follow-ups, questions, or action-items, or you create a request to deliver a post-meeting summary directly to your inbox.

Community

Symbl.ai‘s invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai’s GitHub.

The post Generate enhanced analytics for your audio or video conversations with Dolby.io appeared first on Symbl.ai.

In the age of lightning-fast internet speeds and real-time interactions, there’s simply no room (or tolerance) for delays — particularly with voice/video applications.

Enter WebSocket with its low-latency, bidirectional messaging ability that suits any application designed for real-time communication within a browser. To get you up and running with this popular protocol, here’s a brief overview of what you can use to implement it, and a few clever ways you can take your voice/video app the extra mile.

The why and where of WebSockets

The general-use WebSocket protocol transforms the simplicity of HTTP’s request/response model into a highly reactive, event-driven, duplex protocol.This allows end-users to send and receive messages without delay and lets savvy developers bring real-time communication to anyone with a web browser and an internet connection.

The type of apps that benefit the most from WebSockets typically involve real-time communication between multiple users and/or have to relay constantly-changing server-side data. This includes:

• Instant messaging apps
• Real-time collaboration
• Multiplayer games
• Real-time feeds (sports, social, finance, etc.)
• Location-based apps
• Audio/video calls

If you want to look under the hood and learn how WebSockets actually work, check out our post on WebSockets vs. SIP.

Using WebSockets with open-source libraries

Implementing WebSockets can be tricky if your app needs more than just basic support. Luckily, you can abstract away much of the setup using open-source libraries, including: 

Socket.io

This popular app framework consists of a Node.js server and a Javascript client library for the browser. Unlike vanilla WebSockets, this library can establish a connection even if there’s a personal firewall or antivirus software and will handle a dropped connection automatically.

It also has the benefit of falling back on other technologies like AJAX long polling, Flash Socket, and many others if WebSockets isn’t supported, helping you create real-time apps that work everywhere.

SocketCluster

This is a real-time, client-server messaging framework for Node.js that’s ideal for building chat systems. It can handle an unlimited number of processes/hosts, and is easy to scale after you’ve deployed your app to a Kubernetes cluster. This library takes slightly more effort to install than the easy breezy Socket.io, but it’s still reasonably simple to set up

By the way, if you’re creating brand new schemes for real-time engagements over a WebSocket protocol enabled with Node.js, check out Symbl.ai’s Node.js SDK to make it happen right out of the box. To make your job even easier, Symbl.ai already runs on the most popular microservices, like Twilio, Agora.io, and AWS.

SockJS

SockJS mimics the WebSockets API but uses a SockJS Javascript object instead. Some devs may find the need for this WebSocket emulator questionable, considering most modern browsers natively support WebSockets. But, there are certain environments that might not support the WebSocket protocol — like corporate proxies, for example.

μWebSockets

µWebSockets is described as ‘the obvious starting point for any real-time web project with high demands.’ It’s generally considered the fastest WebSocket server implementation available and is actually used by SocketCluster for its performance and stability.

Taking your app up a notch with APIs

With so many voice/video apps already on the market, it’s worth thinking about elevating yours with dedicated APIs that work over WebSockets. This would give you all the advantages of real-time, two-way communication, while squeezing the most value out of your data.

For example, Symbl.ai’s Real-Time WebSocket API lets you plug in conversation intelligence using a suite of APIs to swiftly upgrade your app with features like:

Real-time intelligence: Analyze audio and surface useful insights when you need them — like whether a customer has called before, what their previous interaction was about and what information to give them next. You can analyze audio either in real time or asynchronously, and also access other useful data like speaker ratios, silence and overlap through Symbl.ai’s Analytics API.

Sentiment analysis: Pick up on people’s tone and word choices to determine their emotional state. Then use that information to help steer the conversation toward a positive outcome. Symbl.ai does this by transcribing speech in real time and analyzing those messages to pull sentiments.

Accurate transcriptions: Capture conversations (from a recording or in real time) with human-level comprehension. With advanced speech recognition and contextual AI, you can get highly accurate notes and summaries with added information like action items, topics, questions, and intents.

Automated actions: Translate insights into action, like automatically schedule a follow-up meeting for specific team members or send an email summary to a customer after a call.

Of course, you could build these capabilities yourself using open-source libraries, but the trade-off is a slower time to market and more effort needed to scale. Your best bet is to harness existing APIs, like Symbl.ai’s, that have already done the heavy lifting for you. Then all you need to do is focus on turning accurate, real-time insights into actionable data.

Implementing WebSockets with Symbl.ai

To try out Symbl.ai’s Streaming API in the browser with your microphone, sign up for a free account. Once you’ve done that, authenticate with cURL or Postman to get a bearer authorization token. Then, add that token to the line below called accessToken. Lastly, open a Chrome browser, hit Control + J + Command,  drop the code with the accessToken in the browser, and hit enter!

/**  * The JWT token you get after authenticating with our API.  * Check the Authentication section of the documentation for more details.  */ const accessToken = "" const uniqueMeetingId = btoa("user@example.com") const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`; const ws = new WebSocket(symblEndpoint); // Fired when a message is received from the WebSocket server ws.onmessage = (event) => {   // You can find the conversationId in event.message.data.conversationId;   const data = JSON.parse(event.data);   if (data.type === 'message' && data.message.hasOwnProperty('data')) {     console.log('conversationId', data.message.data.conversationId);   }   if (data.type === 'message_response') {     for (let message of data.messages) {       console.log('Transcript (more accurate): ', message.payload.content);     }   }   if (data.type === 'topic_response') {     for (let topic of data.topics) {       console.log('Topic detected: ', topic.phrases)     }   }   if (data.type === 'insight_response') {     for (let insight of data.insights) {       console.log('Insight detected: ', insight.payload.content);     }   }   if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {     console.log('Live transcript (less accurate): ', data.message.punctuated.transcript)   }   console.log(`Response type: ${data.type}. Object: `, data); }; // Fired when the WebSocket closes unexpectedly due to an error or lost connection ws.onerror  = (err) => {   console.error(err); }; // Fired when the WebSocket connection has been closed ws.onclose = (event) => {   console.info('Connection to websocket closed'); }; // Fired when the connection succeeds. ws.onopen = (event) => {   ws.send(JSON.stringify({     type: 'start_request',     meetingTitle: 'Websockets How-to', // Conversation name     insightTypes: ['question', 'action_item'], // Will enable insight generation     config: {       confidenceThreshold: 0.5,       languageCode: 'en-US',       speechRecognition: {         encoding: 'LINEAR16',         sampleRateHertz: 44100,       }     },     speaker: {       userId: 'example@symbl.ai',       name: 'Example Sample',     }   })); }; const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false }); /**  * The callback function which fires after a user gives the browser permission to use  * the computer's microphone. Starts a recording session which sends the audio stream to  * the WebSocket endpoint for processing.  */ const handleSuccess = (stream) => {   const AudioContext = window.AudioContext;   const context = new AudioContext();   const source = context.createMediaStreamSource(stream);   const processor = context.createScriptProcessor(1024, 1, 1);   const gainNode = context.createGain();   source.connect(gainNode);   gainNode.connect(processor);   processor.connect(context.destination);   processor.onaudioprocess = (e) => {     // convert to 16-bit payload     const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);     const targetBuffer = new Int16Array(inputData.length);     for (let index = inputData.length; index > 0; index--) {         targetBuffer[index] = 32767 * Math.min(1, inputData[index]);     }     // Send audio stream to websocket.     if (ws.readyState === WebSocket.OPEN) {       ws.send(targetBuffer.buffer);     }   }; }; handleSuccess(stream);

Additional reading

For more information on WebSockets, how to use them and how they compare to similar protocols, check out these sources:

How to use Symbl’s voice API over WebSocket to generate real-time insights

WebSocket or SIP: which is better for your app?

WebRTC or WebSockets: which is right for your app?

How come I can’t cURL a WebSocket?

How to secure your WebSocket connections

The post How to Use WebSockets in Your Voice/Video App appeared first on Symbl.ai.

Symbl.ai operates with CPaaS platforms like Telnyx like a hand in a glove. Symbl.ai’s Conversation Intelligence API platform extends the functionality of CPaaS platforms like Telnyx to encompass features that go beyond mere transcription to programmed artificial intelligence. Symbl.ai’s APIs empower you to analyze recorded calls for questions, follow-ups, action-items, topics, or sentiments on messages.

In the following blog you create a recorded call with Telnyx’s Voice API. After creating a recorded call with Telnyx, you analyze the conversation data from the Telnyx recorded call for sentiment analysis.

A brief overview is the following:

1. Create a webhook server.
2. Configure your new Telnyx developer account to initiate a recorded call with two cURL commands.
3. Configure your new Symbl.ai developer account to analyze that recorded call for sentiments with cURL commands.

Webhooks

Webhooks are a great way to receive real-time updates through the HTTP protocol. To receive real-time updates on your HTTP requests through cURLwith Telnyx, you create a webhook server. To create a web hook server, navigate to https://webhook.site/. At the site there is a line that says, “Your unique URL.” Next to the name is an option to copy the link. Copy the link.

You need your unique URL to set up your webhook server to receive real-time updates on the Telnyx Voice API number you select for initiating the Telnyx Voice API call.

Telnyx

Telnyx is a real-time communications platform with full, feature-rich voice functionalities, making it quick and easy to set up and port numbers around the world, configure messaging, control VoIP and IP network functions, and define how and where communications can be used in real time. The Telnyx API can power a host of specialty applications, from call tracking to cloud-based PBX, dynamic security, and authentication use cases.

Telnyx Voice APIs

Telnyx’s Voice APIs provide you as a developer with access to recorded calls through a single API call.

Steps

To analyze a recorded call with Telnyx, you set up your Voice API configuration with Telnyx’s Call Control within the Telnyx Portal. Just follow these simple steps:

1. Sign up for a Telnyx account.
2. Create an application to configure how you connect your calls.
3. Buy or port a phone number to receive inbound calls, and assign this number to your new application.
4. Create an outbound voice profile to make outbound calls, and assign your application to this new profile.
5. Record the call
6. Generate a Public URL for recorded calls

These steps are summarized on Telnyx’s quickstart guide.

Portal Signup

Let’s begin with the easiest step of all. Head to the signup page to sign up for your free Telnyx account. It’ll give you access to our Mission Control Portal, where you can buy numbers, set up and manage connections and a lot more. The first step is to search and purchase a Telnyx Voice API number.

Search and Purchase

After clinking on numbers, you are provided with the ability to search and buy a number:

After selecting a number, checkout.

Editing your Call Control App

After selecting a number, you navigate to the Call Control App.

After clicking “Create Your First Application”, you edit your Call Control App.

There are three steps to take in editing the Call Control App:

1. Create an App Name.
2. Copy of the Call Control Application number as later the number is used in a cURL command as a uuid.
3. Paste the link you called your “Your unique URL.”

With those three steps out of the way, your Call Control App is ready to enable outbound calls through POST requests with cURL commands.

The First cURL Request

To initiate a Telnyx Voice API call for recording, you make the following POST request:

curl --request POST
   --header "Content-Type: application/json"
   --header "Accept: application/json"
   --header "Authorization: Bearer YOUR_API_KEY"
   --data '{"connection_id": "UUID", "to": "YOUR TELEPHONE NUMBER", "from": "YOUR TELNYX VOICE API NUMBER"}'  https://api.telnyx.com/v2/calls

As you can see above, you will need to change the variables for the following:

1. YOUR_API_KEY
2. YOUR_TELEPHONE_NUMBER
3. YOUR_TELNYX_VOICE_API_NUMBER
4. UUID

The UUID is the Call Control Application number from number 2 in the section above.

After the request successfully POSTs, the webhook server returns a POST request with return data like so:

{
   "data":{
      "event_type":"call.initiated",
      "id":"3d4d3f3a-cfd6-437e-9592-7554a9996a45",
      "occurred_at":"2021-09-14T18:52:40.155477Z",
      "payload":{
         "call_control_id":"ef8ffc44-158c-11ec-8de2-02420a0d4168",
         "call_leg_id":"ef8ffc44-158c-11ec-8de2-02420a0d4168",
         "call_session_id":"ef8ab3d8-158c-11ec-b771-02420a0d4168",
         "client_state":null,
         "connection_id":"1725531088727573914",
         "direction":"outgoing",
         "from":"+16182216200",
         "state":"bridging",
         "to":"+19168739475"
      },
      "record_type":"event"
   },
   "meta":{
      "attempt":1,
      "delivered_to":"https://webhook.site/6919c563-e246-4ec6-a4fd-5918cda3f62a"
   }
}

Here is a view from the webhook server:

The Second cURL Request

To initiate the recording for Telnyx’s Voice API call, you execute the following POST request withcURL:

curl --request POST
   --header "Content-Type: application/json"
   --header "Accept: application/json"
   --header "Authorization: Bearer YOUR_API_KEY"
   --data '{"format": "mp3", "channels": "single"}'  https://api.telnyx.com/v2/calls/{call_control_id}/actions/record_start

As you can see above, you will need to change the variables for the following:

1. YOUR_API_KEY
2. call_control_id

The call_control_id comes directly from the return data from the last POST request.

After the request successfully POSTs, the terminal returns the following data:

{
   "data":{
      "record_type":"recording",
      "recording_id":"c84048c4-e4cd-48f6-a94e-cf8be3fbd884",
      "result":"ok"
   }
}

The next step is to hang up the call. With the call hung up, the webhook server serves the public URL for accessing the recorded call.

Public URLs for Recorded Calls

After recording a call with Telnyx’s Voice API, you make a call with one of the call commands to expose the webhook for the public recording URL (e.g., public_recording_urls) in the call.recording.saved object.

After successfully making the call, the return data POSTs to the webhook server. Within the return data for the JSON, the recording_urls looks like the following before it is posted to the web hook:

{
   "data":{
      "event_type":"call.recording.saved",
      "id":"0ccc7b54-4df3-4bca-a65a-3da1ecc777f0",
      "occurred_at":"2018-02-02T22:25:27.521992Z",
      "payload":{
         "call_leg_id":"428c31b6-7af4-4bcb-b7f5-5013ef9657c1",
         "call_session_id":"428c31b6-abf3-3bc1-b7f4-5013ef9657c1",
         "channels":"single",
         "client_state":"aGF2ZSBhIG5pY2UgZGF5ID1d",
         "connection_id":"7267xxxxxxxxxxxxxx",
         "public_recording_urls":{
            "mp3":"http://example.com/recording.mp3",
            "wav":"http://example.com/recording.wav"
         },
         "recording_ended_at":"2018-02-02T22:25:27.521992Z",
         "recording_started_at":"2018-02-02T22:20:27.521992Z",
         "recording_urls":{
            "mp3":"http://example.com/recording.mp3",
            "wav":"http://example.com/recording.wav"
         }
      },
      "record_type":"event"
   }
}

After posting to the webhook, the recording_urls looks like the following before it is posted to the web hook:

{
   "data":{
      "event_type":"call.recording.saved",
      "id":"47ee6238-6c0d-4f49-bd46-b2b00ea42fbd",
      "occurred_at":"2021-09-14T18:54:58.177200Z",
      "payload":{
         "call_control_id":"ef8ffc44-158c-11ec-8de2-02420a0d4168",
         "call_leg_id":"ef8ffc44-158c-11ec-8de2-02420a0d4168",
         "call_session_id":"ef8ab3d8-158c-11ec-b771-02420a0d4168",
         "channels":"single",
         "client_state":null,
         "connection_id":"1725531088727573914",
         "end_time":"2021-09-14T18:54:57.517467Z",
         "format":"mp3",
         "public_recording_urls":{
            
         },
         "recording_ended_at":"2021-09-14T18:54:57.517467Z",
         "recording_id":"c84048c4-e4cd-48f6-a94e-cf8be3fbd884",
         "recording_started_at":"2021-09-14T18:54:38.008036Z",
         "recording_urls":{
            "mp3":"https://s3.amazonaws.com/telephony-recorder-prod/b880fcdb-d93b-46e9-9c62-2aa7c5b2eb06/2021-09-14/ef8ffc44-158c-11ec-8de2-02420a0d4168-1631645677.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASDC5E2MJDPKMH7UW%2F20210914%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210914T185458Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=b586b5ed26263edb7fd13c7b227c094e4ee9dd45403278d6f9335ee034c4d65d"
         },
         "start_time":"2021-09-14T18:54:38.008036Z"
      },
      "record_type":"event"
   },
   "meta":{
      "attempt":1,
      "delivered_to":"https://webhook.site/6919c563-e246-4ec6-a4fd-5918cda3f62a"
   }
}

Symbl.ai requires nothing more than that URL to handle analyzing the Telnyx recorded call. The great thing about Telnyx is that Telnyx empowers developers to provide options with which to determine the file type of the recorded call hosted on the S3 bucket. In recording_urls above the file type is .mp3. Symbl.ai’s Conversation Intelligence API platform accepts .mp3files right out of the box.

Symbl.ai

Symbl.ai is a conversation intelligence API platform for developers to program the next generation of digital transformations voice, video, message or broadcast through Artificial Intelligence.

How to Sign up for a Free Account with Symbl.ai

Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you should authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl --insecure --request POST "https://api.symbl.ai/oauth2/token:generate"
      -H "accept: application/json"
      -H "Content-Type: application/json"
      -d "{ "type"": ""application""

The post How to Perform Sentiment Analysis on Telnyx Recorded Calls with Symbl.ai appeared first on Symbl.ai.

Running from September 25th, 2021 to September 26th, 2021, TADHack Global, sought to provide support for developers from many companies, among whom were both Telnyx and Symbl.ai, to build programmable communications alongside hundreds of developers across the globe.

TADHacks Global had over 1000 registrations, kicked off a new initiative TADHack TEENS in Sri Lanka (150 registrations), had 3 in-person locations (Berlin, Chicago, and Orlando), and South Africa achieved a MASSIVE 400 registrations, an amazing turn-out at TADHack Orlando at Valencia College.

Sponsors and Prizes

Among the global sponsors, there were: Symbl.ai, jambonz.org, Subspace, AWA Network, and Telnyx. Each provided support and resources to fuel the engagement for the hackathon.

In terms of a standalone prize for Symbl.ai, a team called RescueR (i.e., Ebtesam Al Haque, Doug Moore, Amy Sliwinski, Muntaser Syed, Vincent Tang, Davindra Tulsi, and Chris Woodle), create a submission where an autonomous rescue bot with sonar mapping and telecommunications technology to quicken response time of trapped victims. The same team won Telnyx’s $2500 Verify prize.

Telnyx and Symbl.ai’s Joint Prize

Telnyx and Symbl.ai provided a bonus prize of $4,000 to any team of developers capable of combining Telnyx CPaaS APIs with Symbl.ai’s programmable conversation intelligence APIs. The Joint Symbl.ai for “AI driven communications” became a team with phenomenal success. Of a series of impressive submissions, the team composed of Lily Madar and Steven Goodwin, created Wizard Chess, Colloqui11ly,  and Podcast Annotator.

Wizard Chess, their first submission, is designed to enable people to “play chess with your voice, making it accessible for those with visual impairments or less able motor skills. The code is a basic Node app, running on a server which starts by making a Telnyx call to the conference room, which symbl.ai then joins. From there, anything said into the phone is processed by symbl.ai and passed via websockets to the web page. Computer moves are relayed by voice in the opposite direction to only the human player. (In earlier drafts the voice spoke to everyone, including symbl.ai, but since the moves the computer spoke were invalid for the human, nothing bad happened!)”

In furtherance of their emphasis on accessibility, Colloqui11ly provided an accessible conferencing solution (using TTS and STT). It makes the conference available to all by allowing some users to interact via text message (both to “hear” the chat, and respond) while others get the audio experience.

Last but not least Lily Madar and Steven Goodwin created Podcast Annotator, a hands-free note-taking while listening to a podcast stream. You first start a podcast from the webpage, which initiates a call to your phone. (But it could also be triggered by DTMF tones.) At this point you can listen to the podcast, and say things like “Good point” and “must look that up”. These phrases are transcribed and added to a timeline for later review. Once the podcast ends, this review is sent via SMS.

If you would like to learn more about the event, check out Alan Quayle’s Enterprise Connect 2021 Hackathon Report: https://blog.tadhack.com/2021/09/26/tadhack-global-2021-summary/

The post TADHack Global 2021 Is a Wrap appeared first on Symbl.ai.

The hackathon process was divided into two phases, the first being an ideation phase, the second being an execution phase. During the ideation phase more than 80+ ideas were submitted and over 50 teams submitted completed projects. Upon thorough review of all the submissions, most being very impressive, we, the Symbl.ai team, had a hard time picking the three winners.

The theme of Symbl.ai’s HackerEarth “Integrate to Disrupt” hackathon sought to galvanize developers to access conversational intelligence throughout their voice, video, message or broadcast applications. In the overarching theme of the hackathon, the team at Symbl.ai sought to encourage developers to connect, transform or visualize.

• CONNECT your apps and automate voice enabled workflows. With Symbl.ai, you can process voice, video calls and stream the analytics and outcomes to your day to day apps like Jira, Slack, Trello, Zapier and more.
• VISUALIZE with real-time database and extend using cloud/lambda functions. With Symbl.ai, you can stream all transcription and conversation analytics to a real time database like Firebase, AWS, MongoDB etc and extend your apps with Cloud, Lambda, Realm functions and build cool outcomes.
• TRANSFORM voice or video conversations to another form of representation. With Symbl.ai, you can transform voice or video into chat, live feed, SMS, emoji, gifs, images etc…
•  INNOVATE and build whatever you want. With Symbl.ai, you have access to all of its AI capabilities by hitting some end points… use it to create something completely “out of the box”

Submissions were judged on the basis of a few guidelines. These guidelines were for developers to connect to Symbl.ai through one or more of its source types such as Connect to Symbl.ai through one of its Source Types (i.e., Telephony, Streaming, Async) with a Conversation API together with API from another company. Developers were required to provide a GitHub URL with a link to their code repositories with a YouTube video of no more than three minutes. Below are the winners.

Ultimately, we determined the top three projects and announced the results online through a live stream, the winning submissions were:

1. Desk – by Mehul Nirala
2. BOTALYSIS(CHATBOT & CONVERSATION ANALYZER) / Vishleshan: Video Analyzing Platform – by Team Vulcan and Team TriDev
3. Augmented Transcripts / Drill Down  – by a team led by Sophia Sydir

The first prize winner, Mehul Nirala, created an office utility service built on the comprehensive suite of APIs provided by Symbl.ai. His submission manages the task/action items discussed over meetings/conversations, maps them to appropriate assignees, and presents a summary table with interactive visualizations. His submission connects to the calls using telephone API. After connecting, Symbl.ai processes the calls to generate real-time analyses of sentiments. The data is recorded for storage or processed further through Symbl.ai’s end-to-end solution. Mehula Nirala provided a GitHub repository, a YouTube video tutorial, a slide deck. His submission provided a detailed relation to the themes of the hackathon.

The second place winners, Team Vulcan and Team TriDev, submitted projects that added additional APIs to their technology stacks. In Botalysis, the team submitted a solution using AI and sentiment analysis to assess calls internally for creating feedback of calls and incentivizing customer service hotlines to increase productivity. Their application uses Symbl.ai’s real-time APIs with Google to surface images from speech-to-text transcripts. The other winning team creates a solution for facilitating insights during interviews.

The third place winner, Augmented Transcripts, created an application for analyzing lecture notes within which API calls to external search engines embed links to definitions of key terms. The submission is a solution within the sphere of real-time online education. The other third place winner, Drill Down, is an online analytical cloud-native web application, a new technology to analyze, transcribe, or generate insights from videos. 

We also wanted to give a special shoutout to Ameet Bora for submitting THINKLOUDR.AI: Conversation Intelligence for Think Aloud Testing. THINKLOUDR.AI is designed to serve as an automations platform for automatically generating insights with Symbl.ai’s Conversation Intelligence API platform.

Pushing the limits of Conversation Intelligence

Participants were able to easily use Symbl.ai’s Conversation Intelligence (CI) platform to build and extend applications capable of understanding natural human conversations for different applications. To that extent, Symbl.ai encourages developers who both took part or did not have a chance to take part in Symbl.ai’s “Integrate to Disrupt” hackathon, to continue test the limits of conversation intelligence.

Symbl.ai’s comprehensive suite of products and APIs enable developers like you to easily build and deploy intelligent speech-to-text functionality, extract contextual insights, generate domain-specific insights and intelligence, and access advanced conversation analytics.

To continue your journey through Symbl.ai’s Conversation Intelligence API Developer Platform, Symbl.ai invites you to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai’s GitHub. Do not stop disrupting, connecting, transforming or visualizing with Symbl.ai!

The post The HackerEarth Hackathon “Integrate to Disrupt” appeared first on Symbl.ai.

Symbl.ai, a developer platform for deriving Artificial Intelligence from an analysis of conversation data, is releasing a brand new product called under its experimental wing of “Labs”, for summarization: the Summary API. The API provides developers with the ability to summarize conversation into summaries. The Summary API synthesizes vast amounts of conversation data into a summary.

Automatic summarization is a huge challenge for data scientists. AI models for summarization are required to filter domain-specific language in source texts (scientific research papers, articles, news, legal filings, conversations, or live events). AI models train for the identification of the prominent features in contextually relevant source texts to produce a consistent summary.

Symbl.ai’s Summary API provides a solution to this challenge. The API seeks the most relevant information based upon the context of the conversation. In line with its product suite for generating contextual insights such as follow-ups, questions or action-items, topics and subtopics, or personalized intents, Symbl.ai’s Summary API is an API. It is designed specifically for developers to utilize in their stacks for data science analysis, application development or enhancing real-time communication, engagements or streams.

If you would like to make a call to our Summary API, there are three steps. The first step is to sign up on our platform. The second step is to upload a conversation through our Async APIs with a query parameter for summarization set: `?enableSummary=true`. The third step is to make a call to our Summary API with the conversation ID you obtained from the Async: https://api-labs.symbl.ai/v1/conversations/{{conversation_id}}/summary. If you are unfamiliar with how to make API calls with `cURL` or a programming language like JavaScript or Python, Symbl.ai recommends you check out its collection or public workspace on Postman. Depending upon your set up, you run your APIs calls on a Postman desktop agent or its apps. Symbl.ai’s Postman public workspace is available here.

Currently, the Summary can be enabled with Async APIs. Support for Summarization in real-time will be added soon. However, if you are using Streaming or Telephony API, you can use the refresh parameter (i.e., `?refresh=true`) to generate the Summary after the conversation has ended.

The post Symbl.ai Releases its Summarization API appeared first on Symbl.ai.