Proactive Speech API
Speech Analytics API for the Automotive Industry
© 2019-2020 Proactive Dealer Solutions
Speech Analytics API for the Automotive Industry
© 2019-2020 Proactive Dealer Solutions
The Proactive Speech API offers two specific endpoints which are designed for differing scenarios.
If you are a third-party ISV integrating directly with our solution purely as a back end speech API, start below in the ISV Overview section
If you are using our BDC Central solution and wish to integrate an existing calling vendors recordings with BDC Central please start see the section BDC Central Integration Overview.
1.2 - Documentation Update - Custom Request Support.
1.1 - Documentation Update / Simplified Flow / Payload Additions.
The Proactive Speech API leverages our years of experience in the industry, combining domain expertise, Artificial Intelligence (AI), Machine Learning, and Natural Language Processing (NLP) to help the automotive industry extract critical intelligence from customer calls.
Applications integrate with the Proactive Speech API using a combination of standard REST API interfaces and Webhooks to process call recordings and extract actionable data in near real-time.
Using these analytics, you can provide unique insights to your customers.
A Sample Analytics Dashboard in Proactive Dealer Solutions' BDC Central Powered by the Proactive Speech API
REST API Summary
With the REST API, call recordings are submitted, and machine learning and NLP-based analytics are returned.
The API currenty can calculate over 50 unique indicators and metrics that provide insights into customer calls; and we are adding more with each ensuing release.
Submitting a call to the API is done via a POST with a JSON body describing parameters and processing directives. In order to use the Proactive Speech API and leverage our custom models, redaction, and other features, this specific configuration format.
POST application/json to https://speechapi.bdccentral.com/ingest-v2
Your JSON body should look like this (Note: the "custom_search" section is optional and included here for instructional purposes). The "custom metadata" section is optional and allows you to provide arbitrary data which will be echoed in the response. This is typically used to correlate the response data with the source data in your system of record.
{
"id": "YOUR_PROACIVE_CUSTOMER_ID",
"media_url": "https://xyz.com/some_wav_or_mp3_file.wav",
"speech_api_callback_url": "https://xyz.com/your_callback_handler",
"processing_directives": {
"customer_channel": "left",
"agent_channel": "right"
},
"custom_metadata": {
"some_custom_data": "Whatever you want",
"another_custom_field": "example",
"field_names_and_values_are_up_to_you": "I am a little tea cup"
},
"custom_search": [
{
"name": "summer promotion",
"speaker": "agent",
"search": "are you calling about our summer sales promotion",
"start_time_boundary": 0,
"end_time_boundary": 20000
},
{
"name": "competitor check",
"speaker": "caller",
"search": "hendrick bmw, baker audi, miller acura",
"start_time_boundary": 0,
"end_time_boundary": 0
}
]
}
The Proactive APIs operates using the concept of Web Hooks. In the speech_api_callback_url field above provide a callback URL where you will process the API response. When the Proactive API processing is completed it will then POST the final results to this web hook you provide in speech_api_callback_url.
In this example we have also employed an optional feature of the Proactive Speech API - custom search. Custom search lets you define words or phrases spoken by either the caller or agent that you would like to scan for in the call. You can also define time boundaries (in the above example we are only looking to see if the agent mentioned the summer promotion in the first 20 seconds of the call). The Proactive Speech API callback will contain data indicating the results of these custom searches. We will explore that more when we look at the response schema.
As discussed above, the Proactive Speech API will post all processing results to your provided Web Hook address.
Below we see a typical response.
{
"pds_customer_id":"YOUR_PROACIVE_CUSTOMER_ID",
"pds_call_id":"8c096466-b3b3-460f-bfca-57da34e39143",
"classification":{
"label":"sales",
"class":1,
"confidence":0.87,
"detail_label":"",
"detail_value":0
},
"call_flow":{
"announcement_disclaimer":1,
"receptionist_spoke":1,
"time_to_answer":17101,
"time_to_intended_agent":27172,
"caller_asked_for":"sales manager",
"agent_spoke":1,
"caller_spoke":1,
"went_to_ivr":0,
"went_to_vm":0,
"caller_left_vm":0,
"longest_hold_transfer_time":10071,
"number_of_transfers":1
},
"dealership":{
"name":"Anderson Audi Acura",
"confidence":"high"
},
"customer":{
"name":{
"first":"Kevin",
"last":"Westerly",
"confidence":"high"
},
"phone":{
"number_formatted":"(555) 555-1212",
"number_int":"5555551212",
"type":"cell",
"confidence":"high"
}
},
"predictive_models":[
{
"name":"Appt_Requested",
"label":"Yes",
"confidence":0.56,
"class":1
},
{
"name":"Appt_Set",
"label":"No",
"confidence":0.72,
"class":0
},
{
"name":"Set_Solid_Appt",
"label":"No",
"confidence":0.41,
"class":0
},
{
"name":"First_Name_Inquiry",
"label":"Yes",
"confidence":0.81,
"class":1
},
{
"name":"Last_Name_Inquiry",
"label":"No",
"confidence":0.75,
"class":0
},
{
"name":"Phone_Inquiry",
"label":"Yes",
"confidence":0.88,
"class":1
},
{
"name":"Email_Inquiry",
"label":"Yes",
"confidence":0.76,
"class":1
},
{
"name":"Went_To_Voicemail",
"label":"No",
"confidence":0.88,
"class":0
},
{
"name":"Is_Lead",
"label":"Yes",
"confidence":0.62,
"class":1
},
{
"name":"Agent_Identified_Themselves",
"label":"Yes",
"confidence":0.92,
"class":1
},
{
"name":"Service_Checked_For_Recalls",
"label":"N/A",
"confidence":0,
"class":0
}
],
"attributes":{
"price_inquiry":0,
"payment_inquiry":1,
"lease_mentioned":0,
"test_drive_inquiry":1,
"specific_time_period_interest":0,
"trade_in_mentioned":0,
"credit_mentioned":1,
"specific_stock_number_mentioned":0,
"specific_advertised_vehicle_mentioned":0,
"previous_interaction_mentioned":0,
"customer_provided_email":0,
"customer_provided_phone_number":1,
"customer_provided_vin_number":0,
"customer_provided_mileage":1,
"alternate_number":0,
"check_availability":1,
"confirm_in_writing":0,
"csi_watchlist":0,
"funnel_appt":0,
"promotion_mention":1,
"qualifying_questions":0,
"experimental":{
"objections":{
"price":0,
"payment":0,
"selection":0,
"trade_in_value":0,
"down_payment":0
}
}
},
"vehicles":[
{
"year":"",
"make":"Audi",
"model":"A3",
"confidence":"medium"
},
{
"year":"2018",
"make":"Acura",
"model":"MDX",
"confidence":"high"
}
],
"agent_names":[
"Mark",
"Andy"
],
"advertising_sources":[
"dealer website",
"autotrader"
],
"statistics":{
"total_time":406320,
"agent_speak_time":140349,
"agent_word_count":221,
"call_turn_count":50,
"caller_speak_time":252072,
"caller_word_count":298,
"silence_time":52415
},
"voice_metrics":{
"overtalk_incidents":6,
"agent_overtalk_incidents":4,
"agent_overtalk_ratio":0.22,
"caller_overtalk_incidents":2,
"caller_overtalk_ratio":0.03,
"silence_incidents":2
},
"agent_performance": {
"missed_opportunity": 0,
"agent_identified_themselves": 1,
"agent_identified_themselves_confidence": 0.90571513837468276,
"appt_requested": 1,
"appt_requested_confidence": 0.496521925919558,
"appt_set": 1,
"appt_set_confidence": 0.46710761442675047,
"set_solid_appt": 0,
"set_solid_appt_confidence": 0.37445514460904528,
"acquired_customer_first_name": 1,
"acquired_customer_first_name_confidence": 0.777765579961018,
"acquired_customer_last_name": 1,
"acquired_customer_last_name_confidence": 0.69338627422284671,
"acquired_customer_phone": 1,
"acquired_customer_phone_confidence": 0.66440585149522668,
"acquired_customer_email": 0,
"acquired_customer_email_confidence": 0.061198159723856685,
"agent_offered_to_callback": 0
},
"customer_keywords": {
"newer car": 3,
"diesel option": 2,
"newer lease": 2,
"financial situation": 2,
"cell phone": 1,
"bye bye": 1,
"silverado": 1,
"saturday": 1,
"afternoon": 1,
"perfect": 2
},
"agent_keywords": {
"enjoy improved features": 2,
"free depaula chevrolet": 2,
"vehicle exchange month": 2,
"calling depaula chevrolet": 2,
"depaula chevrolet app": 2,
"depaula chevrolet": 11,
"diesel option": 1,
"depaula dot": 3,
"pre owned": 2,
"phone number": 2,
"parts superstore": 2,
"cell phone": 1,
"monthly payment": 2,
"bye bye": 2,
"smartphone": 2,
"silverado": 1,
"saturday": 3,
"inventory": 2,
"afternoon": 1,
"location": 3,
"problem": 5,
"perfect": 2
},
"transcript":[
{
"speaker":1,
"text":"Thank you for calling Anderson Audi Acura how may I direct your call",
"start_time":17101,
"end_time":22023
},
{
"speaker":2,
"text":"Id like to speak with a sales manager please",
"start_time":23451,
"end_time":26711
},
{
"speaker":1,
"text":"One moment",
"start_time":27123,
"end_time":27664
},
{
"speaker":0,
"text":"[RING] [RING]",
"start_time":27665,
"end_time":29225
}
],
"narrative":"Call was answered by an automated assistant after 4 seconds. The receptionist answered after 17 seconds. Caller asked to speak with a sales manager. Call was transferred and answered after 10 seconds by the intended recipient.",
"narrativeHTML":"Call was answered by an automated assistant after 4 seconds. The receptionist answered after 17 seconds. Caller asked to speak with a sales manager. Call was transferred and answered after 10 seconds by the intended recipient.",
"custom_searches":[
{
"name":"summer promotion",
"value":0
},
{
"name":"competitor check",
"value":1
}
],
"experimental": {
"lead_score": 91,
"customer_sentiment": 0.11,
"agent_sentiment": 0.17
},
}
All Proactive Speech API responses will follow the schema above. This is a stable contract you can code against. While we will add new elements as the API is expanded, we will not remove elements to avoid breaking any serialization assumptions you might make when building out your code to ingest this response.
The pds_call_id echoes the mediaId field that identifies your call in the CallerCX platform. Be sure to store it when you post your request so that you can use this field to correlate the data.
All fields that reference time values are in milliseconds. So above, we can look at the time_to_answer metric and see call was answered by a receptionist after approximately 17 seconds. Interestingly, looking at the time_to_intended_agent metric, we can see that the AI determined the caller didn't actually make it to whom they were looking to speak with until 27 seconds into the call.
The classification section identifies how the API categorized the call. The possible values are as follows:
| failed |
| sales |
| service |
| status check |
| parts |
| other |
| finance |
| body shop |
If a call is classified as failed, The Proactive Speech API will provide classification information to understand what type of failure the call represents. This data will be included in the detail_label and detail_value of the classification section. See the table below for possible values:
| detail_label | detail_value |
| 1 | Unanswered - Neither an IVR system or receptionist answered the call. |
| 2 | Voicemail - Call went to VM but the caller did not leave a message. |
| 3 | IVR Abandon - Customer abandoned the call while navigating IVR system. |
| 4 | Hold Abandon - Customer abandoned the call while on hold. |
| 5 | Transfer Abandon - Customer abandoned the call while on hold during a transfer. |
| Model | Description |
| Appt_Requested | Agent offered appointment to caller |
| Appt_Set | An appointment was set |
| Set_Solid_Appt | Determines whether the appointment was set for a specific date and time |
| First_Name_Inquiry | Agent acquired caller's first name |
| Last_Name_Inquiry | Agent acquired caller's last name |
| Phone_Inquiry | Agent acquired caller's phone number |
| Email_Inquiry | Agent acquired caller's email address |
| Went_To_Voicemail | Call went to voicemail and caller did not leave a message |
| Is_Lead | Determines probability that call is a sales opportunity for sales, service and parts calls |
| Agent_Identified_Themselves | Agent identified themselves by first name |
While the system only inferred two advertising sources on this call (Dealer Website and Autotrader), there are other possible values listed below:
| Advertising Source |
| dealer website |
| autotrader |
| cars.com |
| cargurus |
| carfax.com |
| kelley blue book |
The Proactive Speech API supports integrating calls recorded and stored in third party call management systems so that they may be integrated directly into the product. This API simply requires basic information to link call recordings (stored externally and referenced via URLs) to the applicable entities in BDC Central so that they may be analyzed and presented to users of the product just as if they were recorded using BDC Central's integrated call management solution.
To submit a call recording to BDC Central, make an HTTP POST to the secure endpoint at:
https://speechapi.bdccentral.com/ingestcall
{
"url": "https://api.twilio.com/2010-04-01/Accounts/xxx/Recordings/XEb30226ac27d23c14eea3bc4062c731e0",
"proactive": {
"org_id": "[YOUR_PROVIDED_ORG_ID_KEY]",
"store_id": [ID of the dealership in BDC Central. Format: 54a30dc1-116f-4987-8db3-f57f2a05e7c1],
"tracking_number": [ Tracking Number to associate call with in BDC Central. Format: 111-222-3333]
},
"auth": {
"auth_sid": "optional auth_sid",
"auth_token": "optional auth_token",
"auth_key": "optional auth_key"
},
"metadata": {
"external_id": "Optional ID for correlation to a parent record in your system",
"extended": {
"sample1": "This is a sample. You may add arbitrary metadata to this section.",
"sample2": 42
}
}
}