CodeBaby Markup Language – or CBML – is our proprietary tagging system that allows you to add instructions to direct your avatars while they’re responding to users. Think about it like stage directions in a play or movie production. In your script, you’ll use these tags to “direct” your avatar to use particular “gestures” or animations, control vocal attributes, and interact with the webpage or application where your avatar lives.
Our CBML tags are inserted into your response scripts at the point where you want them to be enabled, and your avatar will perform them where you specify. They work whether you’re using a Natural Language Processor (NLP), such as Google Dialogflow, to control your conversation or if you’re using your own conversation against our CodeBaby Avatar API.
In our Response Generator, once you have added all of your desired CBML tags and previewed the scene by clicking the “Render Scene” button, copy your tagged text from the Response Content text area and paste it into your NLP or your code where you place your responses.
CodeBaby Avatars are pre-programmed with “idle” gestures that help make your avatars look natural when they’re speaking with your customers. But for some conversations, you might want to add gestures in the context of your responses to emphasize points, express empathy, or direct customers’ attention to a certain part of the screen. That’s where our CodeBaby Gesture tags come in handy.
In the Response Generator tab in your CodeBaby Administration Portal, you will see a dropdown of gestures that are available for the Avatar model you have selected. Simply create your response text in the Response Content text area, place your cursor at the point in your conversation where you want to insert your gesture, and click “Insert.” To see the gesture(s) in the context of your conversation, click the “Render Scene” button. If you want to preview a gesture before adding it to your response text, select the gesture from the pulldown and click the “Play” button.
Because we do have default gestures in place, you will likely only add specific gesture tags when you want a specific action performed at a certain point in your response. There is typically no need to add gestures explicitly to all of your responses.
Sometimes you will want your Avatar’s response to trigger an interaction with your site – to scroll to a form she mentions or to highlight a button on the screen, for example. This is when you’d turn to a Page Interaction Tag to help.
Currently, you can add directions to scroll to a DOM object (an id, a class, etc) on your page at a given point in your response. Similar to gesture tags, place your cursor in your response text where you want the scroll action to trigger, select “Scroll to Selector” from the Page Interaction dropdown menu, and click “Insert”.
The Response Generator will insert a formatted tag for you to enter your selector.
Simply replace ___SELECTOR___ with .classname or #idname. Be sure to leave the quotes in place.
You won’t be able to test the interaction until the Avatar performs the scene on your website (our admin tool doesn’t have the same DOM objects as your site). But you can click the “Render Scene” button to ensure that there are no syntax errors on the tag.
You can trigger a style to be applied to a DOM object at a point in the conversation that you choose. The default behavior for this tag is to apply the “codebaby-highlight” style to the DOM object you define (the codebaby-highlight style definition is in most UIs available in the UI tab of the partner portal). You can also change the style to apply to an object by updating “codebaby-highlight” in the tag below with a style that you define. You can add that style and its properties in the UI tab of the partner portal.
You will update ___SELECTOR___ with the .classname or #idname of the object to which you want the new style applied. The tag also allows you to set the length of time for which the tag is applied. In the example below, the style will be applied for 3000 milliseconds before it is removed. To apply a style without a time limit, remove the following: , “time” : 3000
SSML – or Speech Synthesis Markup Language – enables you to adjust the speech the avatar is generating. We currently use Amazon Polly for speech synthesis and have two types of voices available: Standard and Neural. Standard voices are a bit more “flat” and computerized sounding, but the entire set of SSML tags (with the exception of Domain tags) can be used to create vocally nuanced responses. The Neural voices are richer, more robust, and build in as standard some voice inflections, but some SSML tags aren’t supported within the neural synthesis algorithm.
The Partner Portal includes a subset of SSML tags that can be inserted into the desired location of your responses, and you can find all supported tags here: https://docs.aws.amazon.com/polly/latest/dg/supportedtags.html
Because many of the SSML tags apply only to a section of text, rather than just a single trigger point, highlight the text in your response that should be changed by the tag. Other tags, such as breaks, are inserted into a specific area. Once you have added your tags, you can test them by pressing the “Render Scene” button. If the output says “Invalid SSML,” review that you have used tags that are compatible with the voice and type you have selected.
Dialogflow allows additional custom data to be sent with a response, and CodeBaby parses that data to add functionality to your Avatar experience. Each response in Dialogflow has a space where you can add your custom payload, and you can use the Response Generator to format the data for that field.
This tag lets you add clickable buttons to your Avatar’s response.
{
“clickableResponses”: [
“____CLICKABLE RESPONSE TEXT____”
]
}
Replace ______CLICKABLE RESPONSE TEXT_______ with the text you want to appear on the button. Click “Insert” multiple times, and the tool will format the multiple responses correctly. The value of the button will be passed as an intent to Dialogflow, so you can trigger behavior based on your response / route for that intent.
This tag lets you pass additional text to Dialogflow so you can control the routing after the response is delivered (this is needed primarily in Dialogflow ES. Dialogflow CX handles this through page routing.) Select “Client Callback” in the Custom Payload dropdown and click the Insert button.
Let’s take this conversation snippet for example.
Avatar: Welcome! Can you tell me how you’re feeling today?
User: I’m feeling pretty good!
Avatar: That’s great to hear.
Avatar: Let’s get through this quickly so you can get on with your day. Can you tell me what your temperature is?
In this scenario, a “start” trigger would be passed to dialog flow to trigger the original welcome response. This welcome message is listening for the user to provide a response, and based on that response, the avatar will route the user to the appropriate next phrase depending on whether the user is feeling good / bad / so-so. Once the avatar has provided the response to the user (in this case “That’s great to hear! Let’s get through this quickly so you can get on with your day.”), it needs to know where to go to continue the conversation. This is where, in ES, we use Client Callbacks. This is key in creating a bot that is leading the conversation rather than one that is only passively responding.
{
“clientCallBack”: “____TEXT TO PASS TO DIALOGFLOW AFTER THIS RESPONSE____”
}
For this example, let’s say we want to pass back <<getTemperature>> to Dialogflow. You would update the callback to the following:
{
“clientCallBack”: “<<getTemperature>>”
}
That value is passed to Dialogflow and matched to the intent with <<getTemperature>> among its training phrases. The avatar will continue the conversation based on the script in that intent and continue to guide the user through the conversation.
You can pass data that you gather from your users via conversations back to your website to be available to javascript, allowing you to store data remotely, create conditional pathways, and anything else you need to do based on user responses.
To format this data, select “Client Data” from the Custom Payload dropdown and click the “Insert” button.
{
“clientData”: {
“_____YOUR_DATA_NAME_____”: “____YOUR_DATA___Listen_For_The_na-clientData.vidbaby_Event_To_Get_This_Data____”
}
}
You will replace the strings offset by underscores in the above example with your own values.
So let’s say your Avatar’s conversation asks a user for their member ID and their temperature that day. You want to get that data from the conversation so you can save it to that patient’s Electronic Health Record. In this case, you might do something like the following in the custom payload:
{
“clientData”: {
“patientID”: “123456”,
“patientDOB”: “1/2/85”
}
}
Then, to use the data on your front end, you might want to write something like this:
$(document).on(‘na-clientData.vidbaby’, function(event, payload) {
//Any code in here is run every time there is clientData in a responses custom payload
//Check the data we want exists in this batch of clientData, before trying to access it
if(payload.clientData.patientID && payload.clientData.patientDOB){
var patientID = payload.clientData.patientID;
var patientDOB = new date(payload.clientData.patientDOB)
}
})
This custom payload type allows you to create custom website behaviors based on your users’ interactions with the Avatar without needing CodeBaby to do any custom coding.
