In this article, I will show you how you can use your existing backend architecture & APIs, connecting them with your voice apps to offer a new experience to your customers. Voice apps borrow a lot from our general development process, not requiring separate development resources.

What we are building

We are going to build a superhero search app using the Open Source Superhero API.

I have added a wrapper around this API to make it accessible from our Voice App. You can find the code in this gist - https://gist.github.com/akash-joshi/476ead410a244a48e037c138ba2387b0.

Take a look at the completed app below:

We will build this for Voice apps (Alexa and Google). We will be using the platform VoiceFlow, which allows us to build voice apps visually.

Voiceflow Tutorial

VoiceFlow is a visual way to create voice apps, and is very easy to use and understand.

Firstly, create an account there to get started.

After creating an account, create a new project, giving it an appropriate name. For the purpose of this tutorial, we have chosen all English regions as deployment regions.

You will end up on a blank canvas after this. Don't be overwhelmed by all the options present on the screen, this tutorial will guide you through all the relevant blocks required.

In the blocks submenu on the left, you will see several blocks which can be used to build an Alexa or Google skill. Each block performs a certain function, and Voiceflow is based on building Voice apps by combining these blocks.

1. Speak Block

The first block that we will use is the Speak Block. We will use it to make Alexa say something to the user. Drag a speak blog onto the canvas, rename it to Introduction and write a suitable introduction to your app in the text area. I will use "Welcome to Superhero! Say the name of a hero to search!".

At any point, test out your app by clicking on the Play Button.

2. Capture Block

The next block we will use is the Capture block. It is used to capture data from the user's voice and store it into a variable.

Firstly, create a new variable in the 'Variables' submenu on the left by typing a name and pressing enter. Use the name 'hero' for now.

Adding a capture block, name "Input Type" as "Actor" and "Capture input To" as "hero".

Add a speech block after it, saying : "Searching for {hero}. ". We use curly brackets to use a variable in speech. Be sure to enter {hero} by hand so that Voiceflow detects it as a variable. The is a tag which is part of a language called Speech Synthesis Markup Language (SSML). You can read more about it on Amazon's official documentation page.

3. API Block

Click on the Plus icon below your Speak block to add another step to the block. Add an Integration block from the list. After that, click on the integration block & set up the options in this order:

1. Choose an integration - "Custom API" since we will be using a custom API to get some data.

2. I want to - "Make a GET request" since we are using a GET request to get JSON data from an API.

We will use a custom API (https://super-search-akashjoshi.flexiple.now.sh/?hero=)to get the superhero data.

Try navigating to https://super-search-akashjoshi.flexiple.now.sh/?hero=superman in your browser to see what kind of data is returned by the API. Replace superman with any hero that you want to search for.

We replace the hero name with the {hero} variable so that the API fetches the desired hero correctly.

Paste

https://super-search-akashjoshi.flexiple.now.sh/?hero={hero}

into the URL bar. Be sure to type in {hero} by yourself so that Voiceflow detects it as a variable.

However, we aren't done yet. Click on Test Integration to test the API call.

The response from the API has to be mapped into output variables so that they are spoken to the user.

Add variables for name, fullName, born, alignment, work & base from the variables side-bar.

Copy the output path of the JSON file by clicking on the response tab of the Test Integration tab and paste it into the output menu. Do this for all of the following - name, fullName, born, alignment, work & base.

Check the short video below to understand how to map the JSON output with your Voiceflow variables:

In the image above, we can see that the Integration has 2 outputs, one without a prefix text, and one with fail as prefix. The one without the prefix text is a success state output, and the one with fail output is when our API call fails.

Add a speech block saying 'The hero was not found' connected to the fail state. If the API succeeds and a hero was found matching the {hero} variable, all the output variables will be set with the correct values. Otherwise, they will be to the default value of 0.

4. If Block

Add an if block to the canvas, and check whether fullName = 0. If it is 0, connect it to the "Not Found" block.

Watch the short video below to understand how to add conditions to If blocks:

Else, the hero was found. So, speak the hero name by writing in a speech block: "Their hero name is {name}. Their full name is {fullName}. They were born in {born}. They are {alignment}. They work as a {work} from {base}. Do you want to search another hero?"

Again, be sure to type in the variable names so that Voiceflow detects them as variables.

To clear out the variables after the skill completes, add a "Set" block to the canvas and set fullName to 0. This step is necessary, because if the variables are not cleared, the previous answer will be repeated by the skill!

5. Choice Block

We shouldn't end the skill here. We should allow the user a choice on whether they want to search for another superhero. Change the text in the "Not Found" block to ask whether the user wants to search for more at the end.

Add a Choice block to the canvas. The choice block allows us to perform certain actions based on user voice. This block checks whether the user wants to search for another hero. Enter synonyms of Yes for searching further.

For else, add a Flow block, selecting Stop Flow as a flow.

Connect the ' 1 ' output of the Choice block to a speech block asking user to say another Hero name, and connect it to the Capture block. Look at the image below to understand how it's done.

And we are done! Test out your app by clicking on the play button.

6. Deployment

a. Alexa

To deploy your app to the Alexa platform, click on the Publish tab, connect your Amazon developer account and fill the form options according to relevance to your skill (like Description, Skill Name, etc.)

Be sure not to change any of the default invocations that may break your skill during deployment and cause you to resubmit it.

In case you get some part wrong while submitting, the review process is very helpful & they will let you know what went wrong in the submission.

b. Google

Click on the instruction link to see instructions to add the Google Assistant file to Voiceflow. After adding the file, follow the publish to production guide from here . Few caveats in the Google deployment process:

1. Your invocation name cannot have any keywords which may be used during invocation. Eg, you cannot name the action 'Superhero Search', because 'Search' may be used as a invocation name.

2. You need to add a custom privacy policy from Google. You can't use the one from Voiceflow as it has mentions of Alexa or skills, which will cause your action to be declined. You can use a template I built here. It has instructions on how to write your privacy policy and where to keep it.

3. The final point would be to not use the term 'Alexa' or 'skill' at any point in your action description or within the app. If there are any occurrences in the app, replace them with something more generic so you can use the same codebase for Alexa & Google. Replace 'Alexa' & 'skill' with 'Google' & 'actions' in the description and similar places.

What Next?

App ideas:

The interesting point about voice apps is that you can use them as an extension for your existing apps. For example, If you have already built a messaging app like this one, the voice platform can be an easy way of sending and reading messages on your app. For a blog writing platform, a voice app may be an easy way to take notes before actually sitting down to write an article. Even for a product posting platform like ProductHunt or Remote.tools, you can easily integrate voice to read out description and other details on the products.

In such ways, voice apps can be used to enhance user experience.

Useful links:

https://getvoiceflow.com

https://learn.voiceflow.com/articles/2705386-uploading-your-project-to-google-assistant

https://learn.voiceflow.com

https://developer.amazon.com/docs/custom-skills/speech-synthesis-markup-language-ssml-reference.html

Hope you liked this article. I have also written a more extensive article which adds a web-based frontend to this app making it a complete ecosystem.

Happy Coding!

Natural conversations are fluid. That’s one of the joys of face to face human conversation, you never know how the dialog will evolve. But sometimes, there is a natural flow to the conversation. Ask someone where they want to eat, and you expect to hear a restaurant or type of food, not a response about a favorite movie.

One of the frustrations people have with voice assistants is that they sometimes do a poor job understanding what they’re saying. Amazon has a tool to help third party developers provide better recognition. Dialog Management lets you prompt for values to fulfill a request with increased accuracy.

For example, if you are creating a skill that searches for a car rental, Alexa can prompt the customer for a city and dates of travel after they say “find a car.” Alexa’s built-in functionality improves the accuracy of speech recognition in this case as it listens for specific values.

The problem is that the customer has to speak the appropriate words to trigger a Dialog-controlled intent. Amazon recently announced intent chaining as a solution to this problem.

In this blog I’m going to show you how I use this functionality with a skill that lets the user perform a car search. First, let’s review how Dialog Management works within Alexa. Let’s look at a CarSearchIntent that gathers the input needed to kick off a car search.

CarSearchIntent with slots for location and travel dates

As you can see, we have several variations on how a customer can find a car, including slots for Location, PickUpDate, and DropOffDate. We want to make sure the customer provides all three of these slots before we start processing the request. We use Dialog Management to let Alexa handle prompting the customer to provide these.

Location as required slot with prompt

When in Dialog mode, Alexa has a higher chance of accuracy because it is trying to fill slots for this intent. But to get into this mode, the customer has to trigger this intent by saying “Find a car” or a similar phrase. Ideally, we’d drop the customer into this mode as soon as they launched the skill.

Enter Intent Chaining! We can add a Dialog Delegate directive to our response that puts the customer into the dialog flow of CarSearchIntent.

canHandle(handlerInput) {  const request = handlerInput.requestEnvelope.request;  return handlerInput.requestEnvelope.session.new ||    (request.type === 'LaunchRequest');},handle: function(handlerInput) {  return handlerInput.responseBuilder    .addDelegateDirective({      name: 'CarSearchIntent',      confirmationStatus: 'NONE',      slots: {}    })    .speak("Welcome to car search.")    .getResponse();}

The dialog directive allows us to pre-fill some of the slots (for example, we could default the location to the last used search location). What’s interesting to note is that we only specify “Welcome to car search” as the response for this handler. We don’t specify a reprompt. Alexa appends the dialog prompt for CarSearchIntent to our response and uses that as the reprompt. So in this case what the user will hear is “Welcome to car search. Where would you like to pick up the car?” If they don’t answer, they will hear a reprompt of “Where would you like to pick up the car?”

You also have the ability to direct the customer to fill out a specific slot when dropping them into a dialog. Let’s say that we want to guide the user to first specify the pick up date for their car. We can do that using an Elicit Slot directive, as shown in the following code.

canHandle(handlerInput) {  const request = handlerInput.requestEnvelope.request;  return handlerInput.requestEnvelope.session.new ||    (request.type === 'LaunchRequest');},handle: function(handlerInput) {  return handlerInput.responseBuilder    .addElicitSlotDirective('PickUpDate', {      name: 'CarSearchIntent',      confirmationStatus: 'NONE',      slots: {}    })    .speak("Welcome to car search. When would you like to pick up your car?")    .reprompt("When would you like to pick up your car?")    .getResponse();}

In this case, we need to spell out the entire speech and reprompt that the customer will hear. We need to do this since we are controlling how we drop the customer into the dialog management flow.

Chaining intents lets you manage the conversation flow in a more natural way. Use it to make your skill more usable for your customers!

Amazon recently released the Alexa Presentation Language (APL). APL provides a richer display for multimodal skills. It is based on modern frameworks that separate display elements from data sources. It gives you the flexibility to include many visual elements such as graphics, images, and slideshows and lets you tailor the display for different devices.

In this article, I’m going to walk through how I updated one of my skills to use APL. You can also use these tips and techniques if you are creating a new skill.

Most of my skills feature multimodal support using the Display Interface. I decided to learn APL by updating one of my existing skills. I focused on my Video Poker skill because I wasn’t happy with the existing customer experience.

Video Poker presents users with a 5-card poker hand, with an ability to hold and discard cards before drawing to complete a hand. Users can do this by voice command (“keep the first card” or “keep the pair of jacks”), or by touching cards on a visual display. ListTemplate2 was the best way to do this with the Display Interface. However, this came with the limitation of only allowing three cards on the screen at a time and putting numbers beneath each card in the list.

Video Poker using ListTemplate2 Display Directive

Using APL, I modified a ListTemplate2 layout to shrink the size of list items, reduce the spacing between them, and fit a full five card hand on the screen. I was able to remove the numbers from items in the list and put text indicating which held cards in a bold, red font. I was able to optimize the layout for different screen dimensions, such as smaller displays like the Echo Spot.

Video Poker using APL template modified from ListTemplate2

APL Authoring Tool

The way I did this was through the APL Authoring Tool. This handy tool provides a list of different visual designs you can use as a baseline to create compelling visuals for your skill. It also allows you to save and upload layouts, so you can extract them to your skill code, or upload them if you make offline updates. For this use case, I started with the Image Forward List Sample, which is based on ListTemplate2.

Selecting a visual design from the APL Authoring Tool

Once you select this visual design in the tool, you’ll see a generic example of a list with cheese samples. You’ll see the APL document at the bottom of the screen separated into two tabs:

• “Image Forward List Sample” which provides the layout
• “Data JSON” tab which provides a view of the document data.

Editing your APL Document in the Authoring Tool

Take a moment to look through the JSON on both the Image Forward List Sample tab (the layout code) and the Data JSON tab (the content code). You’ll notice in the layout that there are several references to values enclosed in ${} such as ${payload.listTemplate2ListData.listPage.listItems.length}. If you look in the content file, you’ll see that this is a path to a value within the content. This is how APL binds data to the presentation layer and lets you make changes.

Updating Data Sources

As a first step, I wanted to update the data so it would show card images and text relevant to my skill. That way, as I started to update the layout itself, I would be able to see how it would look with my actual images. The Authoring Tool shows your rendered screen images in real time, making it convenient to use as you attempt to perfect your layout. To update the data, I took the following steps:

• Click on the Data JSON tab
• In the listTemplate2Metadata, change the title and logo elements to something relevant to Video Poker
• In this same element, change the url in the backgroundImages.sources fields to point to the background image I wanted to use
• In listTemplate2ListData.listPage, I updated each of the items in the listItems array. Specifically, I updated this array to have 5 items (my cards), with listItemIdentifier and token set to “card.x” (where x ranged from 0–4). I removed the secondaryText since I only wanted one row of text (which would either be blank or say “HELD”). I updated the image.sources to point to URLs containing my card images. For now, I just selected some card images at random — my skill’s code will update the data during gameplay with the actual user’s hand
• Update the hint text to use a transformation. Rather than hard-coding the hint string with the Alexa keyword, you can use a transform to change a hint into one that uses the wake word associated with the device, in case the user has changed it. You do this by removing the hintText from listTemplate2ListData and adding the following into your lastTemplate2Metadata:

"properties": {    "hintText": "select number 1"},"transformers": [    {        "inputPath": "hintText",        "transformer": "textToHint"    }],

Once I had made these to the data sources, my image looked like this — pretty similar to the skill as it exists with the ListTemplate2 display which makes sense since at this point I’m still using the layout based on ListTemplate2 and have only made content changes.

Updated view with relevant Video Poker content

Updating the Layout

Now for the fun part — updating the layout to get five images on the screen at one time. For this part, I made updates to the layout on the “Image Forward List Sample” tab. To make these changes, I clicked on the slider that lets you switch between a visual view of nested APL components to a raw JSON view. I find viewing the full document JSON easier to consume than clicking on each component and editing the JSON within the component. But you can play with it and follow whichever approach seems most natural to you.

Before talking through the changes I made, I wanted to point out some of the elements of this layout template:

• There is a ListTemplate2 node in the JSON which provides two containers in an items node— one that applies to circular screens (like the Echo Spot) and the other for other screen types. In this blog, I’m going to focus on the non-Spot displays, but you should appreciate that you can also make changes specific to different screen layouts.
• Looking at the second container that we’ll be changing, you‘ll see a set of items including an Image (the background image), an AlexaHeader (the title on the screen), a Sequence (which is the list of cards), and an AlexaFooter (the hint at the bottom of the screen)
• You’ll see that the Sequence points to HorizontalListItem, which is another container in this JSON document. It contains items consisting of an Image and two Text elements (the primary text and the secondary text)

With this context in mind, I made the following changes to this document:

• Within the HorizontalListItem, I changed the Image item dimensions — specifically I set height to 40vh and width to 17vw. This sets the height of each card to 40% of the viewport height, and the width to 17% of the viewport width.
• I then updated midWidth to 100. This makes the width of each list item smaller and allows the five card images to appear on the screen
• I changed paddingLeft and paddingRight to 6 to reduce the amount of space between elements
• I added paddingTop and set it to 100 to add some separation between the header and the card images
• I got rid of the secondary Text element since I don’t have two lines of text on my display
• I changed the primary Text element so it doesn’t draw the ordinal. So text in this element changed from <b>${ordinal}.</b>${data.textContent.priamryText.text} to <b>${data.textContent.primaryText.text}.
• Within this same element, I wanted the text to be red and centered. I achieved this by adding a textAlign field with a value of “center,” and a color field with a value of “red.”
• In order to get the hint text from the appropriate location (now part of the metadata rather than the list data), I needed to update the AlexaFooter element to get the hint from ${payload.listTemplate2Metadata.properties.hintText}

Finally, I needed to make the cards in my list selectable, so I could respond when the user touches one of them on the screen. To do this, I needed to change the items that were associated with the Sequence element from a FullHorizontalListItem to a TouchWrapper which in turn contained a FullHorizontalListItem. In code, that means that I changed this:

"item": [  {    "type": "FullHorizontalListItem",    "listLength": "${payload.listTemplate2ListData.listPage.listItems.length}"  }]

to this:

"item": [  {    "type": "TouchWrapper",    "onPress": {      "type": "SendEvent",      "arguments": [        "${data.token}"      ]    },    "item": {      "type": "FullHorizontalListItem",      "listLength": "${payload.listTemplate2ListData.listPage.listItems.length}"    }  }]

Note the onPress element in this item. Specifically, the list of arguments. You can specify an array of different arguments to send to your skill when an item is selected. Because my existing code was processing the token of the selected card, I decided to continue to do the same to minimize the amount of code I needed to change. But you could also pass in ${ordinal} which would tell you the index of the selected item without having to process the token.

Updates to the Skill Code

Once you’ve made changes within the Authoring Tool, you can select the Export Code button which will package up your layout and data files into one JSON file for you. I chose to use two different JSON files in my code, one called main which I used for the layout and another called datasources which I used for the data. I like keeping the separation of layout and content in my source code as a general best practice. It was surprising that Amazon’s Authoring Tool didn’t encourage this too.

Now that we’ve downloaded the document and content, we need to make code changes to incorporate it and update the data as the user plays with our skill. We can do this by manipulating data elements and then passing them back to the skill. I used Alexa’s response interceptor functionality (which I talk about in a separate blog post). I load the data source in from a JSON file, then update the cards and text within the structure before sending it off to Alexa. I do that with the following code:

const main = require('./main.json');const datasource = require('./datasource.json');

function drawTable(handlerInput) {  const event = handlerInput.requestEnvelope;  const attributes = handlerInput.attributesManager.getSessionAttributes();  const game = attributes[attributes.currentGame];  let i;  let cardText;  let url;

  // Update the images  for (i = 0; i < game.cards.length; i++) {    const card = game.cards[i];    url = GetCardURL(card);    cardText = (card.hold) ? 'HELD' : '';    datasource.listTemplate2ListData.listPage.listItems[i]      .textContent.primaryText.text = cardText;    datasource.listTemplate2ListData.listPage.listItems[i]      .image.sources[0].url = url;    datasource.listTemplate2ListData.listPage.listItems[i]      .image.sources[1].url = url;  }  // Give an appropriate hint  if (game.state === 'FIRSTDEAL') {      if (game.cards[0].hold) {        datasource.listTemplate2ListData.hintText = 'discard the first card';      } else {        datasource.listTemplate2ListData.hintText = 'hold the first card';      }      datasource.listTemplate2Metadata.title = 'Select cards to hold or say Deal';    } else {      datasource.listTemplate2ListData.hintText = 'deal';      datasource.listTemplate2Metadata.title = 'Your last hand';    }  }  return handlerInput.responseBuilder    .addDirective({      type: 'Alexa.Presentation.APL.RenderDocument',      version: '1.0',      document: main,      datasources: datasource,    });}

The second place I had to make a code change was to handle the user touching one of the items in my list. In my old code, I was parsing item tokens of the form “card.x” where x is the ordinal position of the card in the list. With a Display Interface, this meant looking for an ElementSelected request. In APL, your code will receive an APL.Presentation.APL.UserEvent, and you can process the request as follows to determine which card was selected:

module.exports = {  canHandle(handlerInput) {    const request = handlerInput.requestEnvelope.request;       // Was this a touch item selected?    if (request.type === 'Alexa.Presentation.APL.UserEvent') {      return ((request.source.type === 'TouchWrapper')        && (request.source.handler === 'Press'));    }    return false;  },  handle(handlerInput) {    let index;    const event = handlerInput.requestEnvelope;    // Was this a touch item selected?    if (event.request.type === 'Alexa.Presentation.APL.UserEvent') {      const cards = event.request.arguments[0].split('.');      if (cards.length === 2) {        index = cards[1];      }            // Do something with the selected card...

    }  },};

With these changes, I had a much cleaner looking Video Poker skill, which is sure to delight my customers more than the older format I was using. I’ve just started scratching the surface in terms of APL’s capabilities. But I can tell it will open up a new world of possibilities for voice-driven multimodal skills! Let me know in the comments about your own learnings with this new framework.

Many of the most engaging smart speaker applications extend beyond the smart speaker itself. When in Rome is an example of an Alexa game that combines voice with a traditional board game. Users navigate the globe sending voice commands to Alexa. Alexa asks questions from people in the countries players visit. But it also plays like a board game, with a physical playing board, tokens, and cards…

I’ve published over a dozen Alexa skills over the last few years. I have run across several patterns and best practices in that time.

One of the more powerful but under-hyped features of the SDK that I’ve made extensive use of in my code is the Response Interceptor. It is part of the Alexa Node SDK. This SDK simplifies Alexa skill development.

When using it you may find your code getting messy with redundant error handling and clean-up tasks. Response interceptors allow you to insert a hook into the flow of handling Alexa intents. This keeps your code clean by performing last-minute actions in one central location before passing the response back to Alexa. This is extremely useful for debugging, resolving common tasks, and generally cleaning up responses to avoid some common errors encountered when using the Alexa ecosystem.

Debugging

To add a response interceptor with the Alexa SDK, you pass a class that implements a process function to the addResponseInterceptors function on your SkillBuilder object as shown in the code sample below. We’ll get to the implementation details of this class in a moment.

This snippet also shows setting a request interceptor function which will run before a request is passed to your Alexa handlers. For example, I find it helps debugging to log each incoming request and the outgoing response. You can do this with the request interceptor and response interceptor as demonstrated in this code snippet. The snippet also shows the syntax for the process function returning an empty Promise.

Resolving Common Tasks

Many of my skills process AMAZON.RepeatIntent so the user can re-hear the last response. If you have a complex skill, especially one that maintains state engines, you should provide additional detail when the user asks Alexa to repeat herself so the user knows exactly where they are. But in simpler skills, it’s fine to just replay the previous response back to the user. A response interceptor allows you to save each outgoing speech and reprompt cue so that you can re-play them easily:

Another common task is handling attributes. The Alexa SDK has several layers of attributes.

• request attributes (only good during a single request)
• session attributes (good for the length of a session)
• persistent attributes (saved in a store like DynamoDB to be used across sessions).

I find juggling all of these can be overwhelming. I structured my code to only use session attributes, which I save to persistent storage at the end of the session. But there are two problems with this as a blanket approach — there are attributes that you may not want to save and there are times you want a value to pass between handlers but not across the session (what request attributes aim to solve).

I get around these problems by using a session and request object off my attributes. You can see an example of the session field in the above code snippet saving lastResponse and lastReprompt. I don’t want to save these persistently, so I clear this entire object before persisting to storage at the end of the session. In a similar fashion, I clear the request object each time at the end of my response interceptor, so those attributes truly remain request-only.

Avoiding Common Errors

One of the limitations that plague many developers is that responses can’t contain more than 5 audio tags. Sometimes it can be difficult to avoid this with dynamic content.

For example in my Blackjack skill, I play a sound for each card that is dealt. Normally not a problem. But when reading out the dealer’s hand, if they end up drawing several cards you can go over this limit. Sure, I could try to catch this when I’m generating the response, but that convolutes the code. Who’s to say whether it hits every edge case?

Far better to remove the excess sound effects as part of the response handler. For this skill, I just remove excess audio files from the end of the response to get the count down to 5.

Another problem that I’ve encountered is that you aren’t allowed to return directives along with the Dialog.Delegate directive if you are eliciting slots from the user. This can be annoying if you are handling button input or using display directives. You might have to check in multiple places before adding directives to your response to make sure you don’t clash with the Dialog.Delegate. Keep your code clean and use a responseInterceptor to filter through your directives before returning instead:

I hope you enjoyed these tips and see the power of using response interceptors. Let me know your own best practices and share your own tips in the comments section!