Using speech recognition and synthesis in Windows 10 to talk to your bot (and have it talk back!)

In my previous posts I have shown how we can use the Microsoft Bot Framework to create smart, intelligent bots that users can interact with using natural language, which is interpreted using the LUIS service (part of Microsoft Cognitive Services).  The bots that we can create using these tools are really powerful but, in most cases users are still typing text when communicating with the bot.  This is perfectly ok in most situations, but it would certainly be fitting for some requirements to actually be able to allow he user to just talk to the bot using speech and have the bot respond in the same way. There are examples of where this can be done on the existing bot channels already, such as users using the Slack mobile app with a voice recognizer function so you don;t have to type your text, but what if we wanted to bake this into our own app?  In this post I am going to provide some basic code that allows a user to converse with a bot using speech in a Windows 10 UWP app.

(more…)

Give your bot some ‘manners’ with the BestMatchDialog

As a follow up to my earlier post which introduced the new BestMatch Dialog, now available via NuGet. One of the best uses for the BestMatch Dialog, and the reason I created it in the first place, is adding ‘manners’ to a bot. i.e. being able to respond to those common things that people say that fall outside of the usual conversation you would handle with your bot. Therefore this post will focus on and show how to use a BestMatch Dialog as a child dialog to respond to general messages like “hello” and “thanks”.  It is amazing what a huge difference handling these sorts of messages can make and how much more natural talking to your bot will feel for your end users. (more…)

BestMatchDialog for Microsoft Bot Framework now available via Nuget

Recently I have been developing a number of bots using the Microsoft Bot Framework, with the LUIS service to allow users to use natural language to interact with them.  One thing that struck me when we released the bots to a wider user base was just how polite everybody was towards them.  The bot was receiving messages like “hi”, “how are you?”, “thanks” and “bye bye” – the only problem was that the bots didn’t know how to deal with these messages.  Sure, they could deal with a ton of more complex messages / intents using LUIS, but wasn’t able to ‘understand’ and provide such common responses.

I set out to think about how I could solve this problem in a re-usable way and whilst doing so I ended up looking through the open source code for the Bot Framework on GitHub to see how dialogs like the LUIS dialog worked. Then, whilst browsing I came across some code in the Node section of the Framework that handled the matching of Intents within the IntentDialog – a few hours later I had my first version of the BestMatchDialog. (more…)

Integrating a LUIS natural language model with your bot using LUISDialog

The LUIS service, part of the Cognitive Services suite, aids you with the task of natural language processing.  In my last post I created a natural language model using Microsoft’s LUIS service and in this post I am going to show you how to hook up the model I created, into a bot created using the Bot Framework and a special type of dialog class, the LUIS Dialog.  If you haven’t got a LUIS model already, go back and work through the last post, it really doesn’t take long.

Update: I have now published a quick video overview of LUIS including how to create your first model.

(more…)

Using the Microsoft LUIS service to build a model to understand natural language input

LUIS, the Language Understanding Intelligence Service, is a part of Microsoft’s Cognitive Services suite.  If you are reading this and you don’t yet have a clear understanding of what Cognitive Services are, head over and read my introduction to Cognitive Services before you continue.

During this post I will introduce you to what the LUIS service is and show you how to build a model to aid in natural language understanding in your applications.

Update: I have now published a quick video overview of LUIS including how to create your first model.  (more…)

Bot Framework Typing Activity – Let users know your bot is responding (and know when they are too)

In the recent major update to the Microsoft Bot Framework (v3), a new type of Activity was introduced, a Typing activity.  This Activity type sites along side others, such as Message activity which is the most commonly used activity to represent most communications between a bot and a user.

So, what is the Typing activity for and how do you use it? (more…)

What are Microsoft’s Cognitive Services?

Some things are just hard. As a developer, tasks such as recognising a face within a photo or understanding speech are just things that, in general, you cannot do on your own own. These sorts of capabilities though, could significantly transform the experiences you offer in your apps.  Imagine being able to write a kiosk app that instantly knew who the user was just by seeing their face or being able to have a user ask a question and understand what they were asking without writing reams and reams on limiting regular expressions.  Some time ago at Microsoft a project got underway to solve this problem, codenamed Project Oxford, the idea was simple – bring the power of machine learning that can only be harnessed by an entity of a similar scale to Microsoft, to developers to use anywhere, anytime.

(more…)

Creating your first bot with the Microsoft Bot Framework – Part 2 – publishing and chatting through Skype

In the previous post of this 2 part tutorial, I demonstrated how you could build a bot using the Bot Framework and the included Bot Application template.  I also went through testing the bot locally using the Bot Framework Channel Emulator.  So, if you haven’t already got a bot working locally ready to publish then check out Creating your first bot with the Microsoft Bot Framework – Part 2 – build and test locally.

In this post I am going to show you how you can register your bot in the Bot Framework Portal, publish it to Azure and then talk with it through Skype. Very cool stuff. (more…)

Create Bota Application

Creating your first bot with the Microsoft Bot Framework – Part 1 – Build and test locally

In my last post, I provided a high level introduction to the Microsoft Bot Framework and the main constituent parts that make up the framework.  If you don’t yet know what the Bot Framework is, or you need a quick refresher, then you should check out that post.

Today though, I am going to show you how you can create your first bot using .NET / C#.  By the end of this post you will have a simple bot running and will be able to test it locally using the Bot Framework Channel Emulator, an excellent tool provided for testing your bots.  (more…)