Integrating Chatbot with FB Messenger using MuleSoft

Author: Praveen Sundar

In this technical article, I will bring about the design details on how MuleSoft, an Integration Platform, can be used to Integrate a Conversational Interface (Chatbot) with FB Messenger via third party Communication Platform. This is made possible because of MuleSoft’s feature of Seamless Integrations using API-led Connectivity approach.

According to Paul Jacobs, Deloitte’s customer experience Transformation Director, there are three technologies that are changing how customers interact with businesses:

Mobile Apps – providing customers a wealth of knowledge and self-service help at their fingertips

Social Media – opening the instant response options for consumers as traditional channels gain a reputation for long lines and bureaucracy

Artificial Intelligence – increasing volume by handling millions of simple transactional queries with a chatbot instead of customer service teams

A truly great customer experience means providing a better service than what the customer can do themselves using these technologies.

INTRODUCTION

For our design, let’s use Google’s Dialogflow – a Natural Language understanding Platform for creating a Conversational Interface & Nexmo – a Communication Platform for message delivery to Facebook (FB) Messenger. We will use below APIs for this purpose:

POST https://dialogflow.googleapis.com/v2/{session=projects/*/agent/sessions/*}:detect  – This serves as a conversational interface for Chatbot. For more info, click here.

POST https://api.nexmo.com/v0.1/messages  – This serves as a communication channel for communication with FB Messenger. For more info, click here.

Data Flow Diagram

1. As any FB user initiates chat in FB Messenger with a business FB page, chat message reaches at the Inbound Message Webhook URL registered in Nexmo (triggers an event) & same event will be consumed at the Listener configured for the same Webhook URL in Mule flow to read the message & other info such as Sender ID, Receiver ID, etc.
2. This message event reaches a transformer which takes care of formatting a request (by extracting only the chat message from the message event) for Dialogflow API (see above – first endpoint).
3. This formatted request reaches Dialogflow API Invoker.
4. Invoker invokes Dialogflow API with the formatted request.
5. Dialogflow API responds with proper response.
6. This response event reaches a transformer which takes care of formatting a request (by extracting only the fulfilment text) for Nexmo API (see above – second endpoint).
7. This formatted request reaches Nexmo API Invoker.
8. Invoker invokes Nexmo API with the formatted request – Nexmo takes care of flashing the response message on the user’s chat window.

DESIGN APPROACH

Let me brief about the approach used here:

1. Set-up your Nexmo Account & Header-based Authentication (BASIC) and Set up your client application by linking your Facebook (FB) Page.

1. Set-up your GCP (Google Cloud Platform) Project & Oauth2.0 Authentication (Access Token) and Create your Dialogflow Agent by linking with the created GCP project.

1. Design a Mule flow using above mentioned APIs in order to integrate with FB Messenger & Deploy the Mule Application on CloudHub.

Set-up your Nexmo Account & your Client Application

Nexmo, the Vonage API Platform is a business unit of Vonage that develops and markets Communication APIs to enable developers and businesses to rapidly innovate the way they communicate with their customers.

Sign up for a Nexmo Account

When you create a Nexmo account, an API key and secret will be created for you. These are located in your account settings in the Nexmo Dashboard. You should always keep these secure and never share these details: be careful when adding it to your codebase to make sure they are not shared with anyone who may use it maliciously.

This is how Dashboard looks:

Set-up Authentication for Messages API

Messages API, one of the APIs used for your integration, supports both JWT and Basic Authentication, you will use Basic Authentication, which is Header-based API Key and Secret Authentication, for our prototyping. You will send your API key & secret while invoking Messages API as below:

Authorization: Basic base64(API_KEY:API_SECRET)

where 

API_KEY : API Key created while creating account.

API_SECRET : API Secret created while creating account.

base64 : Encoding/Encryption used.

Create a Client Application for Messages API

The Messages API allows you to send and, in some cases, receive messages over the following communications channels:

• SMS/MMS
• Facebook Messenger
• Viber
• WhatsApp

You can create Messages and Dispatch application (you need only Messages here) in the Dashboard. This allows you to set webhooks for each application created – this is important for receiving inbound messages and message status updates.

To create your application using the Dashboard:

1. Under Messages and Dispatch in the Dashboard, click Create an application.

1. Enter the Application name, which can be anything you like.

1. Enter the URL for your message status webhook, for example, http://76b262b4.ngrok.io/webhooks/message-status.

1. Enter the URL for your inbound message webhook, for example, http://76b262b4.ngrok.io/webhooks/message-status.

1. Click the link Generate public/private key pair just below the public key text area. This will create a public/private key pair and the private key will be downloaded in the name private.key by your browser.

1. Click the Create Application button. You will be taken to phase 2 of the procedure where you can link Numbers to this application to enable outbound SMS as a communication channel. As you don’t need any, click Next. You will be taken to phase 3 where you can link external accounts such as Facebook to this application. In the next step, you will learn how to link Facebook Messenger. For now, you click Done.

Install Nexmo Command Line Interface (CLI)

The Nexmo CLI requires Node.js 4 or above. If you don’t have Node.js installed on your system, go to https://nodejs.org/en/download/ and download the appropriate installer for your system.

With Node.js installed, you can then install the nexmo-cli package from the npm registry.

npm install nexmo-cli -g   # Using npm

yarn global add nexmo-cli  # Using yarn

If you see an EACCES error when you try to install a package globally, follow the steps outlined at How to Prevent Permissions Errors.

After installation, set up the CLI with your Nexmo API key and secret:

> nexmo setup <api_key> <api_secret>

Credentials written to /Users/yourname/.nexmorc

This will save your credentials to ~/.nexmorc. If you want to use different credentials per project, you can pass the –local  flag as follows:

> nexmo setup <api_key> <api_secret> –local

This will save the config to your local folder instead.

Linking FB page to the Client Application

Only individuals may create a Facebook Profile. However, an individual can use their Profile to create a Facebook Page for a business. 

A Facebook user must initiate communication using Facebook Messenger via the business’s Facebook Page. A message from the business to the Facebook user will otherwise be refused.

Facebook Messenger uses its own form of IDs for the Facebook User and the Facebook Page:

• Facebook User (profile) – Page-Scoped ID (PSID)
• Facebook Page (business) – Page ID

In order to link your FB Page to Nexmo:

• Generate a JWT with the following command, pasting in your Application ID:

JWT=”$(nexmo jwt:generate path/to/private.key application_id=YOUR_APP_ID)”  where private.key is the private key downloaded while creating your Application in the above steps.

• Display the generated JWT & copy the JWT to the clipboard.

• Click the following link when you have your JWT pasted to the clipboard and you are ready to link your Facebook Page to Nexmo: https://static.nexmo.com/messenger/. You will see your Facebook Pages listed:

• Select the Facebook Page you want to connect to your Nexmo account from the drop-down list.

• Paste your JWT into the box labeled 2. Provide a valid JWT token.

• You will receive a message confirming successful subscription.

At this point your Nexmo Account and your Facebook Page are linked. 

NOTE: If at some point in the future you want to further link a specific application to this Facebook Page you can now do this from the Dashboard on the External Accounts tab for the specific Messages and Dispatch application you want to link.

Set-up your GCP Project & Create your Dialogflow Agent

Google Cloud Platform (GCP) is a suite of public cloud computing services offered by Google. The platform includes a range of hosted services for compute, storage and application development that run on Google hardware. GCP services can be accessed by software developers, cloud administrators and other enterprise IT professionals over the public internet or through a dedicated network connection.

Dialogflow is a natural language understanding platform that makes it easy to design and integrate a conversational user interface into your mobile app, web application, device, bot, interactive voice response system, and so on. Using Dialogflow, you can provide new and engaging ways for users to interact with your product. Dialogflow can analyze multiple types of input from your customers, including text or audio inputs (like from a phone or voice recording). It can also respond to your customers in a couple of ways, either through text or with synthetic speech.

The GCP Console is a web UI used to provision, configure, manage, and monitor systems that use GCP products. You use the GCP Console to set up and manage Dialogflow resources.

To use services provided by GCP, you must create a project. A project organizes all your GCP resources. A project consists of a set of collaborators, enabled APIs (and other resources), monitoring tools, billing information, and authentication and access controls. You can create one project, or you can create multiple projects and use them to organize your GCP resources in a resource hierarchy.

Create/Identify a GCP Project

Google Cloud Platform (GCP) projects form the basis for creating, enabling, and using all GCP services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for GCP resources.

You can create a new project using the GCP Console, the gcloud command-line tool, or the projects.create() method.

To create a new project:

• Go to the Manage resources page in the GCP Console:

• On the Select organization drop-down list at the top of the page, select the organization in which you want to create a project. If you are a free trial user, skip this step, as this list does not appear.
• Click Create Project.
• In the New Project window that appears, enter a project name and select a billing account as applicable.
• If you want to add the project to a folder, enter the folder name in the Location box.
• When you’re finished entering new project details, click Create.

To interact with GCP resources, you must provide the identifying project information for every request. A project can be identified in the following ways:

Project name: the customized name you chose when you created the project, or when you activated an API that required you to create a project ID. Note that you can’t reuse the project name of a deleted project.

Project ID: a unique identifier for your project, composed of the project name and a randomly assigned number.

Project number: a number that’s automatically generated by the server and assigned to your project.

To get the project ID and the project number:

• Go to the Dashboard page in the GCP Console: https://console.cloud.google.com/home/dashboard.
• Click the Select from drop-down list at the top of the page. In the Select From window that appears, select your project.

Both the project ID and project number are displayed on the project Dashboard Project info card:

Enable Dialogflow API

You must enable the Dialogflow API for your project. To enable an API for a project:

• Go to the GCP Console API Library page: Use this link https://console.cloud.google.com/apis/library or navigate from main menu to APIs & Services > Library.
• Click Select to choose the GCP project, if not selected already.
• Click the API you want to enable. If you need help finding the API, use the search field.
• Click Enable.

Dialogflow API can be enabled this way as well:

• Go to this page: https://console.cloud.google.com/flows/enableapi?apiid=dialogflow.googleapis.com
• Select your project.
• Click Continue

Set up Authentication for Dialogflow API

Any client application that uses the API must be authenticated and granted access to the requested resources. In order to use Dialogflow API, you need to set up authentication. Google APIs use the OAuth 2.0 protocol for authentication and authorization.

To begin, obtain OAuth 2.0 client credentials from the Google API Console. Then your client application requests an access token from the Google Authorization Server, extracts a token from the response, and sends the token to the Google API (Dialogflow API in our case) that you want to access. 

Obtain OAuth 2.0 credentials from the Google API Console : 

1. Go to the Google Cloud Platform Console.
2. From the projects list, select a project or create a new one.
3. If the APIs & services page isn’t already open, open the console left side menu and select APIs & services.
4. On the left, click Credentials.
5. Click Create Credentials, then select OAuth client ID.
6. Select the appropriate application type for your project and enter any additional information required.
7. If this is your first time creating a client ID, you can also configure your consent screen by clicking Consent Screen. You won’t be prompted to configure the consent screen after you do it the first time.
8. Click Create.

Obtain an access token from the Google Authorization Server :

There are several ways to make this request, and they vary based on the type of application you are building.

Some requests require an authentication step where the user logs in with their Google account. After logging in, the user is asked whether they are willing to grant the permissions that your application is requesting. This process is called user consent.

If the user grants the permission, the Google Authorization Server sends your application an access token (or an authorization code that your application can use to obtain an access token). If the user does not grant the permission, the server returns an error.

Here I will show you how to obtain access token in 2 ways – one using Google OAuth 2.0 Playground (https://developers.google.com/oauthplayground/)& other using Browser/Postman tool.

Using Google OAuth 2.0

1. Select the scope for the APIs you would like to access or input your own OAuth scopes. In our case, select these 2 scopes: 

https://www.googleapis.com/auth/cloud-platform

https://www.googleapis.com/auth/dialogflow

Then click the Authorize APIs button, then it will ask for consent to access your Google Account. Once granted, you will get Authorization Code in the response.

1. Once you got the Authorization Code from Step 1 click the Exchange authorization code for tokens button, you will get a refresh and an access token which is required to access OAuth protected resources.

The above obtained Access Token can be used in the integration you are designing using MuleSoft.

1. Construct your HTTP request by specifying the URI, HTTP Method, headers, content type and request body. Then click the Send the request button to initiate the HTTP Request. This step is just to test the API.

Using Browser/Postman tool

1. Hit the below URL in your browser:

“https://accounts.google.com/o/oauth2/v2/auth?redirect_uri=http://localhost:8081/dialogflow&prompt=consent&response_type=code&client_id=188757296864-kmlu4rmmkhd8eplk3oj93a8pftiru4ta.apps.googleusercontent.com&scope=https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/dialogflow&access_type=offline”

where

redirect_uri: This should be one of the configured when creating OAuth Client ID (See Above)

client_id: This is the Client ID which was created above.

scope: The two scopes must be separated by a single space.

You will be asked for consent to access Google Account for Web Client created before. On clicking Allow button, you will be taken to configured redirect url:

“http://localhost:8081/dialogflow?code=4/uwG2OiLqS29FojAQiTAaqgW08j4sfXTGa N3NG5SR8Nr2hy1XZj_IdwHHrszgm_j64eBfQ twun8Kc1QX1ZeoZs&scope=https://www.googleapis.com/auth/dialogflow%20https //www.googleapis.com/auth/cloud-platform”

where code is the Authorization Code

1. Do a POST on below URL using Postman:

“https://oauth2.googleapis.com/token?code=4/uwEuBngopW0X_hXw2u7kCgTarRLie-nZQBPPnJl9g8yOD_MPT-VEDbJ2nu0kw3QGZeHNtsvJYAOHw1JivCxjFVI&redirect_uri=http://localhost:8081/dialogflow&grant_type=authorization_code&client_id=188757296864-kmlu4rmmkhd8eplk3oj93a8pftiru4ta.apps.googleusercontent.com&client_secret=toO7xx-qPlQFRG2jC8ymcUll”

where

code: Use the Authorization Code generated in the previous step.

redirect_uri /client_id /client_secret: Same as used in previous steps.

Below is the response generated which contains Access Token:

{

  “access_token”: “ya29.Il-3B1fyhi1PgG_YdBsWxPKyIH1JzEVEh2H5ESWrU4sWxxW-D-9oRM_SSpuP_rKo6zbDWG_359CjpJHloQiN6tfq8-cH0FCTdRyzi4UPPVwbqQQcYf4ZqByDk4GZe0CHOg”,

  “expires_in”: 3600,

  “refresh_token”: “1//0gO0TAAFXjruhCgYIARAAGBASNwF-L9Irz01QJkOHGxOaSm-F7yIkC-tMuaU1JD-F6_JG0wBoPOJ6i3kXpuTqJ4GL-Xzd0wzE1fM”,

  “scope”: “https://www.googleapis.com/auth/dialogflow https://www.googleapis.com/auth/cloud-platform”,

  “token_type”: “Bearer”

}

1. Test Dialogflow API using the above generated Access Token in the Authorization header by doing a POST to below URL with a Request:

https://dialogflow.googleapis.com/v2/projects/sunlit-hub-247818/agent/sessions/916442:detectIntent

Sample Request:

{

  “query_input”: {

    “text”: {

      “text”: “reserve a meeting room for six people”,

      “language_code”: “en-US”

    }

  }

}

This whole process of generating Access Token can be automated using MuleSoft.

Create a Dialogflow Agent

A Dialogflow Agent is a virtual agent that handles conversations with your end-users. It is a natural language understanding module that understands the nuances of human language. Dialogflow translates end-user text or audio during a conversation to structured data that your apps and services can understand. You design and build a Dialogflow agent to handle the types of conversations required for your system. 

To create an agent:

1. Go to the Dialogflow Console using this URL: https://console.dialogflow.com/.
2. If requested, sign in to the Dialogflow Console. 
3. Click Create Agent in the left sidebar menu. (If you already have other agents, click the agent name, scroll to the bottom and click Create new agent)
4. Enter your agent’s name, default language, and default time zone.
5. If you have already created a project, enter that project. If you want to allow the Dialogflow Console to create the project, select Create a new Google project.
6. Click the Create button.

This is how your Agent Settings (General) looks like :

An intent categorizes an end-user’s intention for one conversation turn. For each agent, you define many intents, where your combined intents can handle a complete conversation. When an end-user writes or says something, referred to as an end-user expression, Dialogflow matches the end-user expression to the best intent in your agent. Matching an intent is also known as intent classification. For more info on Intents & its related, check-out here.

You can import intents & entities separately by clicking on IMPORT FROM ZIP under Export and Import settings of the agent.

Design a Mule Flow & Deploy the Mule Application

Flow design involves these steps:

1. Configure a HTTP Listener in order to listen to an event (chat message) triggered at the Inbound Message Webhook URL:

1. Set a Variable to store Receiver ID:

1. Set a Variable to store Sender ID:

1. Use a Transformer to format Request for Dialogflow API:

1. Configure a HTTP Requester to invoke Dialogflow API:

1. Use a Transformer to format Request for Nexmo Messages API:

1. Configure a HTTP Requester to invoke Nexmo Messages API:

After the flow design is completed, deploy your Mule Application on CloudHub using Runtime Manager.

Once deployment is done, test the application by initiating a chat with the FB business page.