Introduction

Book convention
Server setup
Telegram and NodeJS
Setup the bot in the Telegram
Setup certificates
The code
List of commands
Slack and NodeJS
Setup the bot in the Slack
The Code
Messenger and NodeJS
Setup the bot in Facebook
The code
Language data preparation
Introduction
Wordnet
Conclusion
Appendix A
Appendix B
Introduction
As in every good book we need to go back in time and start with some history.

If not the first, definitely best known bot aka chatbot is a program called ELIZA, created by Joseph Weizenbaum from MIT Artificial Intelligence Lab
in 1960s. The program was quite simple, but the effect was immense.
People really believed it was a doctor typing on the other side and talked well, typed to it as a one.

(via: http://www.alicebot.org/articles/wallace/eliza.html 23.8.2016)

Smartness has always fascinated people so we will be writing smart bots well, not so fast.
The bots we are talking about are much simpler, but they can have smartness of well, uhm gold fish or something
Anyway lets do it.

We will be doing bots on three major platforms for the communication: Telegram, Facebooks Messenger and Slack.
The technologies are similar, but still a bit different.

We are using NodeJS, but the technology is similar if you prefer other programming languages.
You need to have access to the internet and some possibility to setup a secure web server. Certificates are not a problem as we will see later.
Book convention
To make the book easier to use I tried to mark some words that I thought are important bold and also added code that was written or data returned from
the server in the boxes. The boxes are coloured depending on the code or data in them.

Server setup
We will setup our web server on the Apache platform, but you can, obviously, use other web servers (as Nginx or similar). The only thing we will be
using is a proxy passing of communication, so you can setup similar thing with other web servers.

Just one more thing. Since you need to use an SSL certificate you will need to have a domain name and not the IP address.

Lets setup an Apache proxy server. The syntax is rather simple:

We are creating a https server since the message environments need to use secure communication.
As you can see I left email in ServerAdmin as default. You can obviously put your own email.
Be careful with the selection of port. Here we are using port 8080, but it depends on the port you are using for your NodeJS server (e.g. if the server is
on port 8082, you should use 8082 here).

The same setup will be used for all the chat platforms.
But remember: you need to create a certificate for the domain.

And you might also need to change the port for the localhost server since some traffic might still go to the previous one.
Telegram and NodeJS
Setup the bot in the Telegram
First, you need to set up a bot in Telegram app. You need to connect with BotFather. This is a bot that creates other bots .. how obvious.
To create a bot just use a command
/newbot and BotFather will explain it
to you what you need todo

Lets give the bot a unique name.. The name is not the same as a username and you can use whatever you want. I guess I dont need to remind you,
that it is better to use descriptive names. You can see the name of the bot as a persons name and you will have to add username after that.
As mentioned, the BotFather leads you through the procedure.

When choosing the username you need to add bot at the end. And if you choose a name that is already taken, the BotFather lets you know about
that.
After successfully entering a bot name you will get the success message.

With that you will get some explanation about newly created bot and also a secret token, that should really be secret.

You can add a description, about section and profile pic to the bot and you can do that via commands /setdescription, /setabouttext, /setuserpic.

OK. So this is basically it for creating a bot inside the Telegram.

Lets start coding the server that accepts connection via Telegram and replies with a smart answer.

As said in the introduction, we will be using NodeJS. For all our examples we are using request module, so if it is not installed yet, use npm install
request.

Start creating a bot with this command. The BotFather will lead you through the procedure of creating a new bot.

After adding a name for the bot you will need to add username for the bot and you need to add bot at the end, so the users will know this is the bot.

Just to be clear - to communicate with the bot you are using its name and not username.
Type in the username and you get the following response.

Done! Congratulations on your new bot. You will find it at telegram.me/<name_bot>. You can now add a description, about section and profile
picture for your bot, see /help for a list of commands. By the way, when you've finished creating your cool bot, ping our Bot Support if you want a
better username for it. Just make sure the bot is fully operational before you do this.

Use this token to access the HTTP API:

<secret token code>

For a description of the Bot API, see this page: https://core.telegram.org/bots/api

The secret token code is the token you have to keep secret, so dont publish it no matter what.

To get manual on different commands for your bot just click the link on description of the Bot API.

Use this command to see the token again. You will have to pick for which bot you would like to see the token, if you have more than one bot.

If you forgot your secret token or for some reason need a new one, use this command. You will have to select for which bot you would like to generate

a new token.

Creating new list of commands for the bot can be done with the following command:

What is the list of commands?, you ask. Glad you did. We will explain it later.

Its obvious - if you need to change the bots name use this command.

Describe what the bot is doing.

The bots profile photo.

To change profile photo of the bot. You can upload the photo by clicking the clip button and selecting the image.

Add text that will be visible in command line input box when you type in the name of the bot.

When you enable geo location, the Telegram app will ask you for the location every time you try to communicate with the bot.

To know which of the provided results your users are sending to their chat partners, send @Botfather the /setinlinefeedback command. With this
enabled, you will receive updates on the results chosen by your users.

Please note that this can create load issues for popular bots you may receive more results than actual requests due to caching (see the cache_time
parameter in answerInlineQuery). For these cases, we recommend adjusting the probability setting to receive 1/10, 1/100 or 1/1000 of the results.

You need to send the list of commands that will be shown to the users. Go to new line to enter commands by pressing shift + enter.

Simple command to allow or forbid bot to enter the group.

Enable - your bot will only receive messages that either start with the '/' symbol or mention the bot by username.
Disable - your bot will receive all messages that people send to groups.
Current status is: ENABLED

remove the bot, but you need to confirm it with exact phrase Yes, I am totally sure. (with a punctuation mark at the end).

cancel current operation.

Setup certificates
All the bots are working over the secure SSL connection, so you need to provide the certificate.
And currently - only Telegram has a possibility for you to use self signed certificate, but you need to upload the public part of the certificate, to let the
Telegram trust the certificate.

You can do a simple test if you got the proper token even without letting the server know about the certificates and that is getting the bots name via

curl command:

NOTE: Pay attention to bot prefix for the token. You have to use it.

We can, of course, create a pure javascript solution:

As you can see, we are sending a parameter url by using curly brackets and you can send additional parameters if needed this way.

The result (in JSON format) is found in body, if everything goes well :)

For other objects and methods you can check the Telegram API documentation.

There are two possible ways to get the incoming updates (data from the server) with two different commands: getUpdates and setWebHook.

getUpdates uses long polling which basically means: we ask server and then wait.
On the other hand you give Telegram an URL via setWebHook and when event happens the Telegram sends POST request to the server and gets the
data.

Lets try with getUpdates

For the sake of book space we will use curl, but to use NodeJS example just replace getMe in previous example with getUpdates.

and we get result in JSON format:

So we received nothing. Why? Well, we didnt start the bot

yes. So lets start the Telegram app and start it.

We need to search for the bot username via search input (dont use name) and select it. After clicking Start we can write a message to the bot.

Lets repeat the curl command to see what we will get.

Wooohooo our first response from the Telegram. As you can see, there is already a /start
command in the list.

What happens if we repeat the getUpdates command?

Lets try.

Ooops, we get the same result all over again. One of the things getUpdates does is just sending results every time you ask for them, so if you want to
use that command you will have to check which responses you have already processed.
You can set a limit to the number of messages received (default is 100), via parameters in getUpdates.

But there is a better way to fetch messages (updates as Telegram people call them) from the Telegram and that is by using command setWebhook. We
are not really fetching the messages but rather let Telegram know that whenever something arrives for us, we are ready to receive it.

There is one other thing. You cant use both getUpdates and setWebhook, so by using setWebhook we exclude the possibility to use getUpdates.

Lets first try the Telegram bot by using self-signed certificate. How to create the certificate can be found in Appendix A.

Since we are using self-signed certificate we need to let the Telegram know the public key of our certificate. This can be done by sending the public
key with the url of our secure server that will be used to create a reply to the user query.

The first test would be to use curl:

NOTE: We are using @ symbol for file.

To set the same thing with NodeJS we will have to use request package:

If everything goes well we will get the response:

If we want to turn off the webhook service we just need to send empty request to the api.telegram.org

We are now ready to setup the server and try it.

But since we will be using proper SSL certificate (one of the procedures is also attached in appendix B) we dont need to upload public key of the
certificate so you can set the connection by using only URL. So lets change WebHook to work with that certificate.

Dont forget to remove the connection with your own signed certificate and WebHook first, by using:

And set new connection with the URL:

The code
We can now setup NodeJS bot server on our server to simply reply to whatever has been sent to
it.Since the server is running on the port 8080 we will leave the Apache setup as it is.

Lets explain the code a bit.
The server receives the data from the Telegram. Remember we setup the url for the WebHook.
After receiving the data (POST HTTP method) in JSON format it gets parsed to get the id of the chat, so we can reply to the proper message in
Telegram and we also parse the text that was sent to the server and setup variable return_text.

The value of the variable is returned to Telegram via an API command sendmessage where we need to add chat_id and text values.
List of commands
No, I didnt forget. What is list of commands?
When communicating with BotFather you can use command /setcommands and select for which bot you would like to add commands.
Select one that you have created and the response will say

Lets try with:

You need to enter the list in one message and to move to new line you have to use Shift + Enter. By using only Enter you send the command and we
dont want that, right now.

When using the command you always need to add slash and name of the command. The whole text will be sent from the Telegram to our bot server in
the message body and since the text starts with slash we can assume we are dealing with command.
Slack and NodeJS
Setup the bot in the Slack
You cant use self-signed certificate with Slack, so we will not go into this and we will just use properly signed certificate as shown in Appendix B.

Lets first create a bot in the slack. Bots here are connected to the real time messaging API.
So go to Apps & Integrations in your
slack client (tested on mac)

It takes you to the web page where

you can select Build to start building
the bot.
You will have to decide if the bot will
be used only for your team (that you
created in Slack) or it will be
available to everybody.

If you just want to test it you can create

your own team and create a bot there.



Selecting Incoming WebHooks gives you a possibility to post messages from external source to Slack. Its a HTTP request in JSON format.

Well select Bots for our usage and look at it a bit later.

Slash Commands gives you a

possibility to create slash commands
and when using customised command
the slash will send relevant data to an
external URL (e.g. server).

Outgoing Webhooks is more of a flag

thing - if something is going on, send
data to external URL.

So lets look at Bots now.

Click on Bots and you can now create

a bot username with some restrictions:

Usernames must all be lowercase.

They cannot be longer than 21
characters and can only contain
letters, numbers, periods, hyphens,
and underscores. Most people choose to use their first name, last name, nickname, or some combination of those with initials.

After selecting the name click Add bot integration and the wizard takes you to the next screen where you can change the name of the bot and other
integration settings stuff.

But pay attention to the API token.

Another thing to note is on which

channel is the bot used.

After everything is well and done and

if you made any change just click
Save Integration and you are good to go.
Lets close the web page and setup our slack related bot server.

To add a bot to the slack you need to connect it with a custom trigger that will call the create bot.
Start by clicking on the Manage selection (by the Build menu), then click Custom Integrations and select Slash Commands. Click on it and start
new configuration by clicking on the Add Configuration.

Add a command that will act as a trigger to call the bot and dont forget the slash.
In the URL field you have to add the URL of your bot server (it should be in the secure URL format https://..). You can leave other stuff empty or
default.
The created command will be called /return and the service will do the same as in Telegram. So, it will return what you wrote.

Pay attention to the token - if you need to see if the user is really part of your slack team, but we will leave it for now.
Finish with clicking Save Integration.

After that try to use the command and the bot should respond. You can also check what you get in the command line of the bot.

The Code

We can now customise settings for the bot with autocomplete text (what will show in command line when you type in the custom command), hints
etc

As you can see the server is similar as in Telegram.

We will use Apache again, but this time the NodeJS server will be running on the port 8082, so you need to change the setup procedure for the Apache
accordingly.
Messenger and NodeJS
Setup the bot in Facebook
To setup a messenger bot you need to have Facebook account, of course :)

Open the Facebook in the browser and select Manage Apps under Explore (on the left side). Here we can Add a New App. Lets click it.
Dont forget to select a category, but
you will be notified anyway.
After clicking Create App ID we get
the Product Setup menu. There are a
lot of options you can work on, but we
will concentrate on those related to
bots and Messenger so lets choose it.
After clicking Get Started we will Setup WebHooks.

Add a Product and select Messenger
(Get Started). Click Get Started.

Generate token in Token Generation by

selecting the page and giving a
permission to the app. You will get page
access token which you need to copy in
the server file. Just click on it and it will
be copied to Clipboard.

To set up the WebHook you just need to

add a url (with https://) and verify token
(whatever you pick). Just select all fields
under Subscription. Click Verify and
Save (the server should be running).

You will also need to select a page to subscribe your WebHook to the page events - just pick the same one you used for token generation and
Subscribe.
The code
The code for the server is:

In the Messenger we also need to add a GET responder since Facebook checks if server is still running. Other than that, the procedure for the bot
server is very similar to previous versions for the Telegram and Slack.
We parse received data to get the message sent by the user in variable received_text and use that value as a responder to the user input.
Language data preparation
Introduction
Before you can use the received text data you need to do some housework.
To make things easier we will remove some characters that are not that useful for understanding the received text.
We can easily remove punctuation and lose very little meaning of the sentence.
Next idea is to remove one or all of the following:

100 most common words in language you are using (be careful since we are working with very short text e.g. sentence and you might remove
important information)
remove most re-occurring words in the document (this one is useful for larger texts)
shorter words (e.g. 3 letter words, because they dont carry that much information)

(from https://en.wikipedia.org/wiki/
Most_common_words_in_English)

The next tool we can use is

lemmatisation, which means
changing the word to its basic form
and by using a technique using the
technique called part-of-speech
tagger you can tag the words with
their forms and by using verbs you
can find out what the user wants
well, usually.

One of the useful tools is also a

stemmer. The stemmer tries to cut
end of the words and group similar
words. (e.g. receive, received, receiver
etc. are all cut to the stem receiv.

But for all that we could be using

some tools, if they are available. A
search on the internet shows there is a
natural language processing package
for NodeJS called Natural.

Lets install it with:

As stated in introduction we will pull

out words from the received text. We
will be using a tool called Tokeniser
for that.

The code will produce a list of words from a sentence.

We can search through the list for 100 most common words in language and remove them. This would mean we are removing a, from which would
already cause some troubles, so we will not be using such preprocessing of the text now.

As you can see this part of the process involves some thinking and experimenting.
Since we would have to setup some external files for a POS tagger used in natural we will not use that tool and instead try to cop with a stemmer.

Here is the example for different versions of word receive.

Wordnet
Since this is not a book on natural language processing we will take a really short look into Wordnet.
Official description of it is A lexical database for English which basically means you can get synonyms for words.
So if your user uses synonyms in their sentences to the bot you can try to add Wordnet to your code and try to find a synonym for the words used.
The Wordnet can be found online (https://wordnet.princeton.edu) and is also available for a download.

Lets try our receive example. By searching online we get the following:

As we can see, this is a verb of

examples of different uses and
synonyms that might mean the same.
Conclusion
This concludes our look into bots by using NodeJS. Hope you found it useful and will be on your way coding bots in NodeJS or any other
programming language. The idea is basically the same, so you will not have a lot of problems porting our examples to other programming languages.
Appendix A
Creating our own private certificate is quite simple. The following command will create the whole thing.
Mind you that certificates need to have an expiry date and with this command we are saying that the certificate expires after a year (365 days). To
create a longer lasting certificate you can just add some days to the days part. Sometimes I create a certificate valid for 10 years. This might cause some
security issues, but its ok for testing. Although - if you are testing, why set up a certificate valid for 10 years?

The important thing here is that you add proper domain name to the CN part (without https://).

The certification procedure will fail if you make a mistake here.

After the procedure you will get two files: private and public key in a text format. The private key will start with a BEGIN RSA PRIVATE KEY and
public with BEGIN CERTIFICATE.

To see what is included in a public certificate you can use following command:
Appendix B
We can create an open source free ssl certificate by using https://certbot.eff.org

Select which web server you use and

the platform and you get info how to
setup the certificate.
We will select Apache as a web server
and since it runs on Debian Linux
that will be our Operating System
selection.
We have to install some packages on
our web server to use certbot tool.
The installation procedure depends on
the operating system you use for your
web server, but the command is
always the same:

If you have many virtual servers setup

on the same IP you will get the list of
them and you pick one. The name
should be the same as the name of the domain server.

On Linux the created keys will be located in the directory

/etc/letsencrypt/live under the directory with the domain
name. To use the certificate on the web server (remember we
are using Apache), you have to add fullchain and privkey files.