10 Questions to Ask Your Chatbot Developer
By now you have seen enough proof of how a chatbot can solve your business problem. You have found a chatbot developer, and it is time for you to start vetting them and decide who to use.
From teenagers sat in underwear in their bedrooms (we have all been there) to super-expensive high-end agencies with brick lined walls and neon lights. These are the 10 questions you need to ask every chatbot developer. Some of these will make her sweat, and they should. You may pay her lots of hard-earned money; you need be sure she can deliver.
Got a meeting setup? Download the 20 Must-Ask Questions for your Chatbot Developer Worksheet! Make sure you go to that meeting armed and prepared.
Download the Chatbot Developer Questioning Spreadsheet
Q1) What chatbots have you developed before?
You might think this is a bit of a 'go straight for the jugular' question. Some like to ask it first as it acts as the qualifier before deciding to continue the conversation.
You may be happy with a chatbot developer who's on her first build, or you might want to see a history of success. Enterprise-level clients tend to need approval from the hierarchy and so may need to have the evidence to say "the chatbot developer has worked with Unilever or Johnson & Johnson" or the like.
For the chatbot developer, it can be a tough question to answer, particularly in such a new industry. Few other companies have built enough chatbots to be able to dazzle you with their case studies. Even those that have may not have examples of chatbots in your particular industry.
You, as the client, may have to illicit functions or unrelated successes the chatbot developer has had and put the pieces together to feel confident in their ability.
Ask yourself, does her work history make you feel confident in her ability to deliver?
Q2) What lessons have you learnt from developing chatbots?
Regardless of how the above question is answered (unless the chatbot developer has never created a chatbot before - warning!), she should be able to give you some insight into what makes her a better chatbot developer than others.
Like all other programming disciplines, developing chatbots requires continuous learning. Whether it is how to hit 150 messages per second, the best way to structure a Facebook Messenger campaign or tips on getting the best user engagement. Your chatbot developer needs to prove to you she is at the top of her game.
She should be able to demonstrate she is up-to-date with the rapid changes in the industry. Otherwise, your shiny new chatbot may be obsolete by the time it launches.
Does she give you confidence she is up-to-date with the industry?
Q3) Can you repeat to me what my chatbot will do?
The language of chatbots is new. For starters, even the press and developers cannot decide if it is chatbots, chat bots, bots or chatterbots (FYI, stick with chatbots).
You may find it tough to explain your vision of what your chatbot developer should produce. The terminology is new; it differs based on the delivery channel (FB Messenger vs. Slack, for example) and, if using artificial intelligence, may get exotic quickly.
When explaining what your chatbot needs to do, be sure the chatbot developer creates a project scope. This document should detail what it is she will develop for you. It should include all functions, who owns the data and assets, where the words and media will come from, what happens when your chatbot launches and how maintenance will be carried out.
Projects grow wild quickly, and the project scope is what holds it all together. Without one, neither you nor the developer will be able to see how it is progressing.
A trait of a good developer is their ability to speak human as well as machine. Can she explain what your chatbot will do in a way you can relate to and understand? Does what she said match what you expected and contained in the scope? Is there a scope?
Q4) Who is responsible for writing the content?
Your chatbot needs words, after all, that is how you communicate with it. Equally, depending on what your chatbot needs to do, we could be talking more than just words. Images, emoticons, gifs, video and even audio can all be within the scope.
We are finding it is not just functionality which makes your users engage with your chatbot. It is the chatbot's personality; tone, language and how much they 'like' it. This all comes from copywriting.
We recently developed a chatbot for Unilever which conversed across more than 200 conversation topics using 12,762 words (and this was a small-ish chatbot).
Who is writing this content? You? The developer? If it is the developer, you better make sure she is a high-end copywriter too.
What about images, who is creating them? Are they the correct size for the channel your chatbot is using (both in dimension and file size)? Ask your developer what the best ratio for an image in Facebook Messenger is, does she know? (they will now as they have probably read this!).
Q5) What if my chatbot stops working?
Your chatbot will be hosted somewhere; your developer might not explain (or want to, depending on how she built it) where it is. However, somewhere, there's a server where your chatbot is living. You have spent lots of money having this chatbot developed, what happens if it stops working?
Make sure you ask your developer to include a Service Level Agreement (SLA).
This document should be contractually binding and explain what happens should your chatbot stop working. Most SLAs offer some form of uptime guarantee, typically in the format of a percentage of uptime. You have probably seen it before, something like "99.99% uptime guarantee." It will also include details of any compensation you will be owed should your chatbot go offline.
Ask what her uptime guarantee is. Ask to see and sign an SLA. Do not have a chatbot built without an SLA.
Download the Chatbot Developer Reporting worksheet for free!
Q6) Where is my chatbot hosted?
Depending on what your chatbot intends to do, there may be privacy and data considerations. The location where your data is stored affects which law and legislation it is governed by and what you can do with that data.
It is of particular importance if your chatbot deals with sensitive data, such as financial or medical (ensure you have the adequate legal in place before even considering doing anything with financial or medical data).
Your users will most likely have the right to ask what data you have about them and where it is located. Ask your developer tough questions about data. If your chatbot starts getting attention, it will come up, and it is ultimately your responsibility.
Where will the chatbot data be stored? Who is responsible for that data? What happens when someone wants access to it?
Q7) Are you using any other services to create my chatbot?
During 2016, a multitude of 'do-it-yourself chatbot building platforms' launched (we are talking more than 20 of them within 12 months). These platforms offer variations of drag-and-drop, code free and easy-to-develop simple chatbots. They are designed to build shallow, flow-based and non-artificially intelligent solutions. They are the WordPress of chatbots, good for the masses but not for bespoke high-end brands.
Some of the developers you talk to will take your chatbot idea, charge you lots of money and then use one of these (often free) platforms to develop it.
Depending on your circumstances, this is not necessarily a bad thing, after all, it still saves you from dragging and dropping. The problem comes when things go wrong, as it is not your developer who has the control, it is the platform she used to build it on. If your chatbot goes down, she cannot help. If you need access to your data, she cannot help, and if you want additional functionality her preferred platform cannot do, you guessed it, she cannot help.
Also, if your chatbot requires artificial intelligence (and it should, even if only at a basic level), then ask what service she is using for understanding intent.
Ask which platforms she is using to develop your chatbot. Is she coding it all from scratch or using a third-party solution? If it is using AI, then ask which AI service she is using.
Q8) What if my chatbot gets 100,000 people all messaging at once?
Your chatbot is live and busy sending and receiving messages, congratulations. Now, hopefully, this was covered in your scope document (you did get one right?), but, what happens when it starts getting thousands of users?
Like websites sometimes struggle when facing too many visits at once, if too many people try and talk to your chatbot at once, it could struggle to keep up, reply slowly and even stop working altogether.
This problem means double trouble if your developer is using third-party services. She has no control in scaling up the server infrastructure to deal with the traffic and might receive additional charges which she may pass on to you.
You can make this question fun, ask what happens when your chatbot gets 100,000 users. (Ask a super-high number like this to stress her knowledge of what she will do. If you ask something sensible she will just brush you off with an "our server can handle it" type response.)
Q9) Who owns my chatbot?
Much like the other questions, this should be defined in the scope that you and the developer sign and agree. Broadly speaking (it may be completely different based on your arrangements) chatbot ownership currently falls into one of two categories.
Proprietary or Open Source
If your developer grants you access to the chatbot following an end-user license agreement (commonly known as an EULA), then you are working with them in a proprietary way. The EULA will detail what you are entitled to do with the chatbot and in what ways you can use it. Importantly, you do not own the chatbot; you are simply using it in accordance with the Terms & Conditions of the EULA.
Your developer may elect to open source the chatbot and the code that powers it. This allows you to modify and use it without restriction in accordance the rules on the official OpenSource.org website.
Ask your developer how you will take control of the chatbot. Will you have access to the source code and be able to make changes to it?
Sidenote, if the developer does not give you the source code it may mean she does not have it, i.e. she is using third-party software.
Even if you do not know what to do with the source code, you should ask these questions as it will give you insight into who is building the chatbot.
Q10) What is the best way for me to market my chatbot?
As it is such a new industry, most developers will be able to give you insight on what to do with your chatbot once it is ready. By asking questions about marketing, engagement, and metrics, it will give you an insight into how knowledgeable the developer is about the wider industry.
Even if she only gives you examples of how other companies have done it, i.e. this company uses this viral function or this other company has an excellent way to keep users engaged, it will help you ascertain if your developer is up-to-date with the competition and best practice.
Admittedly, for chatbots that are delivered using modern messaging applications, everyone is still learning. No one knows the best way to onboard users into Facebook Messenger or how to re-engage Telegram subscribers. What you are looking for with this question is the confidence that when someone publishes their results and starts to form best practices, your developer is on it and able to guide you.
If your chatbot is going to be using artificial intelligence, your developer needs to advise you on methods around stopping people breaking it and trying to get Twitter-worthy images of it saying something silly, rude or offensive.
Most developers should have blacklist words and ways to stop chatbots getting stuck in "I do not know" type loops.
Ask questions like how many words are on your blacklist. What is the best way to onboard users? Can you advise me on best practices to keep my users engaged? What is the best way to build virality into the chatbot?
As a handy swipe - here are the 10 questions for a chatbot developer ready to copy-paste.
Q1) What chatbots have you developed before?
Q2) What lessons have you learnt from building chatbots?
Q3) Can you repeat to me what my chatbot will do?
Q4) Who is responsible for writing the content?
Q5) What if my chatbot stops working?
Q6) Where is my chatbot hosted?
Q7) Are you using any other services to create my chatbot?
Q8) What if my chatbot gets 100,000 people all messaging at once?
Q9) Who owns my chatbot?
Q10) What is the best way for me to market my chatbot?
So there we are, our top 10 questions to ask your chatbot developer. Why don't you hit the button below and ask one of our chatbot developers the questions and see how they do? If they fail massively then let me know :).
If you haven't yet, make sure you grab our free worksheet, 20 Must-Ask Questions for your Chatbot Developer. Get a full reporting sheet to work with during your initial meetings. Make sure you ask all these questions!
