Getting Started
Categories:
Welcome to our early access program!
Prerequisites
Docker is required to run this unsupervised learning trial.
WARNING: Docker runs containers that are currently unsupported on machines with the Apple M1 processing chip. Mac users with M1 chips may have trouble using the tool. To find out if your Mac has an M1 chip, navigate to “About This Mac” after clicking the Apple logo on the top left of the screen. The chip is listed under “Processing”.
Docker Settings
After Docker is installed and running, navigate to “Preferences” by clicking the gear icon on the top right of the screen. Then navigate to resources. We recommend adjusting the “Memory” to 8.00 GB. Once adjusted, click “Apply & Restart”.
NOTE: The insights tool has the ability to use a lot of memory, but setting Docker to 8.00 GB is only recommended, not required.
Installation without CLI
To use CLI commands for installation and use, please scroll down to “Installation with CLI”.
We recommend the CLI method for installation and use.
To run the insights tool without the use of the CLI, open the Docker Desktop application installed on the previous step.
Navigate to the “Add Extensions” tab on the left side of the screen.
Search for an extension named “Portainer” and install it. This should only take a few seconds.
When it is installed the extension will appear on the left side of the screen.
Open Portainer and navigate to the “Home” tab. Then click “local” under “Environments.”
Navigate to the “Containers” tab and click on “Add container” at the top right of the screen.
This step is where the container is created in which the insights tool will run.
Name your container “insights” at the top. For “Image” paste in:
servisbot/insights
If the image name is not spelled correctly no image or an incorrect image will be pulled.
Navigate to “+ publish a new network port” below and click it twice.
Two rows will appear with headers “host” and “container.” The rows should look like this:
host:80 container:80
host:3000 container:3000
From there click on “Deploy the Container.” Make sure nothing else has been altered on the page, as it could affect the installation of the insights tool.
A reference image is below. This installation from Docker Hub may take a few minutes to complete. The most recent version of the insights tool will be installed and run on your Docker Desktop app when this process is complete.
The deployed container should look like the image below.
When the container is deployed and running, navigate to localhost in your browser.
If these steps have been completed, the next two steps, “Installation with CLI” and “Setup with CLI” can be skipped. Please navigate to “Running the Tool.”
Installation with CLI
The latest version can be found on Docker Hub.
To pull the current release, after installing docker, use your terminal to run the following command:
docker pull servisbot/insights
Setup with CLI
Once the container is downloaded you can run it with the command:
docker run --rm -p 80:80 -p 3000:3000 -i servisbot/insights
Once it is running, navigate to localhost in your browser to start with the experience.
Running the Tool
Login
To login, use the same business email address used when registering for the trial. The trial will allow running the service using ServisBOT sample data until you have a working API key.
If you choose to upload your own data, you will need to request an API key from this page, using the same business email address you provided when registering for the trial.
Once requested the API key will be generated and sent via email within one business day. If you have somehow reached this documentation prior to officially registering for the trial, please navigate to this landing page and fill out the form. The form only needs to be filled in once.
Select Your Demo
Once signed in, the automated machine learning Insights tool is ready to be used. Select “General Step 1 NLU Automation Demo” to get started.
From here, an outline of the NLU automation process is shown.
Hitting “Next” will navigating you to the next screen and prompt you to upload a bot design.
Upload Your Designs
It is important to make sure that files are formatted correctly.
In order to run the service, the data should be in the ServisBOT common format, we do however support automatic conversion to this format with a number of supported export formats which can be uploaded directly. An error message will appear if the file is not formatted correctly.
ServisBOT Common Format
In the ServisBOT common format, data is arranged by the required columns “Botname”, “intent”, and “utterance”. The order of the columns does not matter but names are case sensitive. For botnames and intents with multiple words, avoid using spaces as separators. Recommended special characters for word separators include hypens “-” and underscores “_”. PascalCase or camelCase are also recommended for compound bot or intent names. In the “utterance” column any character can be included, including emojis.
To convert your data into the ServisBOT common format, see converting to ServisBOT Common Format.
Analyze Data
When the analysis is complete and displays 4 checkmarks this signifies that a report has been generated for the bot design outlining some general statistics of the dataset. It is important to note that at this stage no optimization has been performed.
Once the analysis is complete and there are 4 checkmarks, a report has been generated for the bot design outlining some general statistics of the dataset. It is important to note that at this stage no optimization has been performed.
Navigating to the next page will show a scorecard of the results.
Review Results
This page shows initial accuracy scores of the bot design without any ServisBOT optimization. The scorecard is shown below the loading bar.
Each item on the scorecard will reveal a definition of the term if hovering over it with the mouse.
- Baseline Accuracy: The probability of identifying an intent correctly given a confidence threshold
- The confidence threshold is set at 0.6 (60%) for this tool
- True Positives: The number of utterances correctly classified above the confidence threshold
- An utterance must match to an intent with a confidence of 60% or higher
- False Positives: The number of utterances incorrectly classified
- False Negatives: The number of utterances not classified due to low confidence
- Total Utterances: The total number of utterances in the dataset
- Training Utterances: The number of utterances used to train the model
- Roughly 80% of the data
- Testing Utterances: The number of utterances used to test the model
- Roughly 20% of the data
- Topic Count: The number of topics identified after clustering the data
Intents are split into categories of “Good” and “Bad”.
- Good intents are defined by having more than 5 utterances with those utterances being closely related to one another.
- Bad intents are defined by being undertrained, poorly defined, or having bad samples.
- Undertrained: An intent with fewer than the minimum number of samples required to properly test an intent (fewer than 10 utterances)
- Poorly Defined: An intent with fewer than the minimum number of samples to properly define an intent (fewer than 3 utterances)
- Containing Bad Samples: An intent containing utterances in command form or longform
- Command: an utterance containing 3 or fewer words
- Longform: an utterance containing 20 or more words
While reviewing the scorecard displayed above, a subsequent step in the NLU Automation process, dataset optimization, is running behind the scenes. The goal of dataset optimization is to produce an improved model.
Once the process is complete, navigate to the next screen by clicking the “View Updated Model” button.
Improvement results will be shown along with a side by side comparison of the existing model and the updated model.
Above the scorecards there are “View Accuracy Details” and “View NLU Performance” buttons.
Clicking either button will navigate to a new page for their respective details.
The “View Accuracy Details” button brings up a classification report that shows intents broken down by number of true and false positives, precision, recall, and f1 scores. The table can be sorted by column by clicking the column header. The “Display Comparison” slider on the top right will overlay the existing and new model for a side-by-side comparison.
The “View Model Comparison” button below the expanded classification report will navigate back to the side by side comparison of the models.
The “View NLU Performance” button navigates to a screen with four graphs showing comparisons of the existing and new models. More details about each distribution can be found by hovering over different points on the graphs.
The “View Model Comparison” button below the graphs will navigate back to the side by side comparison of the models.
Once all results have been reviewed, the end of the process has been reached.
The entire process can be started over with new data by clicking “Start Over” button at the bottom of the screen on any of the results pages.
For more information about the tool, customization, or to leave feedback, click the link in the top right corner of any page within the Insights tool.
Personally Identifiable Information (PII)
Usage statistics are collected and shared with ServisBOT to help improve the tool. However, no personally identifiable information such as training phrases or conversation data is collected or shared.
For more information about ServisBOT, click the link in the top left corner of any page within the Insights tool to navigate to our website.
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.