Agents

Agents are AI assistants that can be customised to perform specific tasks. They can be created, edited and hired from the Settings > Agents menu.

Menu Location

Agents can be created and edited from the following menu: Settings > Agent > Hire Agent

Required Details

• Name
• Mode (select one):
  • Retriever
  • Coder
  • Casual
  • Planner
  • Desktop

Mode Descriptions

1. Retriever: Chat agent enhanced by your documents, internet search, and functions
2. Coder: Coding assistant for developers and data scientists, enhanced by programming documentation and code environments
3. Casual: Everyday agent for general-purpose activities, image generation, and file analysis
4. Planner: Agent that can plan and execute tasks leveraging other agents
5. Desktop: Agent that can run on an entire Linux based operating system with the ability to run a browser using mouse and keyboard.

Mode-Specific Settings

Retriever Mode

• Topics (Under Knowledge settings) - required
• Optional features:
  • Step by step reasoning: Forces the agent to explain its reasoning step by step
  • Cross document analysis: Allows the agent to analyse multiple documents separately to cross check and validate information using different sources of information at each step
  • Knowledge Settings: Allows the agent to use documentation to answer questions
  • Internet search: Enables the agent to search the internet for answers

Coder Mode

• Optional features:
  • Knowledge Settings: Allows the agent to use documentation to answer questions
  • Code execution: Enables the agent to execute code
  • Code upload: Allows the agent to analyze source code

Casual Mode

• Optional features:
  • Image generation: Allows the agent to generate images based on user requests
  • Docs upload: Enables the agent to analyze files, bypassing the Knowledge Hub

Planner Mode

• Agent selection: Choose agents to execute tasks
• Planner tools: A new section appears to perform necessary operations

Desktop Mode (Enterprise only)

• Optional features:
  • Assign credentials of type Username and password to dekstop agents to securely login to your web-apps and perform actions.
  • Desktop agents are even more powerful when used by Planner agents to execute multi-step tasks.

Credentials

We strongly recommend to create dedicated user accounts for your dekstop agents. This will prevent and limit the risk of unauthorized access and abuse.

Knowledge Settings

In the Knowledge Settings, you assign the topics to the agent and set the configuration of the agent and the knowledge hub. By default, new agents will not have any topics assigned to them. You will need to assign topics to the agent in order for it to be able to answer questions.

• Topics: The topics that the agent will be able to answer questions about.
• Max history (1-10): How many of the previous conversation will the agent remember to use for context.
• Keywords for knowledge base query: Search through the documentation with the keywords of the question or the full question. ToothFairyAI recommends to use keywords for larger documents sets and full question for smaller documents sets.
• Max keywords (1-10): Max number of keywords to use for the knowledge base query.
• TopK (1-20): Max number of results to return from the knowledge base query.
• Doc TopK (1-20): Max number of documents to extract from the initial knowledge base query by document id. Use this setting to reduce document bias in case especially of conflicting information.
• Min retrieval accuracy: Minimum confidence level for the knowledge base query to return results.
• Recency importance (0-5): How important is the recency of the document in the knowledge base query. This setting is used to give more weight to the most recent documents.
• Keywords importance (0-5): How important are the keywords in the knowledge base query. This setting is used to give more weight to the documents that contain the keywords.

ToothFairyAI's AI Inner Knowledge validation

ToothFairyAI's validation system, will only allow answers from AI Inner Knowledge that are verified. This feature is not how the other Aprovided bys operate. Other systems will use their own reasoning model which does not have these strict fact checking measures in place like what ToothFairyAI does.

The AI Inner Knowledge answers generated by ToothFairyAI are checked by an algorithmic mathematical logic pattern. This pattern references answers provided by what knowledge / documentation is available for the questions asked to it. This is done by converting question texts into 1024 dimension metrics which are then used to compared against an agents selected topics.

The recommended percentage of confidence level used by ToothFairyAI for providing an answer is 60%. This ensures that the output from an agent is accurate to the users questions. However the user can set the minimum retrieval accuracy to a higher percentage if they require a higher level of confidence in the answers provided by the agent.

Functions settings

Functions allow agents to consume APIs and DBs as a data source. Moreover, depending on the type, the function can provide suggestions or generate static responses for greater controllability of the agent's responses including via referencing entire web pages within the chat message. Functions are automatically enabled for retriever agents while for casual and coder agents, the functionality is not available. Functions can be created in the Functions section of the Settings page however to take effect they need to be associated to the agent. Chat and HTML functions, when invoked, override API, DB functions and Knowledge hub documentation. See Settings > Functions for more details.

Functions context

Functions context allows the agent to inject additional information while calling functions and tools using the customer, case and the overall conversation data. Customer and Case data will be present only if the necessary customer and case info is present at the time of calling the function while the chat data will always be present regardless of when the function gets called.

This setting allows the agent to complete a function call using data that is not available in the conversation and/or in the most recent messages.

Three types of context are available

• Customer: The entire JSON object containing the customer information
• Case: The entire JSON object containing the case information
• Chat: Chat metadata such as summary, phone number / email associated to channel, agent id, chat id etc.

Planner tools

Only available for Planner agents.

The section comprises the following fields:

• Available agents for planning: The agents that will be used to execute the tasks. If none are selected, ToothFairyAI will prevent you from saving the agent unless Dynamic agents generation is enabled.
• Planning instructions: Additional custom instructions that the planner can use to better understand the domain in which tasks will be executed and how to plan accordingly.
• Review instructions: Additional custom instructions that the planner can use to better understand how to review the plan and how to correct it if necessary.
• Max execution steps: The maximum amount of steps the planner can execute.
• Max re-attempts: The maximum amount of times the planner can re-attempt a failed task. This applies to all agents including coder agents with code execution enabled
• Approval before execution: When enabled, the planner will require approval before executing a task. This is useful when you want to review the plan before it is executed or when you want to review the execution of a step which requires approval. By default the field set to off
• Allow plan adjustment: When enabled, the planner will be able to adjust the plan if it detects that a task is not executable. This is useful when you want to allow the planner to correct the plan if it detects that a task is not executable. By default the field set to off
• Dynamic agents generation: When enabled, the planner will be able to generate agents on the fly if it detects that a task requires a specific agent to be executed not alredy available in the Available agents for planning field.
• Email on approval: When enabled, the planner will send an email to the user when a task requires approval. This is useful when you want to notify the user that a task requires approval. By default the field set to off
• Email on completion: When enabled, the planner will send an email to the user when a task is completed. This is useful when you want to notify the user that a task is completed. By default the field set to off
• Email on failure: When enabled, the planner will send an email to the user when a task fails. This is useful when you want to notify the user that a task has failed. By default the field set to off

Max runtime

Planner agents can execute autonomously for no longer than 60 minutes as per ToothFairyAI security policies. If your organisation requires more time for the executions, please contact us.

Models availability

Planner agents for now can only use mystica , sorcerer 1.5 and Llama 3.1 Nemotron models due to the complex reasoning required. Our team plans to release more models in the future.

Planning capabilities

Planners cannot perform the following operations:

• Complex powerpoints creation including custom styles and templates.
• PDF conversion to other file types
• Images extraction from docx, ppt and pdf files.
• Creation of videos, animations and 3rd graphics.
• Audio files creation

Lastly, documents can be only processed by Sorcerer and Mystica models, therefore 3rd party models will not be able to process documents and the plan execution might fail if the agent executing the step is configured to use a 3rd party model.

Execution environments and code upload settings

Only available for Coder agents with code execution enabled

Code environments allow agents to leverage predefined docker images and code snippets to execute code. Once assigned to an agent, ToothFairyAI agents will automatically detect and use the most suitable environment for the task at hand.

Code upload allows agents to receive up to 10 source code files at the time. To review which file types are supported see here

Internet search

Only available for agents of type Retriever.

• Allow internet search: When enabled, the agent will be able to search the internet for answers.
• Max search results: The maximum number of search results to return from the internet search for any given mode - therefore if you have both search and news enabled and the max search results parameter is set to 10, the agent will return a maximum of 20 results. This is done to prevent an arbitrary result prioritisation from our side besides the standard SEO ranking.
• Search location: The location to use for the internet search. This will be used to return results more relevant to the selected location.
• Search mode: The search mode to use for the internet search. The available options are search, news, videos, images and shopping. All options can be combined however if this field is left empty the agent will default to search mode only.
• Excluded domains from search: Domains to exclude from the internet search separated by a comma.
• Allow deep search: When enabled, the agent will be able to search the internet for answers in depth while when disabled the agent will only search the summaries of the results. This feature is only available for search and news mode while for the other results no deep search will be conducted. Regardless of how many results are returned, the agent will only be able to perform a deep search on the top three results.

info

When a Retriever agent has only internet search enabled while having no functions associated and no knowledge hub topics connected, the agent will default to searching the internet for answers. The exact search queries used to retrieve the results are shared along with the actual websites in the details section of the answer to provide maximum transparency.

info

The internet search leverages an internal reranking model to return the most relevant results based on the user query and the max search results parameter. The reranking model runs after the webpages have been retrieved from the internet search engine, therefore the list of websites provided are not all considered during the answer generation process.

Agent Instructions

The Agent Character are the input fields about the agent purpose, goals and what it should and should not talk about.

• Agent role and instructions: This is to set the purpose of the agent. Any additional istruction for the agent should be added here.
• Agent tooling guidelines: This is to set the tooling guidelines for the agent. This is useful when you want to force the agent to use a specific tool or when you want to prevent the agent from using a specific tool.
• Default answer: If a response cannot be found, this will be the response that is provided.
• Goals: This is how the agent will fullfil its role.
• Inhibition passage: Subjects that you do not want the agent to talk about.
• Pertinance passage: Subjects to force the agent to focus on.

Agent feedback

When the agent receives feedback from any of the available chats the feedback will be displayed in this section. For performance reasons, only the most recent 10 feedback will be displayed. The context of the conversation and the user feedback is automatically included in the instructions of the agent.

Agent tools

• Summarised memory: When enabled, the agent will only reference a summary of the conversation rather than the whole set of messages.
• Long term memory: When eanbled, the agent will be able to reference information from previous conversations. Not all conversations are used for the long term memory. Only conversations where the user hints the agent to remember or learn from the conversation will be used.
• Multilingual: Enables the multi-language capability for the agent to reply in over 120 languages, however this can impact performance so turn on only if necessary.
• Generate charts: If requested the agent can generate charts, tables and mind maps. This feature is available only when using Sorcerer and Mystica model families. When possible, ToothFairyAI will also generate a downloadable .png file for each chart and graph generated in the message. Below the list of charts and graphs that can be generated:

  - Bar chart
  - Line chart
  - Pie chart
  - Flow chart
  - Mindmap 
  - UML chart
  - Kanban
  - Gantt chart
  - Architecture diagram
  - Quandrant chart (also known as Gartner Magic Quadrant)
• Enable images upload: This setting will allow the user to upload images to the agent. - this feature is available only in retriever and casual mode and only one image can be uploaded for each message. The images must be in .png, .jpg or .jpeg format.
• Enable audio upload: This setting will allow the user to upload audio files to the agent. - this feature is available only in retriever and casual mode and only one audio file can be uploaded for each message. The audio files must be in .wav or .mp4 format.
• Enable video upload: This setting will allow the user to upload video files to the agent. - this feature is available only in retriever and casual mode and only one video file can be uploaded for each message. The video files must be in .mp4 format.
• Enable audio generation: This setting allows agents output to be converted to speech. When the feature is enabled, a small speaker icon will appear next to the copy to clipboard icon. Clicking on the icon will play the audio. Depending on the internet connection, the audio may take a few seconds to load. To guarantee good output quality, ToothFairyAI agents preprocess the text in the message to ensure the audio is generated correctly and it is listeners friendly therefore the text displayed in some cases might not fully match the audio version. This feature is not available for Starter subscription users.
• Enable speech to speech: This setting allows agents to convert speech to text and text to speech dynamically. The user will always need to press the microphone button to start the conversation while the response will be converted to speech automatically. This feature is only for Enterprise subscription users.
• Voice selection: Choose the voice for audio generation, with female British English as the default; you can upload a custom MP3 or WAV file (under 10MB) to personalize your agent's voice. This feature effectively enables deep voice cloning and should be used responsibly and with proper consent. The user can also record 30 seconds of audio to use as the voice for the agent. Currently we natively support only English for Starter and Pro subscriptions while we support the following languages for Enterprises: French, German, Spanish, Italian, Portuguese, Czech, Polish, Russian, Dutch, Turksih, Arabic, Mandarin Chinese
• Enable docs upload: This setting will allow the user to upload documents to the agent. - this feature is available only in casual mode with up to 5 files uploaded at each turn. The docs must be of one of the following formats: .pdf, .csv, .doc, .docx, .xls, .xlsx, .html, .txt, .md .json, .pptx, .ppt
• Enhance answers with NER: This setting will allow the agent to used named entities in the response. - this feature is available only in retriever mode.
• Retrieve images from docs: This setting will allow the agent to retrieve images and display them in the response. - this feature is available only in retriever mode. This feature is available only when using Sorcerer and Mystica model families.

Tools availability for planner agents

The following tools are not available for Planner agents:

• Short term memory
• Multilingual
• Enhance ansers with NER
• Retrieve images from docs

Planner agents must allow for image, docs, video and audio uploads to be enabled in case the user wants to start a plan from one or more files.

Images generation

Only available for agents with casual mode

• Allow images generation: This setting allows the agents to generate new images based on the user input. ToothFairyAI agents generate by default four images at each interaction allowing users to further refine the output.

• Image generation model: This setting allows the user to select the model to be used for image generation.

• Mystica SD is the default model for general purpose image creation;

• Mystica SD realism is a fine-tuned image generation model for photographic generation tasks.

• Flux Pro is an image generation model provided by Black Forest Labs for commercial use cases (not available for Starter and Pro plan).

• Flux Pro 1.1 is the latest image generation model provided by Black Forest Labs for commercial use cases (not available for Starter and Pro plan).

• Flux Pro 1.1 Ultra Realistic is the latest image generation model provided by Black Forest Labs for commercial use cases finetuned for ultra realistic images (not available for Starter and Pro plan).

• Max images generated: This setting allows the user to select the number of images to be generated at each interaction. The default value is 4. When the agent is invoked by the planner agent by default the image generation will produce only one at each step

Flux Pro

When a Flux Pro model is selected ToothFairyAI defaults the Max images generated to 1

Planner

When a Planner agent invokes a Casual agent with Image generation enabled, only one image will be generated at any given step.

Videos & 3D model generation (Enterprise only)

Only available for agents with casual mode

• Allow videos generation: This setting allows the agents to generate new videos based on the user input. ToothFairyAI agents generate by default generate two videos - one with audio and one without. The user can also pass the initial frame of the video to generate by uploading an image file inside the chat. If multiple images are passed only the first one will be used for the start of the video.
• Video generation model: This setting allows the user to select the model to be used for image generation.
• Mystica SD vdeo is the default model for general purpose video creation;
• Allow 3D model generation: This setting allows the agents to generate new 3D models based on the user input. ToothFairyAI agents with this must receive an image inside the chat to generate the 3D model. If multiple images are passed only the first one will be used for the 3D model generation. The agent cannot generate a 3D model based on the prompt alone.

Planner

When a Planner agent invokes a Casual agent with Video generation enabled, only one video will be generated at any given step.

Agent channels

Only available for agents with Channels populated

note

This section allows agents to be connected to one or more communication channels. The agent can be connected to as many channels as you want however each chat can be connected to only one channel at a time.

• Assign custom channels: This is the list of channels the agent can be connected to based on the channels created in the workspace. The user can add or remove channels from the list. Based on the channels selected the user will be able to input the sender phone associated to each channel type
• Phone number: This is the phone number that will be used to send SMS messages to the user. This is required if the agent is connected to a SMS channel.
• Whatsapp number: This is the phone number that will be used to send Whatsapp messages to the user. This is required if the agent is connected to a Whatsapp channel.
• Email address: This is the email address that will be used to send emails to the user. This is required if the agent is connected to an Email channel. (available only for Enterprise)
• Restrict allowed email addresses: This option allows the user to restrict the emails that can be used to interact with the agent. This is useful if the agent is connected to an Email channel and you want to restrict the emails that can be used to interact with the agent. The user can add or remove emails from the list. This is optional.
• Enable agent email : This is the easiest option for the user to interact with any given agent using emails as ToothFairyAI provides a virtual inbox the user can send emails to providing files, images and text to interact with the agent. The agent will respond to the user with the generated response via email providing also a link to the conversation in the ToothFairyAI platform at the bottom of the response.
• Delivery delay: If the agent is connected to a SMS, Whatsapp or Email channel the user will receive the message after the delay. This is useful to simulate a human response. The delay can be between 0 and 120 seconds.

Moderation and Feedback

• Allow feedback: This option allows the user to thumb up or down a response to show if the generated response is correct or not.
• Content moderation: When enabled, the agent will filter out any profanity from the response by responding back to the user with the moderation rule applied when necessary.
• Moderation rules: The list of rules the agent will use to moderate the content. The user can add or remove rules from the list.

note

The moderation of the user message is performed via a combination of a moderation model fine-tuned by the ToothFairyAI team and the base model associated to the agent

Advanced Settings

• Temperature (0.01-1): Determines how creative the response will be, higher meaning most creative. Reduce this number to the lowest value to reduce hallucinations

• Max output tokens (50-4096): Determines the number of tokes (characters set) in the response. The larger the number the longer and more detailed the responsed can be. If the number is set too low some answers might get truncated.

• Prompt enhancement: When enabled, it enhances the user prompt behind the scenes to improve the quality of the generated response. This feature might add a slight delay to the response therefore it is not recommended for public facing agents.

• Show code blocks: When set to false it suppresses the display of any code snippet. This feature is available only for Coder and Planner agents. By default this option is set to true.

• Plain text output: Removes any form of styling from the responses.

• Show response time: Displays the time from the first word to the end of the last word in the response.

• Show detected language: This setting will display the language that what used in the response.

• Restrict access: When enabled, the agent will only be accessible to the users specified in the user access settings and the admins of the workspace. When this mode is enabled the agent will not be accessible to the public through the web-widget and all chats will be private by default.

Multimodal

When Multimodal is enabled, the agent will be able to generate responses with images, charts, tables and mind maps. This feature is available only when using Sorcerer and Mystica model families.

Appearance and Web Widget

Not available for Planner agents

• Splash Logo: A logo you would like displayed in the widget.
• First chat message: The initial message displayed when the chat is started.
• Show agent name: When ticked, the agent name is displayed in the chat message. This is very useful when agent hand-off is enabled.
• Show splash message: When ticked, the first chat message is displayed as splash message in the middle of the chat.
• Input placeholder: Text displayed in the user input field.
• Loading placeholder: Text displayed when the agent is retrieving the answer.
• Disclaimer: A statement that explains rules, limits, or warnings.
• Icons colour in light mode: The global icon colour in light mode. This setting will be applied also in the side menu of Chat agent.
• Icons colour in dark mode: The global icon colour in dark mode. This setting will be applied also in the side menu of Chat agent.
• Splash background colour in light mode: The background colour of the top bar where the logo is displayed in light mode.
• Splash background colour in dark mode: The background colour of the top bar where the logo is displayed in dark mode.
• Favicon: In our latest release, the favicon will now automatically match the icon assigned to the agent. This way the agent setup is streamlined and more predictable for public facing use cases.
• Initial questions: A list of questions that the agent can answer when the chat is started. The user can create up to three questions separated by a semicolon.
• Go to widget: The url of the page where the user can directly interact with the agent. Available only when the agent is Block external usage is ticked off.
• Block external usage: Blocks the widget from public access.

Widget

Chat agents by default are provided with both the public url of the agent if enabled and the code to embed the agent into any web-page as a chat widget.

Icon

When you provide a custom icon to the agent it will be displayed in the side menu of Chat agent and in the web widget.

Hosting and Models

• LLM provider: The provider used to host the agent. Currently only ToothFairyAI is available as a provider for Starter and Pro plans. For Enterprise plans, the user can choose between ToothFairyAI and their own hosting.

• Base model: The model used by the agent to provide answers and generate outputs for the AI agents.

  • Sorcerer: The default model used by any agent fine-tuned by ToothFairyAI.
  • Sorcerer 1.5 (experimental): A quantum leap in the capabilities of the Sorcerer model, offering enhanced performance and accuracy. Request access to this model from the support team.
  • Mystica A larger model more powerful and more accurate however slower fine-tuned by ToothFairyAI.
  • Mistral 7b - FP16: A small but very capable model based created by Mistral team.
  • Qwen 2.5 7b - FP16: The smallest version of the latest open-sourced models provided by the Qwen team.
  • Qwen 2.5 Coder 32b - FP8: The latest open-sourced model designed for programming tasks provided by the Qwen team.
  • Mistral 8x7b - FP16: A larger model provided by Mistral team.
  • Llama 3 8b: The legacy model created by Meta AI in its smallest version. (dismissed on 15/08/2024)
  • Llama 3.1 8b - FP8: The latest model created by Meta AI in its smallest version and larger context window.
  • Gemma 2 9b - FP16: The latest open source model provided by Google.
  • Gemma 2 27b - FP16: The largest open source model provided by Google.
  • Llama 3 70b: The legacy model created by Meta AI in its larger version. (dismissed on 15/08/2024)
  • Mistral 8x22b - FP16: The largest open-sourced model provided by Mistral team.
  • Qwen 2 72b - FP16: The largest open-sourced model provided by the Qwen team.
  • Mistral Large: A larger model closed-sourced provided by Mistral team (not available for Starter plan).
  • Mistral Large 2: The largest and most capable closed-sourced model provided by Mistral team (not available for Starter plan).
  • Llama 3.1 70b - FP8: The legacy model created by Meta AI in its largest version and larger context window.
  • Llama 3.1 Nemotron 70b - FP16 : The latest fine-tuned model created by Nvidia using 70b Llama 3.1 model as base model.
  • Llama 3.1 405b - FP8: The largest open-sourced and state of the art model provided by Meta (not available for Starter plan).
  • Llama 3.3 70b - FP8: The latest open-sourced model created by Meta and larger context window
  • QwQ 32B (experimental) - FP16: The latest open-sourced model created by Qwen team that enables deep reasoning
    FP16 meaning

    The FP16 models are optimized for speed and lower memory usage with 16-bit floating point precision. The FP8 models are optimized for faster speed and lower memory usage with 8-bit floating point precision.

• Functions provider: The provider used to host the functions. Currently only ToothFairyAI is available as a provider for Starter and Pro plans. For Enterprise plans, the user can choose between ToothFairyAI and their own hosting.

• Functions model: The model used by ToothFairyAI to select the functions to be used by the agent.

  • Sorcerer: The default model used by any agent fine-tuned by ToothFairyAI.
  • Sorcerer 1.5 (experimental): A quantum leap in the capabilities of the Sorcerer model, offering enhanced performance and accuracy. Request access to this model from the support team.
  • Mystica A larger model more powerful and more accurate however slower.
  • Llama 3.1 8b - FP8: The latest model created by Meta AI in its smallest version and larger context window.
  • Llama 3.1 70b - FP8: The legacy model created by Meta AI in its largest version and larger context window.
  • Llama 3.1 405b - FP8: The largest open-sourced and state of the art model provided by Meta.
  • Llama 3.1 Nemotron 70b - FP16 : The latest fine-tuned model created by Nvidia using 70b Llama 3.1 model as base model.
  • Llama 3.3 70b - FP8: The latest open-sourced model created by Meta and larger context window

The models list can be expanded by 3rd party providers. Contact us for more information.

Local hosting

For Enterprise plans, the user can choose to host the agent and functions locally on their own servers or on a cloud provider of their choice. Contact us for more information.