Define tests

You can define tests using the following steps:

1. Log in to the Webex AI Agent Studio platform.
2. On the Dashboard, click the scripted AI agent that you've created.
3. Click Testing in the left pane. By default, the Testcases tab appears.
4. Select a test case and click Execute selected tests.

Each row in the table represents a test case having the following parameters:

Execute tests

On the Execution tab, click Execute selected tests to initiate a sequential execution of all selected test cases.

To view test cases with specific outcomes, click the desired result (for example, Passed, Passed with partial match, Failed, Pending) in the summary ribbon. This filters the test case list to display only those matching the selected result.

The session ID associated with each test case is displayed in the results. This allows you to quickly cross-reference test cases and view transaction details. To perform this, choose the Transaction Details option in the Actions column.

Execution history

On the History tab, access all executed test cases.

• Click the Download icon from the Actions column to export the executed test data as a CSV file for offline analysis or reporting.
• Review the specific engine and algorithm settings used for each test case execution. This information helps developers optimize the AI agent's performance.
• To view the advanced algorithm configuration settings used for a particular training engine, click the Info icon next to the training engine name. This provides insights into the parameters and settings that influenced the AI agent's behavior during testing.

Session details of a particular session in the Scripted AI Agent for performing actions

The Transaction Info tab in the Scripted AI Agent for performing actions provides a detailed breakdown of a specific interaction, categorizing information into four sections:

Intents Identified section:

• Displays the intents identified for the customer's query.
• Indicates the confidence level associated with each identified intent.
• Lists the slots that are associated with the identified intent. Click the slot to view additional information about its value and how the system extracts it from the user's query.

Entities Identified section lists the entities that the system extracts from the customer's message and associates it with the active consumer intent. These entities represent the key pieces of information that the AI Agent identified within the user's query.

The Algorithm Results section provides insights into the underlying processes that led to the AI Agent's response. Here's a breakdown of the information displayed:

• List of Intents—Shows the identified intents and their corresponding similarity scores.
• Entity List—Displays the entities that were extracted from the user's message.

The Other Info displays:

1. Processed Query—Indicates the preprocessed version of the customer's input after the AI Agent's natural language understanding (NLU) pipeline processes it.
2. Language detection provider—The provider who offers technology that can automatically identify the language of a given text.
3. Language detected—The language detected by technology. 
4. Agent Handover—Indicates whether an agent handover occurred during the session. Check the Agent Handover by Rules check box if an agent handover was triggered by specific rules.
5. Template Key—Indicates the template key associated with the intent that triggered the AI agent's response.
6. Response Type—Indicates the type of response generated by the AI agent, such as a code snippet or a conditional response.
7. Response Condition—Indicates the specific condition or rule that triggered the AI agent's response.
8. NLU AI Engine—Identifies the NLU AI engine used to process the customer's query (for example, RASA, Switchmatch, or Mindmeld).
9. Vector model—A way of representing text as numerical vectors. 
10. Min threshold Scores—Displays the minimum threshold score.
11. Partial match score difference—The partial match score difference configured in the Handover and Inference settings. The system determines whether a query is out of scope or requires agent intervention based on these values.
12. Debug Logs—Provides a list of debug logs associated with the specific transaction ID. Advanced logs are typically retained for 180 days.

The Metadata tab displays:

• NLP Metadata—Review the preprocessing steps applied to the customer's input in the NLP tab.
• Datastore and FinalDF—Access data related to the session in the Datastore and FinalDF tabs for AI Agents.
• Search Functionality—Use the built-in search bar to find specific utterances within a conversation.

Change Logs

The Change Logs section provides a detailed record of modifications made to your Scripted AI Agent within the past 35 days. To access Change Logs:

1. Navigate to the Dashboard and click the AI agent that you've created.
2. Click the History tab to view the AI Agent's history.
3. Click the Change Logs tab to see a detailed log of changes:
   • Updated At—The date and time the system made the change.
   • Updated By—The user who made the change.
   • Field—The section of the bot where the modification occurred (for example, Settings, Intents, and Responses).
   • Description—Additional details about the change.

Resolve issues

The Issues tab allows you to review and address messages flagged for curation. You can do the following:

• Choose to resolve or ignore issues based on their severity and relevance.
• Examine the original user utterance, the AI Agent's response, and any attached media.

To resolve an issue, you can:

• Link to an existing intent—To connect an issue to an existing intent, select the Link option and search for the desired intent.

• Create new intent—Use the Add to a New Intent option to create a new intent directly from the Curation Console.

• Ignore issues—Resolve or ignore issues to remove them from the Curation Console.

1. You can't link to default intents (welcome message, fallback message, partial match).
2. For a scripted AI agent for performing actions, select the appropriate intent from the drop-down list and tag any relevant entities.
3. After making changes, retrain your AI Agent to ensure that it reflects the new knowledge in its responses.
4. Resolve or ignore multiple issues simultaneously for efficient management.

The Resolved tab displays all issues addressed by the system. You can view a summary of each resolved issue, including whether we linked it to an existing intent, created a new intent, or ignored it. If you see responses that you don't like that the system didn't catch, you can manually add specific examples to the Curation Console.

To add issues from sessions:

1. Identify the Utterance—Locate the utterance that triggered the incorrect response.
2. Check Curation Status—If the issue isn't already in the Curation Console, the system displays the Curation Status toggle.
3. Toggle the Flag—Enable the Curation Status toggle to add the utterance to the Curation Console for review and resolution.