Rovo Agent missreading prompt

[Ugnius Aušra]

Hello,

I have Agent, that has scenario:

You are a Customer Support assistant. Your main goal is to help users identify the most appropriate affected service from a predefined list, based on the summary and description of a Jira issue.

Instructions:
- Given a Jira issue's summary and description, select the most appropriate affected service from the provided list.
- Return only the exact service name, with no additional text or explanation.
- If the issue does not clearly match any service, still select the closest relevant one and return its name.
- Maintain a concise, professional, and neutral tone in your responses.

And created automation rule, where prompt has text:

You are a Customer Support assistant. Your main goal is to help users identify the most appropriate affected service from a predefined list, based on the summary and description of a Jira issue.

Instructions:
- Given a Jira issue's summary and description, select the most appropriate affected service from the provided list.
- Return only the exact service name, with no additional text or explanation.
- If the issue does not clearly match any service, still select the closest relevant one and return its name.
- Maintain a concise, professional, and neutral tone in your responses.

ERP System Development
Description:
ERP System Development refers to all issues related to the Infotrans system.
Infotrans is a comprehensive platform where:

Managers can create orders for clients.

Transport managers coordinate trucks and manage orders.

Users can view a map showing truck positions, routes, and which driver is currently assigned to each truck.

The system includes modules for CRM, bookkeeping, invoicing, claims, pallets, and HR.

This service covers any problems, questions, or requests related to these Infotrans modules and functionalities.

Automation rule looks like this:

I tried to test it, created two requests:

"I need thermoprint"

"Issues with VPN"

Both cases agent returned response, that this is ERP service. 

So where i'm making mistake? How should i improve?

Thank you

[Charlie Misonne · Accepted solution]

Hi @Ugnius Aušra 

I think you created the agent correctly. But you are reentering the agent's definition in the prompt on the automation rule which is not how this is supposed to work.

The prompt in the automation rule is meant to be used just like the Rovo chat window: you select an agent, add a chat message, and get a reply.

Same in automation: you choose an agent, add a prompt (your question) and can process the agent's reply with {{agentresponse}}.

So reformulate your question in the prompt and use {{issue.summary}} to include the summary of the item that triggered the rule.

I hope this helps!

[Ugnius Aušra]

I changed prompt to:

- If the issue does not clearly match any service, do not return any service name.

And it started to not return response every time.

/--

Okay, so you saying, that if i create instructions for scenario and in automation rule prompt, i should not copy same text? I'm little bit confused. 

[Ugnius Aušra]

So i guess prompt in automation rule should look like that:

Given the following Jira issue, what is the most appropriate affected service?
Summary: {{issue.summary}}
Description: {{issue.description}}

[Charlie Misonne]

So you created a "Customer Support Selector" agent in your Jira instance telling it what it is capable of, how it should reply etc.
This agent now exists: you can use it in Rovo chat and in automation rules.

There is no need to redefine the agent in the automation rule. All you need to do is select the agent (which you did) and ask it a question.

Have you tried using your agent in the Rovo chat to see how it replies?

Give it an issue key see if it replies correctly.

Then in the automation rule also specify the key of the work item to be used.
Something like: "What is the affected service for {{issue.key}}"
Since you told the agent to look at an item's summary and description it should be able get that info when you provide the key.

Another question: what is this part about? Are you defining the affected services here? And I only see 1 which is ERP? Where is VPN?

ERP System Development
Description:
ERP System Development refers to all issues related to the Infotrans system.
Infotrans is a comprehensive platform where:

Managers can create orders for clients.

Transport managers coordinate trucks and manage orders.

Users can view a map showing truck positions, routes, and which driver is currently assigned to each truck.

The system includes modules for CRM, bookkeeping, invoicing, claims, pallets, and HR.

This service covers any problems, questions, or requests related to these Infotrans modules and functionalities.

[Charlie Misonne]

So i guess prompt in automation rule should look like that:

Given the following Jira issue, what is the most appropriate affected service?
Summary: {{issue.summary}}
Description: {{issue.description}}

Yes indeed! I think you could also just use {{issue.key}} instead since you told the agent to look at the summary and description of an item.

[Ugnius Aušra]

Oh that makes more sense. :D 

Another question: what is this part about? Are you defining the affected services here? And I only see 1 which is ERP? Where is VPN? 

Yes, in scenario instructions i'm defining ERP service description. Would you recommend to do in different way? I though about to create full page in confluence, fill information about this service, and in scenario change that Agent should look at that page.

Is it good choice?

[Charlie Misonne]

Doing it in Confluence sounds like a good idea.

You can include the URL to the page in the agent's instructions.