My experience building Rovo agents

[#1]

I recently took an amazing course on building custom GPTs (shout out to Hilary Gridley!), and it really got the creative juices flowing for things I could build using Rovo. So I got experimenting, and ended up building a few (no-code of course). Some worked well, others, not so much. Here’s a recap of my experience.

A couple overall tips/take-away lessons

• I don’t use the Rovo agent wizard. I find it summarizes too much for the prompt and removes key details

• Instead I use the Rovo chat (or sometimes event ChatGPT) to generate my prompt, then copy & paste it in to the manual configuration area of Rovo agents

• I always start building my agents/GPTs by defining criteria for what is a good answer/output is.

• Asking “what could make this prompt even more effective” is an easy way to add some quick improvements.

• Rovo Chat or ChatGPT can also help you generate some sample input and output examples to include in the prompt.

• Ask a question - get an answer type Rovo agents seem to work the best. Trying to build a more interactive experience (like talking to a coach) worked less well.

Internal Sales Enablement agent

The problem: We have a long standing problem of employees struggling to find information about the products we make in-house. There are lots of products; battlecards, competitor comparison pages, discount policies are scattered across multiple different spaces, and people don’t know where to find the information they need.

The solution: To give us a leg up, we started by creating a specific space and asked product marketing managers to create battlecards in this space. This gave us a good ‘clean’ source of information. But of course not everything our account managers or consultants need was included in a battlecard.

So I created a Rovo agent that is limited to specific spaces where I know the quality of information is high. In order to create my prompt, I used Rovo chat. This was my first message:

My job is to create a Rovo agent that will help colleagues more easily find information about Confluence and Jira apps. I want this to be a Rovo agnet where the team memmber can submit some questions about an app and receive a helpful output. Your job is to help me make this agent. But I want to be thoughtful and deliberate in our approach. First, help me define what success could look like for a tool like this.

Rovo did a decent job coming up with success criteria, but some of the ideas were not what I wanted, so I gave feedback until I was happy with the list of criteria for good output.

Then I gave some examples of input and asked the Rovo agent to give ideas for output. The result was a good match, so I moved on to asking for a prompt in markdown in a code block.

I asked Rovo chat if there was anything that would make this prompt even more effective, and it gave me a revised prompt.

I went away to test, and realized the Rovo agent wasn’t linking to any of the source material, so I asked Rovo chat to add to the prompt a requirement that the Rovo agent always cite its sources.

Result: It’s pretty good. One colleague said data in macros can sometimes be hard to surface, but so far so good. The improvements we’ve identified have been about missing data/resources, not the way the Rovo agent is performing.

Score: 4.5/5 

Playbook finder

The problem: We have lots of internal procedure playbooks, and not everyone remembers which ones we have and what exactly they’re called.

The solution: Set up a Rovo agent that only looks at the child pages of our playbook parent page to help identify the right Confluence page. I also added an action to create a Jira issue to request a new playbook if the topic is missing.

I started off by working on the success criteria and created a prompt in Rovo chat I could copy into my Rovo agent.

When testing with ideas I knew we did not have as playbooks, the Rovo agent kept trying to suggest playbooks or kept repeating itself. I modified the prompt to make sure it directed the agent to say there was no playbook that existed and to offer to create a Jira issue in a specific project to create the missing playbook.

Here is what my final prompt included:

#**Rovo Agent Prompt: Find Sales & Marketing Playbooks**

You are an expert assistant for product marketing managers and marketing team members. Your job is to help users quickly find relevant sales and marketing playbooks in Confluence that support their work.

##*Success Criteria*:

Ensure comprehensive coverage: Help users find playbooks for all core marketing and sales processes, as identified by team input.

Prioritize usability: Make it easy for users to find, access, and understand playbooks, regardless of their experience level.

Facilitate knowledge sharing: Guide users to playbooks that share best practices and help teams avoid repeating mistakes.

Support quality & standards: Direct users to playbooks that help maintain standards and quality across marketing activities.

Accelerate onboarding: Make it easy for new team members to discover and use relevant playbooks to upskill quickly.

Identify knowledge gaps: Make it easy for missing playbook topics to be identified.

##*Instructions*:

When a user asks for a playbook, search the Sales & Marketing Playbooks directory and related Confluence spaces.

Provide direct links to the most relevant playbooks, along with a brief description of each.

If a user is unsure what they need, ask clarifying questions about their marketing or sales process, channel, or objective.

Highlight playbooks that are especially useful for onboarding or for sharing best practices.

If no matching playbook is found, ask the user if they can describe their need using different words, synonyms, or alternative phrasing to ensure the issue isn’t due to vocabulary or phrasing differences.

If, after clarifying, there is still no relevant playbook, clearly inform the user that there is no playbook available for their request.

If a user indicates a suggested playbook is not a good match, DO NOT suggest it again.

NEVER repeat the same response twice.

Ensure your responses are clear, concise, and actionable.

**If no playbook is found, create an issue in https://xxx.atlassian.net/jira/core/projects/ACASM to create new playbook for the topic**

Results: So far the Rovo agent seems too eager to suggest playbooks and avoids replying that there isn’t one for the topic requested. It was also repeating the same replies, so I tried updating the prompt to instruct it to never reply the same thing twice, but it didn’t work. Maybe I can tweak the prompt with more examples of sample input and output, but for now it gets off topic too easily by suggesting playbooks too far removed from my sample queries. The few times it has understood there is no playbook, the Jira issue it offers to make has a clear summary and description, and it’s picking the right project (I didn’t try configuring more fields).

Score: 3/5

Product launch coach

The problem: we have a very long and elaborate Confluence page laying out best practices for marketing campaigns for product or feature launches. But it’s not very practical for someone looking to get some quick feedback on the feature launch campaign they’re working on.

The solution: Full disclosure, I built this first in ChatGPT and then copied the prompt over to Rovo. But I used the same process: explaining what I wanted, providing reference materials (giving the launch campaign Confluence page as knowledge) to guide the criteria for good output, and then building the prompt.

Results: After some testing, I realized it needed to go step by step, asking the person a question or two, waiting for the input, and then moving along providing feedback based on the reference Confluence page. I was able to get this to work in ChatGPT, but with the Rovo agent, I could never get it to wait for input. It would spit out a bunch of recommendations on what to do for the marketing campaign in a huge, undigestible block.

Score: 1/5

How I share the Rovo agents

I would love for my Rovo agents to be suggested in specific contexts, but I haven’t yet figured out how to set that up.

So for the moment I am sharing the agents by link in meetings, in chat rooms, and individually. I have also added the agents as links on Confluence pages where I think people are looking for information the agent could help them with.