🌱 How a non-dev created a useful Rovo agent

[#1]

Welcome back to our series! We're so glad you're here. In today's post, I’ll detail how and why my teammate helped me create an AI agent.

The scene: I’m the editor for the Atlassian Learning team, and I started using Rovo Search and Rovo Chat for daily support with hunting down resources, checking definitions, verifying how our organization talks about our apps, and more, as soon as Rovo launched. But I was a little more hesitant about Rovo agents.

Building an agent felt intimidating, and I wasn’t sure they could do much for me since I’m not a developer. But I saw my team saving time and energy with agents, and I didn’t want to get left behind.

When our team’s Rovo champion offered to help, I decided to give it a try.

What a customized AI buddy can do for a non-developer

One of my favorite things to do is share all the great work our team does, like how they’re using Rovo Deep Research and Jira, which means I interview a lot of people. I use Loom to record everything, and Loom’s AI automatically creates a transcript, both of which save me a lot of time and energy.

But after each interview, I had to sift through twenty-plus single-spaced pages to pull out themes and key points. I spent hours hunting through transcripts and double-checking so that I didn’t miss anything important.

When Rovo agents first launched, our team’s Rovo champion, Michelle Cacciapaglia, shared lots of different Rovo agent use cases with our team, including writing challenging knowledge check questions for the end of each lesson and creating realistic scenarios and examples within lessons. Agents, Michelle said, are best for structured, repetitive, text-based tasks and custom, specialized tasks that don’t require deep expertise or nuanced human judgment. “If I need to do something complex, or something that has really specific instructions, and it's something that I'm going to do over and over again, I’d choose an agent over Rovo Chat,” she said. "Agents do really well with multiple steps."

I asked Michelle if she thought an agent could help me organize my interview transcripts, and she started coming up with ideas right away.

How we decided what kind of Rovo agent to build

First, Michelle prompted me to name my least favorite tasks: cleaning up a transcript to make it clear who said what, removing the chit chat, and finding every time my interviewee and I talked about a main theme.

Then Michelle asked me what else I’d want, and I came up with a wish list: main themes automatically pulled out for me based on the post’s story and timestamps attached to everything so I could verify and re-listen as needed. I also didn’t want Rovo to pull in any other resources.

First, we searched the existing Rovo agent library to see if what I wanted already existed, but we didn’t find a match.

So Michelle began walking me through the 4-step process she uses to build an agent. Once we finalized the following 4 steps, we put the answers into the agent’s instruction box. “The quality of the output is directly tied to the quality of the inputs we provide,” Michelle explained. “The more specific we can be about what we want the agent to do and how we want it to do it, the better the results will be and the less refining and follow-up prompting we’ll need to do later.”

Michelle explained that there are two ways to create a Rovo agent in Rovo Studio: we could use the chat agent builder to build an agent by answering a few questions, which would help us quickly build a simple agent. Or we could manually configure an agent and fill in its identity, instructions, knowledge, and actions, which would help us build a specialized agent. We chose the second option.

Step 1: Define the agent’s scope and purpose

Including a specific, detailed scope and purpose helps the agent stay focused and deliver what you need.

For the Transcript Organizer agent, we wrote a “context” section that defined its purpose and scope. It began: “You are helping to extract and organize main points to help write articles for two different types of blogs: external and internal.” Then we told the agent to always ask which type of blog post it’s helping with, and we gave it details about each, including the goals of posting. We explained who the audience is for both types of posts, and what to avoid (for example, the external blog posts I write are not marketing materials, and so the focus should never be on selling).

Step 2: Define the agent’s role and persona

Make sure the agent gives you what you’re looking for by defining its role and persona, including:

• Knowledge

• Skills

• Experience

"I think of agents as this like super made-up, better version of myself or somebody else,” Michelle said, “so I will sometimes tell it, 'Hey, you're an expert in A, B, and C.'"

We defined the Transcript Organizer agent’s persona as an experienced, curious blogger and investigative journalist who wants to highlight the great work teams at Atlassian are doing. We also told the agent it has advanced knowledge of Atlassian apps and roles.

We got really specific about the agent’s role by telling the agent who it is and what it should do. We said, “You are a specialized agent focused on analyzing Loom interview transcripts and extracting the main points to shape a cohesive story for a blog post.” Then we told the agent where to put the supporting evidence for each point (below each main point) and what suggestions it should provide (like direct quotes I could use and additional follow-up questions I could ask the interviewee to fill in gaps and take advantage of potential opportunities).

Step 3: Write step-by-step instructions for the agent

Telling your agent to complete one step at a time means you won’t need an extended back-and-forth conversation. It’s also what makes agents more efficient than Chat for certain tasks: once you write your steps, they’re always there, so you don’t have to go back and forth building on each task like you would with Chat.

We told the Transcript Organizer agent that in order to perform its job, it should follow the steps below, and that each step is predicated (and generated) on the results of the previous steps. "If your agent will need specific information in order to give you the right output,” Michelle said, “then don't rely on the human to remember to give the agent what it needs. Tell the agent to ask for it."

We told the Transcript Organizer agent that it first needs three pieces of information:

• A transcript

• The publication location

• What the post is about

The next step we told the agent to take is to analyze the transcript and identify 3 to 5 main ideas. And then in the final step, we told the agent to organize supporting points from the transcript under each main idea.

Step 4: Test, refine, and improve the agent

Using AI, whether you’re using Rovo Chat or a Rovo agent, is a process you continuously improve. So, you’ll write what you think is a complete set of instructions and rules and dos and don’ts, and then you, or you and your team, will try it a few times and adjust as needed.

After testing the Transcript Organizer agent a few times, we realized it was editorializing, summarizing, and adding its own thoughts and opinions. But I didn’t want the agent to do any of the thinking for me; I wanted to know, quickly and clearly, what I had to work with so that I could shape it into a story. So, we added a line to the instructions and told the agent to transcribe verbatim, without summarizing or paraphrasing, and we started really getting what I wanted.

I’ve been using the agent for almost a year now, and it saves me a lot of time (a few hours every time I interview someone, at least). But more than that, when I use the agent, I save my creative energy for story building. Here’s what it looks like (there are a lot more instructions in the box that aren't shown):

We’d love to hear what types of agents you’re building and refining—please tell us in the comments!

Once you've tested out Rovo a bit, consider taking the free, unproctored, open-book exam to earn the Rovo Fundamentals certificate (I’m a proud certificate holder. Tell us below if you’ve earned yours, too). And, learn how you can earn the Rovo Ranger Badge.

Keep an eye out for more "Behind the scenes with Atlassian Learning" posts. 🌱 

Read the whole series

AI use transparency: I used my Transcript Organizer agent to organize information for this post, but everything was written and designed by a human.