30 PRs in 30 Minutes: Automating Route Migrations with Rovo Dev Coding Agents

[#1]

Dear Community Member,

In early February, we shared our internal story about how we utilised Rovo Dev Coding Agents and Jira Automation to enhance our team's efficiency. Prior to the Beta launch, we conducted several in-house experiments, and one particular experiment significantly transformed a tedious manual process into a scalable workflow. Read on to hear how we got there.

The Problem

As part of ongoing efforts to separate our tightly coupled frontend and backend services, my team needed to implement about 40 new API routes. These routes fetch the necessary state for the frontend to function, and while each task was very simple, collectively they posed a significant workload.

The Solution: Rovo Dev Coding Agent and Custom Automation

In the past, our team has used Rovo Dev Coding Agents for straightforward, bitesized tasks but often hit the wall of needing to invest more time than solving the issue manually would have required. So the current initiative to broaden the impact and ‘unleash an army of agents’ was perfect timing. In this way, we set up a repeatable process for automating these repetitive, by-the-books tasks and plan to apply the same flow to future manual work.

How It Works

• Custom Rovo Dev Coding Agent: We configured a single context description, enabling the agent to execute actions without re-defining prompts for each route.

• Jira Automation: Automation rules in Jira were set up to trigger the agent based on metadata updates in the tickets.

• Metadata Integration: Scripts extracted necessary migration metadata—such as route and class information—and incorporated it into Jira tickets.

This setup allowed us to automate the process of running our Rovo Dev Coding Agent, significantly reducing manual inputs and potential errors.

Technical Details

Rovo Dev Coding Agent

For this migration we used an agent we manually created on our internal instance.

There are four parts to the Rovo Dev Coding Agent we built for our needs:

1. Action: Triggers the agent via a free-text prompt. Rovo agent actions | Rovo | Atlassian Support

2. Templated Instructions: Define reusable instructions with templated variables like {route} or {method_name}.

3. Route-Specific Information: Update Jira tickets with specific values for variables such as /signup in the ticket description.

4. Trigger the Agent: This is still an LLM chat agent, so even with the relevant information we need an actual ‘prompt’, which can be done manually via the Rovo chat (useful for debugging) or completely autonomously via Jira’s automation triggers

Jira Automation

Instead of manual triggering via the agent chat, Jira Automations allow us to set conditions to trigger a dev agents whenever a Jira ticket's metadata is updated. This method enabled us to generate numerous code branches by merely adding a label to tickets.

Note the ‘prompt’ field which explicitly adds the Jira ticket as context that the automation is being run against

Route Metadata

So we have ~40 routes that will need essentially the same logic applied to them over and over again but with some unique information to that route: how do we get that info?

Step 1. Metadata Extraction:

Luckily we’d already written some reflective utility functions used for automatically enforcing analytics on our routes, so it was straightforward to write some local kotlin code to iterate through all of our controllers which made the relevant service calls. At that point we just had to format the metadata and push it into the Jira tickets.

Step 2. Applying Metadata:

Rather than manually go through every Jira ticket, we used a script to iterate over the route metadata and update the Jira ticket’s description and labels. Then, with our input conditions looking at label values changing, the automations kicked in, detected an agent needed to be kicked off, and started finishing our project for us.

In the end we ended up using api requests directly instead of the Jira library because we couldn’t get it to grab the epic tickets properly. There’s also some logic to compare the endpoints fetched programatically versus the existing Jira tickets where some were missing or could be ignored.

Results

By leveraging this automation setup, we achieved notable efficiency gains and were able to generate 32 branches within 30 minutes. Although Rovo Dev Coding Agent also produced code with minor issues, these were generally quick to fix and got us ~80% of the way there for each route.

The automation dramatically accelerated our workflow. Pushing out the equivalent number of branches would have taken several weeks, but we managed to learn, experiment with, and automate this workflow over just three days. Once set up, the automation remains mostly static and is highly maintainable, scalable to hundreds of similar tickets unless the workflow significantly changes.

How To: Setting Up and Productionising Rovo Dev Automation

Below are the steps we used when attempting to setup this flow ourselves and what I would recommend others to try out.

1. Identify Use-Case: Choose a narrow, well-defined task that's repetitive and structured.

2. Single Jira Ticket Test: Create a detailed Jira ticket with explicit instructions and manually run Rovo Dev Coding Agent to assess initial output quality.

3. Refine Description: Adjust the ticket's description until Rovo Dev Coding Agent produces reliable results.

4. Create Custom Agent: Use the refined description in a custom agent, ‘templating’ any variable details.

5. Manual Chat Validation: Prompt the custom agent directly via its chat to ensure consistent and useful outputs.

6. Jira Automation with Manual Trigger: Set up a manual-triggered Jira automation to validate agent interaction.

7. Automate Triggers: Transition to some automatic input trigger and validate full automation.

8. Done: Unleash your army of agents.

My recommendation for debugging would be to to eliminate as many variables as possible as you test each component and ensure all required metadata, like the effected service(s) or the Jira ticket in question, is explicitly linked within the agent or prompt.

Looking ahead, we plan to extend similar automation efforts to the frontend, focusing on the calling logic for the new routes. Additionally, we're exploring other use-cases for Rovo Dev Coding Agents, such as automatic feature-flag cleanup and vulnerability fixes, which promise to further improve our DevEx.

Let us know what ideas you have in mind for your automation workflows!

[Waqas Ahmed]

Wonderful efforts!