Ask Alfred: How Claude Fable 5 Helped Me Build a Rovo Agent That Actually Knows Our Jira

[#1]

The problem with "why did my issue do that?"

Picture this: a team member transitions an issue to "In Review." Three things happen automatically. A sub-task gets created. An email goes out. A linked issue changes status. Nobody told them any of this would happen — there's no tooltip, no notification, no breadcrumb. Just... consequences.

They ask the only person who can explain it: the platform admin. He opens the rule tree, mentally decodes customfield_10666 into "Change Object Type," resolves status 10461 to "Active" walks three nested conditional branches, and ten minutes later has an answer.

Multiply that by 61 active automation rules, 242 conditional branches, and a team that uses Jira every day — and you've got a quiet but real support burden sitting on one person's shoulders.

That's the problem Ask Alfred was built to solve.

---

What “Ask Alfred” actually is

Ask Alfred is a Rovo Agent. Its entire knowledge base is a plain-English translation of the instance's own automation export — not a hand-written FAQ, not a general "how Jira automation works" document. Every active rule, every condition, every action, rendered into readable prose by a Python script and fed to the agent as its source of truth.

Ask it "why did my issue just close?" and it walks the rule tree for you. Ask it "what automations run when I create an issue in this project?" and it lists them. Ask it "why didn't the sub-task get created?" and it walks each gate until it finds the one that was likely false.

The key word there is this instance. Not how Jira automation generally works — how your automation is actually configured, right now.

---

The blueprint

The pipeline has three stages that feed the Rovo agent:

Seven inputs feed the script: the automation rules export from Jira, plus six reference exports that resolve the raw IDs the rules are full of — project IDs, user account IDs, custom field IDs, issue type IDs, link type IDs, and status IDs. Three of those six come from the same admin CLI tooling used to manage the instance. Build the extraction layer once, and the agent knowledge falls out of it cheaply.

Three outputs come out the other side:

File	What it is	Who uses it
active_rules_by_project.md	Inventory table — name, trigger, author, rule ID	Humans doing a quick audit
jira_automation_knowledge_base.md	Every active rule in plain English, IDs resolved	The agent's knowledge source
jira_automation_agent_instructions.md	The agent's role, method, guardrails, escalation path	The agent's system prompt

They ship as a set. Never update one without the others.

---

The decision that made it work: script, don't re-reason

Here's the instinct you have to resist: just ask an LLM to read the automation JSON and write the knowledge base.

It's tempting. It's also slow, expensive, and — critically — non-deterministic (meaning: you get different output every time you run it). Every regeneration produces different wording. Diffs become meaningless. And you're asking a model to reliably translate ~190,000 characters of nested JSON without hallucinating a single conditional branch.

The script approach flips this on its head. The translation is deterministic: same inputs, same output, every time. That means:

• Diffs are meaningful. Run a diff on the new inventory file against the old one and you see exactly what changed in the instance — rules added, removed, renamed, flipped on or off. No noise.
• Regeneration takes seconds and costs nothing.
• Hallucination risk on the knowledge base drops to near zero. Every sentence is emitted by a code path you can read.
• The model's intelligence is reserved for the hard bits — validation failures, unrecognised component types, edge cases. That's the right division of labour.

The script is the specification. If you want to change how a trigger type is described, you change the code — not a prompt.

---

What "translation" actually involves

The script works in three layers.

Lookup tables first. Each reference export collapses to a dictionary: project ID → name, account ID → display name, customfield_* → field label, status ID → status name. There's also a hardcoded map for the ~18 system fields Jira doesn't include in the field export. Every raw ID in the rules gets resolved before a single sentence is written.

Recursive renderers second. The rule tree — triggers, conditions, branches, actions — gets walked and mapped to readable sentences. CONDITION_BLOCK becomes **IF** all of these are true: ... **THEN:**. A plain CONDITION becomes Only continues if ... — a hard gate that stops the rule dead. A BRANCH becomes **FOR EACH** ..., with nesting depth carried by indentation. Actions get their own renderer per type: edit field, create issue, create sub-task, transition, send email, link issue (using the link type's own inward/outward phrase, so it reads "this issue blocks it" rather than a raw ID).

Validation before anything ships. The script exits non-zero if it finds the word "unrecognised" anywhere in the output (a component type with no renderer), an empty value after a → arrow, or a leaked raw JSON array. A healthy run prints Validation: PASS. If it doesn't, nothing gets published.

That last one is the one that saves you at 11pm.

Claude Fable 5 did the heavy lifting in building the underlying parsing engine script. It was probably more model than the task strictly needed, but I had credits to burn and it was certainly fun to collaborate on.

---

Trust by construction

Here's the part worth stealing for your own projects.

Every resolved value in the knowledge base carries an inline confidence marker. Status names confirmed against the Jira export say (confirmed via the Jira status export). The one status ID that doesn't appear in the export — a since-retired status referenced by an old rule — says (inferred from this rule's own name). The "Known Limitations" section of the knowledge base auto-updates to name any affected IDs.

The agent instructions then bind the agent to that same distinction. It's instructed to preserve the confirmed/inferred language when it answers, and to say "I can't confirm that" rather than guess.

The chain is: the generator knows its own confidence → it writes that confidence into the knowledge → the agent is instructed to preserve it when it answers.

That's a reusable pattern. The quality of an agent is only as good as the data behind it — and most of the work went into making the source data trustworthy, not into prompting.

---

How it got here (the short version)

This agent didn't arrive fully formed — it went through three distinct versions before it was worth trusting.

The first version was LLM-authored and hand-curated: 9.7 KB, rules grouped by function category, a hand-compiled ID dictionary. Readable and useful — but it only covered a curated subset, couldn't be regenerated identically, and the ID dictionary silently drifted from the instance.

The second version switched to script-generated output and jumped to 192 KB — full coverage of all 61 active rules. But status names were still resolved from a hand-made reference document, so the knowledge base had to warn: "Treat inferred status names as a best guess." Rules rendered as transitions to status ID '10491' (name not resolvable).

The third version swapped the hand-made reference for a real Jira status export. That one change flipped the confidence language across the entire document:

Before	After
"per the technical reference document supplied"	"confirmed via the Jira status export"
"Treat inferred status names as a best guess"	"can be treated as reliable, with the following exception: 10331"
to status ID '10491' (name not resolvable)	to "Peer Review" (confirmed via the Jira status export)

Unresolvable references went from several to one. And that one is honestly labelled.

---

Keeping it current

The knowledge base is a point-in-time snapshot. A rule created after the last generation is invisible to the agent until the next run. Disabled rules are excluded by design — the agent won't describe behaviour that isn't active.

The refresh process is straightforward:

1. Pull fresh exports from Jira and the admin CLI (topics for another article)
2. Run the script; require Validation: PASS
3. Spot-check one simple rule and one complex rule — two minutes
4. diff the new inventory file against the previous one; review what changed
5. Replace all three artefacts together

The agent instructions are a template, not a translation — the project list and escalation contact are the only dynamic insertions. Policy is a human judgement call, not something derivable from JSON.

Set a calendar reminder or automate the pipeline where possible. Future you will be grateful.

---

What transfers

The specific instance here is a Aussie Government Jira with 61 active rules applicable to a few Jira spaces, and one very patient platform admin. But the pattern works anywhere the gap between "what the tool does" and "what users understand" is wide.

The reusable steps:

1. Export the configuration in a machine-readable format
2. Write a deterministic script that translates it to plain English
3. Validate the output before it ships — fail loudly on anything unresolved
4. Label your confidence inline, at the point of generation
5. Constrain the agent to that artefact as its sole source of truth
6. Regenerate on a cadence that matches how often the config changes

The prompting is almost the easy part. The work is in the translation layer — and once that's solid, the agent mostly takes care of itself.

---

Wiring it up in Rovo

The agent configuration itself is deliberately minimal — which is part of the point.

In the Rovo agent builder, the knowledge section is set to Custom knowledge with a single source attached: a Confluence knowledge base named Ask Alfred (Jira automation interpreter). That's the 192 KB plain-English translation file — the one the script generates. No other organisational knowledge is enabled. The agent knows exactly one thing, and it knows it well.

Five Jira skills are also enabled — Get changelog, Get work item, Get editable fields, Get work item types, and Search with JQL — so the agent can look up live issue state when it needs to, rather than reasoning purely from the static knowledge base. The knowledge base explains the rules; the skills let it check the facts.

Three conversation starters are configured to give users a foothold:

• What automations are set up for my project?
• Why did this action trigger an automation?
• How can I troubleshoot unexpected workflow behavior?

They're intentionally broad. The goal isn't to anticipate every question — it's to signal the shape of what the agent can do, so users don't open it and stare at a blank prompt.

---

I’ve since used Ask Alfred on several questions, related to automation behaviour, submitted by end users via JSM support. Each time, Ask Alfred has accurately pinpointed the likely issue and recommended a correct solution or a sensible extension to the existing automation — turning it from an interesting proof of concept into a genuinely useful first stop for support.

I always love hearing about the use cases dreamt up for Rovo Agents. If you've got some great ideas you've put into practise, please share!