nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Where does AI actually earn its place in your workflow — and where do you not trust it?

Howard Berkman

With Rovo now in most Atlassian stacks, I've noticed a lot of us are (rightly) skeptical — every tool suddenly has an "AI" button, and it's hard to tell real value from noise. The pattern I keep seeing: AI is genuinely useful for drafts, summaries, and with Rovo, some good Jira digging, but it'll confidently hand you a wrong answer with the same tone as a right one.

Curious how others here draw that line. What's one place AI has actually saved you time, and one place you've learned not to trust it?

(I ended up putting my own take into a free, no-signup course called AI for Skeptics — happy to share the link if it's useful to anyone.)

Comments

Jordan Wilkinson_1uc1r0

Quite honestly every automation flow I have tried to build with rovo lately has been a big time suck and highly unreliable. It's really frustrating that Atlassian's own AI product seems to know less about Atlassian automation than I do. When I correct it, of course it gives me the typical "good catch" message.

Where it truly shines is when you ask it for help bouncing around ideas given a certain set of boundaries, so it becomes an interesting way of being creative and saving some time from reading hours of material.

Viswanathan Ramachandran

Coming from a Data Center administration background and testing Rovo in my personal site, I treat it like a brilliant, high energy & enthusiastic intern : fantastic at finding needles in the haystack, terrible at taking final responsibility.

Where it earns its place: Writing initial JQL, summarising massive comment threads across linked tickets, and drafting release notes. It cuts out the tedious setup work.

Where I don't trust it: Governance, scheme logic, and anything involving compliance or security settings.

Would love to take a look at your course link.

MeghnaP_LogicLemur Labs

I think the biggest shift for me has been moving from asking “Can AI do this?” to “How do I know it did this well?”

AI has genuinely saved me time on things like summarising information, finding patterns across Jira data, and getting a first draft of something that would otherwise take much longer.

Where I’ve become more cautious is when the output is treated as the final answer, especially for decisions or work where context and accuracy matter. A confident answer doesn’t necessarily mean a correct one.

I also think we’re going to need better ways to measure the value of AI agents. Instead of just looking at tokens, prompts, or number of AI interactions, it’s more useful to understand what work the agent actually performed, how long it took, and whether the outcome was useful.

That’s probably where the conversation around AI needs to go next: from AI adoption → AI accountability and measurable outcomes.

MeghnaP_LogicLemur Labs

Totally agree. I’ve also found AI much more useful as a thinking partner than as something I can blindly trust to execute a workflow.

That “good catch” response after you correct it is painfully familiar 😄

It also got me thinking about how we measure AI value. Usage or number of prompts doesn't tell us much. What really matters is what work the agent actually completed and whether it saved meaningful time.

I recently wrote about this in my article on measuring AI agent ROI in Jira. Might be relevant to what you’re describing.

MeghnaP_LogicLemur Labs

“Brilliant, high-energy intern” is probably the best description I’ve heard 😄

I’ve seen the same thing with Jira AI. It’s great at reducing the tedious first 80% of the work, but that last 20% often needs someone who understands the context and can validate the outcome.

The governance/compliance boundary you mentioned is especially important. That’s where “sounds right” simply isn’t good enough.

Nabeel awan

I have had a similar experience. AI seems most useful when I use it to reduce repetitive first-pass work rather than treating it as an authority.

Summarizing long discussions, brainstorming approaches, drafting JQL, and spotting patterns can save a lot of time. But for permissions, automation logic, security, or anything with real consequences, I think it’s important to verify the result against the actual configuration and documentation.

The brilliant intern comparison is pretty accurate: great at getting you started, but someone still needs to be accountable for the final result.

I also agree that measuring time saved and the quality of the outcome is much more useful than simply counting prompts or AI interactions.

Stephen.Lugton

We have an internal company AI where I've created an agent to read my emails and teams messages and pull out any tasks that I need to focus on. I run it most mornings to create a summary of things I actually have to care about; it's particularly useful if I've had some time off or a busy day and didn't manage to check my emails

I don't trust AI so much when it comes to creating new content, for example my daughter is doing SATs in the autumn term and one thing she hasn't encountered before is Non Verbal Reasoning. To help her learn about this I asked ChatGPT to create a simple guide to explain how to approach each of the topic areas that are likely to be in the tests. To be fair most of what ChatGPT created was ok, but there were enough simple errors that I couldn't rely on it to help teach my daughter.

For instance this figure is meant to demonstrate reflections of a shape:

627447798ce4ff99835e2a17c1d81a44121ebedfa391dff0dcbfd27c26a1153f.png

If you look at the what's happening row you can see how a reflection works, but if you look at the sequence row the sequence doesn't actually show reflections; it looks good but it's just wrong!

Taking this one step further to a full page example, in this case rotations:

8139a385f791e5e264adf6274cd01640a1bdb7dc962e4512beb9e34090a251f1.png

If you look at the rotation guide on the right you can see that 90 degrees clockwise is the same as 180 degrees, and in example 4 the first shape is in the wrong orientation. (There are lots more issues with this page, and I know it's an image but the text only version of this had similar mistakes)

Rodrigo De Sebastian Martin

Hi @Jordan Wilkinson and @MeghnaP_LogicLemur Labs

My name is Rodrigo, I am a Product Manager in the Jira Team. I am very interested in these comments and I would like to have the opportunity to talk to you about these pain points.

We are working on how to improve Jira Experience for Automations and your feedback will be very valuable.

This is my calendar link: https://calendar.app.google/ZiQjQymT7vV9afkm9

Thank you,

Rodrigo

Howard Berkman

Apologies for the slow return — this thread got better than I expected, and I wanted to give it a real read rather than a drive-by.

@Viswanathan Ramachandran — here it is, free, no signup: https://forge-ai-forskeptics.netlify.app/?src=atlassian-skeptics. "Brilliant, high-energy intern, terrible at taking final responsibility" is a better line than anything in the course, and I may be stealing it.

What strikes me reading all of your replies together is how consistently the line lands in the same place. Drafts, summaries, JQL, pattern-finding — fine. Governance, compliance, scheme logic, anything with real consequences — verify against the actual config. Nobody here needed convincing of that; you'd all already learned it the hard way.

@MeghnaP_LogicLemur Labs— the shift from "can AI do this" to "how do I know it did it well" is the one I keep seeing. In Agile work specifically, the failure mode I see most is a sprint summary that reads beautifully and quietly drops the one blocker that mattered. It's not wrong enough to catch on a skim, which is exactly what makes it dangerous. Reading your ROI piece now.

@Stephen_Lugton— the rotation diagrams are a great example, because the text version had the same errors. It's not a rendering problem, it's confident wrongness in a domain where the reader can't easily check. Sorry about the SATs prep.

One thing I'm genuinely curious about, since this group has clearly been through it: has anyone found a review step that actually sticks? Not "be careful" as a principle — an actual habit or checkpoint that survives a busy sprint. That's the part I've had the hardest time making durable for a team, and I'd rather hear what's worked than keep guessing.

Welcome!

It looks like you're new here. Sign in or register to get started.