Where does AI actually earn its place in your workflow — and where do you not trust it?

[Howard Berkman]

With Rovo now in most Atlassian stacks, I've noticed a lot of us are (rightly) skeptical — every tool suddenly has an "AI" button, and it's hard to tell real value from noise. The pattern I keep seeing: AI is genuinely useful for drafts, summaries, and with Rovo, some good Jira digging, but it'll confidently hand you a wrong answer with the same tone as a right one.

Curious how others here draw that line. What's one place AI has actually saved you time, and one place you've learned not to trust it?

(I ended up putting my own take into a free, no-signup course called AI for Skeptics — happy to share the link if it's useful to anyone.)

[Jordan Wilkinson]

Quite honestly every automation flow I have tried to build with rovo lately has been a big time suck and highly unreliable. It's really frustrating that Atlassian's own AI product seems to know less about Atlassian automation than I do. When I correct it, of course it gives me the typical "good catch" message.

Where it truly shines is when you ask it for help bouncing around ideas given a certain set of boundaries, so it becomes an interesting way of being creative and saving some time from reading hours of material.

[MeghnaP_LogicLemur Labs]

Totally agree. I’ve also found AI much more useful as a thinking partner than as something I can blindly trust to execute a workflow.

That “good catch” response after you correct it is painfully familiar 😄

It also got me thinking about how we measure AI value. Usage or number of prompts doesn't tell us much. What really matters is what work the agent actually completed and whether it saved meaningful time.

I recently wrote about this in my article on measuring AI agent ROI in Jira. Might be relevant to what you’re describing.

[Viswanathan Ramachandran]

Coming from a Data Center administration background and testing Rovo in my personal site, I treat it like a brilliant, high energy & enthusiastic intern : fantastic at finding needles in the haystack, terrible at taking final responsibility.

Where it earns its place: Writing initial JQL, summarising massive comment threads across linked tickets, and drafting release notes. It cuts out the tedious setup work.

Where I don't trust it: Governance, scheme logic, and anything involving compliance or security settings.

Would love to take a look at your course link. 

[MeghnaP_LogicLemur Labs]

“Brilliant, high-energy intern” is probably the best description I’ve heard 😄

I’ve seen the same thing with Jira AI. It’s great at reducing the tedious first 80% of the work, but that last 20% often needs someone who understands the context and can validate the outcome.

The governance/compliance boundary you mentioned is especially important. That’s where “sounds right” simply isn’t good enough.

[MeghnaP_LogicLemur Labs]

I think the biggest shift for me has been moving from asking “Can AI do this?” to “How do I know it did this well?”

AI has genuinely saved me time on things like summarising information, finding patterns across Jira data, and getting a first draft of something that would otherwise take much longer.

Where I’ve become more cautious is when the output is treated as the final answer, especially for decisions or work where context and accuracy matter. A confident answer doesn’t necessarily mean a correct one.

I also think we’re going to need better ways to measure the value of AI agents. Instead of just looking at tokens, prompts, or number of AI interactions, it’s more useful to understand what work the agent actually performed, how long it took, and whether the outcome was useful.

That’s probably where the conversation around AI needs to go next: from AI adoption → AI accountability and measurable outcomes.

[Nabeel awan]

I have had a similar experience. AI seems most useful when I use it to reduce repetitive first-pass work rather than treating it as an authority.

Summarizing long discussions, brainstorming approaches, drafting JQL, and spotting patterns can save a lot of time. But for permissions, automation logic, security, or anything with real consequences, I think it’s important to verify the result against the actual configuration and documentation.

The brilliant intern comparison is pretty accurate: great at getting you started, but someone still needs to be accountable for the final result.

I also agree that measuring time saved and the quality of the outcome is much more useful than simply counting prompts or AI interactions.

[Stephen_Lugton]

We have an internal company AI where I've created an agent to read my emails and teams messages and pull out any tasks that I need to focus on.  I run it most mornings to create a summary of things I actually have to care about; it's particularly useful if I've had some time off or a busy day and didn't manage to check my emails

I don't trust AI so much when it comes to creating new content, for example my daughter is doing SATs in the autumn term and one thing she hasn't encountered before is Non Verbal Reasoning.  To help her learn about this I asked ChatGPT to create a simple guide to explain how to approach each of the topic areas that are likely to be in the tests.  To be fair most of what ChatGPT created was ok, but there were enough simple errors that I couldn't rely on it to help teach my daughter.

For instance this figure is meant to demonstrate reflections of a shape:

If you look at the what's happening row you can see how a reflection works, but if you look at the sequence row the sequence doesn't actually show reflections; it looks good but it's just wrong!

Taking this one step further to a full page example, in this case rotations:

If you look at the rotation guide on the right you can see that 90 degrees clockwise is the same as 180 degrees, and in example 4 the first shape is in the wrong orientation.  (There are lots more issues with this page, and I know it's an image but the text only version of this had similar mistakes)