Codegeist Unleashed 2023: How AI Feedback Analyzer Came to Be

[#1]

Another year, another Atlassian Codegeist! This year’s edition, Codegeist Unleashed 2023 was the 4th we participated in. As always, it was a lot of fun and we created three awesome apps! Today, let me introduce you to the AI Feedback Analyzer, telling you a story of how it came to be.

You can also learn more by watching this video:

Your voice, our tool, and the magic of AI, all together in a single app.

Inspiration 

One day, our Product team, reminiscing of heroes from ancient legends, stumbled upon a mysterious power embedded in the voices of our users . These echoes from days gone by beckoned from digital corners, craving recognition. Embracing a user-centric approach, we took on this epic task. Yet, every grand quest requires tools. As time's essence is our most cherished resource, we couldn't scrutinize each voice in detail. That's when the legend of AI, known for its unparalleled text deciphering abilities, came to the rescue. We set forth to design a tool and instantly marry it with Jira Service Management, our feedback battleground, to comprehend every hint and nuance from our users. That’s how the AI Feedback Analyzer was born.

What it does 

Our tool seamlessly integrates with Jira Service Management and Jira Software Cloud projects. With an uncanny precision, it scours through reports and comments, mining for the golden nugget—feedback. Whilst striking this treasure, it puts on its detective hat , identify products and categorize feedback under themes like "functionality", "usability", and so on, atomize them into subcategories, summaries, and solutions to finally measure user emotions. Its masterpiece? A dazzling board elucidating customer insights, with filters for those yearning to dive deep!

How we built it 

Our technological marvel comprises three key pillars. At its core lies the Forge application—a bastion of data security . But every fortress needs connectivity, and that's where our "back office" steps in, bridging the Forge to the enigmatic world of OpenAI Api. Together, they spin a tale of automation and precision, making feedback our foremost business comrade!

Challenges we ran into 

Getting OpenAI's LLM to consistently respond in a predictable manner was like herding cats. But with some clever prompt engineering, we mastered the art. The outcome? Smooth, predictable automations.

Accomplishments that we're proud of 

Delivering such an intricate app in record time stands atop our list. Adopting an 'Ultra Agile' ethos, our sprints were lightning-fast, and meetings turned into strategy huddles. Our team's communication was at warp speed, and the camaraderie? Electrifying! Decisions, both micro and macro, were instantaneous. We were more than just a team; we operated like a finely tuned engine!

What we learned 

This odyssey taught us the art of viewing things from our user's perspective. A revelation? Speed was not paramount. A slight delay was an acceptable trade-off for other victories. What's more, our users schooled us every step of the way, emphasizing that sometimes conventional importance might not resonate with actual needs.

What's next for AI Feedback Analyzer 

This is just the beginning! Post Codegeist, we aim for the Marketplace. Our eyes are set on recruiting early adopters to test our app's mettle in real-world scenarios. Feedback remains our guiding light . Our ultimate aspiration? Perfecting our product to achieve market fit and gathering a dedicated team to steer our creation towards unparalleled heights. The horizon looks promising! 

If you want to learn more, see our Codegeist Submission. Tell us what you think in the comments below!

[Dugald Morrow]

Great post.

Getting OpenAI's LLM to consistently respond in a predictable manner was like herding cats. But with some clever prompt engineering, we mastered the art.

Do you have any tips for others about creating prompts that reduce the variance in format returned by the AI?

[Tomasz Pośpiech]

@Dugald Morrow 
Thanks for appreciating the post :)

The tips are. You need to make the prompt in this form:

Instructions:
You are to do exactly this and that based on the input. Use an example (output) in your answer. Do not make an introduction and an ending.

Input: some text

Output:
Json format { key: "key_desc" }

Then you have to spend some time testing the prompter. Try different texts. Every time the result is a little different from what you expected, you have to add new corrective things to the prompt.