How to implement semantic discovery of Jira / Zephyr BBIs using Rovo + Forge?

[rushikesh.shivajishinde]

Given a query such as login with social, I want the agent to list all relevant BBIs (Building Block Instructions).
BBIs may not use the same wording and could be described as OAuth login, SSO, or external identity provider, which makes JQL keyword search insufficient.

[Jorge Cammarota]

Solution Concept
The Problem

User Input: Queries are made in natural language (e.g., "login with social").

BBI Documentation: BBIs (Building Block Items) are described in Jira/Zephyr with inconsistent wording (e.g., "OAuth login," "SSO with external IdP").

Technical Gap: Standard JQL or keyword searches fail because they rely on exact matches rather than intent or meaning.

High-Level Solution

Semantic Indexing: Generate embeddings (vectors) from Jira/Zephyr issues by processing text from summaries, descriptions, and relevant custom fields.

Query Processing: Convert the user's natural language query into an embedding.

Vector Search: Perform a similarity search (e.g., Cosine Similarity or Dot Product) between the query embedding and the stored BBI embeddings.

Integration: Expose this search capability as a "tool" or API that the Atlassian Rovo agent can invoke during "BBI discovery" tasks.

User Flow: The Rovo agent receives the human query, calls the semantic search tool, and returns a curated list of relevant BBIs.

Implementation Options

Option A: Forge + External Vector Service (Recommended)

This approach provides the granular control needed for specific data types like Zephyr test cases.

• Forge App: Handles reading BBIs from Jira/Zephyr and communicates with external services.

• Vector Infrastructure: Uses services like OpenAI/Cohere (for embeddings) and Qdrant/Elasticsearch/Pinecone (for storage and search).

• Tooling: Exposes a Forge Function or Custom UI backend as a specialized API for Rovo to consume.

Option B: Native Rovo Indexing (Limited Control)

This relies on Rovo’s built-in connectors and RAG (Retrieval-Augmented Generation) capabilities.

• Pros: Lower technical overhead.

• Cons: Often lacks the "fine-tuning" required for complex semantic matching within Zephyr structures or highly specific BBI metadata.
  
  

Modeling BBIs (Building Block Instructions) within Jira and Zephyr is an excellent strategy for creating a structured, machine-readable knowledge base. By treating instructions as discrete data entities, you enable precise retrieval and better AI context injection.

Here is a recommended modeling structure in English to ensure clarity, searchability, and seamless integration with RAG (Retrieval-Augmented Generation) systems.
Jira Project Configuration

To keep the repository clean and manageable, use a dedicated space.

Project Name: BBI Library (Key: BBI)

Issue Type: BBI Instruction

Note: Using a custom issue type allows you to define a specific workflow (e.g., Draft → Internal Review → Published).

Zephyr Integration (Testing & Validation)

Linking BBIs to Zephyr ensures that the instruction isn't just a "guide," but a validated requirement.
Option A: The "Source of Truth" Approach (Recommended)

The Jira Issue remains the master BBI.

In Zephyr, create a Test Case that maps 1:1 to the BBI.

Link the Test Case to the Jira Issue using the "Requirement" link.

AI Impact: When I analyze a BBI, I can "see" the test steps in Zephyr to understand exactly how the logic is verified.

Option B: The "Self-Contained" Approach

Use Zephyr’s Test Case entity as the BBI itself.

The "Steps" section of the Test Case becomes the BBI Instruction steps.

AI Impact: This is more technical; the model focuses on the "Action-Expected Result" flow.

4. Why This English Structure Works for AI

Semantic Weight: Using standard English industry terms (e.g., "OAuth2", "Identity Provider") ensures the embedding models map these BBIs to the correct global concepts.

Filtering Capabilities: By using BBI Category, we can limit the "search space." If you ask for a login solution, I can ignore the "Reporting" category entirely, reducing noise.

Traceability: If a test fails in Zephyr, the AI can trace it back to the specific BBI instruction to suggest where the logic might be flawed.

Example BBI Entry

Summary: Auth - Social Login via OAuth2 (Google/Facebook)

Category: Authentication

Tags: login, oauth, sso, social-auth, external-idp

Description: > "Standard implementation for third-party authentication. Ensure the callback URL is whitelisted in the provider's console. This BBI covers the token exchange and user profile mapping."