Same Top Searches, a Different Marketplace: What's Changed for App Discovery (and What Hasn't)

[#1]

If you sell on the Atlassian Marketplace, here is a small surprise worth sitting with. The list of things people search for has barely moved in four years. The list of things that decide whether they find your app has moved a great deal.

That gap, between a stable demand side and a fast-changing supply side, is the most useful lens I know for reading the last year of Marketplace changes. So this piece is organized around it: what stayed the same, what changed, and what it means for the people writing listings.

The part that didn't change: what customers search for

The clearest public picture of Marketplace search history comes from Rustem Shiriiazdanov's 2025 analysis. Worth naming the provenance plainly. The figures are not scraped, and they are not inferred from search results: they come from Atlassian's own Marketplace reporting API, the top-500 search-keyword report it exposes to authenticated vendors, where each term carries its share of queries. Atlassian does not publish that data openly, so a vendor pulling and charting it is the closest public window we have. Shiriiazdanov works at Actonic, itself a Marketplace vendor, so it is not a neutral source, but the underlying numbers are Atlassian's own measurement of what customers actually typed. The one place he flags his own uncertainty is whether total search volume has risen, which he calls an educated guess rather than a measurement. With that scoped, the pattern is striking.

In December 2021, the top five searches were scriptrunner, structure, jira, automation, and portfolio, and together they made up nearly 57 percent of all searches, with ScriptRunner alone at around 12 percent. By 2026, a more recent pull from the same Atlassian vendor keyword report shows the same five names still at the top, but the share each carries has compressed dramatically: the top query holds roughly two and a half percent of all search share, and the five together account for about twelve percent.

So the names on top persisted. What collapsed was the concentration, and the long tail exploded, which is what you would expect from a maturing, more specific user base. Beneath the stable head, the composition did move. Test management climbed hard, with Xray and Zephyr pushing into the top ranks, and a wave of searches for AI and MCP integrations appeared that the Marketplace mostly could not answer. So the demand side is not frozen. Its head is remarkably steady while its body churns.

Two of those old chart-toppers faded for a specific structural reason: Atlassian absorbed the use case into the product. Automation for Jira began as a Marketplace app, was acquired with Code Barrel in 2019, built into Jira Cloud in 2020, made native in Jira Data Center 9, and eventually removed from the Marketplace entirely. Portfolio for Jira was renamed Advanced Roadmaps in 2020 and bundled into Jira Premium. The demand never disappeared. Atlassian simply moved the supply inside the product. Customers kept typing the word out of habit; the install no longer needed to follow.

What changed, first: the ranking engine

The headline change is how search itself works, and the public record here has two layers worth separating.

Atlassian flagged the direction itself, and early. In an October 2024 post on its Work Life blog, it announced that Marketplace search was moving to a hybrid model pairing keyword matching with natural language processing, so customers could search in plain language rather than guess at exact terms. That part is first-party, and more than a year old.

The detail of how it was rebuilt, and how it now ranks, comes from a more thorough and more recent source: a December 2025 App Central write-up by DevSamurai's Liam Do. It reports that Atlassian migrated Marketplace search from Algolia to OpenSearch in a phased rollout that completed at the end of January 2026, and that ranking now rests on three factors working together: keyword relevance, semantic matching (comparing vector representations of the query and the listing), and engagement signals such as reviews, ratings, installs, and whether you publish support details. In practice, a customer can describe a problem in a full sentence and get sensible results without knowing the exact term a developer used.

Here is where vendors should read carefully, because the official record has not caught up with the partner explainers. That write-up, the clearest public walkthrough available, tells partners reviews and ratings "directly influence your search ranking". Atlassian's own search-ranking documentation still describes the prior system: it names Algolia as the search provider and frames those same signals as tiebreakers that matter only "in the event of a tiebreaker", with relevance dominant. The gap is Atlassian's documentation lagging the change, not the partner getting it wrong. Atlassian has separately published a detailed look at how its Rovo assistant ranks results using popularity and behavioral signals, but that is a different system from Marketplace search.

So two things are true at once. Atlassian signaled the NLP direction itself, and early. But the OpenSearch mechanics and the three-factor model come from a partner, the developer documentation still describes the old engine, and the precise weight of engagement signals has never been published, for Algolia or OpenSearch. Treat the direction as real and the exact weighting as worth watching rather than asserting.

The reason this matters is the obvious counter-argument, and it is a fair one. If engagement signals carry meaningful weight, and those signals scale with install base and review count, the system can favor incumbents over newer apps that are a better semantic fit. This is not a hypothetical. In the comments on that very post, a reader raised exactly this concern, and the author agreed that stronger recency weighting would make discovery fairer. When the person explaining a change concedes its main risk, that risk is worth taking seriously.

None of this is automatically bad news for vendors. For an app that genuinely solves a problem and has earned real reviews, intent-based ranking is an advantage: it rewards fit over keyword games, and the trust and quality signals tightening around it mostly serve customers and honest vendors. The worry is narrow. If engagement weighting is heavy, a strong new app can still sit behind an entrenched one until it builds the installs and reviews the ranker rewards.

What feeds the ranking, and how much of it you control

It clarifies things to sort those inputs by how much sway you actually have over each. The one you control outright is your listing copy, the searchable fields you write yourself. Reviews, ratings, and installs you do not write at all; you earn them, and customers ultimately drive them, though good support and a product worth recommending tilt the odds. Trust badges and support details sit between the two: you qualify for them by meeting a bar rather than by phrasing, and the next section takes those quality signals on directly, since they are where the bar is rising fastest.

Two inputs are worth singling out because they are the easiest to misread. The first is Paid via Atlassian: opting into Atlassian's own billing scored a listing higher in the documented old model, which makes it a commercial decision rather than a copy one, and its weight under OpenSearch has not been published. The second is version recency, which the ranking documentation labels age, and it is the weakest lever of the set. That same documentation says a recently updated app is favored only rarely, as a last-resort tiebreaker once every other signal is equal. It rewards shipping real updates, not how long a listing has existed, and like the rest of the old tiebreaker set its standing under OpenSearch is unclear. Neither one substitutes for the input you control most directly, which is still the words.

What changed, second: trust and quality signals are rising, and they feed ranking

A second cluster of changes raises the bar on listing quality itself, and these signals are part of what the relevance score reads.

Most concretely, on May 20, 2026, Atlassian moved Marketplace app ratings from a four-star to a five-star scale, aligning with the industry-standard system. Historical reviews keep their original scale on the reviews tab; only the aggregate is shown on the new scale, and the public API fields now report on one to five. If you consume those fields anywhere, they changed under you.

Around the same arc, Atlassian began reworking its Marketplace Trust program and shifting how trust badges are displayed, including retiring the Cloud Security Participant badge and leaning harder on Bug Bounty participation. The throughline across ratings, reviews, support details, and trust badges is that the credibility of a listing is no longer cosmetic. It is increasingly an input to whether the listing surfaces at all.

What didn't change: the persistent challenges

For all the movement, two old constraints are exactly where they were.

First, your keyword surface is still your listing copy. There is no dedicated free-text keyword field on the Marketplace. The curated keywords introduced in December 2023, capped at four per app, are for category navigation, not search relevance. Ranking for a phrase still means writing the key searchable fields, your app name, tagline, description, and highlights especially, so they genuinely match intent. OpenSearch changes the craft from keyword-stuffing toward plain clarity, but the surface you write into is the same one you have always had.

Second, competitive visibility remains limited, even though the first-party picture is fairly full. A vendor can see a good deal about its own listing: Atlassian's reporting covers its keyword and zero-result data along with its own install and sales figures, it can connect a Google Analytics property to the listing for traffic and on-page behavior, and it can read its own rank for a phrase simply by searching. What is missing is the comparative layer. All of that is scoped to a vendor's own listings, there is no public feed of what an entire category is searching, and per-listing traffic for competitors is not exposed.

It is worth meeting the obvious objection head on: is that Atlassian's job to provide? Arguably not. A marketplace is under no obligation to tell a vendor who their rivals are or hand over a competitive map. The sharper framing is not about competitor surveillance at all. It is that listing quality is now a ranking input, so understanding the phrases your category actually competes on is part of writing a listing the new engine can rank well.

For context, this is where the Atlassian Marketplace sits relative to its peers. Amazon gives brand-registered sellers category-level search-query share and the top products competing for each query. The Apple App Store and Google Play give a developer their own funnel and leave competitor intelligence to a mature third-party ASO industry. Steam exposes a rich public surface of tags, reviews, and player counts that tools like SteamDB turn into competitive data. The Atlassian Marketplace sits at the narrow end of that range: limited first-party competitive data, and a public surface thin enough that no comparable third-party layer has really formed. Whether that changes is, again, up to Atlassian.

Putting it together

The head of demand has been remarkably stable. Scripting, automation, testing, planning, reporting, time tracking, and diagramming are what people searched for years ago, and broadly still are, even as the categories beneath them churn. What has been rebuilt is everything that turns a search into an install: intent-based ranking, and engagement and trust signals feeding relevance.

For vendors, the job that follows is plain. Describe the problem you solve clearly enough that an intent engine recognizes it, and earn the engagement and trust signals the ranker now reads. The names at the top of the search box may not change much from one year to the next. What it takes to be found under them already has.

There is one more change underway, and it is big enough that it deserves its own piece. Ranking decides who wins the search box, but the search box itself is no longer the only door into your app. Customers increasingly arrive through in-product suggestions, AI assistants, and partner recommendations, channels a vendor can see into very unevenly, if at all. How that discovery fragmentation works, and why it lands hardest on the smallest vendors, is where this analysis goes next.

---

Further reading, all linked above: Atlassian's October 2024 announcement of hybrid keyword-plus-NLP Marketplace search; the Algolia to OpenSearch explainer (DevSamurai, App Central, Dec 2025); Atlassian's Marketplace search-ranking documentation (still describing the prior Algolia model); Atlassian's engineering write-up on Rovo search ranking; the 2025 Marketplace search analysis (Actonic); Atlassian's Code Barrel and Advanced Roadmaps announcements; the Marketplace app-categorization keywords documentation; the Marketplace changelog (five-star transition, May 2026); and the Marketplace Trust program post