We Audited Our Own Six Marketplace Listings — Here's What Fixing Them Actually Did

[#1]

We audited our own six Marketplace listings. Here is what was broken, and what fixing it actually did.

Disclosure: I build Jira apps at Ballon Apps. Every example below is one of our own listings — I am not naming or scoring anyone else's.

We publish six Jira apps. Earlier this year none of them were being found, and the reflex was the usual one: write better copy, add screenshots, chase reviews. Before doing any of that we ran an audit against our own listings and measured the result. Six defects came out. Four of them we had no idea were there, one of them was invisible by design, and the outcome at the end is not the one you would expect from a post like this.

Everything below is measured against the public Marketplace API, anonymously, on dates I have given. You can repeat all of it on your own listings in an afternoon.

First, the method — because one read proves nothing

Two things bit us before we had any findings at all.

The API answers from more than one cache, and they do not agree. Right after we corrected a support link, the same app returned the new value on 9 of 10 reads and the old value on 1. Two consecutive runs of our checking script named different apps as broken while nothing had changed. What told us it was cache and not a config split: every wavering field alternated between exactly the new value and the value that had been there before — never a third, unexpected one. Read every field several times and judge on the majority.

Ranking positions oscillate. We defined "stable" as five identical observations. One term sat permanently between #61 and #62 — adjacent positions with near-equal scores — so it could never satisfy that test and our loop ran to exhaustion reporting "not converged" while the data had been unchanged for hours. A check that cannot go green measures nothing. Require a dominant mode instead, and report an oscillating term as a range.

The six defects

1. The head term was missing from the name — and the description does not save you

This one is worth more than the other five together.

On the term approval (276 Jira Cloud apps, measured 2 Sep 2026) the top of the list is, without exception, apps with approv* in the name. The first app without that stem sits at #26: a time-tracking app with 4,618 installs that does not belong in the results at all. Above it sit five apps with three installs or fewer — including one with a single install at #19.

One install with the word in the name beats 4,618 without it. On a generic head term the name is not a ranking factor, it is an entry requirement.

We then tested which listing fields the search actually reads, using words that appear in exactly one field of our Approval Gate listing:

Search term	Where the word appears	Result
approval	in the name	#22 of 276
sign-off	only in the tagline	#4 of 14
getcompliant	only in the app key	returns exactly our 6 apps
Ballon	only in the partner name	returns all 6
validator	only in the summary	not in the top 150 of 192
transitions	only in the summary	not in the top 150 of 229

Name, tagline, app key and partner name are searchable. The summary is not. All the careful prose we had written into the description was doing nothing for discoverability. If your head term lives only in your long description, as far as search is concerned it does not exist.

Two practical consequences. Measure which term is lexically free, not which is big: pull the top 20 for a term and count how many have it in the name. Zero means one rename makes you the only exact match. That is how we found last comment unclaimed among 850 competitors while comment field was unwinnable — the number one has it in its name twice.

And do not put the same word in two of your own names. Two of ours sat at #4 and #5 on archive; buyers saw two near-identical names from one vendor. Splitting the lanes left one app per term.

2. Zero screenshots — on every version, since the first submission

We assumed the images were there and had simply not rendered. They were not. Across all six listings and fifteen versions: 0 highlights and 0 screenshots, including the oldest version from the original submission day. Nothing had ever been attached.

The trap is where they live: highlights and screenshots are version-scoped, not app-scoped. They sit in the version resource, not the addon resource. That is why our checks kept coming back clean — we were looking at the wrong level. Calibrate your counter against a reference listing before believing a zero: draw.io has 27 screenshots and 3 highlights, ScriptRunner 7 and 3.

3. Missing highlights

Same root cause, separate field, and Atlassian asks for three. We had none on any listing. Adding them is version-level work and, usefully, it survives: we compared the version-level text across four releases of one app and it was byte-identical each time. A correction here is inherited by the version a production deploy creates automatically — it does not get overwritten on your next release.

4. A description that was really release notes

Ours had drifted into changelog voice. Worse, a routine deploy can hand you a version whose release summary is the literal string "Minor version update" while the previous one carried a real description. If your listing has more than one recent build, check that the one you are actually submitting is the one you filled in — we later found an app of ours where the submitted build was completely empty while all the content sat on a build created four seconds earlier.

While rewriting we also found 28 text defects across four listings — words run together, missing spaces after punctuation, double and triple spaces mid-sentence. Nobody had read the rendered page.

5. Support URLs that were broken or redirected

Five of our six listings pointed at the wrong page. Three details matter.

The field is not where you expect. Support lives on the addon resource as vendorLinks.supportTicketSystem; documentation lives on the version resource. Two endpoints, two different screens in the portal. A documentation check will never see a broken support link.

A 200 is not proof. One of our documentation links was missing a path segment, and the site's SPA fallback answered 200 with the Dutch homepage. Assert on the page title, never on the status code.

Redirects get rejected. Atlassian's link checker does not accept a 3xx on a listing field, so the URL you enter has to answer 200 directly — including on the apex if someone types it without www.

Ours pointed at a contact page that belonged to a different product line of ours entirely, with a different support address and a different response time. The reviewer flagged the language; the real defect was the channel.

6. A partner name that did not match

Two listings carried the wrong spelling of our own partner name in the listing text. Ours differs by one letter across platforms for a legitimate reason — the name we use elsewhere failed Atlassian's uniqueness check — and a well-meaning consistency pass is exactly what reintroduces the rejected spelling. If you trade under a brand that is not your registered entity, write it as "(Brand), a brand of (Registered Entity)" and keep it identical in the listing, the support page and the partner profile.

What fixing it actually did — and this is the part that matters

Renaming is cheap and reversible: no review round, the app key stays, installs, reviews and licences come with you, old URLs keep working. So we did it and measured, five observations per term, repeated over days.

App	Term	Before	After
Approval Gate for Jira	approval (276)	outside top 100	#22
Last Comment Field for Jira	last comment (866)	#23	#9
Retention Policy for Jira	retention policy (218)	#4	#3
User Access Review for Jira	user access (1,759)	outside top 300	#35 *

The gains held five days later with no decay.

The first three still stand today. The fourth we later traded away deliberately — see the last paragraph of this section; it now sits at #64.

Installs in that period: none. Not one of the four renamed apps gained a single install. For calibration that the install figure is real and populated, three competitors on the same term sat at 297, 258 and 29.

That is the honest result. Being findable is a precondition, not a cause. We opened the top of the funnel and nothing came through it, which means our bottleneck was never only discoverability. Five days is short for a B2B cycle and we are re-measuring in six weeks; what five days does rule out is the optimistic story, that once you are in the list the installs follow.

There is a second lesson in the same data. When we dropped a stem from a name to make room for a better term, we went from #7 to outside the top 120 on the term we had abandoned. Restoring it in a composite name brought it back to #13 — but cost the new term, which moved from #35 to #64. These are trades, not wins. Decide which lane you want before you rename, because you cannot hold both.

If you want to run this on your own listing

Every check above is one anonymous call against the public Marketplace API plus a careful read of your own portal screens. It takes an afternoon and needs no permissions.

If you would rather have a second pair of eyes, say so in a comment or message me and I will run the same six checks against your listing and tell you what I find. No obligation and nothing to sign up for — I have only ever run this on our own apps and I would like to know whether the pattern holds more widely.