Confluence Public Space Not Being Indexed by Google

[André Hausmann]

Hi we are using our Confluence Cloud space as publicly accessible documentation for our customers:
https://beprint.atlassian.net/wiki/spaces/PL/overview

The space is set to be publicly available without login, and we have created a large amount of content there. However, the pages are not being indexed by Google or other search engines. We have already submitted sitemaps and tried several measures on our side, but none of them lead to indexing.

Could you please check or confirm:

1. Whether this space is allowed to be indexed by search engine crawlers?

2. Whether there are any restrictions or default settings that could prevent indexing (e.g., robots.txt, meta tags, space permissions)?

3. If there is a recommended configuration or documentation for enabling indexing of public Confluence Cloud spaces?

We appreciate your help. Thank you in advance.

[Robert Hean]

Heya @André Hausmann - Sorry to hear you're running into issues!

Curious if you could provide more info, especially what suggests your pages aren't being indexed? Asking as I could find your site by searching for a phrase from one (Specifically "An dieser Stelle erhalten Sie Hilfe zur Benutzung" shows this page https://beprint.atlassian.net/wiki/spaces/PL/overview in search results).

[André Hausmann]

@Robert Hean Thanks for your question.
If you check which pages and content are listed in Google

you can try with this in google

you will see that only the first page is indexed. None of the internal links or subpages from this public Confluence space are being indexed at all.

The landing page itself gets indexed by Google, but none of the internal links or subpages are being indexed. It seems that search engines (and also sitemap tools) are unable to follow any internal links on the public Confluence pages.

We tested this using several sitemap generator tools (both online and offline). All of them can fetch the first public page, but none of them are able to detect or follow the internal links, therefore no deeper content is recognized or indexed.

So even though the content has been public for more than 2 months and we submitted the sitemap multiple times to Google Search Console, none of the underlying pages are appearing in Google search results.

We suspect that Confluence’s public page rendering may be preventing crawlers from accessing internal navigation links.

Could you please check:

Whether public Confluence pages include all required attributes for search engine crawling, and

If there is a known issue or configuration setting required to allow crawlers to follow internal links?

Thank you in advance — I hope the explanation is clear.
We appreciate your help.

[Barbara Szczesniak]

@André Hausmann According to this page: https://support.atlassian.com/confluence-cloud/docs/how-secure-are-public-links/ 

Can someone Google a public link?

Atlassian has taken all necessary steps within its capability to make sure search engines do not index our public links.

This means that public links are not indexed by search engines, which means no one will be able to find the public link in a Google search. They need the actual public link.

Unless something has changed and Atlassian has not updated this page, I think this answers your question.

[André Hausmann]

@Barbara Szczesniak  - Thanks for sharing -
this is really critical - as documentation and its importance are enormous, especially in the area of ​​search engine visibility - this is a real severe limitation of usability.

[Barbara Szczesniak]

@André Hausmann You'll have to take that up with Atlassian.

I'm sure they envisioned this feature to function in an "anyone who has the link" manner, rather than "anyone in the world who finds the link." I would think that you only want your customers in your Confluence space (you could post the link somewhere that your customers have access), not random people who come across it. There may also be some system security concerns; that's not my area, but maybe there's more in that page I linked to or other related pages.

If you do want to have the content available and searchable in Google for the world, you might consider using one of the Atlassian Marketplace apps available to generate a web output of the space (like an online help system) and host that website somewhere accessible to Google.

[André Hausmann]

@Barbara Szczesniak Do you have any experience with a tool that creates a web version?Can you recommend one?

[Barbara Szczesniak]

@André Hausmann I use Scroll Viewport/Scroll Sites from K15t. We only want the content available to our internal users, so we host it on our intranet, but you can host it externally.

See this page for more info: https://marketplace.atlassian.com/apps/1211636/scroll-sites-for-confluence-help-centers-blogs-websites?hosting=cloud&tab=overview 

Disclaimer: You asked which app I use. There are other apps available for you to choose.