One Space Indexed by Google, Another Not - Both Have Anonymous Access Enabled

Question

Hello Community,

I have two spaces in my Atlassian instance. One of them is being indexed and found on Google, while the other is not. Both spaces have anonymous access enabled.

Is there anything else I need to configure to ensure the second space is NOT indexed as well? Or could there be another reason for this behavior?

Robots.txt could be an option, but how do I integrate it?

Best regards,

Anne

Shannon S · Answer

Hi Anne,

Good morning, and welcome to Atlassian Community! It's nice to have you join us.

Unfortunately, modifying the robots.txt file for Confluence Cloud is not currently possible. We have been tracking this requirement in the following feature requests:

* CONFCLOUD-8749 Make Confluence more configureable in how it handles web crawlers such as the Google Search Appliance
* CLOUD-5915 Allow users to modify their robots.txt

You can add a vote and leave your feedback, which will increase visiblity on this requirement for our product team.

Google will index anything available on Confluence publicly. As for why one of your public spaces has not been indexed, it could be that there are restrictions on the pages, despite the space being open to anonymous users, or that it is new content that Google hasn't indexed yet.

Googlebot, the web crawler used by Google, visits websites at different frequencies. Popular and frequently updated sites are crawled more often than less popular or static sites. In general, it can take anywhere from a few hours to several weeks for Google to index new content.

I hope that clarifies things, but if you have any follow-up questions about that, please let me know.

Take care,

Shannon S | Atlassian Cloud Support

anne.demond · Answer

Thank you for your input!

Unfortunately, your response doesn’t provide the concrete steps I was looking for. The mention of “search engine settings” and “robots.txt” is helpful as general concepts, but it doesn’t guide me to a practical solution within Confluence.

If you’re unsure about the exact steps or if this topic is outside your expertise, it’s absolutely fine—no need to feel obligated to reply. I’m hoping to find someone who can provide specific guidance, especially regarding how to configure a robots.txt file or other options within Confluence.

Thank you again for taking the time to respond!

Best regards,
Anne

Sahir Maharaj_28wu8p · Answer

Hello @anne.demond

If one space is indexed by Google while the other is not, the issue might relate to search engine settings or a robots.txt file.

Please also ensure that the non-indexed space isn’t restricted by meta tags like <meta name="robots" content="noindex">.

P.S. I recommend to confirm the space’s permissions and make sure all content is publicly accessible.