One Space Indexed by Google, Another Not - Both Have Anonymous Access Enabled

anne.demond
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
January 15, 2025

Hello Community,

I have two spaces in my Atlassian instance. One of them is being indexed and found on Google, while the other is not. Both spaces have anonymous access enabled.

Is there anything else I need to configure to ensure the second space is NOT indexed as well? Or could there be another reason for this behavior?

Robots.txt could be an option, but how do I integrate it? 

 

 

Best regards,

Anne

 

2 answers

1 vote
Shannon S
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 16, 2025

Hi Anne,

Good morning, and welcome to Atlassian Community! It's nice to have you join us.

Unfortunately, modifying the robots.txt file for Confluence Cloud is not currently possible. We have been tracking this requirement in the following feature requests:

  1. CONFCLOUD-8749 Make Confluence more configureable in how it handles web crawlers such as the Google Search Appliance
  2. CLOUD-5915 Allow users to modify their robots.txt

You can add a vote and leave your feedback, which will increase visiblity on this requirement for our product team.

Google will index anything available on Confluence publicly. As for why one of your public spaces has not been indexed, it could be that there are restrictions on the pages, despite the space being open to anonymous users, or that it is new content that Google hasn't indexed yet.

Googlebot, the web crawler used by Google, visits websites at different frequencies. Popular and frequently updated sites are crawled more often than less popular or static sites. In general, it can take anywhere from a few hours to several weeks for Google to index new content.

I hope that clarifies things, but if you have any follow-up questions about that, please let me know.

Take care,

Shannon S | Atlassian Cloud Support

0 votes
Sahir Maharaj
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
January 15, 2025

Hello @anne.demond

If one space is indexed by Google while the other is not, the issue might relate to search engine settings or a robots.txt file.

Please also ensure that the non-indexed space isn’t restricted by meta tags like <meta name="robots" content="noindex">.

P.S. I recommend to confirm the space’s permissions and make sure all content is publicly accessible.

anne.demond
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
January 15, 2025

Thank you for your input!

Unfortunately, your response doesn’t provide the concrete steps I was looking for. The mention of “search engine settings” and “robots.txt” is helpful as general concepts, but it doesn’t guide me to a practical solution within Confluence.

If you’re unsure about the exact steps or if this topic is outside your expertise, it’s absolutely fine—no need to feel obligated to reply. I’m hoping to find someone who can provide specific guidance, especially regarding how to configure a robots.txt file or other options within Confluence.

Thank you again for taking the time to respond!

Best regards,
Anne

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
PRODUCT PLAN
STANDARD
PERMISSIONS LEVEL
Product Admin
TAGS
AUG Leaders

Atlassian Community Events