nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

advice after anonymous access allowed rampant page and draft creation

Andy Holt_r28i2n

We have discovered a Confluence space that accidentally had the EDITSPACE permission allowed for anonymous users. Doh.

Spammers found this and added >2000 pages. This isn't so bad.

However, web crawlers did web crawler things, and hit the Create Page button approximately 750k times, mostly with URLs like this:

https://confluence.example.com:443/pages/createpage.action?fromPageId=240222349&spaceKey=EXT

We can see some of these pages, by using the ID in the URL, and they show with title $helper.page.title.

The overall size of our database (AWS RDS Aurora PostgreSQL) went from ~20 GB to ~110 GB due to this.

In particular, the confancestors table was previously around 87 MB and is now 84 GB (1000x times increase). That table now has 777 million rows (was about 700k rows).

The space only had a few pages before this started, so I am happy to move those, and then blow away the space entirely. However, on a test clone of our production system, I have initiated an entire space removal, which gets stuck at 2% with an ever-increasing 'Time Remaining'. While this runs, the size of the confancestors table again grows rapidly (added another 30 GB when left running for 1 day), although the row count does not change.

Any advice on a better recovery / removal strategy ?

Accepted answers

All comments

Nicolas Grossi

@Andy Holt You can't recover a backup "before the attack" and change the edit permission on that space and put it back online ?

Nicolas

Kristian Klima_gtcuat

Hi @Andy Holt and sorry for liking your post. OMG....

Seriously, though.

To save your pages, you can use the customize option in space export menu - export just your pages as an XML file, and once you delete the space, reimport it back.

If space deletion doesn't work, you can try to delete the root page with the 'delete child pages option'.

Then you can try using Confluence API and script your way out or you can try a marketplace app:

I found these resources...

Back in day, after a huge migration of thousands of docs to a Confluence Server environment, the team I work with ran a cleanup and I'm pretty sure they did not do that manually (we're talking hundreds if not thousands of spaces).

EDIT: On Cloud, those drafts end up in 'undefined' section of the space settings (next to Archive, Trash...) I've not worked with server for over 2 years now but again, this might be a clue...

Andy Holt_r28i2n

Would be nice - but it was in Sept 2023. Doh (again).

Darryl Lee

Wait... wait. You have AWS RDS Aurora PostgreSQL.

Did you explicitly disable automated backups/snapshots? :-{

Oh wait... that's not possible. But I guess you could've reduced retention period to 1 day.

Their docs say:

You can't disable automated backups on Aurora. The backup retention period for Aurora is managed by the DB cluster.

Andy Holt_r28i2n

No, I haven't disabled automated backups. It's true that we don't have them going back that far, but what I meant was, there's no way we would wind back our Confluence that far (even if we had backups that old), and lose all the work between Sept 2023 and now across all spaces.

Thanks for reading my tale of woe though. :-) I'm now getting help from Atlassian support with removing the space anyway.

Andy Holt_r28i2n

Atlassian support have been very helpful (thank you Shekhar!).

Mostly, this has involved using resolution 2 from this KB article. It's basically, stop Confluence, run a bunch of SQL queries to manually remove the space, clear out the node's index files, restart Confluence, let it automatically rebuild the index (I did not know it did this!) and lastly trigger the scheduled job to repair (rebuild in this case) the confancestors table.

Result, a much smaller, quicker, happier Confluence service. But that's been two weeks' work for me!

Kristian Klima_gtcuat

YAY! Glad to hear that @Andy Holt .

Maybe it's time to look for a solution that will allow you to make your content public without opening your Confluence spaces to anonymous access

Andy Holt_r28i2n

Anon read was always the intent in this space, just not anon edit! :-)

Kristian Klima_gtcuat

The road to hell hell is paved with good permissions intentions

Humans see (a setting), humans do (play with the setting).

Stay safe!

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

confluence

data-center

confluence-data-center