Is there any way to configure confluence to prvent the use of crawlers, even from users with login access?
You probably want to setup a proxy in front of your Confluence to reject these crawlers. There are many guides online to do this with Apache for example.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.