I deleted a repository and set an URL for a redirect page that will be displayed. I missed the information that the redirect page will be public.
Suggestion for Product Improvement
I'd like to suggest an improvement to Bitbucket:
Can you make "We'll display this link to anyone who visits the deleted repository" more prominent, maybe use warning colours and change the wording (for example use "public accessible" rather than "to everyone who visit the deleted repository")?

Solution to remove/change Redirect Page
Anyways, for those who want to get rid of the redirect page (or update the URL):
Create a new repository with the same name and then delete the repository again without setting the URL (or use an updated URL).