I have thousands of attachments on my JIRA system but I don't think it affects JIRA at all. You can say the same with system performance too.
In addition, your best way to handle this is to assess on what you have:
Base on this, I think you can assess how much growth and till when will your system cope up with the growth. This is just for the size itself not performance,
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG