After doing some digging, seems like the "Schedule Issues" permission is what controls adding/ removing backlogs into Active Sprint. However by removing this ability for some users, will it also remove it for prioritizing the backlog?
Example Scenario: The PO needs the ability to prioritize the backlog, but not the ability to add/ remove backlogs into an Active Sprint as it is the Scrum Master who protects the Active Sprint in order to prevent overloading capacity.