How do I completely remove an addon at its underlying data?

[Neal Slensker]

I recently installed a 3rd party addon, and made a bunch of changes to configure it, and somewhere in the process doing so I messed up, and now have caused the addon to not work as designed.  I would like to uninstall the plugin, and I tried doing this through the manage addons.  But when I went to reinstall the plugin it automatically remembered my licensing, and custom changes that I know weren't a part of the out of the box configuration. 

How do I completely remove an addon and its underlying data to start over with a fresh install of the addon?

[Fadoua]

I think you will have to check the installed-plugins directory to make sure that it is no longer there, and run a query on the DB as well because some plugins do use their own tables