how do I setup IM server

[Nic Brough -Adaptavist-]

You install one and then set up Bamboo to talk to it, as per docs.  I'm afraid we can't give you a lot more than that, your question doesn't have any specific detail.  Where are you struggling with the docs or process?