Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Deleted user
0 / 0 points
Next:
badges earned

Your Points Tracker
Challenges
Leaderboard
  • Global
  • Feed

Badge for your thoughts?

You're enrolled in our new beta rewards program. Join our group to get the inside scoop and share your feedback.

Join group
Recognition
Give the gift of kudos
You have 0 kudos available to give
Who do you want to recognize?
Why do you want to recognize them?
Kudos
Great job appreciating your peers!
Check back soon to give more kudos.

Past Kudos Given
No kudos given
You haven't given any kudos yet. Share the love above and you'll see it here.

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Bitbucket Event Queue Filled up After Custom Plugin Containing Push Event Handler

Hi,

 

I've developed a plugin that checks each repository push and the process of my custom event listener may run long according to the amount of change within the push. Especially during project's automated merge push scheduled job hours I started getting "RejectedExecutionException" due full event queue I also increased event queue size to 9000 with no luck. So, what is the best way to handle custom event listener codes in order not to fill up event queue, I may convert processing to a new job schedule if there is no limit for scheduled jobs ? Simply trying a native java thread also did not worked.

 

-thank you very much

-yusuf

1 comment

Hi,

Code that listens for events needs to complete fast as there are a limited number of threads handling requests on the queue; holding onto one of these threads whilst you do stuff is really bad and not something that will hit you until your system is under load.

I fell foul of this, trying to make HTTPS connections in a bit code listening for events. It caused the bb_alert table to swell to 35Gb, filled up the disk and brought Bitbucket down.

You can configure the queue size with event.dispatcher.queue.size which we set to 9000 and the number of threads through event.dispatcher.max.threads.

The property event.dispatcher.queue.rejectioncooldown has no effect from what I could see. Once your queue fills up Bitbucket starts generating recursively ever more events hence the db table bb_alert growing to 35Gb. Our only way out of the full queue situation was a server restart.

I take my hat off though to the Bitbucket developers as the log file very helpfully not only tells you the queue is full but gives you a stack trace for each thread thats servicing the event queue showing what it is currently doing. From this we easily identified the rogue plugin.

Our fix was to decouple our time consuming code by adding a fifo queue with a scheduled task. If our queue filled up we just discarded new events and logged an error.

Comment

Log in or Sign up to comment
TAGS
Community showcase
Published in Bitbucket

Calling any interview participants for Bitbucket Data Center

Hi everyone,  We are looking to learn more about development teams’ workflows and pain points, especially around DevOps, integrations, administration, scale, security, and the related challeng...

631 views 7 4
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you