Hi,
I have a program that uses the REST API to take snapshots of my team's issues a couple of times a day. A snapshot would usually contain several thousand issues.
A snapshot is generated by making a multiple calls to the REST API using JQL, incrementing the startAt value each time by the number of issues returned in the previous call. The program stops when the 'total' number of issues (read from the first response) have been received.
There appears to be a race condition introduced by this iterative process that can cause the program to run forever, or in fact introduce other inconsistencies into the snapshot.
The issue is most easily understood as being a search for all open issues, where a number of issues become closed between reading the initial total and a subsequent attempt to retrieve all open issues. Because some issues have been closed, the attempt to read the initial total will never complete.
Is there a way to handle this race condition?
Community moderators have prevented the ability to post new answers.
I have not had this problem when using a fixed block size. This is how I use it:
block_size = 900 # 1000 is the default maximum block_num = 0 while True: start_idx = block_num*block_size issues = jira.search_issues(jql, start_idx, block_size) if len(issues) == 0: # Retrieve issues until there are no more to come break block_num += 1 for issue in issues: log.info('%s: %s' % (issue.key, issue.fields.summary))
+1
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Can't you just use a timeout thing in order to avoid it?
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You could even set the timeout amount of time proportional to the total number of issues it has to process.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Breaking out of the loop isn't really the problem. It's more the fact that each request is performing a new search and returning the nth page. If anyone updates a JIRA from one request to another in a way that influences the order/number of tickets returned by the search, we end up with either missing or repeated JIRA issues in the assemblage of all the request results. What we want to do is get the results of all the requests as a transaction, or using an asoftime, or get them all in one go without having to iterate, or similar...
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
...or keep track of the last issue processed and if its key doesn't change to a different one in the following 10 seconds (for example), then finish the iteration.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Ok, I understand.
Wouldn't it be good to subscribe to that search to receive the snapshot of the issues twice/day by email and parse its content?
The email can be triggered by clicking on a button as well...
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
What is the purpose for collecting those snapshots?
If it is just for keeping them on a history of archived issues, then I would suggest using filter subscriptions instead, so that you are sent by email a snapshot with the issues and columns you want to store.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Community moderators have prevented the ability to post new answers.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.