Upcoming changes: Introducing per issue limits for list fields in Jira Issues

[Pablo Culebras _Jodocus_io_]

Good morning Atlassian Community,

Right after this announcement, we were approached by one of our customers who wanted to create an early detection system for issues with a worklog count approaching 5,000 entries, using automation rules.

We would like to share this approach with you all to make the transition smoother. Contrary to some proposals, our solution does not require working with Jira's REST API, thus avoiding the storage of access tokens within your rules.

Our solution consists of two rules: the first rule captures the "Work logged - Created" event and forwards the issue information to the second rule via a webhook. This second rule contains the logic of your process (this can be anything you want: sending a notification, creating a new issue, triggering other automation rules, etc.).

Here are the details of both rules:

1 - Worklog Count - Capture Event and Forward to Webhook

This rule is straightforward:

1. Trigger: Capture the "Work logged - created" event
2. Send a web request:
   • Use a placeholder web request URL at first. You will enter the real URL after creating the second rule.
   • Method: POST
   • Web request body: Issue data (Jira format)
   • Headers: Content-Type: application/json

Sending the issue data in Jira's format ensures that the total number of worklogs will be forwarded to the automation rule behind the webhook. Now, let's check the second rule.

2 - Worklog Count - Capture Webhook and Send Email

This rule has a few more components but is fairly simple to set up:

1. Trigger: Incoming webhook
   1. Webhook URL: Replace the placeholder from the first rule with this URL.
   2. Execute this automation rule with: No issues from the webhook. The issue information is already contained within the webhook's request body.
2. Create variable: The variable `worklogCount`will extract and hold the value of the worklog total, using the following smart value expression `{{webhookData.issue.fields.worklog.total}}`.

After that, you can add your own logic. In our customer's case, they wanted to get notified after the number of entries exceeds 4,500.

This approach has a limitation, though. Actions that act on an issue (edit, comment, transition) may not work, since the trigger does not load the issue in the request body into its context. Further research is needed to determine whether a combination of webhook parameters is possible, thus allowing a wider range of actions.

I hope you find this solution helpful. I'm looking forward to reading your comments and feedback on it.

Best regards,
Pablo Culebras
Jodocus GmbH

P.S.: This may be of interest to you @Timo Kinder @Julia Foden 

[Paul York]

Our support request with Tempo has now failed to identify a resolution. Tempo indicated that there are internal discussions but then directed me to subscribe to product updates. Who knows if our systems will break because of this change at the end of this month.

With no way to bulk-move worklogs for JIRA items that are constantly used by our business, this change will break something that works today. If it breaks for our small organisation, who knows what it will do for those larger organisations using Tempo that haven't read/seen this announcement or where the Atlassian email notification fell into a busy or unread mailbox?

I'd echo earlier requests; @Shiraine Liu, can you please provide feedback to your team on the problems with this proposed rollout and pause it until a proper solution is identified? 

[Julia Foden]

Thank you @Pablo Culebras _Jodocus_io_ for your interesting post. I've never used incoming webhook as an automation trigger before.

Thank you also @Shiraine Liu for talking to me earlier. Noting what you told me that until there is a facility for us to 'self-serve' to find tickets approaching the 5000 limit, that we can raise support tickets to request the list of issue ids.

I've also been looking at deleting worklogs older than X months and if this is possible by automation / by API / in Tempo. I know other people here have looked at this as well (or similar). Tempo bulk delete works on 1000 max and does not work on approved worklogs. The only API available is for deleting single worklogs only. And automation can't help here. Am I missing anything? I think there is no solution that will allow us to continue using the existing issue keys - right?

[Stuart Nealis]

@Julia Foden if you look on the second page of comments here, @Maor Levinas  posted what Tempo support suggested as a workaround. If you change the period approval frequency it reopens all worklogs, allowing you to move/delete as needed, then you can change the frequency back and they close again. I will be trying this as well this week in the hopes that we too can retain our existing issue keys. Good luck!

[Paul York]

Hi @Stuart Nealis, the problem with the suggested Tempo bulk edit workaround is that if there is any worklog associated with an account that is now closed or archived the move fails.  Without drilling into each worklog, or filtering specifically by the accounts that are no longer active you can't easily see what may cause the move to fail. 

If you haven't changed your accounts, or you can re-open all of your old accounts, this might be a possible option.

[Maor Levinas]

Hi @Paul York 

You can allow activities on closed accounts by changing the following setting:

Kind regards,
Maor

[Paul York]

Hi @Maor Levinas - that's not a bad shout and one our Tempo support agent hadn't called out.

The only issue I see is that once this box is ticked, it makes all closed and archived accounts selectable for users logging time today so it's not what we really want to do.

If it's our only option, we will try this out-of-hours and apply this setting before attempting to bulk move a number of worklogs before reverting back.

[Stefan]

Hi, 

In an earlier comment, I shared a solution to get the worklog count in eazyBI, which was incorrect. That method was only available for Jira Server/DC. 

This is how you can solve this with eazyBI on Jira Cloud.

Find affected issues

To import the worklog count on Jira Cloud with eazyBI, use global advanced settings to create a new worklog counter measure.

Example: 

[jira.customfield_wlcnt]
name = "Worklog counter"
data_type = "integer"
measure = true
worklog = true
javascript_code = '''
if (worklog.id) {
worklog.customfield_wlcnt = 1
}
'''

Import the new field as a measure in the source data import. And add the measure in a report on issue level dimension. 

I implemented this on all our customer's instances that use both Tempo Timesheets and eazyBI, and it resulted in identifying multiple issues that were either above or close to 5000 worklogs, that might have otherwise been overlooked. 

For the workaround, we use the most elegant method proposed by @Pablo Culebras _Jodocus_io_ 

Regards,

Stefan

[Julia Foden]

Thanks @Stuart Nealis and @Maor Levinas for the answer from Tempo.

Stuart please report back how you get on!

I am uncomfortable about Tempo's suggestion for a couple of reasons:

* Can we believe them that when the period configuration is reverted that it will re-close the reopened timesheets? And ONLY the reopened timesheets

* Period config changes can only be done by site admins. In my site the 'periodic deletion of old worklogs' method would only be feasible if I could outsource it to dept managers with the appropriate project permissions. I could do that with @Pablo Culebras _Jodocus_io_ 's webhook-automation to send them emails but not if it depends on site admins to be changing the period configuration. And it's so messy!

Even if we disregard the problem of messing with the period configuration, another problem is how manual and time-consuming this is for the first mass deletion that must be done before 31st March. I have approx 80 tickets with over 5000 worklogs (or close) and some of them have over 150k. I don't have time to be going into each one and deleting worklogs 1000 at a time. 

@Shiraine Liu if I requested Atlassian to delete all worklogs from before a certain date from a certain list of issues could you do that? In a way that does not require messing with the period configuration and that includes inactive users and closed accounts?

[Stuart Nealis]

Hi @Julia Foden

I spent 2.5 hours last night (off-hours just in case) deleting ~20,000 old worklogs from just a single issue. There is no smooth way to just delete 1,000 at a time, as you have to select date ranges first and then try and bulk edit the worklogs that fall in that range. However, if the number exceeds 1,000, you can't perform any actions and have to go back, reselect a smaller date range, and try again. If there is a better way, I don't know it, but wish I did, as this required me to delete just a month at a time...

Each cycle of deletes takes 11 mouse clicks if done perfectly. I have 40 months of data to remove, on 9 issues, so not only does it feel like carpal tunnel syndrome is on the horizon, it's also nearly insurmountable just from a time perspective. With the number you have, you have no chance of completing it before next week ends (or sanity ends, whichever is first!)

The worklogs would only delete if I switched the approval frequency, and I tested by switching it back to weekly and it errored out due to closed periods, so that part does seem to work.

I opened a ticket with Tempo support and asked if they could delete worklogs from specific issues in specific time ranges, and I'm awaiting a response.

Edit: Tempo support responded and said they don't have access to delete the worklogs :(

I found a post showing how to use Automation to delete all worklogs, but that doesn't help directly because I want to retain the last 3 months in our use case.

TL:DR the manual effort is only useful if you have a very small number of overages, and we need support from Atlassian and/or Tempo to make this happen.

[Julia Foden]

Hi @Stuart Nealis thank you! Sounds like a painful 2.5 hours but it saves me having to do it :)

And thanks for that post you found about using automation to send requests to Tempo's API. At first glance it looks like it should be possible to specify a date range https://apidocs.tempo.io/#tag/Worklogs I'll have a play with this in some spare time (ha!) and will report back. 

Isn't it madness that we are having to scramble for workarounds like this.

[Beth Chubb]

@Stuart Nealis Tempo do have a data retention setting which will delete worklogs (both the Tempo and Jira ones) after a set amount of time. It does apply to all issues though

https://help.tempo.io/timesheets/latest/configuring-tempo-data-retention.

This is what we're using to clear down some old data and after doing a small test last night, it seems to work.

Also, contact @Shiraine Liu to find out when the change will be rolled out for yourselves as it might not be next week (it all depends on your shard) to give yourself more time.

[Julia Foden]

@Beth Chubb that's a great tip about the data retention setting. Unfortunately doesn't help me because in some of my projects (normal projects, not the time-tracking ones) we need to keep the worklog data for longer periods.

I have been experimenting with using automation to call Tempo's API to delete worklogs within a date range per this article and it's good, you can delete up to 5000 at a time BUT you hit the same problem as trying to do it in the Tempo UI - it doesn't delete worklogs from Approved timesheets. This is the killer for me. 

Someone reported that Tempo can't/won't delete worklogs from specified issues. And Atlassian @Shiraine Liu haven't answered on that from what I can see. 

[Paul York]

Hey @Maor Levinas, Last night, I tried the workaround you suggested and it was largely positive, so thank you for that!

For all others, after changing our Tempo settings to allow users to log time for closed and archived accounts and an hour of watching the Tempo bulk editor "progress bar", I managed to move just over 1,000 worklogs from one issue to another. I think we have brought the number of worklogs on our (currently) one problematic issue just under the magic 5,000 threshold and we've now reverted back to blocking the logging of time to closed and archived accounts.

The Tempo worklog bulk editor couldn't move any work logs for an ex-colleague whose JIRA account is no longer active, which may impact others trying something similar. Luckily, they didn't have that many worklogs on our problematic issue, so it wasn't a big impact; however, it did leave the archiving incomplete.

We have no existing API integration with JIRA, but I think we now need to look at what's been suggested earlier in this thread for monitoring if no automated Atlassian solution is available. I suspect we'll be hitting this limitation again in the next few months and will need to redo this activity regularly out of hours which is a really frustrating prospect.

[Shiraine Liu]

Thanks everyone for your patience so far. We had made the initial change log notice and community post ~6 months ago, hoping it would provide sufficient time to update workflows. However, we acknowledge the feedback indicating that there could have been better visibility, specifically for the worklog limits. The intention was never to provide a mere 2 weeks' notice, and we recognise that this represents a significant change management process for many of our customers.

Therefore, we will delay the enforcement of worklog limits until July 2024 to accommodate additional time to update existing workflows required for worklog limits. For further details on specific dates, please refer to "Worklogs Limit Rollout" section.

The limits for all other issue entities will rollout as per the original timelines.

In addition, we are actively working on an API that will allow admins to query issue IDs that are either approaching the limit or already over the limit, which will be released in April 2024. We will share further updates around this on this community post. 

The community post has also been updated to reflect the same. 

[Paul York]

Thank You!

[Ruben Uyterlinde]

Great to hear you have postponed, because of the change I already tried to find a solution, in the limited time.

Would also be nice if smartvalues were added so we can automations. 

The best value for customers would ofcourse be to postpone it indefinitely ;)

[vaughan_dickson]

Hi

In the event that we attempt a diffrent solution for Worklogs Limits and we decide to delete any Worklog that are for example older that 2months old . The fact that , for every Worklog entered there is corresponding deleted transaction. Will this count towards reducing or increasing the limit ?

regards.

[Julia Foden]

Hi @Shiraine Liu 

It would be helpful if you could provide any updates on APIs or any other solutions being developed by Atlassian.

I have requested Tempo for an API to delete multiple worklogs on a ticket, and for it to work on both open and approved worklogs. Failing that an API to change the period configuration. 

If anyone has any info from Tempo on what they are developing, let us know please!

[Iona Augustine]

@Maor Levinas Changing the period configuration from weekly to monthly doesn't reopen all timesheet automatically. It doesn't 'reopen' the approved timesheets to be moved. 

@Shiraine Liu  As some of the members here, our company would like to retain the original internal issue key and willing to compromise by moving the worklogs in bulk manually (periodically) to keep the limit under 5K.

The problem is moving worklogs in bulk will not work for approved timesheet and there is no way to 'reopen' or 'unapproved' these timesheet in bulk (then 're approve' them). The suggested way by @Maor Levinas  on Mar 15 didn't work, unless I missed something.

Thanks.

[Alexandre Analista, Adm Jira e Especialista]

Unfortunately, these limitations imposed by Atlassian are making use cases and the implementation of business rules unfeasible for several customers. I see these limitations in Jira Cloud as a reduction in technology resources and an impediment to deploying customer solutions in Jira.