“RuntimeError: dictionary keys changed during iteration” when attempting to call add_attachment

Arun Ajay December 6, 2019

I am attempting to just add a CSV file to my issues as a test, but I keep receiving the error:

```
RuntimeError: dictionary keys changed during iteration
```


Here is the code (I've removed the parameters for server, username and password:
```
from jira import JIRA

options = {"server": "serverlinkgoeshere"}
jira = JIRA(options, basic_auth=('username', 'password'))
issuesList = jira.search_issues(jql_str='', startAt=0, maxResults=100)
for issue in issuesList:
with open("./csv/Adobe.csv",'rb') as f:
jira.add_attachment(issue=issue, attachment=f)
f.close()
```
I'm at a loss, I'm not changing any dictionary keys in my code. Here is the full error message:

```
Traceback (most recent call last):
File "C:/Users/USER/PycharmProjects/extractor/main/jiraCSVDupdate.py", line 8, in <module>
jira.add_attachment(issue=issue, attachment=f)
File "C:\Users\USER\AppData\Roaming\Python\Python38\site-packages\jira\client.py", line 126, in wrapper
result = func(*arg_list, **kwargs)
File "C:\Users\USER\AppData\Roaming\Python\Python38\site-packages\jira\client.py", line 787, in add_attachment
url, data=m, headers=CaseInsensitiveDict({'content-type': m.content_type, 'X-Atlassian-Token': 'nocheck'}), retry_data=file_stream)
File "C:\Users\USER\AppData\Roaming\Python\Python38\site-packages\jira\utils\__init__.py", line 41, in __init__
for key, value in super(CaseInsensitiveDict, self).items():
RuntimeError: dictionary keys changed during iteration
```

References:
Jira add_attachment example:
https://jira.readthedocs.io/en/master/examples.html#attachments

add_attachment source code:
https://jira.readthedocs.io/en/master/_modules/jira/client.html#JIRA.add_attachment

3 answers

3 votes
Chuck_Phillips February 18, 2020

The bug is in the jira library.  From .../jira/utils/__init__.py:

    for key, value in super(CaseInsensitiveDict, self).items():
if key != key.lower():
self[key.lower()] = value
self.pop(key, None)

Should be something like:

    for key in [x for x in super(CaseInsensitiveDict, self).keys() if x != x.lower()]:
        self[key.lower()] = self.pop(key, None)

(Iterate over a copy of the keys that need to be modified instead of over the dict to be modified.)

Modifying a dict while iterating over it has always been a bug, but Python 3.8 enforces this.

https://cito.github.io/blog/never-iterate-a-changing-dict/

Lance Reichert March 10, 2020

Any timeline on implementing suggested fix?  Being able to use 3.8 would solve some dependency hell I'm currently facing.

John Tse May 21, 2020

Still looking to see if there is a fix as of end of May 2020

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 22, 2020

The fix is to stop using code that does the wrong thing, as given December 2019

Jere P May 29, 2020

The official fix hasn't been merged yet but exists here https://github.com/pycontribs/jira/pull/877/commits/9caed80a031421182102d64ea0d4645947fd2e64

They essentially iterate over the dictionary looking for non lowercase key names, then in the next loop they recreate the key in lowercase and remove the one with uppercase. Two separate loops, rather than modifying the dict while also iterating it.

You need to modify the site-packages/jira/utils/__init__.py file and make the changes you see at the link above, until it gets merged and pushed out officially.

Like Ana Bennett likes this

Hello, @Chuck_Philips. Thanks! It works.

1 vote
Arun Ajay December 23, 2019

Solution. Changed Python Interpreter from 3.8 to 3.7

 

Hope this helps someone else in the future. 

Raz February 27, 2020

Helped me today, thank you!

Assaf Cohen March 3, 2020

What if i can't? (the code runs on a remote vm not under my management)

Renato Teixeira April 30, 2021

This can't be an accepted solution.. I proposed a monkey patch for someone in need.

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
April 30, 2021

I agree - the problem is not the version of Python (nor is it in the Jira API), the problem is in the code, and your patch corrects it.

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
December 7, 2019

You'll need to rework your python code so that it does not alter the object while it is running over it.

This is a slightly different error, but is the same solution - https://hackernoon.com/filtering-dictionary-in-python-3-3eb99f92e6ee

koskov May 15, 2020

Guys, it's really quite weird especially in case Ubuntu 20.04 ('Python 3.8.2' version by default). I need to do custom migrations for some project, but I can't do this due to described problem with Python 3.8.2. So, it would be really helpful to get proper solution ASAP! Please, check Chuck_Phillips suggestion. Is it suitable? Thank you in advance!

koskov May 15, 2020

@Nic Brough -Adaptavist- can you please advise about this part:

from jira import JIRA
jira = JIRA(options, basic_auth=(user, apikey))
project = jira.create_project('TTTP', 'Temporary Toast Test Project')
print(project)

After its execution I get an error also:

'RuntimeError: dictionary keys changed during iteration' 

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 15, 2020

Your code is altering a collection while the collection is being iterated over.

  • Do not do this
  • See the link I gave to see why it's wrong and how to do it the right way
Jere P May 29, 2020

@Nic Brough -Adaptavist-  why are you telling people who are simply using an API provided to them, to stop using it? It is up to the developers to officially fix this problem.

The problem lies in the Jira module, not any persons individual code presented here. Its not their code, its the API code that is behaving badly.

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
May 30, 2020

You need to be clear about what you mean by API.

You need to be clear that there is nothing wrong with the Jira API.  The problem is with the Python module.  Until the Python (module) developers fix that, the only useful advice here is "don't use functions that are broken", or, better, "use the calls the right way"

Suggest an answer

Log in or Sign up to answer