How to extract the issues in a text format or any other light weight format?

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

I am in the process of pulling out information for 30,000 bugs.

Pulling out this information via the usual HTML, Word, Excel does not scale very well.

Any pointers to extracting the issues in a text format or any other light weight format would be very helpful.

2 answers

1 accepted

0 votes
Answer accepted
Bob Swift OSS (Bob Swift Atlassian Apps)
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

You didn't say why you needed a text format - that would help determine what kind of answer you are looking for. For instance, the JIRA Command Line Interface has a text representation of an issue or CSV representation of a list of issues.

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

Hi Bob, I tried JIRA Command Line Interface and found it is very helpful. Thanks for suggesting me about this. I tried the following command jira --action exportData --project "BUG" --version "2.0.0" -- --file "exportDataProject.txt" but i did not get the exact data which i am looking. Could you please suggest me how to import the all the issues of the project?

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
June 25, 2012

The CLI is limited by the same limits that exist within Jira. You can override this internal limit - see https://answers.atlassian.com/questions/64517/jira-export-50000-issues-to-excel?page=1#64520 for how, but bear in mind that it's going to kill performance while running, and unless you've got some decent hardware, probably fail (although not as badly as Excel, which is far more intensive)

I'd recommend answering Bob's question too - why do you think you want 30k issues in text?

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

Hi Bob, I checked getIssueList action as it is returning issues in text format. But I am getting this error.

"Remote error: com.atlassian.jira.rpc.exception.RemoteException: Error occurred during searching: 'You are not allowed to get a result set of more than 1000 results. Current search returns 24320 results'."

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

legal requires that information as part of a internal audit.

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
June 25, 2012

Then they're wasting your time, and their own. Jira is searchable, auditable and can tell them a lot more than 30k plain text records. Are they really going to pay someone to read all of that? I very much doubt it, they are almost certainly going to either search it, import it into something to help them with their audit, or they're just ticking boxes on a to do list (on that third one, give them a database backup and leave them to it)

I've been here a few times before - demands for audit information are often unqualified because they've come through too many layers of managers. They often have little or no idea what they're asking for, or what they're actually looking for. Ask to talk to one of their auditors directly, or the techies running the systems they'll be using - you'll almost certainly get a much better answer. One good question - "how about we give your auditors read-only Jira accounts?"

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

Hi Nic, Thanks for your comments as they are viable. I will check with my team regarding this.

Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
June 25, 2012

Sorry, they may come across as quite grumpy, and I do not really mean to sound like that.

My point is that it's well worth asking what they're looking for directly - it might save both you and them a LOT of time :-)

On a more positive note, the last time I had auditors ask for something like that, I went straight back with the "what are you looking for" question, and it turned out we could do everything they needed with a simple JQL query to look for a couple of words in comments, description, summary or a couple of custom text fields, followed by them reading the issues with the history (I suspect they used a Word export too). They were delighted not to have to manually search 100,000 records, and I was very happy to add a couple of accounts and not have to do any exports!

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

I heard that it is for internal audit but the actual reason i got now from my team is for a court trial. I am actually trying to confirm that information now. So if they really require the issues in a text file, then i have to work on a solution for this.

I hope the information getting using the command line interface tool would be sufficient, but the restriction of 1000 isssues made me struck. This requires a change in configuration and JIRA restart in order to get all the 30k issues which is not feasible for me at this point. I had raised a flag about this to my manager and waiting for his reply.

Bob Swift OSS (Bob Swift Atlassian Apps)
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 25, 2012

All good discussion here and clearly trying to avoid having to do this is the best approach. But, if you have to, then I would use the getIssueList to a file with the append parameter and a JQL query that returns issues created in a date range (like: created > "-15" AND created <= "-10")) and loop through all intervals using a simple groovy (or other) script or using run action with a manually edited file.

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 26, 2012

Added the parameter jira.search.views.max.unlimited.group = jira-administrators to jira-config.properties file and restarted the server. I executed the getIssueList action but i got ReadTimeOutException.

I tried again and now i am getting the following exception.

Remote error: ; nested exception is:

org.xml.sax.SAXParseException; lineNumber: 384183; columnNumber: 15; XML document structures must start and end within the same entity.

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 27, 2012

Hi Bob/Nic,

Any ideas please?

Thanks, Srinivas..

Bob Swift OSS (Bob Swift Atlassian Apps)
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 27, 2012

Both the timeout and parse exceptions are server problems, probably a result of such a large result. I would go back to my suggestion above about subsetting to reasonable size chucks of data and appending to a single file or produce a weekly or monthly files. You don't have to change the default JIRA settings and it should be equivalent.

srinivasp
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 27, 2012

Thanks for your information.

0 votes
Nic Brough -Adaptavist-
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
June 25, 2012

I'd grab it with xml and whack it through an xslt transform.

Suggest an answer

Log in or Sign up to answer