We're updating the issue view to help you get more done. 

Run CLI action runFromCSV taking longer each time to import issues into jira

Description

We are using CSV Macro on a confluence page to import data into jira. While the csv macro works fine for smaller load but with larger files and consecutive user calls, the import gets slower and slower. We did a few test runs using a number of different users each uploading a same file of 500 issues from the confluence page.
It turns out that with each import run the execution time increase and it took a lot more time to execute the last few import runs. I have summarised the times below. We also observed high CPU usage during this time. Unfortunately we have not been able to figure out the root cause as yet and therefore would like you to share your thoughts as to why its taking longer when more calls are initiated to import data, even though the import runs sequentially and not in parallel.

Also find attached the snapshots of the high CPU and memory usage

Test Run Summary (times are reported based on the first and the last issue created based on the log file generated)

Users

Time Taken

Time taken to create One Issue

Note

User One

5 mins

0.58 sec

User Two

9 mins

1.04 sec

User Three

9 mins

1.04 sec

User Four

12 mins

1.38 sec

User Five

13 mins

1.48 sec

User Six

16 mins

1.69 sec

User Seven

22 mins

2.68 sec

User Eight

51 mins

6.18 sec

Not all issues were created for this run - 4 errors

User Nine

1h 44mins

12.48 sec

Not all issues were created for this run - 4 errors

Please note we ensured no other activity was done on the server while the import runs were executing.

Environment Details

Jira

v7.2.1 on Centos 7 (EC2) - using authentication on crowd

Confluence

5.10.2 on same server as jira - using authentication on crowd

JIRA CLI COnnector  

6.1.0

Run CLI Actions for JIRA

6.1.0-SNAPSHOT

Confluence CLI connector:  

6.1.0-SNAPSHOT

Run CLI Actions in Confluence

 6.1.0-SNAPSHOT

Database

Postgres 9.2.18

Some questions we need answered are

  • Is there a load limit on csv macro when processing more than a couple of import runs?

  • Is there any log kept by the plugins that we can make use of in identifying the cause of slowness?

  • Can we profile the import runs through csv macro somehow?

  • Any thoughts on the high CPU usage We noticed it doesn't go above 90%?

Environment

Status

Assignee

Bob Swift

Reporter

Asad Ahmed

Priority

Major