Uploaded image for project: 'JIRA Command Line Interface (CLI)'
  1. JCLI-1229

Run CLI action runFromCSV taking longer each time to import issues into jira

    Details

    • Type: Support Request
    • Status: Closed
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Sprint:

      Description

      We are using CSV Macro on a confluence page to import data into jira. While the csv macro works fine for smaller load but with larger files and consecutive user calls, the import gets slower and slower. We did a few test runs using a number of different users each uploading a same file of 500 issues from the confluence page.
      It turns out that with each import run the execution time increase and it took a lot more time to execute the last few import runs. I have summarised the times below. We also observed high CPU usage during this time. Unfortunately we have not been able to figure out the root cause as yet and therefore would like you to share your thoughts as to why its taking longer when more calls are initiated to import data, even though the import runs sequentially and not in parallel.

      Also find attached the snapshots of the high CPU and memory usage

      Test Run Summary (times are reported based on the first and the last issue created based on the log file generated)

      Users Time Taken Time taken to create One Issue Note
      User One 5 mins 0.58 sec  
      User Two 9 mins 1.04 sec  
      User Three 9 mins 1.04 sec  
      User Four 12 mins 1.38 sec  
      User Five 13 mins 1.48 sec  
      User Six 16 mins 1.69 sec  
      User Seven 22 mins 2.68 sec  
      User Eight 51 mins 6.18 sec Not all issues were created for this run - 4 errors
      User Nine 1h 44mins 12.48 sec Not all issues were created for this run - 4 errors

      Please note we ensured no other activity was done on the server while the import runs were executing.

      Environment Details

      Jira v7.2.1 on Centos 7 (EC2) - using authentication on crowd
      Confluence 5.10.2 on same server as jira - using authentication on crowd
      JIRA CLI COnnector   6.1.0
      Run CLI Actions for JIRA 6.1.0-SNAPSHOT
      Confluence CLI connector:   6.1.0-SNAPSHOT
      Run CLI Actions in Confluence  6.1.0-SNAPSHOT
      Database Postgres 9.2.18

      Some questions we need answered are

      • Is there a load limit on csv macro when processing more than a couple of import runs?
      • Is there any log kept by the plugins that we can make use of in identifying the cause of slowness?
      • Can we profile the import runs through csv macro somehow?
      • Any thoughts on the high CPU usage We noticed it doesn't go above 90%?

        Attachments

        1. cli-increasing-load.png
          46 kB
        2. error.txt
          59 kB
        3. memory-usage.png
          33 kB

          Activity

            People

            • Assignee:
              bswift Bob Swift
              Reporter:
              asadahmed Asad Ahmed
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: