Feature - Confluence testing

Description

Included with GINT is an extension that provides simple testing of Confluence with particular focus on testing macros that render content on Confluence pages.

This extension is also an example of how the GINT framework can be extended to provide integration testing of diverse environments. Confluence macro testing involves creating or updating Confluence pages with content using the macro(s) being tested. The rendered content can be analysed for correct behavior. A specific benefit of this approach is the ability to automate most testing and inter-mix manual testing or verifications that are too difficult or not cost effective to fully automate.

Wiki

Confluence testing is easiest using wiki markup, For testing legacy wiki macros, install Wiki Markup for Confluence on the Confluence server being tested.

Confluence 4.x or higher

  • By default, you can continue to use wiki markup
  • Storage format can optionally be specified
    • testcase.storageFormat = true
  • To test legacy (migrated) macros continue to work in wiki markup, run some or all of your test with parameter wiki=true
    • Recommend running the same test twice, the second time with -Dwiki=true
      Use a different space name for the wiki test - example
    • If you have testcases that should not be run in legacy mode, condition those testcases to not be included when wiki is set
  • Legacy (migrated) macros are tested by GINT automatically embedding your macro markup inside of the Wiki Markup for Confluence macro that prevents your macro from being automatically converted



How does it work?

Testcases define a subset of page content related to the macro being tested. The GINT extension uses this information to generate a page using the Confluence CLI. Test pages are rendered using the Confluence CLI. Rendered macro data and page data is made available to be analysed using standard GINT capabilities like automatic comparisons

By default, testcases use a special Confluence macro testing command generator that produces the necessary CLI commands based on relatively simple testcase parameters relating to Confluence testing. Additionally, normal Confluence CLI testcases can also be used for setting up other Confluence pages, attachments, or anything else supported by the Confluence CLI - see Command generator for Confluence. The command generator is provided as a convenience to make testcases more readable and easier to understand, however, normal CLI commands can also be specified directly using the cmd parameter.

Usage

  1. Functions are provided to do some standard things for macro testing
    • getAddSpaceTestcase and getRemoveSpaceTestcase - normally used for setUp and tearDown
    • getSpaceName and setSpaceName - normally just use the default name constructed from prefix and test name
    • getSpaceKey and setSpaceKey - normally just use the default key constructed from the space name
    • getParentName and setParentName - page's parent, normally just use the default (home)
    • isServerAvailable and getServerInfo - for getting version and other information about the configured Confluence server
  2. noTearDown is set to true by default so that the Confluence space and pages that are created by the testing are preserved after the test completes. This helps for doing manual verifications and debugging. An explicit tearDown target can be run to remove the space.
  3. See dependencies above. GINT requires the confluenceCli property be defined and point to the CLI - normally set in the user's or project's gint.properties file.
  4. Normal Confluence CLI actions work by default together with special testcase definitions:
    • macro - specifies a macro to be tested, the page content is generated automatically based on the macro and other testcase parameters. The output from the macro is available for analysis using standard testcase parameters like datafailData.
    • render - specifies a name that either:
      • references another testcase name - the test will render the page associated with the referenced testcase. Either name or ext parameter must also be provided so the testcase has a unique name. ext is used together with the render parameter to construct a testcase name. 

        Example
        gint.add('example', [
            [macro: 'info', ext: 'WithTitle', description: 'Creates a page with info macro that has a title',
                parameters: [title: 'My title'],
                data: ['<b>My title</b>'],
            ],
            [name: 'exampleRender', description: 'Render the page created by the infoWithTitle testcase',
                render: 'infoWithTitle', 
                data: ['<b>My title</b>'],
            ],
        ])
      • provides a partial name (used together with ext parameter to form a testcase name) for creating and rendering a page based on content provided as a testcase parameters
    • See detailed information - Command generator for Confluence and Command generator for Confluence Macro

Example

This example is from the GINT integration test. It includes code to bypass the test if suitable Confluence configuration is not available allowing allowing the integration test to run error free on any system.

GintForConfluence

  • includeTool << org.swift.tools.GintForConfluence includes the gint framework with special extensions for Confluence testing.

Simplest macro test

The simplest testcase is [macro: 'recently-updated'] which tests that the macro runs without an error. By default, macro errors will automatically cause the testcase to fail.

confluenceTest.gant

includeTool << org.swift.tools.Helper             // helper object for general utility functions
includeTool << org.swift.tools.GintForConfluence  // gint framework with extensions for Confluence macro testing

gint.initialize(this) // required

// cli or confluenceCli must be set
if (!gint.isServerAvailable()) {
    message 'warning', 'Confluence server is not configured/available for this test. Bypass testing.'
} else {
    gint.addSetUp(gint.getAddSpaceTestcase())       // creates a default space for pages
    gint.addTearDown(gint.getRemoveSpaceTestcase()) // required when addSpaceTestcase is added

    gint.add('recently-updatedMacro', [
        [macro: 'recently-updated'],  // tests that the recently-updated macro is available on this confluence instance!
    ])

    gint.add('infoMacro', [
        [macro: 'info', body: 'my body', data: 'my body'],
        [macro: 'info', ext: 'Title', parameters: 'title=My title', body: 'my body', data: ['My title', 'my body']],
        [macro: 'info', ext: 'NoIcon', parameters: 'icon=false', body: '', failData: 'information.gif'],
        [macro: 'info', ext: 'Compare', description: 'Compare macro output between releases', body: 'my body', compare: true],
    ])

    gint.add('errorHandling', [
        [macro: 'include', ext: 'NotFound', description: 'macro errors are automatically detected - page not found',
            expected: -1,
            parameters: 'NOT_FOUND|',
            data: [
                'The included page could not be found',
            ],
        ],
    ])
}
gint.finalizeTest() // final preparations for running tests

Run

= = = = = =   confluencetest started at Sun Sep 12 14:07:50 CDT 2010   = = = = = =

     [info] Confluence Server Info.version  . . . . . : 3.1.0
     [info] Confluence Server Info.build  . . . . . . : 1722
     [info] Confluence Server Info.url  . . . . . . . : http://localhost:8107
     [info] space name  . . . . . . . . . . . . . . . : zconfluencetest
    [start] removeSpace as tearDown
   [ending] removeSpace ignoring failure
 [complete] removeSpace - 1.326 secs

    [start] addSpace
   [ending] addSpace
 [complete] addSpace - 1.139 secs

    [start] recently-updated
    [start] info
    [start] infoTitle
    [start] infoNoIcon
    [start] includeNotFound
   [ending] recently-updated
 [complete] recently-updated - 2.797 secs

   [ending] info
 [complete] info - 2.806 secs

   [ending] infoTitle
 [complete] infoTitle - 2.712 secs

   [ending] includeNotFound
 [complete] includeNotFound - 2.735 secs

   [ending] infoNoIcon
 [complete] infoNoIcon - 2.865 secs

     [info] Successful testcases  . . . . . . . . . . : 5    <<< TEST SUCCESSFUL
     [info] Total testcases . . . . . . . . . . . . . : 5
     [info] Elapsed run time  . . . . . . . . . . . . : 8.444 secs

= = = = = =   confluencetest completed at Sun Sep 12 14:07:58 CDT 2010 = = = = = =

BUILD SUCCESSFUL
Total time: 9.59 seconds

Example - test attachments macro

Add another section to the example above. This shows a more complicated example. In this case we add an attachment to the page and verify the behavior of the attachments macro.

def attachmentName = 'MyAttachment.txt'
def attachmentPage = "--space ${gint.getSpaceName()} --title attachments"
gint.add('attachmentsMacro', [
    [macro: 'attachments'],  // creates attachments page
    [action: 'addAttachment', parameters: attachmentPage + " --name ${attachmentName} --content 'x'", depends: ['attachments']],
    [action: 'renderPage', file: true, parameters: attachmentPage, depends: 'addAttachment', output: [data: attachmentName]],
    [action: 'removeAttachment', parameters: attachmentPage + " --name ${attachmentName}", depends: ['renderPage']],
    [action: 'renderPage', ext: 'AfterRemove', file: true, parameters: attachmentPage, depends: 'removeAttachment',
        output: [failData: attachmentName]],
])

Run

gant -f confluenceTest.gant

= = = = = =   confluencetest started at Sun Sep 12 13:59:08 CDT 2010   = = = = = =

     [info] Confluence Server Info.version  . . . . . : 3.1.0
     [info] Confluence Server Info.build  . . . . . . : 1722
     [info] Confluence Server Info.url  . . . . . . . : http://localhost:8107
     [info] space name  . . . . . . . . . . . . . . . : zconfluencetest
    [start] recently-updated
    [start] info
    [start] infoTitle
    [start] infoNoIcon
    [start] includeNotFound
    [start] attachments
   [ending] attachments
 [complete] attachments - 2.752 secs

    [start] addAttachment
   [ending] recently-updated
 [complete] recently-updated - 3.496 secs

   [ending] info
 [complete] info - 3.773 secs

   [ending] infoTitle
 [complete] infoTitle - 3.726 secs

   [ending] infoNoIcon
 [complete] infoNoIcon - 3.606 secs

   [ending] includeNotFound
 [complete] includeNotFound - 3.475 secs

   [ending] addAttachment
 [complete] addAttachment - 1.434 secs

    [start] renderPage
   [ending] renderPage
 [complete] renderPage - 0.922 secs

    [start] removeAttachment
   [ending] removeAttachment
 [complete] removeAttachment - 0.909 secs

    [start] renderPageAfterRemove
   [ending] renderPageAfterRemove
 [complete] renderPageAfterRemove - 1.139 secs

     [info] Successful testcases  . . . . . . . . . . : 10    <<< TEST SUCCESSFUL
     [info] Total testcases . . . . . . . . . . . . . : 10
     [info] Elapsed run time  . . . . . . . . . . . . : 10.801 secs

= = = = = =   confluencetest completed at Sun Sep 12 13:59:18 CDT 2010 = = = = = =

BUILD SUCCESSFUL
Total time: 11.94 seconds

Example - using render support

The example above can be improved and simplified using the render support. The code is more concise, but more importantly, the render support automatically handles the same attachment processing that a macro testcase does to make the rendered macro data available for analysis. This works because the render references a macro testcase. This means that the data comparisons are done only on the rendered attachments macro data and not the entire rendered page as in the first example.

def attachmentName = 'MyAttachment.txt'
def attachmentPage = "--space ${gint.getSpaceName()} --title attachments
gint.add('attachmentsMacro', [
    [macro: 'attachments'],  // creates attachments page
    [action: 'addAttachment', parameters: attachmentPage + " --name ${attachmentName} --content 'x'", depends: ['attachments']],
    [render: 'attachments', ext: 'AfterAdd', output: [data: attachmentName]], // automatically depends on attachments testcase
    [action: 'removeAttachment', parameters: attachmentPage + " --name ${attachmentName}", depends: ['attachmentsAfterAdd']],
    [render: 'attachments', ext: 'AfterRemove', depends: 'removeAttachment', failData: attachmentName],
])