possible full reindex schedule with scriptrunner?

[ChristopherChilds]

According do the documentation it if a good idea to perform a full re-index once in a while, say once every two months. 

But typically this can only be performed when the system is not needed. This would ideally happen at the weekends in the evening. As no-one wants to open the Work Laptop to click full re-index, is it possible to automate it somehow? maybe with script runner?

I have searched and I do not see anythig for this, only for re-indexing issues in Projects, and not for the configuration.

[Clark Everson · Accepted solution]

Hi @ChristopherChilds 

I would not recommend trying to script re-indexing. I see you are on Server, so for now on your version one part of it is something could go wrong and you would want to make sure you have someone on call for it, thus having that person click the button is the way to go.

If you choose to stay locally hosted and not move to Cloud by 2024, then with data center it supports multiple nodes with zero down time. However, you need to take one node offline and sync it back up.

It is possible to automate through something like Ansible but I would recommend considering which path (Cloud versus Data Center) you plan on going with before investing time in automation of re-indexing as well as reporting you will need in case it fails.

Best,
Clark

[ChristopherChilds]

Many thanks for the answer.

I had considered indexing to be a benign thing to do, and did wonder why there was no automation. This makes sense.

I have seen there a list of issues to look out for when performing re-index.

https://confluence.atlassian.com/jirakb/troubleshoot-index-problems-in-jira-server-203394752.html

Out of interest, how has your personal experience been with full re-indexing?
Can you provide some estimates of when to perform this.

This is a brief summary of our Server (I am not sure what relevant information is useful):

Users 1600

Projects: 371 (of which around 220 are considered active)

[VF · Accepted solution]

Hi,

I have developed a ScriptRunner script that starts a background reindex using the reindex REST APIs. The script can then be set up as a SR job that runs e.g. once a week. 

For a full reindex just replace the type=BACKGROUND_PREFERRED with type=FOREGROUND

Hope it helps.

/**

 * Start Jira reindex through a REST API call 

 * The script first verifies that there is no other reindex running

**/

import com.atlassian.sal.api.ApplicationProperties

import com.atlassian.sal.api.UrlMode

import com.atlassian.sal.api.net.Request

import com.atlassian.sal.api.net.TrustedRequestFactory

import com.onresolve.scriptrunner.runner.customisers.PluginModule

import groovy.json.JsonSlurper

import groovyx.net.http.URIBuilder

import com.atlassian.sal.api.net.Request

@PluginModule

TrustedRequestFactory trustedRequestFactory

@PluginModule

ApplicationProperties applicationProperties

final reindexAPI = '/rest/api/2/reindex?type=BACKGROUND_PREFERRED'

final reindexStatusAPI = '/rest/api/2/reindex/progress'

def urlIndexStatus = applicationProperties.getBaseUrl(UrlMode.CANONICAL) + reindexStatusAPI

def urlStartIndex = applicationProperties.getBaseUrl(UrlMode.CANONICAL) + reindexAPI

def requestIndexStatus = trustedRequestFactory.createTrustedRequest(Request.MethodType.GET, urlIndexStatus)

def requestStartIndex = trustedRequestFactory.createTrustedRequest(Request.MethodType.POST, urlStartIndex)

def host = new URIBuilder(urlIndexStatus).host

log.info("Host: " + host)

def indexRunning = false

try {

    requestIndexStatus.addTrustedTokenAuthentication(host)

    def responseBody = requestIndexStatus.execute()

    def responseAsMap = new JsonSlurper().parseText(responseBody) as Map

    

    if( (int) responseAsMap['currentProgress'] < 100){

        log.info("Index in progress. Percentage complete: "+ responseAsMap['currentProgress'])

        indexRunning = true

    }

}catch(def error){

    log.info("No index running")

}

if( !indexRunning ){

    log.info("No reindex in progress. Attempt to start a new reindex")

    

    try {

        requestStartIndex.addTrustedTokenAuthentication(host)

        def responseBody = requestStartIndex.execute()

        def responseAsMap = new JsonSlurper().parseText(responseBody) as Map

        log.info("Index started")

        log.info(responseAsMap)

    }catch(def error){

        log.info("Reindex not started. " + error)

    }

}else{

    log.info("NO new reindex started")

}

[Jakob F_ Gormsen]

Thanks

[Tomas Vaicekauskas]

Hello @VF ,

Maybe you can help with Jakob error? Comment bellow. Maybe you know from the experience the root cause of that 411 error. 

[VF]

It looks like the API calls are failing. And the explanation I could find in atlassian community is that this could be a proxy problem solvable by setting the content length of the call to 0. Please read more here:
https://community.atlassian.com/t5/Jira-questions/411-error-while-trying-to-POST-to-an-api/qaq-p/1064740

So what you could try is set an empty body to the request. For example by adding the following line right below line 58 requestStartIndex.addTrustedTokenAuthentication(host) :
requestStartIndex.setRequestBody("")

However I can not test if this works as I can not reproduce the error on our servers. Hope this helps.

[Jakob F_ Gormsen]

Thanks for the replay.

I have changed to this, and now I get another Error code.

if( !indexRunning ){

    log.info("No reindex in progress. Attempt to start a new reindex")

    try {

        requestStartIndex.addTrustedTokenAuthentication(host)

        requestStartIndex.setRequestBody("") //Added 

        def responseBody = requestStartIndex.execute()

        def responseAsMap = new JsonSlurper().parseText(responseBody) as Map

        log.info("Index started")

        log.info(responseAsMap)

 

    }catch(def error){

        log.info("Reindex not started. " + error)

    }

 

}else{

    log.info("NO new reindex started")

}

Error that I get:

2022-06-24 13:51:41,777 INFO [runner.ScriptBindingsManager]: Host: jira.customer.com

2022-06-24 13:51:41,793 INFO [runner.ScriptBindingsManager]: No index running

2022-06-24 13:51:41,793 INFO [runner.ScriptBindingsManager]: No reindex in progress. Attempt to start a new reindex

2022-06-24 13:51:41,809 INFO [runner.ScriptBindingsManager]: Reindex not started. com.atlassian.sal.api.net.ResponseStatusException: Unexpected response received. Status code: 415

Br Jakob

[VF]

Hi Jakob

Please replace the line you have just added with these 2:

requestStartIndex.addHeader("Content-Type", "application/json")

requestStartIndex.setRequestBody(new JsonBuilder([]).toString())

These should handle the content format.

[Jakob F_ Gormsen]

Hi

And it works :-)

I had to import the class, though.

import groovy.json.JsonBuilder //Added

Thank you for helping, it's appreciated.

Br Jakob

[VF]

Hi Jakob

Yes, the JsonBuilder library is needed. Glad you figured that one out and that it works. 

[Jakob F_ Gormsen]

Thanks again

[VF]

Hi Jakob. I have sent you a LinkedIn request.

[Venkataramana]

Good One

[Joel Batac]

Can the script be used for JIRA Data Center?

[Nic Brough -Adaptavist-]

It could, but don't do it.

You should not have any need to re-index regularly.  If you think you do, then you actually have another, deeper, problem that you need to debug and fix.

[Joel Batac]

Thanks Nick. Due to regular customization we do (we're enterprise) we re-index weekly . Can you elaborate the possible problems we may encounter?

[Nic Brough -Adaptavist-]

Just that the system will be unusable during re-indexing, if you're doing it properly.

[Joel Batac]

Hi Nick,

 I'm trying to create an email notification for indexing status. I have the code below but the while loop is not working as expected. It's just spins around. Any advice?

    requestIndexStatus.addTrustedTokenAuthentication(host)

    def responseBody1 = requestIndexStatus.execute()

    def responseAsMap1 = new JsonSlurper().parseText(responseBody1) as Map

    def percentStatus = (int)responseAsMap1['currentProgress']

    while  (percentStatus < 100){

        sleep(10*15000) //for testing purposes, this will be 15 minutes interval

        requestIndexStatus.addTrustedTokenAuthentication(host)

        def responseBody = requestIndexStatus.execute()

        def responseAsMap = new JsonSlurper().parseText(responseBody) as Map

        def responseOutput = (int)responseAsMap['currentProgress']

        percentStatus = responseOutput

        def body = "Jira indexing progress status: " + percentStatus +"% complete."

        //sendEmail (emailAddr, subject, body) 

        log.warn( "Jira indexing is currently in progress with " + percentStatus +"% complete.")

        //return( "Jira indexing is currently in progress with " + percentStatus +"% complete.")

       }

[Nic Brough -Adaptavist-]

I would guess that your percentStatus is not being filled with a percentage number that you can usefully compare with 100.

[Jakob F_ Gormsen]

Can't get it to work.
Can you help?

2022-06-09 00:30:00,542 INFO [runner.ScriptBindingsManager]: Host: jira.customer.com 22022-06-09 00:30:00,637 INFO [runner.ScriptBindingsManager]: No index running 32022-06-09 00:30:00,637 INFO [runner.ScriptBindingsManager]: No reindex in progress. Attempt to start a new reindex 42022-06-09 00:30:00,668 INFO [runner.ScriptBindingsManager]: Reindex not started. com.atlassian.sal.api.net.ResponseStatusException: Unexpected response received. Status code: 411

[Vivian.HOAREAU]

Hello

same error for me:

2022-09-07 06:54:27,431 INFO [runner.ScriptBindingsManager]: Host: jira-uat.1place.com
2022-09-07 06:54:27,461 INFO [runner.ScriptBindingsManager]: No index running
2022-09-07 06:54:27,461 INFO [runner.ScriptBindingsManager]: No reindex in progress. Attempt to start a new reindex
2022-09-07 06:54:27,463 INFO [runner.ScriptBindingsManager]: Reindex not started. com.atlassian.sal.api.net.ResponseException: java.net.UnknownHostException: jira-uat.1place.com

Do you have an idea ? 

fyi: jira server V8.20.11