Can you add a retry on failure option for the trigger-pipeline pipe

[Jay Seletz]

We use the trigger-pipeline pipe in several different pipelines to run a long list of builds across different repos each night.  Sometimes there is an unexpected error that is not a real build failure - a network issue pulling an artifact or an image, a timeout making an internet call, a failure in the Atlassian pipeline runner, etc.

It would be great to have an option to re-run failed trigger-pipeline steps so that the overall pipeline can proceed if the re-run succeeds.

This pipe is already really useful, but this would be really helpful. 

[Theodora Boudale]

Hi Jay!

Thank you for the suggestion. I can create a feature request for this, I would just like to ask a few questions first to better understand your use case:

If the trigger-pipeline pipe is used in repo A and it triggers a build in repo B, then:

1. I assume that you have the WAIT variable in the pipe set to true, so if the build in repo B fails, then the build in repo A fails as well?

2. Are you looking for a retry option in the trigger-pipeline pipe? This is what you mention in the title, but later on, you mention

option to re-run failed trigger-pipeline steps

so I'm a bit confused. The option to rerun a failed step already exists on the UI. Is this something that would work for you, or do you want the retry option in the pipe, so you won't have to manually rerun the whole step?

Kind regards,
Theodora

[Jay Seletz]

Hi Theodora,

Thanks for the reply. 

1. Yes, we have the WAIT variable set to true, and yes that will fail the repo A build as well.
2. Yes, I'm looking for a retry option within the trigger-pipeline pipe so I don't need to manually re-run the failed step(s) in pipeline A.

Thanks,

Jay

[Theodora Boudale]

Hi Jay,

Thank you for the details. I went ahead and created a feature request in our issue tracker:

• https://jira.atlassian.com/browse/BCLOUD-22872

I would suggest adding your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

Implementation of new features is done as per our policy here and any updates will be posted in the feature request.

Please feel free to let me know if you need anything further.

Kind regards,
Theodora

[Igor Stoyanov]

@Jay Seletz hi. I also have a question. If pipeline fails, why it should not be in failed state on retry? Only example i could imagine is to wait for some infrastructure become available.

And if pipeline could change state from FAIL to SUCCESS after some time, why don't add a condition in pipeline in repoB to run until SUCCESS state, so there is no need to re-run pipeline in repoA. In example:

while not (infrastructure created) and not max retries:
   wait for the infra
   retry += 1
logger(SUCCESS state)

Regards, Igor

[Jay Seletz]

We have a pipeline that runs over 60 pipelines in grouped sequences each night.  Occasionally we'll see an intermittent failure on one or more steps, and we just use the "Re-run failed steps" option on the pipelines to pick up where it left off and it typically completes.  This is usually due to either a network failure, the Pipe API call to Bitbucket hitting an intermittent failure, or a flaky unit test that fails maybe 1% of the time so not worth fixing.  I'm trying to simulate/automate the "Re-run failed steps" so we can reduce manual intervention.  I see this PR is merged, not sure if this is implemented yet, or if this is how it was implemented?  https://bitbucket.org/atlassian/trigger-pipeline/pull-requests/46

[Igor Stoyanov]

@Jay Seletz . Yes, this feature was released. The reason of this feature was:

Running bitbucketpipelines/trigger-pipeline:5.1.2 from a bitbucket
pipeline step to run a pipeline in another repo is intermittently
failing with a "Read timed out error". This is happening triggering
pipelines in different target repos, so it's not specific to a target
repo, and re-running the step usually succeeds.

You could try the latest 5.4.0 version of the pipe, and check if your problem is fixed. No changes to variables needed.

Regards, Igor

[Jay Seletz]

Hi @Igor Stoyanov - I don't see any new variables documented in the README.md for this - how do I configure retries?

[Igor Stoyanov]

@Jay Seletz hi.

You could not configure this `retry` variable. It will work without any changes to the pipe configuration.

Regards, Igor

[Jay Seletz]

From the PR, looks like retry was added for failures when retrieving pipeline status.  That's great, as we see a fair number of these, but I'm also still looking for this feature to be added: https://jira.atlassian.com/browse/BCLOUD-22872 - re-run when the triggered pipeline being run by the pipe fails.

[Jay Seletz]

Also, I'm seeing that if there's an API error trying to initially trigger the pipeline, the pipe fails.  The retry is only implemented when retrieving the pipeline status after it's already triggered.  If the trigger call itself fails, there is no retry.  Sadly the Bitbucket API call to trigger the pipeline does fail with some frequency.