How can I re-run a pileline after editing the bitbucket-pipelines.yml file.

[Sam Rolfe]

I am deploying to multiple sites on custom servers using yml file with variables through pipelines.

However I shut down one of those sites but did not update the yml file - the pipeline fails as the site is no longer responding.

So I want to edit the bitbucket-pipelines.yml file and remove that site and rerun the pipeline.

Can not do this - does anyone know how I could achieve this?

[Theodora Boudale]

Hi Sam,

What kind of pipeline are you using in your bitbucket-pipelines.yml file? Is it a default pipeline, a branches pipeline, a tags pipeline, a pull-requests pipeline, or a custom one (reference: https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/)?

Do you have more than one branch in your repo and if so, did you edit the bitbucket-pipelines.yml file in a branch other than the one where the pipeline was previously running?

Kind regards,
Theodora

[Sam Rolfe]

I am using default pipeline and only one branch.

I noticed in other docs there was previously an "edit" option available, this does not appear in the bitbucket interface,  and there is one in Microsofts version as well (can't remember what it is called).

Thanks for taking a look.

[Theodora Boudale]

Hi Sam!

I noticed in other docs there was previously an "edit" option available, this does not appear in the bitbucket interface, and there is one in Microsofts version as well (can't remember what it is called).

Are you referring to the Edit option when viewing a file on Bitbucket Cloud website? This option should be visible at the top right corner of a file, please see the screenshot below for reference:

Please keep in mind that if you select an older hash from the commit hash dropdown, the option will be disabled.

Is this the Edit option you are referring to? If so, can you post a screenshot to show us what it looks like for you?

I am not sure what you are referring to by "Microsoft's version", can you please explain?

Can you also please confirm if you have indeed changed your bitbucket-pipelines.yml file?

Please keep in mind that if you click the Rerun pipeline option of a previous build, or if you run a pipeline manually for an older commit, the version of the bitbucket-pipelines.yml file in that old commit will be used.

The new version of the bitbucket-pipelines.yml file (if you have pushed a new one) will be taken into account for builds running on this new commit with the changed yml and any newer commits.

Kind regards,
Theodora

[Sam Rolfe]

Your last part is exactly my situation. I have updated the bitbucket-pipelines.yml file as it had incorrect info.

So when I try and run PREVIOUS (as you guessed) pipelines it uses the old bitbucket-pipelines.yml and not my newly edited one.

So my local repository assumes everything was pushed to bitbucket fine, however bitbucket is stuck and can not push to remote websites.

[Theodora Boudale]

Hi Sam,

What you are describing is expected behavior.

A Pipelines build that runs for a certain commit, will use the version of the bitbucket-pipelines.yml file that exists on that specific commit.

Assume a commit graph like the following, where A, B, and C are commit hashes and C is the newest commit.

A - B - C

Commit B has the old version of the bitbucket-pipelines.yml file.
Commit C has the new version of the bitbucket-pipelines.yml file.

If you rerun the build that ran for commit B, it is going to use the old version.

If you pushed commit C with the new version, and you have a default pipeline, a new build should have been triggered for commit C. This one will use the new version of the bitbucket-pipelines.yml file.

Can you see in your list of builds a build for the newer commit that has the new version of the bitbucket-pipelines.yml file?

If you look at the list of builds on the Pipelines page of the repo, the commit hash is mentioned for each build, next to the branch name.

Kind regards,
Theodora

[Sam Rolfe]

Yes thanks, I can see that this is happening. 

I was hoping there was a way to around it. A way to re-run the previous pipeline with the new yml file.

As it stands I use Eclipse which pushes to Bitbucket which deploys to many sites.

So my local version is up to date as it has successfully pushed to Bitbucket - however bitbucket has NOT deployed do to an "error" in the pipeline.

As mentioned - all this is as expected.

However I am now left with an orphaned deployment that can not be fixed.

The only solution I have is to use a list of previously pushed files in my local, edit everyone of those files locally and push them again with the new yml file already added.

Here is a screen grab.

[Theodora Boudale]

Hi Sam,

I am not sure I understand what problem you are trying to fix.

You mentioned in your first post that you shut down one of your servers and the pipeline was failing as this server was not responding.

You edited then the bitbucket-pipelines.yml file, I assume that was in order to remove the deployment to the server you shut down.

Doesn't the deployment to the rest of your servers succeed now if you deploy the code of your repo from the latest commit?

Kind regards,
Theodora

[Sam Rolfe]

No, because I edited the YML file to fix the broken server issues. So running the pipeline again does not work.

I can not recommit my local as it is already up to date and synced locally with remote bitbucket.

So local files on my desktop are committed, bitbucket repository is up to date, pipelines have FAILED.

[Sam Rolfe]

So the only way to get files to be pushed to remote websites is to manually make changes in each file, so that they can be pushed again.

The perfect solution would simply be able to edit the YML file for a pipeline that had failed.

It appears there is no way to do this - but it would be helpful.

Here is AZURE pipeline editor editing the buildfile for re-deployment.

This is what I was hoping was available, or something similar, some way of editing the buildfile on a pipeline so that I can just re-run it with the fixes in place.

[Theodora Boudale]

Hi Sam,

I am not talking about rerunning the failed pipeline. If we assume the same commit graph again:

A - B - C

The Pipelines build for commit B with the old yml file failed.

Doesn't the new build for commit C, with the new yml file, deploy to your servers?

On the Pipelines page of the repo (where you see a list of all your builds) there is an option Run pipeline at the top right corner. You can use that to trigger a pipeline manually for master.

This should run the same build that was triggered automatically for the latest commit of master branch, commit C.

Kind regards,
Theodora

[Sam Rolfe]

Theodora the last commit for branch C as it were - would simply be the UPDATED YML file. which would do nothing,

Did you look at the attached image this explains the problem perfectly.

I appreciate you trying to help but the problem is really very simple.

A)

First pipeline fails with 50+ files due to YML

B)

New pipeline with YML 

I then can NOT run A) as it has OLD YML file associated with it.

B)

IN order to utilize new YML file I must push ALL FILES from my local repo again and this works.

I want to be able to run A) again. That is all.

WHY?

Because pushing all files from local repo again could take a very long time to be able to identity them.

There is no other option here - I need a way to run the failed pipeline again. How can this be done?  

[Theodora Boudale]

Hi Sam,

I think my confusion here has to do with the fact that I don't know what exactly your yml file does and how you deploy. I assumed you deploy everything from the repo's source code. Perhaps you are deploying only files that were changed in a certain commit, which is why you are looking to rerun the previous pipeline?

If we assume again the commit graph

A - B - C

it is not possible to rerun the pipeline B with a yml definition that doesn't exist in commit B.

What might help (emphasis on 'might', because I don't know the specifics of your build) would be to rewrite history in the repo with an interactive git rebase in order to edit commit B and change the yml file in that commit.

Please keep in mind that if you do that, the commit hashes of commit B and all its children will change. You will then have a commit graph like the following:

A - D - E

After you push your changes, you can then open commit D (previously commit B) from the Commits page of the repo on Bitbucket Cloud website, and on the right sidebar you will see the option Run pipeline. You can then trigger a pipeline for commit D and it will use the new yml file that you edited during the interactive rebase.

Please keep in mind that if you rewrite history, you can push with the --force flag only. I strongly recommend taking a backup of the repo before doing that, as pushing with the --force flag is a destructive operation. You can take a backup of the repo by cloning it with the --mirror flag.

The drawback with this approach is that since commit hashes will change, then:

• The Pipelines builds that ran for commits B and C will reference commits that will no longer exist in the repo. Actually, they will exist for a while, as dangling commits, but they will not be reachable from any branch and when a garbage collection runs they will get removed.
•  If you have an integration with Jira and Jira tickets referencing commits B and C, the Jira tickets will continue to reference commits B and C (along with the new commits hashes) since they got indexed in the Jira database.

If you don't want to do that, I would need some more details on what exactly your build does, why running a pipeline on commit C does not solve your problem, and why you would need to push all files again, to see if there is an alternate solution.

Kind regards,
Theodora

[Sam Rolfe]

Thanks again for your all your help, replies and time.

My process is from Eclipse -> commits locally and pushes to Bitbucket repository -> Bitbucket pipeline takes new changes and pushes them to remote websites via SECURE FTP.

So say website www.aaa.com and www.bbb.com have their files automatically. 

Does that make sense?  

Because the build file exists independently of the working files I was hoping it could be done. 

Honestly it seems like its something that should be possible.

Is there a change I can make to my flow to make this possible in the future?  

[Theodora Boudale]

Hi Sam,

Thank you for the information.

Just to give you some context, for every step of a Pipelines build, a Docker container starts and your repo gets cloned in this container (unless you have disabled cloning in the yml file). Then, the commands of that step's script are executed. When the build finishes (either with success or failure), then the container gets destroyed.

I would like to ask for some clarification:

Bitbucket pipeline takes new changes and pushes them to remote websites via SECURE FTP.

• Does the pipeline push to the websites the source code files of your repo?
• Or do you have a filter that pushes to the websites only the files that were changed in that specific commit?
• Or does it push files that were generated during the build and do not exist in your repo?

In case you want to push to your website all the files from the source code of your repo or files that exist in a certain directory of your source, you could configure this with sftp and they should be pushed with the next build.

I am trying to understand why this will not work in this case.

Kind regards,
Theodora