How does a pipeline, with trigger conditions, determine which steps to run

Question

I have a mono-repo with several services in it (in separate folders).  I've used a condition to make sure that only when files within that folder have been modified, do I then build and deploy that service.

My build pipeline has something like this step:

- step:
  name: Deploy 
script:
  - export SERVICE_NAME=my-api
  - pipe: atlassian/aws-ecs-deploy:1.12.1
variables:
  condition:
    changesets:
      includePaths:
      - "MyProject.Shared.*/**"
      - "MyProject.SubProject.*/**"

This appears to work OK, and I can see that unmodified steps are skipped in the pipeline build view.

My question is how does the pipeline builder determine what has changed?

I was presuming it would compare between the current build, and the last time that build worked, but it appears to do a comparison between the current build and the previous one.

Is that correct?

The problem I have, is that if the pipeline fails for some any reason that doesn't require a change to that code to fix it, then my service will not be deployed, as the code being monitored hasn't changed.

Why would the fix be outside a code change?  Maybe the infrastructure wasn't deployed before the build, or the image repository hasn't been created yet, or other things outside of my control.

Mark C · Answer

Hi @Neil Scales

So that we can investigate this further and find alternatives for you, the best I can suggest is for you to reach out to us by raising a support request using this link. -https://support.atlassian.com/contact/

* Select "Technical issues and bugs"
* Select "Bitbucket"
* Enter your paid workspace ID (e.g. bitbucket.org/workspace_id)

Regards,
Mark C

Neil Scales_ywcg32 · Answer

OK, so only changes in the most recent commit are considered, I can explain that to others.

I can't use an 'after-script' because I don't want to deploy things that haven't built properly.

I'll enhance my scenario:

Several of my steps are parallel.

I have service #1 and service #2.   Both services are build in separate but parallel steps (because the steps have different conditions). If both services are built, there is a following set of steps to deploy the services (again in parallel, based on the same conditions are the build steps).  Services have to be deployed together, if one of the services fails to build, then I don't deploy the other one.

Something like this:

- parallel
 - step:
  - build service #1
  - condition:
   - includePath: /service1
 - step
  - build service #2
  - condition:
   - includePath /service2
-parallel
 -step
  - deploy service #1
  - condition:
    - includePath: /service1
 - step
  - deploy service #2
  - condition:
    - includePath: /service2

I run the pipeline and service #1 builds, but service #2 doesn't.

I fix service #2, without modifying any code that service #1 relies on.

I run the pipeline, and service #1 is not built because it hasn't changed, but service #2 does get built.

Then the pipeline moves onto the deployment stage, but skips service #1 because it's conditions mean it wasn't built.

At this point, I can't run another pipeline to just deploy everything, because the conditions mean it will just build and deploy service #2 (I've opened a separate question for this).

Mark C · Answer

Hi @Neil Scales,

Welcome to the community.

About Conditional steps, Pipelines determine changes based on commits.

Condition

The condition option prevents a step or stage from running unless a condition or rule is satisfied. Currently, the only condition supported is changesets. Use changesets to execute a step or stage only if one of the modified files matches the expression in includePaths. The file match pattern specified in the includePaths is relative to the $BITBUCKET_CLONE_DIR directory.

Changes that are taken into account

In a pull-requests pipeline, all commits are taken into account, and if you provide an includePath list of patterns, the step or stage will be executed when at least one commit change matches one of the conditions. The format for pattern matching follows the glob patterns.

For other types of pipelines, only the last commit is considered. If you push multiple commits to branch at the same time or if you push multiple times to given branch, you might experience non-intuitive behavior when failing pipelines turn green only because the failing step or stage is skipped on the next run.

Reference: Stage options - Condition

For failed builds that don't require a code change, you may want to consider using after-script configuration which allows you to run a set of commands/scripts regardless of the status of the build step (Failed or Success).

You can check these links for more information about after-script:

* After scripts now available for Bitbucket Pipelines
* Step options - After-script

Regards,
Mark C