Is cosmos db emulater supported on bitbucket pipeline?

s.barbuto November 7, 2022

Hi,

the scenario we are working on, uses cosmos db as database on a C# .NET application. The integration tests - locally - use the cosmos db emulator spun up through docker. It's necessary to reproduce the same behavior on the pipeline when the PR is submitted.

 
image: mcr.microsoft.com/dotnet/sdk:6.0

definitions:
services:
cosmos-db:
image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
variables:
AZURE_COSMOS_EMULATOR_PARTITION_COUNT: '10'
AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE: 'true'
AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE: '127.0.0.1'
steps:
- step: &stepTests
name: Run Tests
services:
- cosmos-db
script:
dotnet test
The db's connection string points to 127.0.0.1 on the port 8081
"ConnectionString": "AccountEndpoint=https://127.0.0.1:8081/;AccountKey=secretKey"

The result of it, is that connections are refused and consequently  tests fail. As side note, the cosmos db tab  - during the execution - seems stuck, it hangs and  shows "Starting".

Screenshot from 2022-11-07 09-32-01.pngScreenshot from 2022-11-07 09-34-00.png

Questions here are:

Is the service correctly configured? Does the connection string point to the right address?

Is cosmos db emulator supported by the pipeline?

 

Thanks in advance.

 

Best regards

Sante

 

1 answer

1 vote
Theodora Boudale
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 9, 2022

Hi Sante,

We haven't tested Pipelines with every database service available and we don't have a guide on how to use cosmos db.

I have tried running a build in Pipelines based on the bitbucket-pipelines.yml file you posted here. In the script of my step I used the following command instead to check if the service is running:

- curl -k https://127.0.0.1:8081/_explorer/emulator.pem

In the beginning, I was seeing errors in the output of curl. Services in Pipelines have no mechanism to wait for service startup, so I added the command sleep 30 before the curl command, in case the service needed more time to start up, but I was still seeing a "Connection refused" error.

I checked the documentation here https://learn.microsoft.com/en-us/azure/cosmos-db/linux-emulator?tabs=sql-api%2Cssl-netstd21 and I saw that 3 GB of memory is required for the service container, so I adjusted the memory used for the service in Pipelines. After that change, and with a sleep command at the beginning of my script, the curl command started succeeding.

This is the bitbucket-pipelines.yml file I used:

image: mcr.microsoft.com/dotnet/sdk:6.0

pipelines:
default:
- step:
size: 2x
script:
- sleep 30
- curl -k https://127.0.0.1:8081/_explorer/emulator.pem
services:
- cosmos-db

definitions:
services:
cosmos-db:
image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
memory: 3072
variables:
AZURE_COSMOS_EMULATOR_PARTITION_COUNT: '10'
AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE: 'true'
AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE: '127.0.0.1'

I would suggest trying to run your build by allocating 3072 MB to the service (as above) and also adding a sleep command at the beginning of the step.

Instead of using a sleep command, another option would also be to write a script that checks if the connection to https://127.0.0.1:8081 succeeds. If it does, exit the script so the rest of the commands in the step's script are executed. If not, use a sleep command for e.g. 10 seconds, then try again (e.g. 3 more times) and if it fails after these 3 times, then exit the script with exit code 1 so that the build fails and doesn't continue.

I would also recommend using size: 2x for the step, if you allocate 3072 MB of memory to the service. Regular steps (without size: 2x) have 4096 MB of memory in total. The build container is given 1024 MB of the total memory, so the remaining memory for a regular step is 3072 MB. If all of that is given to the service container, then there may not be enough memory left for the build to run.

With size: 2x the step will have 8192 MB of memory in total. The build container is given 1024 MB, we allocate 3072 MB to the service and the rest remains available for the build. Please keep in mind that 2x pipelines will use twice the number of build minutes.

Please feel free to let me know how it goes.

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events