Pipelines fail to publish to pypi when package contains yaml files

Question

I'm attempting to integrate pypi-publish into my build pipelines. I already had pipelines running for testing and building my python package into a wheel and pyz file. When I add script: - python -m pip install tox - tox -e pyz - pipe: atlassian/pypi-publish:0.2.13 variables: PYPI_USERNAME: $PYPI_USER PYPI_PASSWORD: $PYPI_PASSWORD DISTRIBUTIONS: 'bdist_wheel' REPOSITORY: 'https://test.pypi.org/legacy/' I get an error for Operation not permitted
running build
running build_py
running egg_info
writing src/pyaware.egg-info/PKG-INFO
writing dependency_links to src/pyaware.egg-info/dependency_links.txt
writing requirements to src/pyaware.egg-info/requires.txt
writing top-level names to src/pyaware.egg-info/top_level.txt
reading manifest file 'src/pyaware.egg-info/SOURCES.txt'reading manifest template 'MANIFEST.in'
warning: no files found matching '*.json' under directory 'src/pyaware' warning: no previously-included files matching '__pycache__' found under directory 'src'
warning: no previously-included files matching '.idea' found under directory 'src'
warning: no files found matching '.git' under directory 'src' writing manifest file 'src/pyaware.egg-info/SOURCES.txt'
error: [Errno 1] Operation not permitted: 'build/lib/pyaware/examples/I33.yaml'
✖ Failed to build distributionsINFO: Building distributions: ['bdist_wheel']
running bdist_wheel I then removed my examples folder containing the yaml file and I hit the same problem elsewhere in my project error:
[Errno 1] Operation not permitted: 'build/lib/pyaware/devices/ampcontrol/ipb.yaml'
✖ Failed to build distributionswarning: no files found matching '.git' under directory 'src'
writing manifest file 'src/pyaware.egg-info/SOURCES.txt' I tried changing from tox to setup.py to check if there was some weird thing going on there script: - python setup.py bdist_wheel - pipe: atlassian/pypi-publish:0.2.13 variables: PYPI_USERNAME: $PYPI_USER PYPI_PASSWORD: $PYPI_PASSWORD DISTRIBUTIONS: 'bdist_wheel' REPOSITORY: 'https://test.pypi.org/legacy/' The setup.py bdist_wheel build worked successfully but the pypi-publish failed in the same way. Anyone have some insights?

Halyna Berezovska · Answer

@Ryan Parry-Jones  glad you succeeded.

IF you have time, can you share logs of publishing failure, not building?

Those logs you put, they were during building process, so we need specific logs to see if there is an issue with this pipe

Ryan Parry-Jones · Answer

Hi Galyna,

Sorry for being unhelpful but I can't really spend the time at the moment to reproduce with a minimum failing example.

In order to fix the issue I had to not use

- pipe: atlassian/pypi-publish:0.2.13

- step: 
   name: Deploy 
   caches: 
     - pip 
   script: 
     - python -m pip install tox twine 
     - tox -e pyz 
     - twine upload dist/* -u __token__ -p $PYPI_TOKEN 
   artifacts: 
     - pyaware.pyz - dist/**

Seemed to fix the issue. It also worked with setup.py instead of tox.

Which suggest that it isn't an access issue with the setup.py directories.

Halyna Berezovska · Answer

@Ryan Parry-Jones  hello again!

Could you please share your case in detail? I have tried this pipe out and it worked for me with *.yml and *.yaml files.

I guess the case may be the following: your script has created some folders that cannot be reached by setup.py tools to write there. You can check the permissions for these folders and then try again.

Looking forward to hearing from you, thanks for using pipes.

Cheers, Galyna