Pipelines fail to publish to pypi when package contains yaml files

[Ryan Parry-Jones]

I'm attempting to integrate pypi-publish into my build pipelines.

I already had pipelines running for testing and building my python package into a wheel and pyz file.

When I add

script:
  - python -m pip install tox
  - tox -e pyz
  - pipe: atlassian/pypi-publish:0.2.13
    variables:
      PYPI_USERNAME: $PYPI_USER
      PYPI_PASSWORD: $PYPI_PASSWORD
      DISTRIBUTIONS: 'bdist_wheel'
      REPOSITORY: 'https://test.pypi.org/legacy/'

I get an error for Operation not permitted

INFO: Building distributions: ['bdist_wheel']
running bdist_wheel
running build
running build_py
running egg_info
writing src/pyaware.egg-info/PKG-INFO
writing dependency_links to src/pyaware.egg-info/dependency_links.txt
writing requirements to src/pyaware.egg-info/requires.txt
writing top-level names to src/pyaware.egg-info/top_level.txt
reading manifest file 'src/pyaware.egg-info/SOURCES.txt'reading manifest template 'MANIFEST.in'
warning: no files found matching '*.json' under directory 'src/pyaware'
warning: no previously-included files matching '__pycache__' found under directory 'src'
warning: no previously-included files matching '.idea' found under directory 'src'
warning: no files found matching '.git' under directory 'src'
writing manifest file 'src/pyaware.egg-info/SOURCES.txt'
error: [Errno 1] Operation not permitted: 'build/lib/pyaware/examples/I33.yaml'
✖ Failed to build distributions

I then removed my examples folder containing the yaml file and I hit the same problem elsewhere in my project

warning: no files found matching '.git' under directory 'src'
writing manifest file 'src/pyaware.egg-info/SOURCES.txt'error: 
[Errno 1] Operation not permitted: 'build/lib/pyaware/devices/ampcontrol/ipb.yaml'
✖ Failed to build distributions

I tried changing from tox to setup.py to check if there was some weird thing going on there

script:
  - python setup.py bdist_wheel
  - pipe: atlassian/pypi-publish:0.2.13
    variables:
      PYPI_USERNAME: $PYPI_USER
      PYPI_PASSWORD: $PYPI_PASSWORD
      DISTRIBUTIONS: 'bdist_wheel'
      REPOSITORY: 'https://test.pypi.org/legacy/'

The setup.py bdist_wheel build worked successfully but the pypi-publish failed in the same way.

Anyone have some insights?

[Halyna Berezovska]

@Ryan Parry-Jones  hello again!

Could you please share your case in detail? I have tried this pipe out and it worked for me with *.yml and *.yaml files.

I guess the case may be the following: your script has created some folders that cannot be reached by setup.py tools to write there. You can check the permissions for these folders and then try again.

Looking forward to hearing from you, thanks for using pipes.

Cheers, Galyna

[Ryan Parry-Jones]

Hi Galyna,

Sorry for being unhelpful but I can't really spend the time at the moment to reproduce with a minimum failing example.

In order to fix the issue I had to not use

- pipe: atlassian/pypi-publish:0.2.13

 - step: 
   name: Deploy 
   caches: 
     - pip 
   script: 
     - python -m pip install tox twine 
     - tox -e pyz 
     - twine upload dist/* -u __token__ -p $PYPI_TOKEN 
   artifacts: 
     - pyaware.pyz - dist/**

 Seemed to fix the issue. It also worked with setup.py instead of tox.

Which suggest that it isn't an access issue with the setup.py directories.

[Halyna Berezovska]

@Ryan Parry-Jones  glad you succeeded.

IF you have time, can you share logs of publishing failure, not building?

Those logs you put, they were during building process, so we need specific logs to see if there is an issue with this pipe

[Halyna Berezovska]

Thanks for the reporting the case! We will investigate your issue as soon as we can. At the first look it seems to be worked with yml, but perhaps, there is a separate issue