I am trying to configure a bitbucket pipeline to unit test our python code. However when I try and use pip to install python if fails. The first problem was that it said that numpy wasn't install despite it being before gdal in the requirements.txt file. So I moved the GDAL install out from the requirements file and tried installed it afterwards. The pipeline definition is
# This is a sample build configuration for Python.
# Check our guides at https://confluence.atlassian.com/x/x4UWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: python:3.6.8
clone:
depth: 1
pipelines:
custom:
test:
- step:
name: Run unit tests
caches:
- pip
script: # Modify the commands below to build your repository.
- python -V
- pip install --upgrade pip
- pip install -r requirements2.txt
- pip freeze
- pip install GDAL
- python -m unittest discover -s test
The requirements2.txt file contains
numpy==1.17.2
pandas==0.25.1
parse==1.12.1
python-dateutil==2.8.0
pytz==2019.3
six==1.12.0
The output of the commands is
+ pip install --upgrade pip
Collecting pip
Downloading https://files.pythonhosted.org/packages/30/db/9e38760b32e3e7f40cce46dd5fb107b8c73840df38f0046d8e6514e675a1/pip-19.2.3-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 19.1.1
Uninstalling pip-19.1.1:
Successfully uninstalled pip-19.1.1
Successfully installed pip-19.2.3
+ pip install -r requirements2.txt
Collecting numpy==1.17.2 (from -r requirements2.txt (line 1))
Downloading https://files.pythonhosted.org/packages/e5/e6/c3fdc53aed9fa19d6ff3abf97dfad768ae3afce1b7431f7500000816bda5/numpy-1.17.2-cp36-cp36m-manylinux1_x86_64.whl (20.4MB)
Collecting pandas==0.25.1 (from -r requirements2.txt (line 2))
Downloading https://files.pythonhosted.org/packages/73/9b/52e228545d14f14bb2a1622e225f38463c8726645165e1cb7dde95bfe6d4/pandas-0.25.1-cp36-cp36m-manylinux1_x86_64.whl (10.5MB)
Collecting parse==1.12.1 (from -r requirements2.txt (line 3))
Downloading https://files.pythonhosted.org/packages/84/4d/5578a484f4a3ec5d4440a95731a104a96a12a6fb57385292d3e7b74d8f6d/parse-1.12.1.tar.gz
Collecting python-dateutil==2.8.0 (from -r requirements2.txt (line 4))
Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
Collecting pytz==2019.3 (from -r requirements2.txt (line 5))
Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
Collecting six==1.12.0 (from -r requirements2.txt (line 6))
Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Building wheels for collected packages: parse
Building wheel for parse (setup.py): started
Building wheel for parse (setup.py): finished with status 'done'
Created wheel for parse: filename=parse-1.12.1-cp36-none-any.whl size=22341 sha256=cada607fad94b4bf7661c09817a8d868139f02bfaf2823c5e82a377f2d66d506
Stored in directory: /root/.cache/pip/wheels/16/f1/d5/72f7e2c22ab3d106bb6f7d727552b5e027b832284f4704b106
Successfully built parse
Installing collected packages: numpy, pytz, six, python-dateutil, pandas, parse
Successfully installed numpy-1.17.2 pandas-0.25.1 parse-1.12.1 python-dateutil-2.8.0 pytz-2019.3 six-1.12.0
<span><strong>+ pip freeze</strong><br>numpy==1.17.2<br>pandas==0.25.1<br>parse==1.12.1<br>python-dateutil==2.8.0<br>pytz==2019.3<br>six==1.12.0<br><br><strong>+ pip install GDAL==2.4.2</strong><br>Collecting GDAL==2.4.2<br>Downloading https://files.pythonhosted.org/packages/dc/d5/90339b48bdcabc76124eaa058a32d796963a05624eb418ed5ea5af7db9fa/GDAL-2.4.2.tar.gz (564kB)<br>ERROR: Command errored out with exit status 1:<br>command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-xnjkin2o/GDAL/setup.py'"'"'; __file__='"'"'/tmp/pip-install-xnjkin2o/GDAL/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info<br>cwd: /tmp/pip-install-xnjkin2o/GDAL/<br>Complete output (72 lines):<br>running egg_info<br>creating pip-egg-info/GDAL.egg-info<br>writing pip-egg-info/GDAL.egg-info/PKG-INFO<br>writing dependency_links to pip-egg-info/GDAL.egg-info/dependency_links.txt<br>writing top-level names to pip-egg-info/GDAL.egg-info/top_level.txt<br>writing manifest file 'pip-egg-info/GDAL.egg-info/SOURCES.txt'<br>Traceback (most recent call last):<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 151, in fetch_config<br>p = subprocess.Popen([command, args], stdout=subprocess.PIPE)<br>File "/usr/local/lib/python3.6/subprocess.py", line 729, in __init__<br>restore_signals, start_new_session)<br>File "/usr/local/lib/python3.6/subprocess.py", line 1364, in _execute_child<br>raise child_exception_type(errno_num, err_msg, err_filename)<br>FileNotFoundError: [Errno 2] No such file or directory: '../../apps/gdal-config': '../../apps/gdal-config'<br><br>During handling of the above exception, another exception occurred:<br><br>Traceback (most recent call last):<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 237, in get_gdal_config<br>return fetch_config(option, gdal_config=self.gdal_config)<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 154, in fetch_config<br>raise gdal_config_error(e)<br>__main__.gdal_config_error: [Errno 2] No such file or directory: '../../apps/gdal-config': '../../apps/gdal-config'<br><br>During handling of the above exception, another exception occurred:<br><br>Traceback (most recent call last):<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 151, in fetch_config<br>p = subprocess.Popen([command, args], stdout=subprocess.PIPE)<br>File "/usr/local/lib/python3.6/subprocess.py", line 729, in __init__<br>restore_signals, start_new_session)<br>File "/usr/local/lib/python3.6/subprocess.py", line 1364, in _execute_child<br>raise child_exception_type(errno_num, err_msg, err_filename)<br>FileNotFoundError: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'<br><br>During handling of the above exception, another exception occurred:<br><br>Traceback (most recent call last):<br>File "<string>", line 1, in <module><br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 431, in <module><br>setup(**setup_kwargs)<br>File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup<br>return distutils.core.setup(**attrs)<br>File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup<br>dist.run_commands()<br>File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands<br>self.run_command(cmd)<br>File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command<br>cmd_obj.run()<br>File "/usr/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 296, in run<br>self.find_sources()<br>File "/usr/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 303, in find_sources<br>mm.run()<br>File "/usr/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 534, in run<br>self.add_defaults()<br>File "/usr/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 570, in add_defaults<br>sdist.add_defaults(self)<br>File "/usr/local/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults<br>self._add_defaults_ext()<br>File "/usr/local/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext<br>build_ext = self.get_finalized_command('build_ext')<br>File "/usr/local/lib/python3.6/distutils/cmd.py", line 299, in get_finalized_command<br>cmd_obj.ensure_finalized()<br>File "/usr/local/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized<br>self.finalize_options()<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 287, in finalize_options<br>self.gdaldir = self.get_gdal_config('prefix')<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 243, in get_gdal_config<br>return fetch_config(option)<br>File "/tmp/pip-install-xnjkin2o/GDAL/setup.py", line 154, in fetch_config<br>raise gdal_config_error(e)<br>__main__.gdal_config_error: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'<br>----------------------------------------<br>ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.<br><br>Has anyone encountered the same problem and know how to fix it?<br><br>Thanks in advance,<br>Ian</span><span><strong>+ python -V</strong><br>Python 3.6.8<br><br></span>
