Bug 25849 - Getting Python pip working properly on M7 to install awsebcli
Summary: Getting Python pip working properly on M7 to install awsebcli
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 7
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Python Stack Maintainers
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-12-09 23:39 CET by William Kenney
Modified: 2019-12-11 02:07 CET (History)
4 users (show)

See Also:
Source RPM: python2-pip 19.0.3-1.mga7
CVE:
Status comment:


Attachments

Description William Kenney 2019-12-09 23:39:43 CET
Description of problem:

Tested in a Vbox Plasma client.

Ref:
https://docs.aws.amazon.com/cli/latest/userguide/install-linux.html
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3.html
https://docs.aws.amazon.com/AmazonS3/latest/dev/setup-aws-cli.html
https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html

Description of problem:

Using Python pip I am attempting to install AWS CLI
If all works as advertised the following pip commands should install the Python awsebcli module(s):
$ pip install awsebcli --upgrade --user

The following steps should work but don't:

Verify version of Python
[root@localhost wilcal]# urpmi python
Package python-2.7.17-1.1.mga7.x86_64 is already installed

Verify that pip is installed correctly.
$ pip --version
pip 19.0.3 from /usr/lib/python3.7/site-packages/pip (python 3.7)

Use pip to install the EB CLI.
$ pip install awsebcli --upgrade --user

Verify that the EB CLI installed correctly.
$ eb --version
EB CLI 3.16.0 (Python 3.7.5)

To upgrade to the latest version, run the installation command again.
$ pip install awsebcli --upgrade --user

$ aws configure

[adminuser] 
aws_access_key_id = adminuser access key ID 
aws_secret_access_key = adminuser secret access key 
region = region

That's the way it should work.
Comment 1 William Kenney 2019-12-09 23:40:38 CET
In VirtualBox, M7, Plasma, 64-bit

Before trying to install awsebcli the following occurs:

[root@localhost wilcal]# urpmi python2
Package python-2.7.17-1.1.mga7.x86_64 is already installed
[root@localhost wilcal]# urpmi python2-pip
Package python2-pip-19.0.3-1.mga7.noarch is already installed
[wilcal@localhost wilcal]# pip --version
Traceback (most recent call last):
  File "/usr/bin/pip", line 7, in <module>
    from pip._internal import main
ModuleNotFoundError: No module named 'pip'
[root@localhost wilcal]# pip install awsebcli --upgrade --user
Traceback (most recent call last):
  File "/usr/bin/pip", line 7, in <module>
    from pip._internal import main
ModuleNotFoundError: No module named 'pip'
Comment 2 William Kenney 2019-12-09 23:41:32 CET
Moving on I installed Python3

install task-python3 and python3-pip

For task-python3
The following 9 packages are going to be installed:

- glibc-devel-2.29-19.mga7.x86_64
- kernel-userspace-headers-5.3.13-2.mga7.x86_64
- lib64python3-devel-3.7.5-1.mga7.x86_64
- lib64python3.7-testsuite-3.7.5-1.mga7.x86_64
- lib64xcrypt-devel-4.4.6-1.mga7.x86_64
- python3-docs-3.7.5-1.mga7.noarch
- task-python3-1-2.mga7.noarch
- tkinter3-3.7.5-1.mga7.x86_64
- tkinter3-apps-3.7.5-1.mga7.x86_64
- tkinter3-apps-3.7.5-1.mga7.x86_64

For python3-pip
The following package is going to be installed:

- python3-pip-19.0.3-1.mga7.noarchc

[root@localhost wilcal]# urpmi python3
Package python3-3.7.5-1.mga7.x86_64 is already installed
[root@localhost wilcal]# urpmi python3-pip
Package python3-pip-19.0.3-1.mga7.noarch is already installed

Now this works:

[wilcal@localhost ~]$ pip --version
pip 19.0.3 from /usr/lib/python3.7/site-packages/pip (python 3.7)

Now try to install the AWS awsebcli modules in a user terminal:

[wilcal@localhost ~]$ pip install awsebcli --upgrade --user
Collecting awsebcli
  Downloading https://files.pythonhosted.org/packages/a6/26/e723d628db031a434466597f2310779a46d41e7548a83e04188c55aaa051/awsebcli-3.16.0.tar.gz (242kB)
    100% |████████████████████████████████| 245kB 4.2MB/s 
Collecting botocore<1.14,>1.13.0 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/1a/69/abe9276bc3699aa89cd5f743982aa9d9ce929d7f95b4900bba60922747b0/botocore-1.13.35-py2.py3-none-any.whl (5.8MB)
    100% |████████████████████████████████| 5.8MB 2.6MB/s 
Collecting cement==2.8.2 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/70/60/608f0b8975f4ee7deaaaa7052210d095e0b96e7cd3becdeede9bd13674a1/cement-2.8.2.tar.gz (165kB)
    100% |████████████████████████████████| 174kB 5.9MB/s 
Collecting colorama<0.4.0,>=0.3.9 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting future<0.17.0,>=0.16.0 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/00/2b/8d082ddfed935f3608cc61140df6dcbf0edea1bc3ab52fb6c29ae3e81e85/future-0.16.0.tar.gz (824kB)
    100% |████████████████████████████████| 829kB 6.1MB/s 
Collecting pathspec==0.5.9 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/84/2a/bfee636b1e2f7d6e30dd74f49201ccfa5c3cf322d44929ecc6c137c486c5/pathspec-0.5.9.tar.gz
Requirement already satisfied, skipping upgrade: python-dateutil<2.8.1,>=2.1 in /usr/lib/python3.7/site-packages (from awsebcli) (2.8.0)
Collecting PyYAML<=3.13,>=3.10 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
    100% |████████████████████████████████| 276kB 20.2MB/s 
Collecting requests<2.21,>=2.20.1 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl (57kB)
    100% |████████████████████████████████| 61kB 8.0MB/s 
Requirement already satisfied, skipping upgrade: setuptools>=20.0 in /usr/lib/python3.7/site-packages (from awsebcli) (41.0.0)
Collecting semantic_version==2.5.0 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/bd/8d/49a968bafda84c2f1c39a9ed429e37cb75cc03896e8d6b873001e6456fad/semantic_version-2.5.0-py3-none-any.whl
Collecting six<1.12.0,>=1.11.0 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting termcolor==1.1.0 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.24.1 in /usr/lib/python3.7/site-packages (from awsebcli) (1.24.3)
Collecting wcwidth<0.2.0,>=0.1.7 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/7e/9f/526a6947247599b084ee5232e4f9190a38f398d7300d866af3ab571a5bfe/wcwidth-0.1.7-py2.py3-none-any.whl
Collecting docker-compose<1.24.0,>=1.23.2 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/1e/6c/bf9879305530c4b765ef4eb3be76202788ca1037aec74d2c0ec73191d467/docker_compose-1.23.2-py2.py3-none-any.whl (131kB)
    100% |████████████████████████████████| 133kB 21.2MB/s 
Collecting blessed>=1.9.5 (from awsebcli)
  Downloading https://files.pythonhosted.org/packages/fe/31/ab33f900a948491470bbd7f204a1887dcd4d7f4dbba6160eab5d55a84875/blessed-1.16.1-py2.py3-none-any.whl (63kB)
    100% |████████████████████████████████| 71kB 6.8MB/s 
Collecting jmespath<1.0.0,>=0.7.1 (from botocore<1.14,>1.13.0->awsebcli)
  Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting docutils<0.16,>=0.10 (from botocore<1.14,>1.13.0->awsebcli)
  Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)
    100% |████████████████████████████████| 552kB 24.0MB/s 
Requirement already satisfied, skipping upgrade: idna<2.8,>=2.5 in /usr/lib/python3.7/site-packages (from requests<2.21,>=2.20.1->awsebcli) (2.7)
Collecting certifi>=2017.4.17 (from requests<2.21,>=2.20.1->awsebcli)
  Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
    100% |████████████████████████████████| 163kB 21.5MB/s 
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.7/site-packages (from requests<2.21,>=2.20.1->awsebcli) (3.0.4)
Collecting docker<4.0,>=3.6.0 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/09/da/7cc7ecdcd01145e9924a8ccbe9c1baf3a362fc75d4cb150676eb5231ea60/docker-3.7.3-py2.py3-none-any.whl (134kB)
    100% |████████████████████████████████| 143kB 21.8MB/s 
Collecting texttable<0.10,>=0.9.0 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/02/e1/2565e6b842de7945af0555167d33acfc8a615584ef7abd30d1eae00a4d80/texttable-0.9.1.tar.gz
Collecting cached-property<2,>=1.2.0 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/3b/86/85c1be2e8db9e13ef9a350aecd6dea292bd612fa288c2f40d035bb750ded/cached_property-1.5.1-py2.py3-none-any.whl
Collecting docopt<0.7,>=0.6.1 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz
Collecting websocket-client<1.0,>=0.32.0 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)
    100% |████████████████████████████████| 204kB 12.5MB/s 
Collecting dockerpty<0.5,>=0.4.1 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/8d/ee/e9ecce4c32204a6738e0a5d5883d3413794d7498fe8b06f44becc028d3ba/dockerpty-0.4.1.tar.gz
Collecting jsonschema<3,>=2.5.1 (from docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/77/de/47e35a97b2b05c2fadbec67d44cfcdcd09b8086951b331d82de90d2912da/jsonschema-2.6.0-py2.py3-none-any.whl
Collecting docker-pycreds>=0.4.0 (from docker<4.0,>=3.6.0->docker-compose<1.24.0,>=1.23.2->awsebcli)
  Downloading https://files.pythonhosted.org/packages/f5/e8/f6bd1eee09314e7e6dee49cbe2c5e22314ccdb38db16c9fc72d2fa80d054/docker_pycreds-0.4.0-py2.py3-none-any.whl
Installing collected packages: jmespath, docutils, botocore, cement, colorama, future, pathspec, PyYAML, certifi, requests, semantic-version, six, termcolor, wcwidth, docker-pycreds, websocket-client, docker, texttable, cached-property, docopt, dockerpty, jsonschema, docker-compose, blessed, awsebcli
  Running setup.py install for cement ... done
  Running setup.py install for future ... done
  Running setup.py install for pathspec ... done
  Running setup.py install for PyYAML ... done
  Running setup.py install for termcolor ... done
  Running setup.py install for texttable ... done
  Running setup.py install for docopt ... done
  Running setup.py install for dockerpty ... done
  Running setup.py install for awsebcli ... done
Successfully installed PyYAML-3.13 awsebcli-3.16.0 blessed-1.16.1 botocore-1.13.35 cached-property-1.5.1 cement-2.8.2 certifi-2019.11.28 colorama-0.3.9 docker-3.7.3 docker-compose-1.23.2 docker-pycreds-0.4.0 dockerpty-0.4.1 docopt-0.6.2 docutils-0.15.2 future-0.16.0 jmespath-0.9.4 jsonschema-2.6.0 pathspec-0.5.9 requests-2.20.1 semantic-version-2.5.0 six-1.11.0 termcolor-1.1.0 texttable-0.9.1 wcwidth-0.1.7 websocket-client-0.56.0
[wilcal@localhost ~]$ 

aws intalls but does not run.

[wilcal@localhost ~]$ aws configure
bash: aws: command not found
Comment 3 William Kenney 2019-12-09 23:43:22 CET
At first look Python2 pip does not work at all
and on Python3 works only partially.
Comment 4 Lewis Smith 2019-12-10 19:43:28 CET
Hello Bill

The last bit of the otherwise good Python3 experience looks trivial, a PATH question. Can you explore that, Bill?
But for Python2...
BTW It looks as if Mageia 7 is the last for Python2, so better to go for Python3 where possible henceforth.

Assigning to the Python group; CC the registered maintainer & most recent committers.

Assignee: bugsquad => python
CC: (none) => geiger.david68210, jani.valimaa, mageia

Comment 5 William Kenney 2019-12-10 20:19:58 CET
Thanks Lewis. I'm no Python expert but I'm trying to find ways that
Mageie 7 & AWS S3 can work with each other. I continue to try this
and that but I don't think I can even pip install basic Python
games using Python2.

I agree on the PATH situation. Problem is I can't seem to find where
Python3 is fully installing the awsebcli packages. And thanks for
referring this to the Mageia Python Group. This just has to be a
fairly simple solution.

I'll continue to post notes on this bug as I stumble along.
Comment 6 William Kenney 2019-12-10 20:30:42 CET
Now for the really good news. Somehow initially stumbling along
in a Vbox M7 Client, 64-bit Plasma, I actually got it to
successfully install and work. I should have kept better
notes on how I did that so what I am doing now is trying to
rediscover that process.

I have full Mageia 7 & AWS S3 CLI functionality. I can upload
and download 100's if not 1000's of files using sync. I have an S3
bucket named wilcal.backup.bucket and with that I can fully communicate
with using AWS cli in my local Vbox client. So it can work.
I just have to rediscover how I initially set things up.
But I have found that Python2 pip seems to be mostly non-functional.

Thanks again
Comment 7 Jani Välimaa 2019-12-10 20:40:02 CET
awsebcli doesn't install binary called aws, but eb and ebp. One can find them from ~/.local/bin if pip install is used with --user.

awscli is needed for aws binary.
Comment 8 papoteur 2019-12-10 21:04:22 CET
(In reply to William Kenney from comment #6)
> But I have found that Python2 pip seems to be mostly non-functional.
> 
Hell Bill,
pip is now the command for Python3, and pip2 is for Python2.
pip --version
pip 19.0.3 from /usr/lib/python3.7/site-packages/pip (python 3.7)
pip2 --version
pip 19.0.3 from /usr/lib/python2.7/site-packages/pip (python 2.7)

CC: (none) => yves.brungard_mageia

Comment 9 William Kenney 2019-12-10 21:33:47 CET
Thanks guys. Gives me more things to tinker with.

FWIW Mageia & AWS S3 buckets love each other. :-))
I hope to have this easily documented and posted on the discussion forums.
Comment 10 William Kenney 2019-12-11 02:07:46 CET
Works every time in 64-bit Plasma, Gnome and Xfce

Solution:

In the MCC install task-python3 & python3-pip

In a root terminal verify version of Python3 & python3-pip
[root@localhost user]# urpmi python3
Package python3-3.7.5-1.mga7.x86_64 is already installed
[root@localhost user]# urpmi python3-pip
Package python3-pip-19.0.3-1.mga7.noarch is already installed

In a user terminal verify that pip is installed correctly and working
[user@localhost ~]$ pip --version
pip 19.0.3 from /usr/lib/python3.7/site-packages/pip (python 3.7)

In a user terminal use pip to install the eb cli
( eb = Elastic Beanstalk / cli = Command Line Interface )
[user@localhost ~]$ pip install awsebcli --upgrade --user
Lots of things install

In a user terminal verify that eb cli installed correctly and is working.
[user@localhost ~]$ eb --version
EB CLI 3.16.0 (Python 3.7.5)

In a user terminal use pip to install the aws files
[user@localhost ~]$ pip install awscli --upgrade --user
Lots of things install

In a user terminal use pip to again install awscli looking
for any updates to the aws files
[user@localhost ~]$ pip install awscli --upgrade --user

In a user terminal verify that aws installed and is working correctly
[user@localhost ~]$ aws --version
aws-cli/1.16.300 Python/3.7.5 Linux/5.3.13-desktop-2.mga7 botocore/1.13.36

In a user terminal configure aws
[user@localhost ~]$ aws configure
AWS Access Key ID [None]: xxxxxxxxxxxxxxxxxxxxxx
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxxxxxxxxxxxxx
Default region name [None]:  ( cr )
Default output format [None]: ( cr )

In a user terminal list your s3 buckets
[user@localhost ~]$ aws s3 ls
Lists the buckets in your aws s3 account

Status: NEW => RESOLVED
Resolution: (none) => FIXED


Note You need to log in before you can comment on or make changes to this bug.