Bug 10102 - Unable to install Python packages with C extensions via pip and virtualenv
Summary: Unable to install Python packages with C extensions via pip and virtualenv
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 3
Hardware: x86_64 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Mageia Bug Squad
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: python-2.7.5 10989
Blocks:
  Show dependency treegraph
 
Reported: 2013-05-15 05:01 CEST by Dorian Pula
Modified: 2013-08-17 19:14 CEST (History)
3 users (show)

See Also:
Source RPM: lib64python-devel-2.7.3-7.mga3, python-pip-1.3.1-2.mga3, python-virtualenv-1.7.1.2-4.mga3
CVE:
Status comment:


Attachments
Output from PIP (10.51 KB, text/x-log)
2013-05-15 05:04 CEST, Dorian Pula
Details

Description Dorian Pula 2013-05-15 05:01:37 CEST
Description of problem:
Installing Python packages via pip inside a virtualenv fails if the Python package uses a C extension.  gcc is unable to find the headers for the Python dev package and the installation fails.

Version-Release number of selected component (if applicable):
- lib64python-devel-2.7.3-7.mga3
- python-pip-1.3.1-2.mga3
- python-virtualenv-1.7.1.2-4.mga3

How reproducible:


Steps to Reproduce:
1. Make sure lib64-python-devel, python-pip, python-virtualenv, gcc are installed.

2. Create a virtualenv using:
 virtualenv --distribute $venv

3. Activate the virtualenv with:
 source $venv/bin/activate

4. Attempt to install fabric (which has a C extension dependency) using:
 pip install fabric


Reproducible: 

Steps to Reproduce:
Comment 1 Dorian Pula 2013-05-15 05:04:06 CEST
Created attachment 3977 [details]
Output from PIP

Added output from pip when trying to install the fabric Python package.
David Walser 2013-05-15 06:02:00 CEST

CC: (none) => makowski.mageia, nicolas.lecureuil, oliver.bgr

David Walser 2013-05-15 18:53:13 CEST

CC: (none) => lev

David Walser 2013-05-20 12:45:32 CEST

CC: (none) => cazzaniga.sandro

Sandro CAZZANIGA 2013-05-20 13:04:09 CEST

Assignee: bugsquad => cazzaniga.sandro

Comment 2 Sandro CAZZANIGA 2013-05-20 13:27:02 CEST
I take a look on virtualenv, can someone look at python-pip ?
Sandro CAZZANIGA 2013-05-20 13:27:15 CEST

Assignee: cazzaniga.sandro => bugsquad

Lev Givon 2013-05-20 14:07:40 CEST

CC: lev => (none)

Comment 3 Philippe Makowski 2013-06-01 15:02:29 CEST
I don't think it is python-pip the problem

outside virtualenv all is ok , but inside vitualenv distutils don't provide correct links

inside virtualenv :
gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags build/temp.linux-x86_64-2.7/src/MD2.o -o build/lib.linux-x86_64-2.7/Crypto/Hash/_MD2.so

outside virtualenv :
    gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags build/temp.linux-x86_64-2.7/src/MD2.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/Crypto/Hash/_MD2.so

you see outside virtualenv we have :
-L/usr/lib64 -lpython2.7 

seems the same kind of bug than 9395
Comment 4 Sandro CAZZANIGA 2013-06-01 18:04:25 CEST
So it should just been a fix in linking in python3.
Comment 5 Philippe Makowski 2013-06-01 18:15:20 CEST
not really, we have the same kind of bug with Python3, but now here it is with Python2 under virtualenv, but both bugs are very close.
Comment 6 Philippe Makowski 2013-06-10 22:59:18 CEST
by the way, looking at this bug, I see that virtualenv is bundling its own copies of distribute and pip, presumably to install when it creates a virtualenv.

rpm -ql python-virtualenv |grep tar.gz
/usr/lib/python2.7/site-packages/virtualenv_support/distribute-0.6.24.tar.gz
/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.1.tar.gz


and pip-1.1 have a security issue CVE-2013-1888 mga #10007

It should be better if we can avoid bundling distribute and pip
Comment 7 Philippe Makowski 2013-06-12 16:42:47 CEST
Dorian,inside the virtualenv, what return :

python-config --ldflags

what about trying before pip install fabric :

LDFLAGS="$LDFLAGS `python-config --ldflags`"

?
Comment 8 Philippe Makowski 2013-06-20 15:00:52 CEST
For the record :
inside the virtualenv, edit the file :
 lib/python2.7/config/Makefile 

and remove "-Wl,--no-undefined" from the LDFLAGS def
and then no problem, you can install fabric
Comment 9 Philippe Makowski 2013-06-20 18:41:13 CEST
bug is in the python package (2 and 3), we need to change them to remove "-Wl,--no-undefined" from LDFLAGS in their respective ..../config/Makefile

it should also resolve bug 9395
Comment 10 Philippe Makowski 2013-06-22 13:09:53 CEST
for Python2 and mga3, this bug should be solved with python-2.7.3-7.1.mga3
Python3 need to be fixed too
Philippe Makowski 2013-07-17 18:57:19 CEST

Depends on: (none) => 10761

Comment 11 Dorian Pula 2013-08-13 01:12:27 CEST
I can verify that virtualenv (1.9.1-1.1.mga3) and C extensions work for Python 2.7.5 (2.7.5-1.1.mga3).  I will test against Python 3, when I have the chance to do so.
Comment 12 Dorian Pula 2013-08-13 01:38:18 CEST
Still having issues on Python 3 though, as reported above: 3.3.0-4.mga3
Comment 13 Philippe Makowski 2013-08-13 13:45:51 CEST
Should be solved in next update 3.3.0-4.2.mga3
Comment 14 Dorian Pula 2013-08-13 21:00:16 CEST
Just an update, installations of C extensions on Python 3 work after removing "-Wl,--no-undefined" in CONFIG_ARGS and CONFIGURE_LDFLAGS from /usr/lib64/python3.3/config-3.3m/Makefile .

Looking forward to 3.3.0-4.2.mga3 :)
Sandro CAZZANIGA 2013-08-14 07:11:18 CEST

CC: cazzaniga.sandro => (none)

David Walser 2013-08-15 04:00:56 CEST

Depends on: (none) => 10989

Comment 15 David Walser 2013-08-17 19:14:34 CEST
Fixed in the Bug 10989 update:
http://advisories.mageia.org/MGASA-2013-0250.html

Status: NEW => RESOLVED
Resolution: (none) => FIXED


Note You need to log in before you can comment on or make changes to this bug.