Bug 10102

Summary:	Unable to install Python packages with C extensions via pip and virtualenv
Product:	Mageia	Reporter:	Dorian Pula <dorian.pula>
Component:	RPM Packages	Assignee:	Mageia Bug Squad <bugsquad>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Normal	CC:	mageia, makowski.mageia, oliver.bgr
Version:	3
Target Milestone:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Source RPM:	lib64python-devel-2.7.3-7.mga3, python-pip-1.3.1-2.mga3, python-virtualenv-1.7.1.2-4.mga3	CVE:
Status comment:
Bug Depends on:	10761, 10989
Bug Blocks:
Attachments:	Output from PIP

Description Dorian Pula 2013-05-15 05:01:37 CEST

Description of problem:
Installing Python packages via pip inside a virtualenv fails if the Python package uses a C extension.  gcc is unable to find the headers for the Python dev package and the installation fails.

Version-Release number of selected component (if applicable):
- lib64python-devel-2.7.3-7.mga3
- python-pip-1.3.1-2.mga3
- python-virtualenv-1.7.1.2-4.mga3

How reproducible:


Steps to Reproduce:
1. Make sure lib64-python-devel, python-pip, python-virtualenv, gcc are installed.

2. Create a virtualenv using:
 virtualenv --distribute $venv

3. Activate the virtualenv with:
 source $venv/bin/activate

4. Attempt to install fabric (which has a C extension dependency) using:
 pip install fabric


Reproducible: 

Steps to Reproduce:

Comment 1 Dorian Pula 2013-05-15 05:04:06 CEST

Created attachment 3977 [details]
Output from PIP

Added output from pip when trying to install the fabric Python package.

David Walser 2013-05-15 06:02:00 CEST

CC: (none) => makowski.mageia, nicolas.lecureuil, oliver.bgr

David Walser 2013-05-15 18:53:13 CEST

CC: (none) => lev

David Walser 2013-05-20 12:45:32 CEST

CC: (none) => cazzaniga.sandro

Sandro CAZZANIGA 2013-05-20 13:04:09 CEST

Assignee: bugsquad => cazzaniga.sandro

Comment 2 Sandro CAZZANIGA 2013-05-20 13:27:02 CEST

I take a look on virtualenv, can someone look at python-pip ?

Sandro CAZZANIGA 2013-05-20 13:27:15 CEST

Assignee: cazzaniga.sandro => bugsquad

Lev Givon 2013-05-20 14:07:40 CEST

CC: lev => (none)

Comment 3 Philippe Makowski 2013-06-01 15:02:29 CEST

I don't think it is python-pip the problem

outside virtualenv all is ok , but inside vitualenv distutils don't provide correct links

inside virtualenv :
gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags build/temp.linux-x86_64-2.7/src/MD2.o -o build/lib.linux-x86_64-2.7/Crypto/Hash/_MD2.so

outside virtualenv :
    gcc -pthread -shared -Wl,--as-needed -Wl,--no-undefined -Wl,-z,relro -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags build/temp.linux-x86_64-2.7/src/MD2.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/Crypto/Hash/_MD2.so

you see outside virtualenv we have :
-L/usr/lib64 -lpython2.7 

seems the same kind of bug than 9395

Comment 4 Sandro CAZZANIGA 2013-06-01 18:04:25 CEST

So it should just been a fix in linking in python3.

Comment 5 Philippe Makowski 2013-06-01 18:15:20 CEST

not really, we have the same kind of bug with Python3, but now here it is with Python2 under virtualenv, but both bugs are very close.

Comment 6 Philippe Makowski 2013-06-10 22:59:18 CEST

by the way, looking at this bug, I see that virtualenv is bundling its own copies of distribute and pip, presumably to install when it creates a virtualenv.

rpm -ql python-virtualenv |grep tar.gz
/usr/lib/python2.7/site-packages/virtualenv_support/distribute-0.6.24.tar.gz
/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.1.tar.gz


and pip-1.1 have a security issue CVE-2013-1888 mga #10007

It should be better if we can avoid bundling distribute and pip

Comment 7 Philippe Makowski 2013-06-12 16:42:47 CEST

Dorian,inside the virtualenv, what return :

python-config --ldflags

what about trying before pip install fabric :

LDFLAGS="$LDFLAGS `python-config --ldflags`"

?

Comment 8 Philippe Makowski 2013-06-20 15:00:52 CEST

For the record :
inside the virtualenv, edit the file :
 lib/python2.7/config/Makefile 

and remove "-Wl,--no-undefined" from the LDFLAGS def
and then no problem, you can install fabric

Comment 9 Philippe Makowski 2013-06-20 18:41:13 CEST

bug is in the python package (2 and 3), we need to change them to remove "-Wl,--no-undefined" from LDFLAGS in their respective ..../config/Makefile

it should also resolve bug 9395

Comment 10 Philippe Makowski 2013-06-22 13:09:53 CEST

for Python2 and mga3, this bug should be solved with python-2.7.3-7.1.mga3
Python3 need to be fixed too

Philippe Makowski 2013-07-17 18:57:19 CEST

Depends on: (none) => 10761

Comment 11 Dorian Pula 2013-08-13 01:12:27 CEST

I can verify that virtualenv (1.9.1-1.1.mga3) and C extensions work for Python 2.7.5 (2.7.5-1.1.mga3).  I will test against Python 3, when I have the chance to do so.

Comment 12 Dorian Pula 2013-08-13 01:38:18 CEST

Still having issues on Python 3 though, as reported above: 3.3.0-4.mga3

Comment 13 Philippe Makowski 2013-08-13 13:45:51 CEST

Should be solved in next update 3.3.0-4.2.mga3

Comment 14 Dorian Pula 2013-08-13 21:00:16 CEST

Just an update, installations of C extensions on Python 3 work after removing "-Wl,--no-undefined" in CONFIG_ARGS and CONFIGURE_LDFLAGS from /usr/lib64/python3.3/config-3.3m/Makefile .

Looking forward to 3.3.0-4.2.mga3 :)

Sandro CAZZANIGA 2013-08-14 07:11:18 CEST

CC: cazzaniga.sandro => (none)

David Walser 2013-08-15 04:00:56 CEST

Depends on: (none) => 10989

Comment 15 David Walser 2013-08-17 19:14:34 CEST

Fixed in the Bug 10989 update:
http://advisories.mageia.org/MGASA-2013-0250.html

Status: NEW => RESOLVED
Resolution: (none) => FIXED