Bug 11283 - Python headers included are bungled, probably due to multi-arch issues
: Python headers included are bungled, probably due to multi-arch issues
Status: RESOLVED FIXED
Product: Mageia
Classification: Unclassified
Component: RPM Packages
: 3
: i586 Linux
: Normal Severity: normal
: ---
Assigned To: Philippe Makowski
:
:
:
:
: 11785
:
  Show dependency treegraph
 
Reported: 2013-09-24 19:07 CEST by Denis Bitouzé
Modified: 2015-05-08 12:08 CEST (History)
3 users (show)

See Also:
Source RPM: python-2.7.5-1.2.mga3.src.rpm
CVE:
Status comment:


Attachments
correct virualenv.py (94.51 KB, text/plain)
2013-09-28 11:48 CEST, Philippe Makowski
Details

Description Denis Bitouzé 2013-09-24 19:07:05 CEST
Description of problem:
Plone (not provided by Mageia) installation (with its installer) fails as if the Python headers included with Mageia were bungled, probably due to multi-arch issues: "pyconfig.h:2:32: error: #include nested too deeply" error (BTW, no problem on Ubuntu for instance).

How reproducible:
Follow the instructions here:
  http://plone.org/documentation/manual/installing-plone/installing-on-linux-unix-bsd/running-the-unified-installer
to try to install Plone

Steps to Reproduce:
1. Run the following commands:

wget https://launchpad.net/plone/4.3/4.3.2/+download/Plone-4.3.2-UnifiedInstaller.tgz
tar xfj Plone-4.3.2-UnifiedInstaller.tar.bz2
cd Plone-4.3.2-UnifiedInstaller/

2. Run the installer for a default installation:

./install.sh standalone

3. You'll see error messages like the following one:

/usr/include/multiarch-dispatch.h:74:56: error: #include nested too
deeply In file included from /usr/include/multiarch-dispatch.h:74:0,

The (almost) complete backtrace is as the following:
#############################################################################################
Testing /usr/bin/python2.7 for Zope/Plone requirements....
/usr/bin/python2.7 looks OK. We'll try to use it.

Rootless install method chosen. Will install for use by system user
bitouze

Detailed installation log being written
to /home/bitouze/InstallationsLocales/Plone-4.3.2-UnifiedInstaller/install.log
Installing Plone 4.3.2 at /home/bitouze/Plone

Creating python virtual environment, no site packages.
New python executable in /home/bitouze/Plone/Python-2.7/bin/python2.7
Also creating executable in /home/bitouze/Plone/Python-2.7/bin/python
Installing
Setuptools....................................................................................................................................................................done.
Installing
Pip...............................................................................................................................................................................................................................................done.
Skipping libjpeg build Unpacking buildout cache
to /home/bitouze/Plone/buildout-cache Copying Plone-docs Copying
buildout skeleton Fixing up bin/buildout
Building Zope/Plone; this takes a while...
Buildout returned an error code: 1; Aborting.
Buildout failed. Unable to continue

Installation has failed.
See the detailed installation log
at /home/bitouze/InstallationsLocales/Plone-4.3.2-UnifiedInstaller/install.log
to determine the cause. $
more /home/bitouze/InstallationsLocales/Plone-4.3.2-UnifiedInstaller/install.log
Detailed installation log Starting at sam. sept. 21 16:54:30 CEST 2013
Creating directory '/home/bitouze/Plone/zinstance/parts'.
Creating directory '/home/bitouze/Plone/zinstance/develop-eggs'.
Getting distribution for 'ZODB3==3.10.5'.
warning: build_py: byte-compiling is disabled, skipping.

warning: install_lib: byte-compiling is disabled, skipping.

warning: install_lib: byte-compiling is disabled, skipping.

warning: easy_install: byte-compiling is disabled, skipping.

Got ZODB3 3.10.5.
Getting distribution for 'zope.interface==3.6.7'.
warning: build_py: byte-compiling is disabled, skipping.

warning: install_lib: byte-compiling is disabled, skipping.

warning: install_lib: byte-compiling is disabled, skipping.

warning: easy_install: byte-compiling is disabled, skipping.
[...]
Got AccessControl 3.0.8.
Installing instance.
Getting distribution for 'Pillow==1.7.8'.
warning: no previously-included files found matching '.hgignore'
warning: no previously-included files found matching '.hgtags'
warning: no previously-included files found matching 'BUILDME.bat'
warning: no previously-included files found matching 'make-manifest.py'
warning: no previously-included files found matching 'SHIP'
warning: no previously-included files found matching 'SHIP.bat'
warning: no previously-included files matching '*' found under
directory 'Tests' warning: build_py: byte-compiling is disabled,
skipping.

In file included
from /home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2:0,
from /usr/include/multiarch-dispatch.h:74,
from /home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2,
from /usr/include/multiarch-dispatch.h:74,
from /home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2,
from /usr/include/multiarch-dispatch.h:74, [...]
[same lines, many times]
[...]
[then:]
                 from /usr/include/python2.7/pyconfig.h:2,
                 from /usr/include/python2.7/Python.h:8,
                 from _imaging.c:75:
/usr/include/multiarch-dispatch.h:74:56: error: #include nested too
deeply In file included from /usr/include/multiarch-dispatch.h:74:0,
                 from /home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2,
                 from /usr/include/multiarch-dispatch.h:74,
[...]
[same lines, many times]
[...]
[then once more:]
                 from /usr/include/python2.7/Python.h:58,
                 from _imaging.c:75:
/home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2:32:
error: #include nested too deeply In file included
from /usr/include/multiarch-dispatch.h:74:0,
from /home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2,
from /usr/include/multiarch-dispatch.h:74, [...]
[same lines, many times]
[...]
[then once more:]
                 from /usr/include/python2.7/Python.h:58,
                 from _imaging.c:77:
/home/bitouze/Plone/Python-2.7/include/multiarch-i386-linux/python2.7/pyconfig.h:2:32:
error: #include nested too deeply In file included from
libImaging/Imaging.h:14:0, from _imaging.c:77:
libImaging/ImPlatform.h:14:2: erreur: #error Sorry, this library
requires support for ANSI prototypes. libImaging/ImPlatform.h:17:2:
erreur: #error Sorry, this library requires ANSI header files.
libImaging/ImPlatform.h:55:2: erreur: #error Cannot find required
32-bit integer type In file included from _imaging.c:77:0:
libImaging/Imaging.h:90:5: erreur: unknown type name ‘INT32’
libImaging/Imaging.h:264:16: erreur: unknown type name ‘INT32’
libImaging/Imaging.h:264:16: erreur: unknown type name ‘INT32’
libImaging/Imaging.h:395:15: erreur: expected ‘=’, ‘,’, ‘;’, ‘asm’ or
‘__attribute__’ before ‘ImagingCRC32’ _imaging.c: In function
‘getlist’: _imaging.c:406:10: erreur: ‘INT32’ undeclared (first use in
this function) _imaging.c:406:10: note: each undeclared identifier is
reported only once for each function it appears in _imaging.c:411:25:
erreur: expected expression before ‘)’ token _imaging.c:418:25: erreur:
expected expression before ‘)’ token _imaging.c: In function
‘getpixel’: _imaging.c:470:7: erreur: unknown type name ‘INT32’
_imaging.c: In function ‘getink’: _imaging.c:558:11: erreur: ‘INT32’
undeclared (first use in this function) _imaging.c:558:17: erreur:
expected expression before ‘)’ token _imaging.c: In function
‘_histogram’: _imaging.c:989:9: erreur: unknown type name ‘INT32’
_imaging.c: In function ‘_point’:
_imaging.c:1152:9: erreur: unknown type name ‘INT32’
_imaging.c:1164:48: erreur: ‘INT32’ undeclared (first use in this
function) _imaging.c: In function ‘_putdata’:
_imaging.c:1295:22: erreur: ‘INT32’ undeclared (first use in this
function) _imaging.c:1322:49: erreur: expected expression before ‘)’
token _imaging.c: In function ‘_getextrema’:
_imaging.c:1822:9: erreur: unknown type name ‘INT32’
_imaging.c: In function ‘_draw_ink’:
_imaging.c:2266:5: erreur: unknown type name ‘INT32’
_imaging.c: In function ‘_crc32’:
_imaging.c:2797:12: erreur: expected ‘=’, ‘,’, ‘;’, ‘asm’ or
‘__attribute__’ before ‘crc’ _imaging.c:2797:12: erreur: ‘crc’
undeclared (first use in this function) _imaging.c:2804:13: erreur:
expected ‘)’ before ‘INT32’ error: Setup script exited with error:
command 'gcc' failed with exit status 1 An error occurred when trying
to install Pillow 1.7.8. Look above this message for any errors that
were output by easy_install. While: Installing instance.
  Getting distribution for 'Pillow==1.7.8'.
Error: Couldn't install: Pillow 1.7.8
*************** PICKED VERSIONS ****************
[versions]

*************** /PICKED VERSIONS ***************
#############################################################################################


Reproducible: 

Steps to Reproduce:
Comment 1 Philippe Makowski 2013-09-27 23:59:09 CEST
for me it is a bug in the pillow install script later versions don't have this problem we package it without problem report this to plone or tweak their setup to use a better version of pillow 1.7.8 is an old version
Comment 2 Philippe Makowski 2013-09-28 11:48:04 CEST
Created attachment 4382 [details]
correct virualenv.py

I did some tests, the problem is in the virtualenv they are setting up, that have include/multiarch-x86_64-linux/python2.7 -> /usr/include/python2.7/
instead of 
include/python2.7 -> /usr/include/python2.7/

so if in their /packages/virtualenv-1.10.1.tar.gz you put this virtualenv.py instead, all is ok
Comment 3 Denis Bitouzé 2013-09-28 18:35:46 CEST
(In reply to Philippe Makowski from comment #2)
> Created attachment 4382 [details]
> correct virualenv.py
> 
> I did some tests, the problem is in the virtualenv they are setting up, that
> have include/multiarch-x86_64-linux/python2.7 -> /usr/include/python2.7/
> instead of 
> include/python2.7 -> /usr/include/python2.7/
> 
> so if in their /packages/virtualenv-1.10.1.tar.gz you put this virtualenv.py
> instead, all is ok

I'll try it, thanks!

As it works like this with Ubuntu, is there a way to have something which works in all cases?
Comment 4 Philippe Makowski 2013-09-30 15:08:35 CEST
yes, report the problem to upstream virtualenv
it seems that these lines (1009:10019) in virtualenv.py (https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1009)

        multiarch_exec = '/usr/bin/multiarch-platform'
        if is_executable_file(multiarch_exec):
            # In Mageia (2) and Mandriva distros the include dir must be like:
            # virtualenv/include/multiarch-x86_64-linux/python2.7
            # instead of being virtualenv/include/python2.7
            p = subprocess.Popen(multiarch_exec, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
            stdout, stderr = p.communicate()
            # stdout.strip is needed to remove newline character
            inc_dir = join(home_dir, 'include', stdout.strip(), py_version + abiflags)
        else:
            inc_dir = join(home_dir, 'include', py_version + abiflags)

are causing more trouble that solving a problem that don't exist anymore under Mageia
Comment 5 Philippe Makowski 2013-09-30 15:15:48 CEST
pull request to upstream : https://github.com/pypa/virtualenv/pull/472
Comment 6 Denis Bitouzé 2013-09-30 18:08:38 CEST
(In reply to Philippe Makowski from comment #5)
> pull request to upstream : https://github.com/pypa/virtualenv/pull/472

I confirm Plone can be installed smoothly on Mageia 3 as soon as virtualenv.py is changed as you said.

Thanks!
Comment 7 Philippe Makowski 2013-12-01 17:48:00 CET
Updated packages in core/updates_testing:
========================

python-virtualenv-1.10.1-1.2.mga3.noarch

Source RPMs:
python-virtualenv-1.10.1-1.2.mga3.src 

This also fix mga#11785
Comment 8 Dave Hodgins 2013-12-05 18:23:40 CET
This bug should be closed when bug 11785 is pushed, as it includes
python-virtualenv-1.10.1-1.2.mga3.src.
Comment 9 claire robinson 2013-12-12 15:29:27 CET
Assigning Philippe again as python-virtualenv is being updated in bug 11785 which is also assigned to QA

Could you update the advisory there please Philippe to reflect the fixes here too.
Comment 10 Philippe Makowski 2013-12-12 19:18:50 CET
Suggested advisory:
========================

Updated python-virtualenv packages fix :

- security vulnerabilities:
  Changed behavior of ssl.match_hostname() to follow RFC 6125 (mga#11785).
  References:
  https://bugs.mageia.org/show_bug.cgi?id=11785
  http://bugs.python.org/issue17997#msg194950

- inc_dir settings to avoid #include nested too deeply" error
  References:
  https://bugs.mageia.org/show_bug.cgi?id=11283


Updated packages in core/updates_testing:
========================

python-virtualenv-1.10.1-1.2.mga3.noarch

Source RPMs:
python-virtualenv-1.10.1-1.2.mga3.src
Comment 11 claire robinson 2013-12-12 20:07:49 CET
Sorry Philippe if it wasn't clear, I was meaning on the other bug. 

We have two bugs assigned for the same package update at the moment and will be using bug 11785 where python-virtualenv is part of the security update rather than this one.

Please leave this one assigned to yourself to avoid the confusion.
Comment 12 Philippe Makowski 2013-12-21 08:59:55 CET
Done
Comment 13 Denis Bitouzé 2015-05-08 10:59:50 CEST
Unfortunately, this bug still arises with Plone 4.3.4 on Mageia 4.
Comment 14 Philippe Makowski 2015-05-08 11:41:54 CEST
(In reply to Denis Bitouzé from comment #13)
> Unfortunately, this bug still arises with Plone 4.3.4 on Mageia 4.

again see my comments, I can't do anything if virtualenv insist on not applying the patch. And why Plone insist on installing their own virualenv.

We patch virtualenv in our package, if you use our package or vitualenv with this patch (http://svnweb.mageia.org/packages/cauldron/python-virtualenv/current/SOURCES/virtualenv-1.10.1-mga-fix_inc_dir.patch?view=markup), all is ok. Report upstream yourself to virtualenv and or plone.
Comment 15 Denis Bitouzé 2015-05-08 12:08:56 CEST
OK, sorry for the noise.

Note You need to log in before you can comment on or make changes to this bug.