Bug 21441 - Mageia's /usr/bin/perl is much slower than a perlbrew 5.26.0 one or the old 5.22.x perl
Summary: Mageia's /usr/bin/perl is much slower than a perlbrew 5.26.0 one or the old 5...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: x86_64 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Shlomi Fish
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-08-03 17:16 CEST by Shlomi Fish
Modified: 2017-09-07 21:15 CEST (History)
6 users (show)

See Also:
Source RPM: perl-5.26.0-2.mga7.src.rpm
CVE:
Status comment:


Attachments

Description Shlomi Fish 2017-08-03 17:16:55 CEST
Description of problem:

While running this program and any other program I tried:

#!/usr/bin/perl

use strict;
use warnings;

my $x = 0;
for my $i (1 .. 100_000_000)
{
    $x = $x * 2 + 1;
}

I am getting this:

shlomif@telaviv1:~$ time perl time-me-1.pl 

real    0m21.258s
user    0m21.257s
sys     0m0.000s
shlomif@telaviv1:~$ perlbrew use 5.26.0-mt

A sub-shell is launched with 5.26.0-mt as the activated perl. Run 'exit' to finish it.

shlomif@telaviv1:~$ time perl time-me-1.pl

real    0m6.500s
user    0m6.499s
sys     0m0.001s


=====

It also happens if I remove the strict and warnings statements.

I also recall the old 5.22.3 mageia perl being faster.

Version-Release number of selected component (if applicable):

Cauldron.
Comment 1 Shlomi Fish 2017-08-03 17:31:29 CEST
I've now rebuilt and installed perl from mgarepo and i am still getting the same results.
Comment 2 Shlomi Fish 2017-08-03 17:55:39 CEST
After patching the mgarepo perl with the below patch, rebuilding and installing, the problem was fixed:

shlomif[rpms]:$mageia/perl$ svn di
Index: SPECS/perl.spec
===================================================================
--- SPECS/perl.spec     (revision 1134389)
+++ SPECS/perl.spec     (working copy)
@@ -1,5 +1,5 @@
 %define threading 1
-%define debugging 1
+%define debugging 0
 %define _disable_ld_no_undefined 1
 %bcond_with int64
Comment 3 Shlomi Fish 2017-08-03 18:20:41 CEST
(In reply to Shlomi Fish from comment #2)
> After patching the mgarepo perl with the below patch, rebuilding and
> installing, the problem was fixed:
> 
> shlomif[rpms]:$mageia/perl$ svn di
> Index: SPECS/perl.spec
> ===================================================================
> --- SPECS/perl.spec     (revision 1134389)
> +++ SPECS/perl.spec     (working copy)
> @@ -1,5 +1,5 @@
>  %define threading 1
> -%define debugging 1
> +%define debugging 0
>  %define _disable_ld_no_undefined 1
>  %bcond_with int64

i should note that doing that broke all perl XS extensions (and urpmi) but improved the speed of the XML-LibXML's test suite.
Comment 4 Frédéric "LpSolit" Buclin 2017-08-03 18:56:50 CEST
Slowness confirmed.

On Mageia 6, perl-5.22.3-3.mga6,        real hw:  6.030s
On Mageia 6, self-compiled perl 5.22.4, real hw:  5.609s
On Mageia 6, self-compiled perl 5.26.0, real hw:  3.717s

On Cauldron, perl-5.26.0-2.mga7,           VBox: 14.260s
On Cauldron, self-compiled perl 5.26.0,    VBox:  3.895s

(the perf boost between 5.22 and 5.26 is pretty impressive!)

CC: (none) => LpSolit

Comment 5 Marja Van Waes 2017-08-03 23:15:55 CEST
Thanks for working on it, Shlomi :-)

Assignee: bugsquad => shlomif
CC: (none) => marja11

Comment 6 Nicolas Lécureuil 2017-08-05 22:08:47 CEST
Thierry any idea about this ?

CC: (none) => mageia, thierry.vignaud

Comment 7 Nicolas Lécureuil 2017-08-05 22:24:35 CEST
i think we can disable debuging but i would like to have your approval first.
Comment 8 Thierry Vignaud 2017-08-06 14:19:04 CEST
Yeah it has been enabled by error
Comment 9 Thierry Vignaud 2017-08-06 14:20:58 CEST
Done in SVN.
I fear this might need a boostrap build though (to be confirmed)
I that's the case, I would suggest we investigate the use of int64 by default on 32bit like FC does (I've added support but disabled by default through %bcond)
Comment 10 Nicolas Lécureuil 2017-08-06 21:32:47 CEST
just pushed on the BS ( to see if this break the BS or not ).

i don't know all this enough to work on the use of int64.

Shlomi Fish can you help thierry on this ?
Comment 11 Nicolas Lécureuil 2017-08-06 23:09:25 CEST
I: [iurt] [iurt] --------------- Command failed, full output follows ---------------
I: [iurt] [iurt] D: [iurt_root_command] chroot
/usr/bin/perl: symbol lookup error: /usr/lib/perl5/vendor_perl/5.26.0/x86_64-linux-thread-multi/auto/Math/Int64/Int64.so: undefined symbol: Perl_pad_sv
I: [iurt_root_command] ERROR: chroot
D: [iurt_root_command] chroot
/usr/bin/perl: symbol lookup error: /usr/lib/perl5/vendor_perl/5.26.0/x86_64-linux-thread-multi/auto/Math/Int64/Int64.so: undefined symbol: Perl_pad_sv
I: [iurt_root_command] ERROR: chroot


how to fix ? 

by rebuilding it on a clean chroot like first bootstraping ?
Comment 12 Nicolas Lécureuil 2017-08-07 00:38:25 CEST
perl breakage reverted, sorry.


"I that's the case, I would suggest we investigate the use of int64 by default on 32bit like FC does (I've added support but disabled by default through %bcond)"


How to work on this ?
Comment 13 Thierry Vignaud 2017-08-07 12:17:43 CEST
64bit build is OK but on 32bit, we need to find out the right missing -D option used by FC in order to get rid of -int64 (else file lists differs between 64 & 32bit arches).
Though FC has also moved !binary modules to /usr/share
Comment 14 Frédéric "LpSolit" Buclin 2017-08-20 01:47:46 CEST
(In reply to Thierry Vignaud from comment #13)
> 64bit build is OK but on 32bit, we need to find out the right missing -D
> option used by FC in order to get rid of -int64 (else file lists differs
> between 64 & 32bit arches).

For now, could we revert this change? Performance matters more than supporting 64bit integers on 32 bit machines. If the want 64bit support, then they should use a 64bit machine.
Comment 15 Thierry Vignaud 2017-08-26 08:55:03 CEST
That means 2 painful bootstrap instead of only one.
Actually it looks I already did all the work from a quick chroot test.
So here it comes:
http://svnweb.mageia.org/packages?view=revision&revision=1147909
Then Nicolas or another sysadmin will have to use iurt/rebuild_perl_iurt
Comment 16 Nicolas Lécureuil 2017-08-26 09:19:23 CEST
ok i will do this, this week end.

you have increased the release of the rpm builded with rebuild_perl_iurt ?
Comment 17 Nicolas Lécureuil 2017-08-26 09:31:54 CEST
build pb:

URPM.c: In function 'XS_URPM__Transaction_check':
URPM.xs:2603:7: warning: variable 'translate_message' set but not used [-Wunused-but-set-variable]
   int translate_message = 0;
       ^
URPM.xs: In function 'XS_URPM_parse_rpm':
URPM.xs:3033:31: error: 'RPMVSF_NOPAYLOAD' undeclared (first use in this function)
                   vsflags |= (RPMVSF_NOPAYLOAD | RPMVSF_NOSHA1HEADER);
                               ^
URPM.xs:3033:31: note: each undeclared identifier is reported only once for each function it appears in
URPM.xs:3037:46: error: 'RPMVSF_NOSHA256HEADER' undeclared (first use in this function)
                   vsflags |= (RPMVSF_NOMD5 | RPMVSF_NOSHA256HEADER);
                                              ^
URPM.xs: In function 'XS_URPM_get_gpg_fingerprint':
URPM.xs:3125:9: warning: implicit declaration of function 'pgpPubkeyKeyID' [-Wimplicit-function-declaration]
         pgpPubkeyKeyID (pkt, pktlen, fingerprint);
         ^
URPM.c: In function 'XS_URPM__Package_EVR':
URPM.xs:1508:10: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result]
          asprintf(&res, "%d:%s", epoch, version);
          ^
Makefile:343: recipe for target 'URPM.o' failed
make: *** [URPM.o] Error 1
error: Bad exit status from /home/iurt/rpmbuild/tmp/rpm-tmp.MTUi63 (%build)
Comment 18 Thierry Vignaud 2017-08-26 11:15:51 CEST
That's b/c URPM is now targeting rpm-4.14.
I can workaround a compat patch
Comment 19 Nicolas Lécureuil 2017-08-26 12:47:15 CEST
can you just modify the rebuild script for updates_testing to be enabled ?
Comment 20 Nicolas Lécureuil 2017-08-26 21:40:55 CEST
i just commited for this ( i forgot to add the bug reference btw ).

Rebuild in progress
Comment 21 Thierry Vignaud 2017-08-26 23:32:27 CEST
I've restored URPM' support for rpm-4.13.x in svn
Comment 22 Thierry Vignaud 2017-08-27 00:01:38 CEST
(In reply to Nicolas Lécureuil from comment #20)
> i just commited for this ( i forgot to add the bug reference btw ).

"svn propedit svn:log --revprop -rXXX" is your friend :-)
(already done)
Comment 23 Thierry Vignaud 2017-08-28 11:14:54 CEST
(In reply to Nicolas Lécureuil from comment #20)
> i just commited for this ( i forgot to add the bug reference btw ).
> 
> Rebuild in progress

What's the status, Nicolas?
Comment 24 Nicolas Lécureuil 2017-08-28 11:19:58 CEST
in progress ( i will start arm in some minutes ).

Do i upload first 64 and 32 bit and then arm when available or do i upload all at once ?
Comment 25 Thierry Vignaud 2017-08-28 11:24:44 CEST
I guess all at once would be better (it would be bad to still have to doble bootstrap arm :-) )
Comment 26 Thierry Vignaud 2017-08-29 11:43:59 CEST
So what's the status :-) ?
Comment 27 Nicolas Lécureuil 2017-08-29 17:46:52 CEST
32 and 64bits and armv7 OK

i am doing armv5 now
Comment 28 Nicolas Lécureuil 2017-08-30 00:00:48 CEST
done.

I am rebuilding the hdlist now.
Comment 29 Thierry Vignaud 2017-08-31 17:06:48 CEST
The fixed Perl has been moved to core/release and a bunch of perl modules have already been rebuilt for it too (WIP).
We did both disabling debuging & enabling 64bit integers on 32bit arches in order to prevent another later bootstrap for 32bit arches

I've fixed the in main perl so that it's not installed in "*-64int".

Though there are still references to "-64int" in rebuilt modules.
eg:
/usr/lib/perl5/vendor_perl/5.26.0/i386-linux-thread-multi-64int/URPM.pm

I think this can be safely fixed later, w/o breaking anything, providing we temporary add the proper directories to @INC
Ideally this should be saved before the big glibc-2.26/gcc-7 rebuild :-)

CC: (none) => jquelin, shlomif
Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 30 Mageia Robot 2017-09-07 21:15:15 CEST
commit 4920fbf71ee20c8cfd44fa6a73c06f2159932cff
Author: Thierry Vignaud <thierry.vignaud@...>
Date:   Thu Sep 7 20:24:35 2017 +0200

    adapt to URPM-5.14 (using perl native ints)
    
    See mga#21441
---
 Commit Link:
   http://gitweb.mageia.org/software/drakx/commit/?id=4920fbf71ee20c8cfd44fa6a73c06f2159932cff

Note You need to log in before you can comment on or make changes to this bug.