Description of problem: If URPMI is set to use aria2 as downloader, there is no download progress displayed Version-Release number of selected component (if applicable): How reproducible: Every time URPMI downloading packages. Steps to Reproduce: 1. Set downloader in /etc/urpmi/urpmi.cfg to aria2. 2. Install some package via urpmi command. Reproducible: Steps to Reproduce:
CC: sysadmin-bugs => (none)Assignee: bugsquad => thierry.vignaudComponent: Release (media or process) => RPM Packages
Confirmed here (Mageia Caulrdon i586)
CC: (none) => piscestong
aria2-1.19.0 must have changed its output, thus breaking urpmi parsing code
Severity: normal => minorSource RPM: (none) => urpmi
Hi, I and Shlomi Fish were going a little deeper to this bug and found that effectively the problem is in aria2c. In particular starting from aria 1.16 (i.e. MGA3) the output of the progress download string of aria2c changed from: [#1 SIZE:176.0KiB/2.5MiB(6%) CN:3 SPD:256.22KiBs ETA:09s] to [#2c8dae 496KiB/830KiB(59%) CN:1 DL:84KiB ETA:3s] so the urpmi parser had to be rewritten. I and Shlomi rewrite the parser, and the result is shown in this patch in attach.
CC: (none) => ghibomgx
Created attachment 7112 [details] patch for urpmi progress downloading string with aria2
adding myself to CC.
CC: (none) => shlomif
Status: NEW => ASSIGNED
Keywords: (none) => PATCH
CC: (none) => doktor5000
Comment on attachment 7112 [details] patch for urpmi progress downloading string with aria2 >--- /usr/lib/perl5/vendor_perl/5.20.1/urpm/download.pm.orig 2015-05-18 05:18:06.000000000 +0200 >+++ /usr/lib/perl5/vendor_perl/5.20.1/urpm/download.pm 2015-10-10 21:41:05.178272050 +0200 >@@ -694,7 +694,7 @@ > "/usr/bin/aria2c", $options->{debug} ? ('--log', "$options->{dir}/.aria2.log") : @{[]}, > '--auto-file-renaming=false', > '--ftp-pasv', >- '--summary-interval=0', >+ '--summary-interval=1', > '--follow-metalink=mem', > $medium->{mirrorlist} ? ( > '--metalink-enable-unique-protocol=true', # do not try to connect to the same server using the same protocol >@@ -762,8 +762,38 @@ > propagate_sync_callback($options, 'start', $file) > if !$options->{is_retry}; > } >+ >+ # >+ # aria2c > 1.14 changed the output of the progress download string >+ # it's no longer what follows: > #parses aria2c: [#1 SIZE:176.0KiB/2.5MiB(6%) CN:3 SPD:256.22KiBs ETA:09s] >- if ($buf =~ m!^\[#\d*\s+\S+:([\d\.]+\w*).([\d\.]+\w*)\S([\d]+)\S+\s+\S+\s*([\d\.]+)\s\w*:([\d\.]+\w*)\s\w*:(\d+\w*)\]$!) { >+ # if ($buf =~ m!^\[#\d*\s+\S+:([\d\.]+\w*).([\d\.]+\w*)\S([\d]+)\S+\s+\S+\s*([\d\.]+)\s\w*:([\d\.]+\w*)\s\w*:(\d+\w*)\]$!) { >+ # >+ # but here is the new one from aria2c 1.16 and beyond: >+ # >+ #parses aria2c: [#2c8dae 496KiB/830KiB(59%) CN:1 DL:84KiB ETA:3s] >+ # >+ # so the old parser no longer works. The following string parser has been rewritten by Shlomi Fish with use of >+ # multiline mode and comments for better readability, should >+ # aria2c progress download string change again: >+ # >+ if ($buf =~ m! >+ ^\[\#[\dA-Fa-f]+ # match #2c8dae >+ \s+ >+ ([\d\.]+\w*) # Match 496KiB >+ / >+ ([\d\.]+\w*) # Match 830KiB >+ \s* \( (\d+) % \) # Match (59%) >+ \s+ >+ CN:(\S+) # Match CN:1 >+ \s+ >+ DL:(\S+) # Match DL:84KiB >+ \s+ >+ ETA:(\w+) >+ \]$ >+ !msx >+ ) >+ { > my ($total, $percent, $speed, $eta) = ($2, $3, $5, $6); > #- $1 = current downloaded size, $4 = connections > if (propagate_sync_callback($options, 'progress', $file, $percent, $total, $eta, $speed) eq 'canceled') {
(
CC: (none) => dominidomini
CC: (none) => luigiwalser
commit e5cdde09bc13d9d39400feadc9703a4194253d0f Author: Shlomi Fish <shlomif@...> Date: Mon Jul 11 10:53:26 2016 +0200 adapt to aria2c-1.16+ output (mga#16500) aria2c > 1.14 changed the output of the progress download string it's no longer what follows: "[#1 SIZE:176.0KiB/2.5MiB(6%) CN:3 SPD:256.22KiBs ETA:09s]" but here is the new one from aria2c 1.16 and beyond: "[#2c8dae 496KiB/830KiB(59%) CN:1 DL:84KiB ETA:3s]" So the old parser no longer works... The string parser has been rewritten by Shlomi Fish with use of multiline mode and comments for better readability, should aria2c progress download string change again Dug & fixed by Giuseppe Ghibò <ghibomgx@gmail.com> and Shlomi Fish <shlomif@shlomifish.org> --- Commit Link: http://gitweb.mageia.org/software/rpm/urpmi/commit/?id=e5cdde09bc13d9d39400feadc9703a4194253d0f
Fixed in git Thanks for the patch and sorry for the response delay
Status: ASSIGNED => RESOLVEDResolution: (none) => FIXED
commit a0067ef8d20f28f6947e2669a700b07319a147d5 Author: Shlomi Fish <shlomif@...> Date: Mon Jul 11 10:53:26 2016 +0200 adapt to aria2c-1.16+ output (mga#16500) aria2c > 1.14 changed the output of the progress download string it's no longer what follows: "[#1 SIZE:176.0KiB/2.5MiB(6%) CN:3 SPD:256.22KiBs ETA:09s]" but here is the new one from aria2c 1.16 and beyond: "[#2c8dae 496KiB/830KiB(59%) CN:1 DL:84KiB ETA:3s]" So the old parser no longer works... The string parser has been rewritten by Shlomi Fish with use of multiline mode and comments for better readability, should aria2c progress download string change again Dug & fixed by Giuseppe Ghibò <ghibomgx@gmail.com> and Shlomi Fish <shlomif@shlomifish.org> --- Commit Link: http://gitweb.mageia.org/software/rpm/urpmi/commit/?id=a0067ef8d20f28f6947e2669a700b07319a147d5
*** Bug 17988 has been marked as a duplicate of this bug. ***
CC: (none) => herman.viaene