Description of problem: I use wget to download large files of astronomical images. If the filesize is shorter the 2GB, the file if correctly downloaded. If the size is greater that 2GB, only the first 2GB are downloaded. Version-Release number of selected component (if applicable): wget 1.21.1 How reproducible: Always on mageia 8, 32 bits version, on a i586 oldtimer Steps to Reproduce: 1. Create a file of say 3GB on a http server 2. Try to download it with wget 3. Very integrity
Summary: wget do not complete downloading of files greater them 2GB => wget do not complete downloading of files greater then 2GB
What's the output of "mount|grep ^/dev" on the system where wget is being run?
CC: (none) => davidwhodgins
/dev/sdb1 on /run/media/raoul/200E4E0C0E4DDB82 type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2) More than 4GB files are present on this external had disk drive.
While it shows it's handled by fuse, it doesn't show the file system type which is what I was looking for. What's the output (as root) of "blkid /dev/sdb*"?
blkid /dev/sdb* /dev/sdb: PTUUID="77878535-9d90-4fa5-9662-cc0d1dfd3e67" PTTYPE="gpt" /dev/sdb1: BLOCK_SIZE="512" UUID="200E4E0C0E4DDB82" TYPE="ntfs" PARTLABEL="Elements" PARTUUID="29dfac0f-24f6-4139-9f06-1551d14b4ba0"
Assigning to all packagers as wget has no specifically registered maintainer. Please also test using curl to download a file thats being truncated with wget to see if that works.
Assignee: bugsquad => pkg-bugs
Also, please include the output of "df -h|grep sdb| and "df -i|grep sdb".
Here are the results of the asked actions: df -h|grep sdb /dev/sdb1 3.7T 3.4T 256G 94% /run/media/raoul/200E4E0C0E4DDB82 df -i|grep sdb /dev/sdb1 259M 3.0M 256M 2% /run/media/raoul/200E4E0C0E4DDB82 curl -o curled.zip https://www.grosfichiers.com/jCR7[...] ls -S curled.zip 2604645063 curled.zip unzip -t curled.zip says it's ok wget -O wgetted.zip https://www.grosfichiers.com/jCR7[...] ls -s wgetted.zip 2147483647 wgetted.zip I also redirected the output of wget in a file, using -o option. Results of the end of the listing is: 2096850K .......... .......... .......... .......... .......... 99% 6.04M 0s 2096900K .......... .......... .......... .......... .......... 99% 6.27M 0s 2096950K .......... .......... .......... .......... .......... 99% 5.55M 0s 2097000K .......... .......... .......... .......... .......... 99% 6.30M 0s 2097050K .......... .......... .......... .......... .......... 99% 6.35M 0s 2097100K .......... .......... .......... .......... .......... 99% 5.88M 0s 2097150K . 100% 1.24M=5m56s 2021-05-20 19:56:41 (5.76 MB/s) — « wgetted.zip » sauvegardé [2147483647/2147483647] unzip -t complains about the End-of-file missing signature. Playing with the wgetted.zip filesize: ln(2147483647+1)/ln(2) is exactly 31. So it seems that there is a limitation to 31 bits. I don't know if it is in wget or in the grosfichiers.com server. Can someone try with a different server ? Downloading with firefox is ok.
Bug confirmed. On a Mageia 8 i586 install, wget shows ... $ wget http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso --2021-05-20 17:06:55-- http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso Resolving mirror.math.princeton.edu (mirror.math.princeton.edu)... 128.112.18.21 Connecting to mirror.math.princeton.edu (mirror.math.princeton.edu)|128.112.18.21|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2147483647 (2.0G) [application/octet-stream] Saving to: ‘Mageia-8-i586.iso’ Note the 2.0G length On a Mageia 8 x86_64 install it shows ... wget http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso --2021-05-20 17:07:36-- http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso Resolving mirror.math.princeton.edu (mirror.math.princeton.edu)... 128.112.18.21 Connecting to mirror.math.princeton.edu (mirror.math.princeton.edu)|128.112.18.21|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4398635008 (4.1G) [application/octet-stream] Saving to: ‘Mageia-8-i586.iso’ In my case both file systems are using ext4, so ntfs is not relevant.
Severity: minor => major
> So it seems that there is a limitation to 31 bits. > I don't know if it is in wget or in the grosfichiers.com server. Well, given that "curl -o curled.zip https://www.grosfichiers.com/jCR7[...]" works, it seems that there is no issue with grosfichiers.com ? regards
CC: (none) => boulshet
On a Mageia 7 i586 install the size is correct, so this bug has been introduced in Mageia 8. $ wget http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso --2021-05-20 19:22:18-- http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso Resolving mirror.math.princeton.edu (mirror.math.princeton.edu)... 128.112.18.21 Connecting to mirror.math.princeton.edu (mirror.math.princeton.edu)|128.112.18.21|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4398635008 (4.1G) [application/octet-stream] Saving to: ‘Mageia-8-i586.iso’
Please test with wget-1.21.1-1.1.mga8 in core/updates_testing.
Assignee: pkg-bugs => qa-bugs
Advisory: ======================== Updated wget package fixes an file size issue. The updated wget package fixes an issue for downloading files >= 2GB on i586 arch by applying a patch from Fedora. References: https://bugs.mageia.org/show_bug.cgi?id=28948 ======================== Updated packages in core/updates_testing: ======================== wget-1.21.1-1.1.mga8 from SRPM: wget-1.21.1-1.1.mga8.src.rpm
CC: (none) => ouaurelien
Fixes the issue. Validating the update $ wget http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso --2021-05-22 13:22:18-- http://mirror.math.princeton.edu/pub/mageia/iso/8/Mageia-8-i586/Mageia-8-i586.iso Resolving mirror.math.princeton.edu (mirror.math.princeton.edu)... 128.112.18.21 Connecting to mirror.math.princeton.edu (mirror.math.princeton.edu)|128.112.18.21|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4398635008 (4.1G) [application/octet-stream] Saving to: ‘Mageia-8-i586.iso’
Keywords: (none) => validated_updateWhiteboard: (none) => MGA8-32-OKCC: (none) => sysadmin-bugs
Keywords: (none) => advisory
An update for this issue has been pushed to the Mageia Updates repository. https://advisories.mageia.org/MGAA-2021-0123.html
Status: NEW => RESOLVEDResolution: (none) => FIXED