Bug 18411 - Isodumper fail getting formats of changed USB stick -> write is bogus despite its own check say OK
Summary: Isodumper fail getting formats of changed USB stick -> write is bogus despite...
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 5
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard: MGA5-32-OK advisory
Keywords: validated_update
Depends on:
Blocks:
 
Reported: 2016-05-08 18:57 CEST by Morgan Leijström
Modified: 2016-09-21 22:39 CEST (History)
4 users (show)

See Also:
Source RPM: isodumper-0.51-3.mga6.src.rpm
CVE:
Status comment:


Attachments
Screenshot of gparted viewing the stick that was isodumped boot.iso over Stepensons rocket (36.73 KB, image/png)
2016-05-08 18:58 CEST, Morgan Leijström
Details
Screenshot of gparted viewing the first stick that was isodumped 32bit boot.iso over 64bit boot iso (24.26 KB, image/png)
2016-05-08 19:08 CEST, Morgan Leijström
Details
Screenshot of gparted viewing the first stick isodumped again after reboot = verified stick works OK (23.80 KB, image/png)
2016-05-08 21:10 CEST, Morgan Leijström
Details
udisks2.py output when things went wrong (5.79 KB, application/gzip)
2016-08-09 11:56 CEST, Morgan Leijström
Details
udisks2.py output after reboot, go wrong also here (5.93 KB, application/gzip)
2016-08-09 15:11 CEST, Morgan Leijström
Details

Description Morgan Leijström 2016-05-08 18:57:13 CEST
What i did, using isodumper only:

put a boot.iso on a USB stick
when i used it i realise i had dowloaded the wrong arch, so back to flashing:

put the correct boot.iso on the same stick.
Result: Still the former iso is on stick!?? the one i overwrote!!
-very confused-
Downloaded cauldron network install 32 bit again but from another mirror,
closed and restarted isodumper, let it flash the iso, 
booted it and it again reports it is wrong arch (but it is a 32 bit P4 CPU)

Put in *another* stick and downloaded the cauldron 32 bit boot iso to that.

That iso have been used for "Stephensons Rocket" (version of SteamOS)
When i booted on that it boot "Stephensons Rocket" installer!!

Now i got really anxious about where the iso i downloaded really ended up.
Launched gparted and saw all drives was OK.
But the stick was pretty funny partitionned:
The original boot partition is left intact, and isodumper have placed a new partition a bit further into the drive.

Also, looking in the terminal from where gparted got launched i see: 

  "
Enhetens handtag påstår att den fysiska blockstorleken är 2048 byte, men Linux påstår att den är 512 byte.
Partition(s) 1 on /dev/sdc have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  As a result, the old partition(s) will remain in use.  You should reboot now before making further changes.
   "

I took a screenshot of gparted, will attach it next.

The first part in swedish translates to: The handle of the unit say the physical blocksize is 2048 byte, but Linux say it is 512 byte.

Looking in the log of isodumper i see confusing lines:

First line, yes it did stil display the no longer connected stick by name, and i first selected it just to see it it showed some error, it did not.
I did not want to test further as sda2 is a PV for my system LVM...
Why is it showing that alternative?  Even if i close and open isodumper and if i press the update button. what is /org/freedesktop/UDisks2/block_devices/ and what do that have to do with sda2, and why is it stil listed anyway - Corsair VoyagerGT got unplugged long ago.

Then, i chosed the other alternative /dev/sdc_1
Why that "_1" part?

__from isodumper log window:__
MÃ¥lenhet: Corsair VoyagerGT (/org/freedesktop/UDisks2/block_devices/sda2) 0Mb
MÃ¥lenhet: SanDisk Extreme (/dev/sdc_1) 30533Mb
Avbild: /home/morgan/Hämtningar/boot-nonfree32.iso
MÃ¥lenhet: SanDisk Extreme (/dev/sdc_1) 30533Mb
Kör kopia från/home/morgan/Hämtningar/boot-nonfree32.iso till /dev/sdc_1
Skrev: 1% 786432 bytes
---8<--
  cut away all percent between 1 and 99...
---8<--
Skrev: 99% 52166656 bytes
Avbildboot-nonfree32.iso skrevs till /dev/sdc_1
Bytes skrivna:52428800
SHA1-värde:c767926ed32e130d9154c418803f2f6dd2ab432f
MD5-värde:bac947228ca455d2e58bd3057966689e


I really wanted to proceed with installing Cauldron, so at this point i rebooted and then wrote the iso to a third stick, no problem.
Comment 1 Morgan Leijström 2016-05-08 18:58:52 CEST
Created attachment 7767 [details]
Screenshot of gparted viewing the stick that was isodumped boot.iso over Stepensons rocket
Comment 2 Morgan Leijström 2016-05-08 19:08:25 CEST
Created attachment 7768 [details]
Screenshot of gparted viewing the first stick that was isodumped 32bit boot.iso over 64bit boot iso

Why the first unallocated 1,37MiB?
Why sdc*2*

I forgot to tell:
The system on where isodumper is run is a fully updated Mageia 5 64 bit.
Comment 3 Morgan Leijström 2016-05-08 21:10:30 CEST
Created attachment 7769 [details]
Screenshot of gparted viewing the first stick isodumped again after reboot = verified stick works OK
Rémi Verschelde 2016-05-10 11:55:49 CEST

Assignee: bugsquad => yves.brungard_mageia

Comment 4 papoteur 2016-05-11 08:32:14 CEST
Hello Morgan,
Thanks for the report.
I think that the mess is related to information reported by Udisks2, because at some  point, there are not coherent with the real state of the devices.
It seems that the writing has been done on a partition instead of a device.
I have to check the device selection using Udisks2 information to avoid such behaviour.
Comment 5 Morgan Leijström 2016-07-06 16:30:11 CEST
Could you please have a look at this.
Running on current cauldron 64 bit:

It still fail i.e i plugged in a 16GB stick with stick with one large ntfs partition, dumped mga6 sta1 on it, but the result was it had a one partition of strange type 16GB large, and installer boot fail.

Tried writing again after reboot, same stick, same file and this time the result is a FAT16 partition of 4 megabyte.

In both cases isodumper say it wrote and verified OK.

Then I used dd to make the stick instead and that just worked.
(at a bit higher bitrate too)
Comment 6 papoteur 2016-07-10 17:50:36 CEST
Hi Morgan,
In case you encounter again the problem, please give the output of this program :
http://people.canonical.com/~zyga/udisks2.py
use it in command line like :
python3 udisk2.py > udisk2-output.txt
- one time when you see the problem
- one time after a reboot, and with the stick in place.

Isodumper works like dd. The only difference can be the name of the destination path. It comes from what udisks2 reports.
Comment 7 Morgan Leijström 2016-07-11 23:49:36 CEST
OOPS!!
I downloaded that .py in Firefox and lazyfully clicked it in firefox download list thinking i should see it in a text editor - but apparently it got executed by something somehow - screen immediately went black, monitor indicator lamp flash as in standby, and myself slaps my forehead... killing (session, not myself...) by ctrl-alt-backspace works, but as soon as i log into plasma again screen is black again. Ctrl-Alt-Fn terminals work.

I can use other DE.


Have you run this script in Cauldron KDE Plasma yourself?

BTW should the .py be run as root?
Comment 8 Morgan Leijström 2016-07-11 23:55:47 CEST
Updating bug variables to the fact it lately is about current version on cauldron; adding "MGA5TOO" and will check there again when fixed on cauldron/mga6.

Version: 5 => Cauldron
Source RPM: isodumper-0.44-1.mga5.src.rpm => isodumper-0.51-3.mga6.src.rpm
Whiteboard: (none) => MGA5TOO

Comment 9 papoteur 2016-07-12 09:24:22 CEST
I Morgan,
I found that the cauldron release has a bug, which is not the one you found first.
Marja will, I hope, give a try, and then I will publish it on cauldron.
Note that the verification only calculate the sums MD5 and SHA1 without checking them against a reference.
You can get the sum with :
sha1sum boot.iso
md5sum boot.iso
Comment 10 papoteur 2016-07-12 09:29:10 CEST
Note also that the bug is only with small images.
Comment 11 Mageia Robot 2016-07-12 22:42:18 CEST
commit 6ccd9019ef1364ee3de83a1a1f05d5d4ed72a6c5
Author: Papoteur <papoteur@...>
Date:   Tue Jul 12 22:42:00 2016 +0200

    Correction of writing tiny images (mga#18411)
---
 Commit Link:
   http://gitweb.mageia.org/software/isodumper/commit/?id=6ccd9019ef1364ee3de83a1a1f05d5d4ed72a6c5
Comment 12 papoteur 2016-07-13 08:46:09 CEST
Morgan,
can you test it ?
One way to do it get isodumper.py here
http://gitweb.mageia.org/software/isodumper/tree/lib/isodumper.py?id=6ccd9019ef1364ee3de83a1a1f05d5d4ed72a6c5
and to put it in /usr/lib/isodumper/ as root.
Comment 13 papoteur 2016-07-25 09:47:44 CEST
Release 0.52 is out with the correction.
Reopen this if not solved.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 14 Morgan Leijström 2016-08-09 11:35:58 CEST
Weirdnesses...

Put netinstall .iso on a 16GB stick.
(it was before containing the sta1 DVD put there by dd)
Result now: it contain a 4GB fat16 partition! (as viewed by gparted)

(afterthought: isodumper may have shown target not o be the driv but a partition on it, see below, i do not remember if i read it carefully.)

Then I put the netinstall .iso on the stick using dd.
Gparted say there is now only a 2MB fgat16, sdd2.
Very strange as the iso file is much larger...

BUT i used that stick to boot and install cauldron successfully.

So either gparted also is broken or it get strange data from system, or the iso file contain a weird image with error in partition table partition but it somehow works anyway.

Just for trying i plugged in the same stick in same machine and tried to run isodumper again.  It said write error at 4%.
Opening my eyes i see the target selection box say /dev/sdd2  !  (should be without the "2".

Of course there is problem at 4% when writing 50MB to a 2MB partition... :)

   What is strange is:

* why i got that 2MB partition on the USB stick by using dd to put the netinstall boot image there.   Error in netinstall iso?

* why isodumper selects sdd2 instead of sdd.  And that is the only option in that selection box.

Status: RESOLVED => REOPENED
Resolution: FIXED => (none)

Comment 15 Morgan Leijström 2016-08-09 11:56:30 CEST
Created attachment 8320 [details]
udisks2.py output when things went wrong

Log from running
# python3 udisks2.py > udisk2-output.txt
+ waiting a couple seconds
+ then removing the stick
+ waiting some seconds
+ then reinserting the stick
Ctrl-C
Comment 16 Morgan Leijström 2016-08-09 15:11:51 CEST
Created attachment 8321 [details]
udisks2.py output after reboot, go wrong also here

Now i had the stick in while booting.
Soon started udisks.py again, output attached.

Ran Isodumper and the only option now is sdd1 (before boot it selected was sdd2, but why never sdd ?)
Gparted however still say there only exist sdd2 (still 2MB) and no other partition.
So it did not write where drive partition table is.
Also when trying to again opening isodumper it can select sdd1 only. 
But how could it select and say it wrote to sdd1 when gparted say sdd1 do not exist??

removed the stick, pause, attached it again
Isodumper stil offer sdd1 only, gparted say only sda2 exist.
Another computer verifies there is only one partition sda2 of 2MB.

And anyhow: isodumper should write to disk, not partition!
Comment 17 Morgan Leijström 2016-08-09 15:23:25 CEST
Verified the behaviour on laptop, and also with other sticks of different size and original partitioning: I never offer to write to the drive any time, always to sdx1 or sdx2 !

Sidenote: the segmentation fault at closing is still there:
<_M_> [ui] YUILoader.cc:104 deleteUI(): Shutting down UI
/usr/libexec/isodumper: rad 16:  4384 Segmenteringsfel        python3 $DIR/isodumper.py $1
Comment 18 Mageia Robot 2016-08-21 19:38:00 CEST
commit 9cb191880de2767f1c0b2d6bd535e9f92b491b98
Author: Papoteur <papoteur@...>
Date:   Sun Aug 21 19:28:06 2016 +0200

    Better detection of the device (mga#18411), avoiding to write to a partition.
    
    The device is obtained through Drive list of udisks2, but without
    /dev/sdxx name. This is got trough Block list, but without direct
    correspondance. Thus, we get the first block which has 'Drive' the
    selected Drive, and suppress the trailing digits.
    Clean unused code
---
 Commit Link:
   http://gitweb.mageia.org/software/isodumper/commit/?id=9cb191880de2767f1c0b2d6bd535e9f92b491b98
Comment 19 Mageia Robot 2016-08-21 20:29:51 CEST
commit 39f0d84e9c117e94c6b01898c2d4a74c51c6cab3
Author: Papoteur <papoteur@...>
Date:   Sun Aug 21 19:52:12 2016 +0200

    Better detection of the device (mga#18411), avoiding to write to a partition.
    
    The device is obtained through Drive list of udisks2, but without
        /dev/sdxx name. This is got trough Block list, but without direct
        correspondance. Thus, we get the first block which has 'Drive' the
        selected Drive, and suppress the trailing digits.
        Clean unused code
---
 Commit Link:
   http://gitweb.mageia.org/software/isodumper/commit/?id=39f0d84e9c117e94c6b01898c2d4a74c51c6cab3
Comment 20 papoteur 2016-08-21 20:49:54 CEST
Thanks Morgan.
The output of udisks2.py helped me to figure out what was wrong.
I reported also the correction to the Mageia 5 release.
Papoteur
Comment 21 papoteur 2016-08-27 15:26:50 CEST
Suggested advisory:
========================

Updated isodumper package fixes bug in the naming of the detected device. The correction ensures that no partition name is used.

References:
https://bugs.mageia.org/show_bug.cgi?id=18411
========================

Updated packages in core/updates_testing:
========================

isodumper-0.48-1.mga5.noarch.rpm

Source RPM: 
isodumper-0.48-1.mga5.src.rpm

Assignee: yves.brungard_mageia => qa-bugs

Comment 22 Herman Viaene 2016-08-29 11:31:36 CEST
MGA-32 on AcerD620 Xfce
No installation issues.
To be sure I understood the issue correctly, here is what I did:
Used a USB3.0 stick, made two partitions on it using gparted on desktop PC. Made sure the partitions were visible in the Acer laptop.
Started Isodumper, checked I could only select sdb device, not sdb1 or 2.
Dumped netboot iso on the stick and checked this one booted correctly.
One thing strikes me (but did not investigate further): gparted now lists the stick as one unallocated area (except 4Mb in front), but udisks shows device sde and parts sde1 and sde2 with a different udisk id, but both having the same "Mageia 5 ....." label.

CC: (none) => herman.viaene

Herman Viaene 2016-08-29 11:32:13 CEST

Whiteboard: MGA5TOO => MGA5TOO MGA5-32-OK

David Walser 2016-08-31 20:37:59 CEST

Version: Cauldron => 5
Whiteboard: MGA5TOO MGA5-32-OK => MGA5-32-OK

papoteur 2016-09-03 17:37:37 CEST

CC: (none) => yves.brungard_mageia

Comment 23 Dave Hodgins 2016-09-12 00:01:12 CEST
gparted doesn't properly show iso images on a block device, especially where
the a fat partition (required for uefi boot) is inside of the first partition
which contains the boot image.

Advisory added to svn. Validating the update.

Keywords: (none) => validated_update
Whiteboard: MGA5-32-OK => MGA5-32-OK advisory
CC: (none) => davidwhodgins, sysadmin-bugs

Comment 24 Mageia Robot 2016-09-21 22:39:06 CEST
An update for this issue has been pushed to the Mageia Updates repository.

http://advisories.mageia.org/MGAA-2016-0112.html

Status: REOPENED => RESOLVED
Resolution: (none) => FIXED


Note You need to log in before you can comment on or make changes to this bug.