Bug 15422 - Update Isodumper to v0.41
Summary: Update Isodumper to v0.41
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 4
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard: has_procedure advisory mga4-32-ok MGA...
Keywords: validated_update
Depends on:
Blocks:
 
Reported: 2015-03-04 14:21 CET by papoteur
Modified: 2015-04-03 21:38 CEST (History)
7 users (show)

See Also:
Source RPM: isodumper
CVE:
Status comment:


Attachments

Description papoteur 2015-03-04 14:21:37 CET
Description of problem:
The release adds a feature to write images in a format for booting with UEFI. This is useful for upgrading from Mageia 4 to Mageia 5.


Reproducible: 

Steps to Reproduce:
Comment 1 Rémi Verschelde 2015-03-04 14:24:18 CET
I'll add the advisory later today.

CC: (none) => geiger.david68210, remi
Assignee: bugsquad => qa-bugs

Rémi Verschelde 2015-03-04 14:24:29 CET

Source RPM: (none) => isodumper

Comment 2 claire robinson 2015-03-05 16:57:37 CET
Nice work, it's really a useful piece of software.

A couple of comments:

It's impossible to resize the window. When an iso is selected the iso name is truncated in the button.

The sync operation takes alot longer (with my well used usb stick anyway) to complete, after the Success message appears saying to remove the stick. It might be wise to build in a 60second-ish delay to allow sync to finish before displaying the Success message.


Verified that with UEFI ticked it had created a vfat formatted partition and copied the iso contents to it..

# blkid | grep MGALIVE
/dev/sdc1: LABEL="MGALIVE" UUID="2870-CC11" TYPE="vfat" PARTUUID="00004736-01" 

$ ls /var/run/media/claire/MGALIVE/
autorun.inf  boot.catalog  dosutils/  EFI/  isolinux/  x86_64/

Tested a few other dumps, backups and formats, no issues.
Comment 3 claire robinson 2015-03-05 17:02:07 CET
I was testing mga4 64 but I don't have EFI hardware to verify the dump so have not added the Ok.

Can somebody test this with EFI hardware please.

Whiteboard: (none) => has_procedure

Comment 4 Samuel Verschelde 2015-03-05 17:05:53 CET
About the message being displayed too early, this is a bug in my opinion, since users can be invited to remove the stick too early. A fixed delay would only make it too long for others with efficient hardware without ensuring it works for everyone.

Papoteur : if you mount the stick for copying the files (which I guess you do), do you use the sync mount option? This allows to solve disk caching issues IIRC. Or, you could simply not display the message until you've managed to unmount the key.
Comment 5 claire robinson 2015-03-05 17:13:37 CET
Yes, indeed. I'll add feedback marker for now as it is likely to cause problems.

Still would like people with EFI to Ok this one though please.

Whiteboard: has_procedure => has_procedure feedback

Comment 6 Dave Hodgins 2015-03-05 21:59:33 CET
To ensure the device has actually finished being written, use
ps -Afl |grep usb-storage|grep -v grep
and ensure D (device wait) is not in the second column of any of the
returned lines. Probably best to repeat the check once per second, and
wait a few seconds of no device waits, as it can go into sleep mode
for short periods.

CC: (none) => davidwhodgins

Comment 7 papoteur 2015-03-07 23:16:37 CET
A correction will be uploaded with:
A wait for unmount the USB key after writing.
A new calcultation for progressbar based on file size instead of number.
More occasions to refresh the window.
Papoteur
Comment 8 Ben McMonagle 2015-03-08 03:51:04 CET
uefi option enabled.
wrote Mageia-5-rc-x86_64-DVD.iso to usb,
boot usb in efi mode, installed from usb in efi mode - ok

check ok for me.

CC: (none) => westel

Comment 9 papoteur 2015-03-08 08:19:20 CET
(In reply to papoteur from comment #7)
> A correction will be uploaded with:
> A wait for unmount the USB key after writing.
> A new calcultation for progressbar based on file size instead of number.
> More occasions to refresh the window.
> Papoteur

Pushed on mga repos.
The writing function is altered, thus this should be checked.
Comment 10 Rémi Verschelde 2015-03-08 20:17:25 CET
New version to test:
  isodumper-0.41-1.mga4


It fixes some of the above-mentioned issues:
 - added a wait for unmount the USB key after writing.
 - new calcultation for progressbar based on file size instead of
number in UEFI mode.
 - added occasions to refresh the window.
 - replace the shutil.copy2 with built-in instructions.
 - update English (United Kingdom) translation.

Summary: Update Isodumper to v0.40 => Update Isodumper to v0.41
Whiteboard: has_procedure feedback => has_procedure

Comment 11 Rémi Verschelde 2015-03-08 21:36:38 CET
Suggested advisory:
===================

Updated isodumper package provides support for Mageia 5 UEFI-compatible ISOs

  In preparation for the final release of Mageia 5 with UEFI support, isodumper
  has been updated to support writing ISOs to USB sticks following our UEFI
  procedure. This new feature is being made accessible to Mageia 4 users through
  the normal updates channel, in order to ease the upgrade path to Mageia 5.

  Notable changes since isodumper 0.25:
   - UEFI feature, creates a FAT32 partition and copies the contents of the ISO
   - Support to format devices in FAT32, NTFS or ext4
   - Support for SD cards
   - Many UI improvements
   - New translations and translation updates
   - Many other bug fixes

References:
===========
- http://gitweb.mageia.org/software/isodumper/tree/CHANGELOG?id=0.41
Comment 12 Ben McMonagle 2015-03-10 07:36:16 CET
enable uefi option

backup - ok
write to Mageia-5-rc-x86_64-DVD.iso usb device - ok
check format - fat32, named drive MGALive
boot usb device uefi mode - ok

note: when attempting to increase the vertical size of the information window to read more info, the whole Gui increases in size, including  buttons.
is this an expected result?
Comment 13 Dave Hodgins 2015-03-10 17:28:46 CET
(In reply to papoteur from comment #9)
> (In reply to papoteur from comment #7)
> > A correction will be uploaded with:
> > A wait for unmount the USB key after writing.
> > A new calcultation for progressbar based on file size instead of number.
> > More occasions to refresh the window.
> > Papoteur
> 
> Pushed on mga repos.
> The writing function is altered, thus this should be checked.

Assuming the only change is the addition of os.fsync(fdst) in the raw_write
and files_write routines, I don't think this will work for non-efi installs.

Also, assuming the python file system sync works in the same way as the sync
command from the coreutils package, it will work for efi installs, as the
files are being written to a mounted vfat filesystem.

For a non-efi install, the image is being written to a device. There is no
mounted file system involved, so the sync command has no affect. I found that
out the hard way, back when I was working on copyiso2usb. Never got around
to adding the check, but manually used comment 6, with the intention of adding
it to the script, but had to stop work on the project for various reasons.
Comment 14 papoteur 2015-03-11 14:01:19 CET
Thanks Dave for yours comments, it's interesting.
For UEFI format, thus the files_write routine, the os.fsync(fdst) seems to have no effects. I added a wait for the umount command, and this is the solution for waiting the end of writing:
            self.process = Popen(['umount', temp_dir ], shell=False, stdout=PIPE)
            while True :
                self.process.poll()
                if self.process.returncode != None:
                    break
I didn't notice any problem in the raw_write routine. I will check again.
Comment 15 papoteur 2015-03-11 17:44:46 CET
@Dave,
I just tested the dump mode writing operation (raw_write routine), with and without the instruction
os.fsync(ofc)
with: the progress bar progresses smoothly. After attaining 100%, the check operation is started. The check is OK.
without: the progress bar attains 100% quickly and the led's device blinks more time. Then the check operation is started. The check is OK.
Comment 16 Antoine Dumondel 2015-03-19 18:01:55 CET
Suggested advisory:
==================

When you widen the main window the banner space is biggest than the rest. Specificly biggest than the details space. And in this last one we only can scroll to see what happen (if we didn't have opened isodumper on a terminal or viewing the log). Maybe is it possible to change that ?

After following step by step the process (wiki.mageia.org/en/QA_procedure:Isodumper), it seems no bug appeared. Everything is OK.

CC: (none) => antoine.dumondel
Whiteboard: has_procedure => MGA4-64-OK

Comment 17 Lewis Smith 2015-04-02 20:49:18 CEST
Tested MGA4 x64 real hardware, using an x64 KDELive ISO image, 8Gb clé USB.

1] Dumped the *.iso file/image to the clé. Using its size (n * 2048) to steer 'dd', read it back into another disc file. MD5summed that, it gave the correct MD5 checksum, so it was OK.

2] Formatted the clé USB FAT32.
I had started by intending to dump the ISO *contents* to the clé for EFI booting, and choosing the ISO image file, expected the formatting to be a necessary first step. I was surprised that after formatting the clé, that was that; no continuation with the dump.
ISOdumper does a host of useful things, but the flow is not alwys evident.

3] Dumped the ISO 'for EFI' (i.e. its contents) to the previously formatted clé USB.
What would happen if the clé was *not* already formatted? Would that be done automatically, or would it fail?
For the EFI ISO *contents* copy, it would be nice if this fact was mentioned in the progress bar text.
At the end of the progress bar, the device remained visibly busy a *long time* as residual data was written. Correctly, ISOdumper did not signal the copy end until the device was quiet.
It would be re-assuring if during the long period when the progress bar is finished but not the write, that the bar text changed to 'finalising' or something in the same spirit.
I mounted the same ISO image locally, and compared its contents on disc with those dumped to the clé USB. Counts of files from 'tree' were identical; more precisely, so were the total sizes from 'ls -lR'. So this also is OK.

MGA4-64-OK

CC: (none) => lewyssmith

Comment 18 claire robinson 2015-04-03 14:01:56 CEST
Validating. Advisory uploaded.

Please push to 4 updates

Thanks

Keywords: (none) => validated_update
Whiteboard: MGA4-64-OK => has_procedure advisory mga4-32-ok MGA4-64-OK
CC: (none) => sysadmin-bugs

Comment 19 Mageia Robot 2015-04-03 15:12:01 CEST
An update for this issue has been pushed to Mageia Updates repository.

http://advisories.mageia.org/MGAA-2015-0033.html

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 20 Lewis Smith 2015-04-03 21:38:31 CEST
Addition to Comment 17, late in the day, but worth knowing:
1] The clé USB produced in test 3 successfully booted to a working Live KDE system. (I should have tried that before).
2] Repeated Test 3 to produce an EFI bootable clé USB from the latest Gnome x64 DVD ISO (contents) with the clé UNformatted (still with an ISO image), showed that ISOdumper does indeed format it correctly. It says so in its log, but without precisions like label or filesystem. So there is *no* need to pre-format the clé USB as per Test 2.
All of which supports the validity of OK'ing this update.

Note You need to log in before you can comment on or make changes to this bug.