Bug 23654 - allow isodumper to have the option to enable a "persistence partition"
Summary: allow isodumper to have the option to enable a "persistence partition"
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 6
Hardware: All Linux
Priority: Normal enhancement
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard: MGA6-64-OK MGA6-32-OK
Keywords: IN_RELEASENOTES6, advisory, validated_update
Depends on:
Blocks:
 
Reported: 2018-10-08 21:01 CEST by ben mcmonagle
Modified: 2019-02-20 21:58 CET (History)
10 users (show)

See Also:
Source RPM: isodumper
CVE:
Status comment:


Attachments

Description ben mcmonagle 2018-10-08 21:01:15 CEST
Description of problem: from a discussion with Martin Whitaker:

 "you mentioned in the past that the Mga 7 lives
    would have "persistence" as a feature/function.

In fact that feature is included on the 6.1 64-bit Live ISOs. I backported
the fix that allows the ISOs to be used with multibootusb, and the
persistence feature came with it." 

As the latest Live ISOs (and probably Mga7 Live ISOs onwards) now have this function available, would it be feasible to add this function as an option to Isodumper.

Many other distros Lives have the ability to have persistence enabled for a USB .

I am prepared to assist with the testing of this feature.



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. enable Isodumper to have the option to add a persistence partition  
2.
3.
Comment 1 ben mcmonagle 2018-10-08 21:05:58 CEST
meant to add:

"It would certainly be nice if Isodumper supported it. The procedure is
fairly straightforward - after you've dumped the ISO, just add another
partition on the USB stick, format it (ext4 filesystem is probably best)
and set the partition label to "mgalive-persist". But both diskdrake and
gparted don't allow you to create the extra partition - they see the
iso9660 partition and assume the device is optical media - so you have to
use fdisk and mkfs."
Comment 2 Morgan Leijström 2018-10-08 22:12:08 CEST
Persistence on live isos as USB stick is working "out of the box"?!
Did we announce this nice feature properly?

CC: (none) => fri

Comment 3 ben mcmonagle 2018-10-08 23:06:29 CEST
(In reply to Morgan Leijström from comment #2)
> Persistence on live isos as USB stick is working "out of the box"?!
> Did we announce this nice feature properly?

according to MartinW, it came as a "bonus" with the fix for use with "multibootusb".

as the 6.1s were an rollup of existing updates, no need really to announce the change for the persistence feature as some work is needed to be done to make a persistent USB.

Likely it will be announced for the Mga7 Lives though.

So, my thoughts are, as this function is now available, we have the excellent Isodumper, is this a chance to improve it for release with Mga7, and at the same time, update the Mga6 isodumper to enable the new Mga7 Lives to be written to USB with persistence available. 


do you agree a useful function for Isodumper if possible?


comment 1 advises the likely procedure, so not quite "out of the box" :).

I guess, if possible, the GUI will have a tick box with "do you want to enable persistence?",
and user entry textbox for the partition label, so that it could be used for other distro Lives that support persistence, with "mgalive-persist" as the default.
Marja Van Waes 2018-10-09 09:39:46 CEST

CC: (none) => marja11, yves.brungard_mageia
Assignee: bugsquad => mageiatools

Comment 4 papoteur 2018-10-09 10:20:38 CEST
Hello,
This would be surely a great feature. I can work on that for Isodumper.
It can help if a manual process of the creation of persistence is described here.
Comment 5 Martin Whitaker 2018-10-09 21:57:58 CEST
(In reply to papoteur from comment #4)
> It can help if a manual process of the creation of persistence is described
> here.

I've just done it like this:

sudo dd if=Mageia-6.1-LiveDVD-Xfce-x86_64-DVD.iso of=/dev/sdf bs=1M
sudo fdisk /dev/sdf << EOF
n
p
3


w
EOF
sudo mkfs.ext4 -L mgalive-persist /dev/sdf3

The two blank lines in the commands to fdisk select the default values, which cause the persistence partition to fill the remaining free space.

parted can't be used, because it treats the iso9660 partition as free space.

For 6.1 this is an experimental feature. Known limitations are:

 - only works on the 64-bit ISOs (this is already fixed for mga7)
 - if you run draklive-install and choose to remove unneeded locales
   and drivers, that will remove them from the live system as well
 - although you can install kernel updates, the live system will
   always boot from the original kernel

CC: (none) => mageia

Comment 6 papoteur 2018-10-18 21:57:45 CEST
Thanks Martin,
This works fine :)
Is the persistent partition always the third?
Comment 7 Martin Whitaker 2018-10-19 00:40:20 CEST
(In reply to papoteur from comment #6)
> Thanks Martin,
> This works fine :)
> Is the persistent partition always the third?

It doesn't have to be - it searches for the 'mgalive-persist' label.
Comment 8 papoteur 2018-12-15 18:00:38 CET
The option has been added in Isodumper 1.05.
The label is coded in 'mgalive-persist'.
Comment 9 papoteur 2018-12-15 18:00:59 CET
Thus closing

Resolution: (none) => FIXED
Status: NEW => RESOLVED

Comment 10 ben mcmonagle 2018-12-15 23:01:10 CET
had a quick look;
created a live Xfce64 with persistence on a 8GB stick and rebooted.

under /run is mgalive directory.

how-ever saving a document to, for example, "documents" or the desktop and then rebooting the system does not restore the document.

am I doing something wrong?
Comment 11 Martin Whitaker 2018-12-16 15:21:41 CET
(In reply to ben mcmonagle from comment #10)
> am I doing something wrong?

That should work. If you plug the USB stick into a working system, can you mount the 3rd partition. It should have an ext4 filesystem labelled 'mgalive-persist'. If the live boot detected it, it should contain two top-level directories, 'work' and 'memory', and any new/modified files should appear in the filesystem under 'memory'.

I've not got a cauldron system on real H/W at the moment, so can't test the new Isodumper just yet.
Comment 12 James Kerr 2018-12-16 16:59:47 CET
I used isodumper on my cauldron system to burn 7 beta Xfce iso with a persistent partition to an 8GB usb stick and booted to the live system. I created a text file and saved it to Documents. I installed kernel-devel which was not already installed.

I shut down the live system.

On rebooting into the live system, the file that I had saved was present in Documents and kernel-devel was reported as installed.

It looks to me as though the persistent partition is working as intended.

CC: (none) => jim

Comment 13 ben mcmonagle 2018-12-18 07:16:10 CET
must have been the stick.

wrote to another stick, booted into the live session, created and saved a document. 
rebooted system back to the live- saved document there.

sorry for the noise.

looks like something for the next Mageia Blog, as it is working great.
Comment 14 Morgan Leijström 2019-01-03 23:11:48 CET
Should be updated in mga6 too!

- Users of mga6 should be able to facilitate the persistence of already released Mageia 6.1 isos, and of course the coming Mageia 7.

Resolution: FIXED => (none)
Status: RESOLVED => REOPENED
Whiteboard: (none) => MGA6TOO

Comment 15 Morgan Leijström 2019-01-03 23:29:17 CET
... and added to release notes of both mga6(.1) and mga7 in sections

https://wiki.mageia.org/en/Mageia_6_Release_Notes#Available_installation_media , where it talks about USB flash drive

and https://wiki.mageia.org/en/Mageia_6_Release_Notes#Isodumper

And of course in https://wiki.mageia.org/en/Installation_Media#Dump_Mageia_ISO_on_a_USB_flash_drive

Also in the manual, at https://doc.mageia.org/installer/6/en/content/Select-and-use-ISOs.html#d4e159 i propose to change "You can use a graphical tool like IsoDumper" to "The graphical tool IsoDumper which is included in Mageia lets you safely dump the iso to USB and make it bootable, and you also have the option to let it use the rest of the stick for persistent user documents as well ass settings and system updates. Whis works for iso images of MAgeia 6.1 and later."

(...when the mga6 isodumper is updated that is)

Keywords: (none) => FOR_RELEASENOTES6

Comment 16 Martin Whitaker 2019-01-04 10:59:24 CET
On the 6.1 ISOs, the persistence partition isn't unmounted when you shut down, so fsck always has to repair it on reboot. This is unlikely to cause a problem, as all writes should have been flushed before shutdown, but even so, I'd prefer not to advertise this feature for 6.1.
Comment 17 Morgan Leijström 2019-01-04 11:49:40 CET
Could that be solved by an update (saved persistently) ?
Comment 18 ben mcmonagle 2019-02-04 23:20:49 CET
as no comments for a month, I am going to close.

please re-open if you feel more could be done.

Status: REOPENED => RESOLVED
Resolution: (none) => FIXED

Comment 19 ben mcmonagle 2019-02-04 23:22:43 CET
BTW, thanks Martin and Papoteur.

works great.
Comment 20 papoteur 2019-02-05 16:21:51 CET
Release 1.94 is submitted as update in Mageia 6, to allow to create Mageia 7 Live images with persistence.

Resolution: FIXED => (none)
Keywords: FOR_RELEASENOTES6 => IN_RELEASENOTES6
Assignee: mageiatools => qa-bugs
Status: RESOLVED => REOPENED

Thomas Backlund 2019-02-05 19:37:39 CET

Whiteboard: MGA6TOO => (none)
Version: Cauldron => 6
CC: (none) => tmb

Comment 21 ben mcmonagle 2019-02-06 00:00:25 CET
(base system Mga6-32, plasma5 DE, sddm DM)\
kernel 4.18.20-desktop-1.mga6


# urpmi isodumper
Use of uninitialized value in null operation at /usr/lib/perl5/vendor_perl/5.22.3/i386-linux-thread-multi/URPM/Resolve.pm line 1852.
To satisfy dependencies, the following packages are going to be installed:
  Package                        Version      Release       Arch    
(medium "Core Updates (distrib3)")
  python3-gnupg                  0.3.8        2.1.mga6      noarch  
(medium "Core Updates Testing (distrib5)")
  isodumper                      1.06         1.mga6        noarch  
  isodumper-gtk                  1.06         1.mga6        noarch  
  python3-pydbus                 0.6.0        1.mga6        noarch  
412KB of additional disk space will be used.
252KB of packages will be retrieved.
Proceed with the installation of the 4 packages? (Y/n) y

invoke isodumper.
choose Mageia-7-beta1-Live-Xfce-i586,
enable persistence

no progress bar percentages until 100% complete
no write percentage in info window until complete

reboot into Live system
create and save .txt document.
reboot into live - saved text document opened and text -ok
Comment 22 papoteur 2019-02-06 08:35:42 CET
Hum,
Launched isodumper --gtk, and yes, found the same behaviour. Window is not refreshed during the writing, in contrast with Qt one.
Comment 23 papoteur 2019-02-06 09:29:28 CET
Should be OK now with 1.07 release, to package.
Comment 24 David GEIGER 2019-02-06 17:26:19 CET

Packages in 6/core/updates_testing:
========================
isodumper-1.07-1.mga6.noarch.rpm
isodumper-qt-1.07-1.mga6.noarch.rpm
isodumper-gtk-1.07-1.mga6.noarch.rpm

python2-pydbus-0.6.0-1.mga6.noarch.rpm
python3-pydbus-0.6.0-1.mga6.noarch.rpm


Source RPM: 
========================
isodumper-1.07-1.mga6.src.rpm
python-pydbus-0.6.0-1.mga6.src.rpm

CC: (none) => geiger.david68210

Comment 25 Herman Viaene 2019-02-07 14:42:33 CET
MGA6-32 MATE on IBM Thinkpad R50e
No installation issues
Used it to dump M7 Xfce Live on 4Gb (really 3.7) USB stick.
Result as seen from MCC:
1 partition 2.2 Gb for the iso
1 partition MGALIVE-ESP vfat 4 Mb
1 unused space 1.5 Gb
Just for curiosity I tried in MCC to increase the size of the MGALIVE-ESP partition, but as soon as I clicked the "Resize" button, I get an error:
Bad FAT: Chain outside filesystem for /EFI /BOOT propagated.
Anyway, I will try to boot this laptop with the stick.

CC: (none) => herman.viaene

Comment 26 Herman Viaene 2019-02-07 17:01:19 CET
My mistake: I did not set the option for persistent partition. I ldumped the USB stick again, with the option set, and all works as expected.
Comment 27 James Kerr 2019-02-18 10:08:40 CET
on mga6-64 plasma

packages installed cleanly:
- isodumper-1.07-1.mga6.noarch
- isodumper-gtk-1.07-1.mga6.noarch
- isodumper-qt-1.07-1.mga6.noarch
- python3-gnupg-0.3.8-2.1.mga6.noarch
- python3-pydbus-0.6.0-1.mga6.noarch

used isodumper-qt to write mga7-beta1-xfce-live to 8GB stick

mgalive-persist exists and is mountable

booted live system
installed kernel-devel and dependencies
created a text file and saved it to Documents
increased dpi to 120
switched mouse to single-click
disabled mga welcome

shut down live system

booted live system on different hardware

kernel-devel was reported as installed
the text file that I had created was present and readable
all three UI preferences that I had changed were preserved

OK for mga6-64
Comment 28 James Kerr 2019-02-18 11:04:12 CET
on mga6-32  xfce

packages updated cleanly:
- isodumper-1.07-1.mga6.noarch
- isodumper-gtk-1.07-1.mga6.noarch
- python3-gnupg-0.3.8-2.1.mga6.noarch
- python3-pydbus-0.6.0-1.mga6.noarch

used isodumper-gtk to burn 7-beta1-Live-Xfce-i586

The progress bar functioned correctly
and percentage completed was correctly reported

OK for mga6-32
Comment 29 James Kerr 2019-02-18 11:45:50 CET
It would be good to have this available 
for people burning mga7 beta 2 live iso's

I have added OK's and validated

Suggested advisory:
************************************************
isodumper has been enhanced to optionally allow 
the creation of a persistent partition

This enables changes made to the live system
to be preserved through a reboot or restart 
of the live system.
************************************************

package list in comment 24

Whiteboard: (none) => MGA6-64-OK MGA6-32-OK
Keywords: (none) => validated_update
CC: (none) => sysadmin-bugs

Dave Hodgins 2019-02-20 20:24:13 CET

CC: (none) => davidwhodgins
Keywords: (none) => advisory

Comment 30 Mageia Robot 2019-02-20 21:58:37 CET
An update for this issue has been pushed to the Mageia Updates repository.

https://advisories.mageia.org/MGAA-2019-0014.html

Resolution: (none) => FIXED
Status: REOPENED => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.