Unsure whether this is for isodumper or Live ISOs, so assigning to David G as the registered isodumper maintainer, CCing MartinW.
The kali reference is good "Adding USB Persistence with LUKS Encryption".
Thought: do the Live ISOs cope with LUCKS?

Assignee: bugsquad => geiger.david68210
CC: (none) => lewyssmith, mageia

Yes, it is possible to use LUKS with a USB memory stick. 

I am using that today with Kali Linux. It is very useful to encrypt the "persistent" partition (typically /dev/sdb3 on the memory stick device).
It works very well and is easy to use.

I think that Kali achieves this by having a "Live USB encrypted with persistence" option in their initial grub menu. I also note they have a "Live USB with persistence" (for un-encrypted).

I have been trying to figure out of there is a way to manually add the "union mount" between the RO root and the RW persistent partition which is the key to this working at an early stage of the boot process (hit "e" perhaps?).

The union mount would essentially be: 

    mount --union /dev/sdb3 /

where "/" is the already mounted RO root and "/dev/sdb3" would be the persistent partition.

I have duplicated and documented most of the Kali method in my local wiki ready to share this but I am slightly stuck at getting the union mount done early in the boot process. Any advice much appreciated. Merci d'avance

for ref: 

1) union mount documented here => https://lwn.net/Articles/312641/ 
2) How to create a LUKS encrypted persistent USB Kali image:
   https://docs.kali.org/downloading/kali-linux-live-usb-persistence

I think this is in two parts:

1) isodumper to be enhanced to create a LUKS encrypted persistent partition (as an additional option).

2) Grub menu on the Live ISO image to offer two additional options:
   a) "boot Live USB with persistence"
   b) "boot Live USB encrypted with persistence"

Assigning to our developer isodumper tool @papoteur :)

Assignee: geiger.david68210 => yves.brungard_mageia

To make this work, the script that mounts the live filesystem would need to be modified to enable it to detect and mount encrypted partitions. If you have a cauldron system, look at /usr/lib/dracut/modules.d/90mgalive/mgalive-root.sh.

You can experiment by adding "rd.break" to the boot command line (remove "splash quiet" and possibly "rd.luks=0"). This will stop just before the pivot to the live filesystem, so mgalive-root,sh will have already run, but you can undo what it has done.

Thanks Martin. I have been doing some experimenting.

My first approach was to try to replicate the actions of isodumper as manual steps but I have not been able to fully identify every command/action yet.

So I switched to using isodumper and then trying to modify the 3rd (persistent) partition. The idea occurred to me that it could be a simple and elegant solution to add a 4th LUKS encrypted partition because the union mount will "stack" the n+1'th unioned FS over the previous ones. This would have the advantage of not needing to change anything in the existing isodumper process for creating a persistent (3rd) partition.

I then discovered that isodumper only offers the option to use all the remaining (unused) space on the USB for the 3rd persistent partition. It would have been neat to have the option to have a smallish 3rd partition and have most of the unused remaining space allocated to the 4th (LUKS) encrypted partition.

After several attempts to manually reduce the persistent (3rd) filesystem and partition (using parted's resizefs and resize2fs) I only managed to make the USB partially bootable (it gets stuck waiting (apparently forever) for some boot steps to complete).

In the course of all this, I have observed some odd things:
a) diskdrake can't "see" any partitions on the USB
b) neither can gparted which seems to report that there is no partition table
c) both fdisk and parted can "see" the partitions on the USB
d) after running isodumper, "fdisk -l /dev/sdb" shows the 3 partitions as expected but after running parted's resizefs and resize2fs the first one, /dev/sdb1, vanished leaving only sdb2 and sdb3. Not sure what's going on here.

Things which could help in isodumper:
a) In the log, display exact commands used at each step (so they could be understood and possibly run manually).
b) Add the LUKS steps to simply encrypt the 3rd partition (see the kali doc for the precise commands for this (ref: https://docs.kali.org/downloading/kali-linux-live-usb-persistence )).
c) if b) (above) is to complex or liable to break things then have an option to use a (smallish) part of the remaining USB space for a 3rd persistent partition and then be able to use what's left for a 4th LUKS encrypted partition.

Hi Paul,
What isodumper does for writing the 3rd partition is what Martin gave:
https://bugs.mageia.org/show_bug.cgi?id=23654#c5
This is done here:
http://gitweb.mageia.org/software/isodumper/tree/backend/raw_write.py#n227
You can see the succession of commands that are sent like in the fdisk interactive mode to create the partition. One of them takes the default size of the partition. If you want to change it by specifying a specific size, this is the place to do it.

Merci Papoteur,
I am unsure how to do the "please type in your LUKS pass-phrase" bit in raw_write.py. I will try manually and make notes to share.

(In reply to Paul Blackburn from comment #6)
> In the course of all this, I have observed some odd things:
> a) diskdrake can't "see" any partitions on the USB
> b) neither can gparted which seems to report that there is no partition table
> c) both fdisk and parted can "see" the partitions on the USB

This is because our ISO images are all hybrid ISOs, so appear as both an iso9660 filesystem and a MBR/DOS partitioned system. I think the higher level tools see the iso9660 filesystem and look no further.

> d) after running isodumper, "fdisk -l /dev/sdb" shows the 3 partitions as
> expected but after running parted's resizefs and resize2fs the first one,
> /dev/sdb1, vanished leaving only sdb2 and sdb3. Not sure what's going on
> here.

The first partition covers/protects the actual iso9660 filesystem. For the 64-bit ISOs, its partition type is set to 0x00 (Empty). It would seem parted doesn't preserve such partitions.

(In reply to Paul Blackburn from comment #8)
> I am unsure how to do the "please type in your LUKS pass-phrase" bit in
> raw_write.py. I will try manually and make notes to share.

The drakx tools appear to do it in two different ways. The first is by running a command like this:

    echo -e $passphrase | cryptsetup luksFormat $device

The second is by writing the passphrase to a temporary file, then running a command like this:

    cryptsetup luksFormat $device --key-file $temp_file_path

Thanks Martin et merci papoteur,

I have made a simple stand-alone script to prompt for the LUKS pass phrase.
ref: https://pastebin.com/UVJqh4Aw

I am aiming to make this into a function for isodumper and need to integrate it into the input/output method isosumper is already using in _do_persistence() and to add the additional LUKS-format and LUKS-open steps.

So now trying to understand how to do the equivalent of "getpass" in the isodumper user i/o context.

Not made much progress with approach in comment 10 (above).

Tried a "plan b" which is to add a 4th partition (calling it "mgalive-LUKS").

I have now reached the point where I am trying to figure out what goes into /etc/fstab to have mgalive-LUKS be union mounted with root and "mgalive-persist" (3rd partition).

When booted up with the mgalive-persist USB (not using 4th partition), "fdisk -l" shows the following:

Device     Boot   Start       End   Sectors   Size Id Type
/dev/sdb1  *          0   4789035   4789036   2.3G  0 Empty
/dev/sdb2       4789036   4797227      8192     4M ef EFI (FAT-12/16/32)
/dev/sdb3       4798464   8992767   4194304     2G 83 Linux
/dev/sdb4       8992768 242483199 233490432 111.3G 83 Linux


Looking at the existing /etc/fstab:
none / overlay defaults 0 0

Wondering if the following would be feasible?
none / overlay defaults 0 0
/dev/sdb4 / overlay defaults 0 0


This seems to be a bit tricky as one would wish for any updates in the unioned filesystems to be accessible early in the boot sequence (so that, for example, kernel updates take effect).

Any ideas or guidance would be welcome.

I put some notes in my wiki draft page here:
https://wiki.mageia.org/en/User:Codegazer#Notes_on_adding_LUKS_disk_encryption_to_Mageia_live_persistent_USB

Tried adding the following line to /etc/fstab :

/dev/sdb4 / overlay defaults 0 0

Then rebooted, hit "e" at the Mageia boot screen and removed "rd.luks=0" then ctrl-x to continue boot but there was no prompt for the LUKS pass phrase in the boot sequence.

I will have a look at this once we get Mageia 7 released.

Thanks Martin, 
as best as I can understand the way the Mageia persistent USB is working is that somewhere in the boot process, if a partition labelled "mgalive-persist" is found then that is union mounted with / (which is what provides the access to read/write and save configuration/data on the 3rd USB partition).

I found that if I tried to LUKS encrypt the mgalive-persist partition then the "mgalive-persist" label becomes inaccessible (because it is encrypted) and so access to the persistence partition fails.

I also tried adding a 4th partition "mgalive-LUKS" which I hoped to union mount on top of the union between / and mgalive-persist but it appears that having a second "overlay" line in /etc/fstab is "duplicate error".

If I have an entry in /etc/fstab to mount the mgalive-LUKS encrypted partition at /mnt/mgalive-LUKS/ then the boot process stops to prompt for the LUKS pass phrase and the mount works correctly.

So, I seem to be stuck trying to figure out how to union mount mgalive-LUKS in the boot process.

As you rightly point out, this enhancement request can wait until after mga7.

Hello Martin,

Just wondering is there is anything I can do to progress this?

Would it be possible during the boot-from-USB process to LUKSopen any encrypted partitions on the USB drive? 

This would be similar to how LUKS encrypted partitions are handled when booting from disk.

I think the LUKSopen step would need to occur before processing the "mgalive-persist" drive otherwise and encrypted "mgalive-persist" partition could not be identified as "mgalive-persist" (because it would be encrypted).

Created attachment 11392 [details]
Modified mgalive-root.sh to support encrypted persistent partition

Created attachment 11393 [details]
Patch to drakiso gui example to support encrypted persistent partition

To build a Live ISO that supports an encrypted persistent partition, follow the instructions for example 2 at https://wiki.mageia.org/en/Draklive2#Quickstart, except after step 1 (making the working copy), apply this patch to the working copy and copy the modified mgalive-root.sh (attachment 11392 [details]) into the files directory, e.g.

% cp -r /usr/share/drakiso/examples/gui my-iso
% cd my-iso
% patch -p3 < <path to download directory>/drakiso-encryption.patch
% cp <path to download directory>/mgalive-root.sh files/

To create a Live USB stick with an encrypted persistent partition:

1. Dump your new ISO onto the USB stick.
2. Use fdisk to add a third partition on the USB stick.
3. Encrypt the third partition, e.g.
     cryptsetup luksFormat /dev/sdb3
4. Format the third partition, e.g.
     cryptsetup open /dev/sdb3 crypt_sdb3
     mkfs.ext4 -L mgalive-persist /dev/mapper/crypt_sdb3
     cryptsetup close crypt_sdb3

Hello Martin,

Thank you. It works brilliantly!

I followed your steps and managed to create a LUKS encrypted 3rd partition on USB. 

I made a few errors along the way creating the ISO but got there in the end. ;-)

My errors specifically:
a) in the custom live ISO process, don't chose "region=all" because it makes the ISO too big.
b) Do not add any files to the config/files directory. I mistakenly had a copy of mgalive-root.sh in there with a date added to the name.

I made some notes about how to create an encrypted Mageia live USB on my wiki draft page:

https://wiki.mageia.org/en/User:Codegazer#Appendix-3:_Encrypted_3rd_persistent_partition_on_persistent_Mageia_live_USB

Thanks both for opening this way.
I can indeed to manage the option of creation the 3rd partition in luks format with isodumper. For what I understand, our ISO need to be adapted for benefit such a way.

@Paul, good to hear it worked for you. Yes, region=all makes a big difference. The CJK fonts are a major factor in that.

One warning - if you ever update the dracut package on your Live ISO, that will overwrite the modified mgalive-root.sh file, and on the next boot it won't find the persistent partition. If that happens, mount the partition manually (your DE file manager should be able to do that), navigate to /memory/lib/dracut/modules.d/90mgalive/ and delete the mgalive-root.sh file you find there. That will expose the old version. Alternatively, just add dracut to /etc/urpmi/skip.list in your live system to prevent dracut being updated.

N.B. Never modify the persistent partition like that when it is in use by the live system.

@Papoteur, yes, we'd need to rebuild our Live ISOs to support this. Let's plan to support it in the next release.

Hello,
I have pushed some updates to Isodumper which add an option for encrypted persistent partition.
I have tested the creation, but not the final usage, as I'm to lazy to build an ISO for this purpose.
Paul, do you want to test it?
For that, you will have to clone the repo with git, 
git clone git://git.mageia.org/software/isodumper
and 
cd isodumper
make install
If you have already run isodumper during the session, do
systemctl restart magiback
to run the last backend.

Version: 6 => Cauldron

Salut papoteur,

Merci. Oui, I will test it. I think it will take some time.

Allons y.

Salut papoteur.

I followed your steps. All went OK except for the "make install":

[mpb@zbook isodumper (master)]$ /bin/sudo make install
[sudo] password for mpb: 
# for binary file script isodumper on /usr/bin
mkdir -p /usr/bin
install -m 755 isodumper /usr/bin
# for backend magiback listening DBus on /usr/bin/
install -m 755 backend/magiback /usr/bin
# for DBus conf file on /etc/dbus-1/system-d
# to launch magiback on request
mkdir -p /etc/dbus-1/system.d
install -m 644 backend/org.mageia.Magiback.conf /etc/dbus-1/system.d
# for DBus conf file on /usr/share/dbus-1/system-d
# to provide objects and interfaces
mkdir -p /usr/share/dbus-1/system-services
install -m 644 backend/org.mageia.Magiback.service /usr/share/dbus-1/system-services
# for systemd service on /usr/lib/systemd/system
mkdir -p /usr/lib/systemd/system/
install -m 644 backend/magiback.service /usr/lib/systemd/system/
# for policy file isodumper on /usr/share/polkit-1/actions/
# to have authentication with polkit (use for mageia policy)
mkdir -p /usr/share/polkit-1/actions
install -m 644 polkit/org.mageia.isodumper.policy /usr/share/polkit-1/actions
install: cannot stat 'polkit/org.mageia.isodumper.policy': No such file or directory
make: *** [Makefile:77: install] Error 1



I had a quick look at the makefile but it was not clear what the issue is to me.
Please advise: should I just go ahead and try anyway or shoudl we resolve this makefile issue forst?

Kind Regards
Paul

OK,
Before the make install, try to do from the isodumper directory:
tools/po-update.sh
this will generate the translated polkit/org.mageia.isodumper.policy

Created attachment 11470 [details]
Example of running isodumper's tools/po-update.sh

Make some progress. 
Needed to install: "urpmi intltool" then, in isodumper directrory, able to run:
tools/po-update.sh
(see attachment for details)

Created attachment 11471 [details]
running "make install" in isodumper directory

Running "make install" in the isodumper directory gets much further.
Please see attachment for details.

Ends with:

install -m 644 $locale/LC_MESSAGES/isodumper.mo /usr/$locale/LC_MESSAGES/isodumper.mo; \
done
install: target '/usr/share/locale/en_GB/LC_MESSAGES/isodumper.mo' is not a directory
make: *** [Makefile:124: install] Error 1

SVP advise.

Hi Paul,
I will check on my side.
But I think that the installation is enough advanced so that you launch isodumper (as user).

Created attachment 11473 [details]
log showing draklive2 on Mageia 7 attempt to create ISO (fails)

Salut papopeur,

So good news: the new isodumper worked well.

Some progress: I was able to run your new isodumper and create a LUKS encrypted persistent USB using the ISO I previously generated (see comment 18, above).

I like the updates on the isodumper window. Nifty! ;-)

I also tried using a "standard" Mageia 7 live ISO but of course that does not work because some configuration changes are needed in the drakiso configuration files.


I have also been trying to do the complete process from scratch: creating a new ISO with drakiso but unfortunately I have some snags with this (see attachment).

I just need to understand why I am not able to run drakiso to create a new ISO image.

Merci!

PS: I have documented what I managed to do today here:
https://wiki.mageia.org/en/User:Codegazer/Encrypted_live_persistent_USB_with_Mageia_7

Hi Paul,
Hmm, there is still an error displayed:
Error while doing persistent partition. Re-reading the partition table failed.:
Device or resource busy.
Have you checked that the partition is really in force?

Hi Paul,

What filesystem is your working directory contained in?
What is the output from

  sudo draklive2 --boot -v 5

BTW, I recommend not running draklive2 as root, particularly if you are using the GUI. If you start it as a normal user, it will use sudo itself when necessary. But if you do this now, you'd have to make a clean start, because everything in your working directory is currently owned by root.

Created attachment 11479 [details]
output from: draklive2 --boot -v 5

Hello Martin,

Thank you.

Here is the output from "draklive2 --boot -v 5" (attached).

I followed your guidance and rebuilt the ISO as non-root.
It has worked. :-)

I also installed another related RPM: drakiso-null-bootloader.
I think that helped. 

I will try isodumper to a USB in the morning.

Salut papoteur,

Having created a new ISO in light of Martin's guidance (above), I have used isodumper to write this to USB and it worked fine. It is very nifty! ;-)

I have updated isodumper screenshot in my draft wiki page at:
https://wiki.mageia.org/en/User:Codegazer/Encrypted_live_persistent_USB_with_Mageia_7 and I am updating this at the moment.

One good thing I noticed is that the USB isodumper made works on different systems: one old one with legacy boot and a new one with EFI.
I mention this because a previous encrypted persistent mgalive would only work for me on the legacy machine (not the new EFI one).

Kind Regards
Paul

Some more thoughts: I was thinking about what I would need to do for a new release of Mageia to upgrade/re-install on an existing Mageia LUKS encrypted USB.

With a HD or SSD based system, it is easy to separate the /home/ partition from root /.
This allows users to do a fresh overwrite install and preserve what data they have in /home/.

This makes me wonder is there is a benefit in having a 4th partition for LUKS encrypted /home/?

The 3rd partition would need to be large enough to accommodate updates to the system (kernel, RPMs, system configuration).

Also, a 4th partition would not be union mounted with the 1st and 3rd but would be a separate mount point.
This has another advantage because the 4th partition can be mounted and accessed independently (eg when not booted up from USB but just plugging the USB into a system for example running from DH/SSD).


The other way would be to simply get a new USB for a new release of Mageia. However, one is then faced with the issue: how to copy /home/ from the old USB where it is in a union mount of 1st and 3rd partitions. I think one would have to be booted up from the old USB to access the contents of /home/.

Another alternative would be to use a two USB solution: USB 1 for the system / and USB2 for /home/.
However, having two USBs could be messy: how would /home/ from 2nd USB get mounted?

Hello,
You to compare a system on a stick build from a Live ISO, and a system installed on the stick as target. 
I thinkk that the system you have in mind would be better fit with the second method, where you will free to choose the configuration.

An update for this issue has been pushed to the Mageia Updates repository.

https://advisories.mageia.org/MGAA-2020-0043.html

Status: NEW => RESOLVED
CC: (none) => tmb
Resolution: (none) => FIXED