Bug 1023 - missing packages in rescue mode
Summary: missing packages in rescue mode
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: Installer (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: Normal enhancement
Target Milestone: ---
Assignee: Thierry Vignaud
QA Contact:
URL:
Whiteboard:
Keywords: PATCH
: 4824 (view as bug list)
Depends on:
Blocks: 1994
  Show dependency treegraph
 
Reported: 2011-04-28 11:46 CEST by AL13N
Modified: 2012-03-06 21:55 CET (History)
8 users (show)

See Also:
Source RPM: drakx-installer-rescue
CVE:
Status comment:


Attachments
svn diff /soft/drakx/trunk/rescue (3.47 KB, patch)
2011-10-23 03:05 CEST, AL13N
Details | Diff

Description AL13N 2011-04-28 11:46:04 CEST
- sshd (or dropbear, since it's been packaged)
- passwd (to set password for sshd starting)
- vim (only vi seems to be in rescue)

i hope this can fit...
Comment 1 Thierry Vignaud 2011-04-28 14:29:53 CEST
vim won't fit.

CC: (none) => thierry.vignaud
Component: Release (media, process) => Installation
Source RPM: (none) => drakx-installer-rescue
Severity: major => enhancement

Comment 2 AL13N 2011-04-28 15:00:17 CEST
that is sad... :-(

vi is ... not as easy as vim
Comment 3 AL13N 2011-05-19 00:07:15 CEST
still confirmed as missing on rescue mode on mageia-1-dual-rc*.iso
Comment 4 AL13N 2011-05-19 02:35:26 CEST
i did some testing around with spturtle and almost got it working...

what did i do:
 - install /usr/sbin/dropbear (200K) and /usr/bin/dropbearkey (75K)
 - create directory /etc/dropbear
 - used dropbearkey to create rsa and dss host key
 - i used a different machine to generate a password to put into a new /etc/shadow file (i found out that busybox actually has a passwd command; perhaps enabling that would be enough)
 - had to adjust root in /etc/passwd to have '*' instead of blank password
 - then i ran dropbear -F -E (only for testing)
 - when i logged in, openpty was failing

reading up on that issue, there seem to be 2 possibilities:
1. mounting /dev/pts during boot (and not afterwards, which supposedly doesn't work)
2. enable legacy pty support on busybox

i have no idea what kind of pty support is on the busybox in the rescue mode. but it would be nice if someone could test this... (i suspect step 1 would work)

ideally it would be nice to modify the rescue menu to add this to do the steps for you, but even having these files (and enabling busybox passwd and possibly legacy support) but be a big usability improvement of rescue mode.
Comment 5 AL13N 2011-05-19 11:12:12 CEST
forgot to add that i had to add /bin/bash to a file called /etc/shells
AL13N 2011-07-02 13:28:26 CEST

Blocks: (none) => 1994

Comment 6 Jari S 2011-07-17 12:49:15 CEST
I've found out that the only thing in addition to sshd missing are full set of fsck tools for all filesystems as well as other filesystem related tools like btrfs command utility. Also gdisk would be good addition since it is more easy to use in low res modes than gparted (too long help page). These usually come handy when you really need rescue mode i.e. no network, no second computer to browse the net.

CC: (none) => lihamakaroonilaatikko

Comment 7 AL13N 2011-07-17 12:59:46 CEST
actually, imho, i think we need to have the absolute minimum of required tools to more or less comfortably rescue a system.

we need to keep in mind that this rescue is on ALL media, and thus cuts deep into the available space for CD media.

especially the dual arch CD (where the rescue is also the biggest advantage), because it cuts twice into the available space.

i agree on filesystem tools, but not where there is already one that does the trick (even though it would be harder to do so).

- btrfs utility would be paramount
- fsck for all filesystems too
- but IMHO as long as there is a partitioner (fdisk), i don't see the need to include other partitioners, perhaps gparted and gdisk should be stripped from it, if they are on it.
Comment 8 Jari S 2011-07-17 13:15:57 CEST
gdisk is for gpt partitions where fdisk is for mbr partitions and cannot handle gpt. I do agree that we do not need multiple tools just to fulfill personal preferences. Maybe gparted is to be replaced with gdisk. However, this is probably best for someone better qualified to compare the maturity of these programs.
Comment 9 AL13N 2011-07-17 22:52:33 CEST
that was indeed what i meant to say.

but what the hell is gpt partitions?

i've always worked with fdisk for everything that i do, even loopbacked stuff
Comment 10 Sander Lepik 2011-07-17 23:05:07 CEST
http://en.wikipedia.org/wiki/GUID_Partition_Table - you meet them when you get some BIG HDDs in your system :)

CC: (none) => sander.lepik

Comment 11 Jari S 2011-07-17 23:12:34 CEST
Or SSD's which need to be aligned to get best performance.
Comment 12 AL13N 2011-07-17 23:12:49 CEST
Ah, i have no 3TB disks yet, and at dayjob the hardware raid cards of our clients are mostly using 300GB SAS disks in RAID5.

Thanks for the heads up.

Does this in fact mean that 3TB disks which aren't split up can only be booted with EFI mobos?
Comment 13 andré blais 2011-10-06 14:06:36 CEST
Not at all.
GPT can use a legacy BIOS.
Linux doesn't use the BIOS to access the disk.  (Only for the beginning of the boot process.)
If you want to dual-boot Microsoft and Linux with a GPT table on a legacy BIOS system, you have to emulate an MBR partition table for Microsoft.  (And keep the ms partitions in the lower 2TB.)
Gdisk creates "hybrid" GPT disks with MBR emulation quite well.

As a rescue tool, gdisk is ideal for GPT partitioned disks, being equivalent of MBR-specific fdisk.

The GPT format makes more efficient use of disk space, as well as having a backup partition table at the end of the disk.
The default table size allows 128 partitions.
"Extended" partitions do not exist under GPT.  So no flaky daisy-chaining of partition allocation as in MBR tables.
So GPT is a better choice, even if one doesn't have a 3TB disk.

My first usage was on a 80G disk with about 8 partitions.
Gdisk is able to convert the type of partition table in place, as long as there is space for the backup table at the end of the disk.  The main table at the beginning easily fits into the first 63 sectors available on an mbr-partitioned disk.

CC: (none) => andre999

andré blais 2011-10-09 07:25:24 CEST

CC: andre999 => andre999mga

Comment 14 AL13N 2011-10-17 12:40:52 CEST
how does this affect chainloading? what about small media? ie: floppy, usb sticks, etc...

can the tools work with MBR if there is an MBR (i'd not like to converted).

i'm not adverse to when having a clean or partitionless disk, to use this format instead of MBR. but no conversions with existing partitions.
Comment 15 AL13N 2011-10-17 12:41:48 CEST
of course providing the tools can be replaced and don't take more space
Comment 16 andré blais 2011-10-18 07:57:10 CEST
gpt is just the partition table at the beginning of the disk (and a backup at the end).
It doesn't affect the partitions themselves.
Chainloading works with gpt.  (I've chainload to a ntfs ms-xp partition, using a hybrid gpt table.  The hybrid aspect is only needed since ms-xp doesn't function directly with gpt.
BTW, if you need more than 2 partitions (on the same physical disk) for an OS that needs MBR (such as with most versions of Ms-windows), I would recommend to _not_ use GPT.  Since GPT can only reliably protect 2 MBR-emulated partitions (per disk). (Without getting complicated, that is.)
The systems I converted used primarily Linux, and only had 2 (or fewer) Ms partitions.
Gpt on floppies should be possible, unless the floppy driver blocks it somehow.  But it wouldn't be very useful, since there would only be 1 partition on a floppy.
Gpt on Usb sticks would be much like gpt on SSD drives.
Unless you have more than 4 partitions on a media (and thus an extended partition if using MBR), or a very large disk, there is not much advantage to using GPT.
 
The first few gpt disks I set up were automatic conversions from mbr using gdisk, leaving all the partitions intact.  You can go back and forth (between mbr & gpt) using gdisk, without problems.
The only tricky part is to ensure that there is about 32M free at the end of the disk for the gpt backup table.
Unless you are using some form of raid.  Then the location of the backup gpt table can be more complicated, but it is documented.
Comment 17 AL13N 2011-10-23 03:04:21 CEST
ok, back to topic and some good news...

I've got it working for me on my x86_64 system, svn diff for /soft/drakx/trunk/rescue/ follows.

What've I done:
---------------
 - add dropbear dropbearkey
 - add /etc/shells
 - write a extremely simple passwd program
 - make /etc/passwd writable
 - mount /dev/pts
 - symlink /dev/ptmx to /dev/pts/ptmx
 - write a startssh script
 - mention startssh in etc/issue
 - add screen

this added 692KB to the rescue.sqfs file.

I've tested by putting that file on my local repos and booted from an boot.iso, chose rescue, then got via network the rescue.sqfs file.

It may not be the best way, but it works for me. If some can give me write access to /soft I can commit this, so that the rescue will work nicely for mga2 :-)
Comment 18 AL13N 2011-10-23 03:05:30 CEST
Created attachment 995 [details]
svn diff /soft/drakx/trunk/rescue
Manuel Hiebel 2011-10-23 12:54:28 CEST

Keywords: (none) => PATCH

AL13N 2011-10-23 14:18:19 CEST

CC: (none) => pterjan

Comment 19 Pascal Terjan 2011-10-23 14:30:24 CEST
====
 ln -s /tmp/stage2/etc/* /etc 2>/dev/null
+# make passwd changable
+rm -f /etc/passwd
+cp /tmp/stage2/etc/passwd /etc/
====

I think it would be better to place it initially at the right place than creating the link + removing it at boot time + copying the file

Also, it would probably be better to have some explanations asking to select a password for the ssh server rather than the prompt "Password:" as I wouldn't know which password I am supposed to enter.
Comment 20 AL13N 2011-10-23 15:19:34 CEST
the passwd script is just needed for startssh script. i wanted something small to generate a password hash, perhaps i should make it "Give a password: " , is that ok?

or perhaps i should not call this script passwd? but something else? how about 'passwdgenhash' ?

about the /etc/passwd file, i'm not sure how to do this, i mean, /tmp/stage2/ is a mount at that time and the passwd file is there and i think it should be there, no? that's the correct place for it, and imho i would have to do bigger changes than just to fix this... if you have a suggestion, i'm up for it though...

btw: have you also had time to test it?
Comment 21 Thomas Backlund 2011-10-24 23:56:44 CEST
So,
this is now available in the rescue image on cauldron mirrors.

It seems to work, but one thing more to fix:

when it generates the ssh keys it prints them to stdout too, wich is uggly.
So please check and fix that too.

CC: (none) => tmb

Comment 22 Thierry Vignaud 2011-10-25 17:57:52 CEST
Also, all perl files should pass perl_checker.
You should clean genpasswd accordingly.

See /usr/share/doc/perl_checker/perl_checker.html
Comment 23 AL13N 2011-10-25 20:02:00 CEST
(In reply to comment #22)
> Also, all perl files should pass perl_checker.
> You should clean genpasswd accordingly.
> 
> See /usr/share/doc/perl_checker/perl_checker.html

ah, thanks, i tried to clean it up, but it complains that:

join '', $foo == $foo

but i have this part: 

join('', ('.', '/', 0..9, 'A'..'Z', 'a'..'z')[rand 64, rand 64])

directly from the crypt perldoc...?
Comment 24 AL13N 2011-10-25 20:16:22 CEST
(In reply to comment #21)
> So,
> this is now available in the rescue image on cauldron mirrors.
> 
> It seems to work, but one thing more to fix:
> 
> when it generates the ssh keys it prints them to stdout too, wich is uggly.
> So please check and fix that too.

I wanted to print out the fingerprint so that when you connect remotely, you can check it it's correct and secure.

If you still think it should be removed, i'll go ahead and do it.
Comment 25 Thomas Backlund 2011-10-25 20:49:13 CEST
(In reply to comment #24)
> 
> I wanted to print out the fingerprint so that when you connect remotely, you
> can check it it's correct and secure.
> 
> If you still think it should be removed, i'll go ahead and do it.


The intention is good, but unfortunately it gives a very unfinished look,
so  please remove it.

The "attack vector" is not that big as the attacker must first find out the ip, and then the password.

And it's up to the user of the rescue ssh mode to set a strong password.
Comment 26 AL13N 2011-10-25 21:04:17 CEST
done, anyone any idea about how to finish this perl_checker thing?
Comment 27 AL13N 2011-10-25 21:22:53 CEST
ok, i made it perl_checker safe, but i had to replace:

join('', ('.', '/', 0..9, 'A'..'Z', 'a'..'z')[rand 64, rand 64])

by 

my @salt = ('.', '/', 0..9, 'A'..'Z', 'a'..'z')[rand 64, rand 64];
join('', @salt)

@tv: is this a perl_checker bug?
Comment 28 Thierry Vignaud 2011-10-26 03:49:37 CEST
Yes but that makes the code clearer anyway...
Comment 29 AL13N 2011-10-26 19:39:24 CEST
(In reply to comment #28)
> Yes but that makes the code clearer anyway...

maybe so, but still, this exact line comes from the online perldoc of crypt function... perhaps maybe this could be taking into account for perl_checker?
Comment 30 Thomas Backlund 2011-10-27 18:39:00 CEST
Ok, this is mostly fixed, only the gdisk is still missing.

I'll take a look on what kind of diskspace it will add
Comment 31 andré blais 2011-10-28 14:28:40 CEST
It's about 350k (according to rpmdrake).
The latest version upstream is 0.8.1 (I've been planning to update it.)
0.7.2 in cauldron
0.6.14 in mga1
Comment 32 Thomas Backlund 2011-10-28 18:44:53 CEST
I took the liberty of updating gdisk to 0.8.1 so I could close this bug.

All changes for ssh support and gdisk is now available in the rescue image in cauldron.

Status: NEW => RESOLVED
Resolution: (none) => FIXED

Comment 33 AL13N 2012-02-25 16:24:47 CET
i just retested the ssh rescue with mageia 2 beta1 and it fails again with the openpty failed, even though /dev/ptmx and /dev/pts/ptmx exist...

i also get the following in the console:

INIT: cannot execute "/sbin/agetty"
INIT: cannot execute "/sbin/agetty"
INIT: cannot execute "/sbin/agetty"
INIT: Id "s0" respawning too fast: Disabled for 5minutes

i did a manual "mount -t devpfs devpts /dev/pts" and after that, it worked again...

i also noted that the /etc/passwd entry for root had / as path and not /root, so dropbear complained about permissions not being right for / (donno why the uid of / is 500:501 ...?

Status: RESOLVED => REOPENED
Resolution: FIXED => (none)

Comment 34 AL13N 2012-02-25 16:25:47 CET
i noted that /dev/pts was mounted in the /tmp/stage2 dir... but apparently that's not enough
Comment 35 AL13N 2012-02-25 17:10:02 CET
/sbin/agetty is just not on the rescue, so rescue over serial line see bug 2052, is thus impossible
Comment 36 AL13N 2012-02-25 17:12:32 CET
ok, i committed all the fixes and updated the NEWS file to reflect a new rescue image update. but i don't know how to release it.

I'm assigning to tv to make the rescue image.

if anyone can document such things in the wiki, it'd be nice.

Status: REOPENED => ASSIGNED
Assignee: bugsquad => thierry.vignaud

Comment 37 AL13N 2012-02-25 21:56:20 CET
these tests were with the mga2 beta1 boot.iso x86_64

i've retested with the DVD mga2 beta1 x86_64 and that one has an additional extra problem:

in rescue mode, the /dev/ contains /dev/dev which contains most stuff.

a direct result is that /dev/urandom cannot be found when generating host keys for dropbear.

i notice that /dev/dev/urandom does exist...

i'm changing the rc.sysinit to copy the /dev correctly, in the hopes of fixing this issue.

i think i didn't have this issue with the boot.iso , because of the preconfigured network settings so i could actually fetch the rescue.

@tv: again, i've committed, and this time, i tried to pay attention to your comments, but i'm fairly sure some extra steps are required. I'd be nice to have some kind of documentation on the wiki about this.
Comment 38 Thierry Vignaud 2012-02-25 23:15:44 CET
I did the proper fix
Comment 39 AL13N 2012-02-26 00:06:23 CET
no offense, but your changes aren't totally clear to me:

you removed the copy of dev that i moved earlier, so that i would have a /dev/urandom

is this still going to work?

also i really hope that this time the /dev/pts and the likes are mounted normally and not in /tmp/stage2/dev/pts like it is now...

since i've seen a difference between the boot.iso and the dvd iso, i'd like to do some proper testing, but i donno how to generate those isos to test it...
Comment 40 Thierry Vignaud 2012-02-26 00:15:24 CET
We don't need those nodes since we're using udev now
Comment 41 AL13N 2012-02-26 16:31:54 CET
i've used boot.iso with the updated rescue.sqfs on the mirrors, but it's not working at all...

loading program, then exactly nothing happens, i only see:


proceeding, please wait...
Comment 42 Thierry Vignaud 2012-02-26 16:59:18 CET
It worked when locally tested but genereated squash image still list /dev as to be symlinked in /usr/share/symlinks.
I'll fix
Comment 43 AL13N 2012-02-26 18:14:57 CET
ugh, i still had some typos in the startssh command and didn't take into account / appearing in generated passwords.

I committed this, and i also noticed that the mount points are now ok. so from boot.iso it works.

@tv: can you release this? thanks!

of course, I can't test from DVD until likely livecd generation or mageia 2 beta 2.
Comment 44 Thierry Vignaud 2012-02-26 21:07:44 CET
Done

Status: ASSIGNED => RESOLVED
Resolution: (none) => FIXED

Comment 45 AL13N 2012-02-27 08:18:18 CET
thanks!

we'll need to test the livecd and the beta2 DVD...

can we have these sort of things in the procedure for the pre-QA testing?

1. boot into rescue mode
2. test restore bootloader
3. test mount partitions
4. test console
5. test loadkeys
6. test drvinst
7. test startssh

maybe also test text installer?

CC: (none) => ennael1

Comment 46 Thierry Vignaud 2012-03-06 21:55:41 CET
*** Bug 4824 has been marked as a duplicate of this bug. ***

CC: (none) => herbert


Note You need to log in before you can comment on or make changes to this bug.