Description of problem: After verification isodumper creates persistence partition, then isodumper crash/exits, its window vanishes, log file not written. Version-Release: isodumper-1.21-1.mga7 How reproducible: No change if i start it as user or root, and if i put it in USB2 or USB2 port only the speed change. * This bug do not appear if i do not check the box to make persistence * Steps to Reproduce: System is Mageia 7 64 bit Plasma, fully updated also to testing Write mga8b1 live plasma 64 bit iso to stick, * with persistence * After verification, a fraction of a second after display show "100%" isodumper just vanishes. In Konsole from where i started isodomper as root, i see: g-io-error-quark: GDBus.Error:unknown.TypeError: __init__() got an unexpected keyword argument 'sterr' (36) In journal i see it creates the partition for persistence, then hit that TypeError okt 04 03:20:59 svarten.tribun magiback[12377]: Välkommen till fdisk (util-linux 2.33.2). okt 04 03:20:59 svarten.tribun magiback[12377]: Ändringar kommer att förbli endast i minnet, till dess att du beslutar dig för att skriva dem. okt 04 03:20:59 svarten.tribun magiback[12377]: Var aktsam innan du använder skrivkommandot. okt 04 03:20:59 svarten.tribun magiback[12377]: Kommando (m för hjälp): Partitionstyp okt 04 03:20:59 svarten.tribun magiback[12377]: p primär (2 primär, 0 utökat, 2 ledigt) okt 04 03:20:59 svarten.tribun magiback[12377]: e utökad (behållare för logiska partitioner) okt 04 03:20:59 svarten.tribun kernel: sdd: sdd1 sdd2 okt 04 03:20:59 svarten.tribun magiback[12377]: Välj (standard p): Partitionsnummer (3,4, standardvärde 3): Första sektorn (6773316-60555263, standardvärde 6774784): Sista sektorn, +/-sektorer eller +/-storlek{K,M,G,T,P} (6774784-60555263, standardvärde 60555263): okt 04 03:20:59 svarten.tribun magiback[12377]: Skapad en ny partition 3 av typen ”Linux” med storlek 25,7 GiB. okt 04 03:20:59 svarten.tribun kernel: sdd: sdd1 sdd2 sdd3 okt 04 03:20:59 svarten.tribun magiback[12377]: Kommando (m för hjälp): Partitionstabellen har ändrats. okt 04 03:20:59 svarten.tribun magiback[12377]: Anropar ioctl() för att läsa om partitionstabellen. okt 04 03:20:59 svarten.tribun magiback[12377]: Synkroniserar diskar. okt 04 03:21:00 svarten.tribun plasmashell[11931]: file:///usr/lib64/qt5/qml/QtQuick/Controls/Button.qml:99: TypeError: Type error But the same TypeError is also in journal five more times in quick succession at an earlier time in same run, without apparent problem. Sidenote: at the very launch of isodumper, in console it outputs: <bound method YCheckBox.value of <yui.YCheckBox; proxy of <Swig Object of type 'YCheckBox *' at 0x7f96c16ed510> >> ---------------------------- Output from fdisk -l after run: Disk /dev/sdd: 28,9 GiB, 31004295168 byte, 60555264 sektorer Disk-modell: DT Elite G2 Enheter: sektorer av 1 * 512 = 512 byte Sektorstorlek (logisk/fysisk): 512 byte / 512 byte I/O-storlek (minsta/optimal): 512 byte / 512 byte Disketikettstyp: dos Diskidentifierare: 0x00000000 Enhet Start Början Slutet Sektorer Storlek Id Typ /dev/sdd1 * 0 6765123 6765124 3,2G 0 Tom /dev/sdd2 6765124 6773315 8192 4M ef EFI (FAT-12/16/32) /dev/sdd3 6774784 60555263 53780480 25,7G 83 Linux
Assignee: bugsquad => yves.brungard_mageia
Hi Morgan, Thanks for the report. Yes, I reproduce the crash. Now, I have to debug it.
Good hunting! Persistance is not working on that stick. (actually i have never tested persistance before but now i need for another bug) Was it something more isodumper was supposed to do to the stick, but died before? I stumbled on key in comment 0: "USB2 or USB2" -> "USB2 or USB3"
Persistance is a major feature -> rising severity For the persistance functionality: (and there is a journal of current boot) Bug 27356 - Live 8b1 Persistance is not working
Severity: normal => majorSee Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=27356
Fixed in git. Error could be seen in /var/log/magiback.log
Thank you. That was agile :) Did this error affect the result on the memory stick? If not, there is something else causing Bug 27356.
Yes, it causes problem. The instruction mkfs.ext is thus not executed.
OK So can you make a new version with this fixed and I can test and use, please?
I have tagged a release 1.22 of isodumper. Can someone package it?
QA Contact: (none) => pkg-bugsStatus: NEW => ASSIGNEDCC: (none) => geiger.david68210
Done for both Cauldron and mga7!
Advisory ===================================== Isodumper 1.22 The program crashed when making the persistence partition. The update fix this problem. =====================================
Assignee: yves.brungard_mageia => qa-bugs
It dont crash anymore when making a persistent partition, and i have also tried encryption OK. But: 1) The (qt) interface it is not translated anymore 2) For one of the sticks i tried there are lots of I/O errors at the time i think it tries to validate persistent partition (?): okt 05 23:13:17 svarten.tribun magiback[6174]: Skapar ett filsystem med 30521441 4 k-block och 7634944 inoder okt 05 23:13:17 svarten.tribun magiback[6174]: Filsystems-UUID: a428ce25-b733-413d-a3be-f2af48358cde okt 05 23:13:17 svarten.tribun magiback[6174]: Superblockkopior lagrade på block: okt 05 23:13:17 svarten.tribun magiback[6174]: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, okt 05 23:13:17 svarten.tribun magiback[6174]: 4096000, 7962624, 11239424, 20480000, 23887872 okt 05 23:13:17 svarten.tribun magiback[6174]: [57B blob data] okt 05 23:13:17 svarten.tribun magiback[6174]: [55B blob data] okt 05 23:13:20 svarten.tribun magiback[6174]: Skapar journal (131072 block): klar okt 05 23:13:20 svarten.tribun magiback[6174]: [96B blob data] okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#17 device offline or changed okt 05 23:13:20 svarten.tribun kernel: print_req_error: 60 callbacks suppressed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: buffer_io_error: 7425 callbacks suppressed okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#18 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#19 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: ldm_validate_partition_table(): Disk read failed. okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#16 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#17 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#18 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#19 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#16 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sdc: unable to read partition table okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#19 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#16 device offline or changed okt 05 23:13:20 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 okt 05 23:13:20 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#17 device offline or changed okt 05 23:13:20 svarten.tribun kernel: ldm_validate_partition_table(): Disk read failed. okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#18 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#19 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#16 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#17 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#18 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sdc: unable to read partition table okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#22 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#23 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#4 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#5 device offline or changed okt 05 23:13:20 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#6 device offline or changed The above happens with stick Corsair voyager GTX 128GB, for both USB2 and USB3 connections But none of the kernel lines happens with another stick, the Kingston DT Elite G2 32 GB Another difference between the sticks is that when i insert them, for Kingston kernel tells: okt 05 23:22:17 svarten.tribun kernel: sd 10:0:0:0: [sdc] No Caching mode page found okt 05 23:22:17 svarten.tribun kernel: sd 10:0:0:0: [sdc] Assuming drive cache: write through Maybe that difference is a clue. Both sticks works, and the partitions also look OK in mageia 8 diskdrake. But is there some checking going wrong with all that kernel lines in journal, or what is it? Both sticks are very little used before.
Source RPM: isodumper-1.21-1.mga7.src.rpm => isodumper-1.22Assignee: qa-bugs => yves.brungard_mageia
Advisory ===================================== Isodumper 1.23 The program crashed when making the persistence partition. The update fix this problem. The persistence creation process has now a step of waiting for the partition table to be known by the kernel. =====================================
? I still get tens of lines of errors like in #11, from line sd 10:0:0:0: [sdc] tag#4 device offline or changed etc And yes isodumper say it is 1.23
Assignee: qa-bugs => yves.brungard_mageia
:( I don't have that here. Perhaps I must with other sticks. Can you give the relative timing of the errors?
No Caching mode page found Assuming drive cache: write through doesn't seem to be the trigger. oct. 10 22:25:42 YZenbook.home kernel: usb 1-3: new high-speed USB device number 10 using xhci_hcd oct. 10 22:25:42 YZenbook.home kernel: usb 1-3: New USB device found, idVendor=058f, idProduct=6387, bcdDevice= 1.03 oct. 10 22:25:42 YZenbook.home kernel: usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3 oct. 10 22:25:42 YZenbook.home kernel: usb 1-3: Product: Mass Storage oct. 10 22:25:42 YZenbook.home kernel: usb 1-3: Manufacturer: Generic oct. 10 22:25:42 YZenbook.home kernel: usb 1-3: SerialNumber: 7AE7D6C9 oct. 10 22:25:42 YZenbook.home kernel: usb-storage 1-3:1.0: USB Mass Storage device detected oct. 10 22:25:42 YZenbook.home kernel: scsi host4: usb-storage 1-3:1.0 oct. 10 22:25:42 YZenbook.home mtp-probe[18074]: checking bus 1, device 10: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-3" oct. 10 22:25:42 YZenbook.home mtp-probe[18074]: bus: 1, device: 10 was not an MTP device oct. 10 22:25:42 YZenbook.home mtp-probe[18088]: checking bus 1, device 10: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-3" oct. 10 22:25:42 YZenbook.home mtp-probe[18088]: bus: 1, device: 10 was not an MTP device oct. 10 22:25:43 YZenbook.home kernel: scsi 4:0:0:0: Direct-Access Generic Flash Disk 8.07 PQ: 0 ANSI: 2 oct. 10 22:25:43 YZenbook.home kernel: sd 4:0:0:0: [sdc] 7823360 512-byte logical blocks: (4.01 GB/3.73 GiB) oct. 10 22:25:43 YZenbook.home kernel: sd 4:0:0:0: [sdc] Write Protect is off oct. 10 22:25:43 YZenbook.home kernel: sd 4:0:0:0: [sdc] Mode Sense: 03 00 00 00 oct. 10 22:25:43 YZenbook.home kernel: sd 4:0:0:0: [sdc] No Caching mode page found oct. 10 22:25:43 YZenbook.home kernel: sd 4:0:0:0: [sdc] Assuming drive cache: write through oct. 10 22:25:43 YZenbook.home kernel: sdc: sdc1 oct. 10 22:25:43 YZenbook.home kernel: sd 4:0:0:0: [sdc] Attached SCSI removable disk oct. 10 22:26:22 YZenbook.home polkitd[3456]: Operator of unix-session:1 successfully authenticated as unix-user:root to gain TEMPORARY authorization for action org.mageia.Magiback.Isodumper.write for system-bus-name::1.146 [python3 isodumper.py] (owned by unix-user:yves) oct. 10 22:26:59 YZenbook.home kernel: sdc: sdc1 sdc2 oct. 10 22:27:06 YZenbook.home kernel: sdc: sdc1 sdc2 oct. 10 22:27:06 YZenbook.home kernel: sdc: sdc1 sdc2 sdc3 oct. 10 22:27:36 YZenbook.home kernel: sdc: detected capacity change from 4005560320 to 0
I forgot to say localisation works again :) Below is the output of # journalctl -fko short-unix Inserting stick into a USB3 port: 1602364196.572838 svarten.tribun kernel: usb 6-1: new SuperSpeed Gen 1 USB device number 4 using xhci_hcd 1602364196.590834 svarten.tribun kernel: usb 6-1: New USB device found, idVendor=1b1c, idProduct=1a0e, bcdDevice= 1.00 1602364196.591165 svarten.tribun kernel: usb 6-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1 1602364196.591646 svarten.tribun kernel: usb 6-1: Product: Voyager GTX 1602364196.591874 svarten.tribun kernel: usb 6-1: Manufacturer: Corsair 1602364196.592085 svarten.tribun kernel: usb 6-1: SerialNumber: 511190606267A0D6018B 1602364196.596881 svarten.tribun kernel: scsi host10: uas 1602364196.597241 svarten.tribun kernel: scsi 10:0:0:0: Direct-Access Corsair Voyager GTX 0 PQ: 0 ANSI: 6 1602364196.598840 svarten.tribun kernel: sd 10:0:0:0: [sdc] Spinning up disk... 1602364197.659842 svarten.tribun kernel: .ready 1602364197.659973 svarten.tribun kernel: sd 10:0:0:0: [sdc] 250069680 512-byte logical blocks: (128 GB/119 GiB) 1602364197.660246 svarten.tribun kernel: sd 10:0:0:0: [sdc] Write Protect is off 1602364197.660479 svarten.tribun kernel: sd 10:0:0:0: [sdc] Mode Sense: 43 00 00 00 1602364197.660710 svarten.tribun kernel: sd 10:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA 1602364197.660945 svarten.tribun kernel: sd 10:0:0:0: [sdc] Optimal transfer size 33553920 bytes 1602364197.670833 svarten.tribun kernel: sdc: sdc1 sdc2 sdc3 1602364197.671836 svarten.tribun kernel: sd 10:0:0:0: [sdc] Attached SCSI disk In isodumper i select that stick, the 8b1 xfce 64 bit iso, and persistent partition, and start it. After writing, (start verifying): 1602364255.565836 svarten.tribun kernel: sdc: sdc1 sdc2 Appearing after isodumper have verified the stick: 1602364305.020069 svarten.tribun kernel: sdc: sdc1 sdc2 1602364305.248835 svarten.tribun kernel: sdc: sdc1 sdc2 sdc3 1602364305.249832 svarten.tribun kernel: sdc: sdc1 sdc2 sdc3 1602364310.461835 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.462578 svarten.tribun kernel: print_req_error: 57 callbacks suppressed 1602364310.463507 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.463563 svarten.tribun kernel: buffer_io_error: 6913 callbacks suppressed 1602364310.463604 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.465589 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.465846 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.465886 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.466303 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.466547 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.466587 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.466625 svarten.tribun kernel: ldm_validate_partition_table(): Disk read failed. 1602364310.466668 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.467195 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.468420 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.468460 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.469093 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.469131 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.469608 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.469901 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.469946 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.469982 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.470234 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.470574 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.470631 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#28 device offline or changed 1602364310.470890 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.470933 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.471333 svarten.tribun kernel: sdc: unable to read partition table 1602364310.471389 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#3 device offline or changed 1602364310.471648 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.471994 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.472042 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#0 device offline or changed 1602364310.472296 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 1602364310.472336 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 1602364310.472669 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#1 device offline or changed 1602364310.472930 svarten.tribun kernel: ldm_validate_partition_table(): Disk read failed. 1602364310.473000 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#2 device offline or changed 1602364310.473228 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#3 device offline or changed 1602364310.473690 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#0 device offline or changed 1602364310.473914 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#1 device offline or changed 1602364310.474281 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#2 device offline or changed 1602364310.474507 svarten.tribun kernel: sdc: unable to read partition table 1602364310.481835 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#8 device offline or changed 1602364310.482171 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#9 device offline or changed 1602364310.485835 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#10 device offline or changed 1602364310.486497 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#11 device offline or changed 1602364310.486769 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#8 device offline or changed
1602364310.463604 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read That looks like either a bad usb stick, or a problem with the connection somewhere between it and the motherboard.
CC: (none) => davidwhodgins
I was also thinking that, but: There are *no* buffer errors while writing the whole iso file, nor when reading it, or creating partitions. Only after the actual work is done. And that kind of errors comes only after it have reported device is offline, but that said have no idea about how synchronous and self aware the handling of USB drives is... I also tested on a USB2 port, on same machine, identical result except speed. Also, the stick boot and runs OK also with persistence. (Did not look into the log though)
What is isodumper trying to do in the end? Can i execute some similar commands in CLI, to see if i get similar result in journal?
The last journal entry in my test was Oct 10 19:16:49 magiback[14548]: Syncing disks. No errors in journal in my test. Konsole output (I ran it after using su - out of habit) ... # isodumper <bound method YCheckBox.value of <yui.YCheckBox; proxy of <Swig Object of type 'YCheckBox *' at 0x7f303bd73510> >> Target Device: Kingston DT 101 G2 (/dev/sde) 14.912 GiB Image : /s3/m8/Mageia-8-beta1-Live-Xfce-x86_64/Mageia-8-beta1-Live-Xfce-x86_64.iso Executing copy from /s3/m8/Mageia-8-beta1-Live-Xfce-x86_64/Mageia-8-beta1-Live-Xfce-x86_64.iso to /dev/sde Image Mageia-8-beta1-Live-Xfce-x86_64.iso successfully written to /dev/sde Bytes written: 2810071040 The sha3 sum check is OK and the sum is signed Added persistent partition # blkid /dev/sde* /dev/sde: UUID="2020-07-19-12-04-36-00" LABEL="Mageia-8-b1-Live-Xfce-x86_64" TYPE="iso9660" PTTYPE="dos" /dev/sde1: UUID="2020-07-19-12-04-36-00" LABEL="Mageia-8-b1-Live-Xfce-x86_64" TYPE="iso9660" PTTYPE="dos" /dev/sde2: SEC_TYPE="msdos" LABEL_FATBOOT="MGALIVE-ESP" LABEL="MGALIVE-ESP" UUID="4E93-AEE1" TYPE="vfat" /dev/sde3: LABEL="mgalive-persist" UUID="cc5d17df-ee66-427b-a549-56ce76f35e10" TYPE="ext4" ]# mount /dev/sde3 /e3 [root@x3 ~]# ll /e3 total 16 drwx------ 2 root root 16384 Oct 10 19:16 lost+found/ You can test a usb using using something like dd bs=1M if=/dev/urandom of=/dev/sdz Replace sdz with correct device id. This will write until it fails when the device is full.
Neither dd nor isodumper cause any problem with writing Mageia iso to this usb stick (Corsair Voyager GTX). Also if i let isodumper only format the stick, there is no error reported. Now i have realised the problems do not have to do with making persistence; I always (for this stick) see the same errors coming at end of isodumper operation with or without that box checked. Also note there are never errors during writing or verifying the iso - but afterwards - and if it should create persistence it also succeeds with that. Also note, isodumper dialog say it completed OK. So maybe all is well regarding isodumper but kernel/whatever is trying to communicate with that stick for other reasons (is it only trying to read partition table and all other errors are results of that?) but the stick is shut down? Maybe not properly shut down? Maybe bug in stick? Kernel? Another detail/bug: if after the failure i keep the stick plugged in and isodumper open, i can in isodumper select the stick in dropdown but it fail if i try to i.e format it.
Timing, making a live iso with persistence: Last part of isodumper log, (and dialogues say it have succeeded): 2020-10-11 03:52:09 DEBUG Start doing persistence partition 2020-10-11 03:52:09 DEBUG Persistence thread started 2020-10-11 03:52:10 DEBUG New partition created 2020-10-11 03:52:14 INFO Persistent partition done 2020-10-11 03:52:14 DEBUG Finished 2020-10-11 03:52:14 DEBUG Starting unmounting 2020-10-11 03:52:14 INFO Inga partitioner är monterade. ( = No partitions are mounted ) At the same time, from # journalctl -fko short-iso-precise : 2020-10-11T03:52:14.885834+0200 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#15 device offline or changed ...yes all like before, see i.e comment 16, snipping many lines here... 2020-10-11T03:52:14.914726+0200 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#13 device offline or changed Could be interesting to have microseconds resolution in isodumper log too. Anyway, i tend to say isodumper is good to release, but you know more about how this works than me :)
The hardware errors are clearly being recovered from ok, so validating the update.
Keywords: (none) => validated_updateWhiteboard: (none) => MGA7-64-OKCC: (none) => sysadmin-bugs
For information, when doing persistence, operations are: fdisk to add a partition in the remaining space; udevadm settle --timeout=15 to wait for propogation mkfs.ext4 The message "Persistent partition done" is then printed unmount partitions of the device eject the device What I can do is to add waiting after mkfs and before unmount.
Hello, I have pushed on git some modifications that add waiting steps. If you want test them, you have to replace backend/raw_write.py and backend/magiback then stop magiback.service I found a problem that encrypted partition is not unmounted with other partitions of the if it is mounted. I look for a method to enumerate it.
Thank you for keeping hammering :) I see commit for microseconds too. =========================== Sidenote: http://gitweb.mageia.org/software/isodumper/about/ is empty. Maybe copy from https://github.com/papoteur-mga/isodumper ? =========================== Not familiar with git, i downloaded the files... http://gitweb.mageia.org/software/isodumper/plain/backend/magiback http://gitweb.mageia.org/software/isodumper/plain/backend/raw_write.py ( to /home/morgan/Hämtningar/ ) ...and figured out what to do with them: # mv /usr/bin/magiback /usr/bin/magiback_1.23 # mv /home/morgan/Hämtningar/magiback /usr/bin/magiback # chown root:root /usr/bin/magiback # chmod +x /usr/bin/magiback # cd /usr/lib/python3.7/site-packages/isodumper/ # mv raw_write.py raw_write.py_1.23 # mv /home/morgan/Hämtningar/raw_write.py . # chown root:root raw_write.py # systemctl stop magiback.service =========================== # isodumper I selected the 64 bit 8b1 xfce live, persistence, not encrypted. Same stick as before. Isodumper dialogue reported success. Same errors in journal, starting 18 milliseconds after isodumper logs it start unmounting. Strange: it logs it start unmounting, then the same *micro* second ! , it logs no partitions are mounted?! Last part of /var/log/magiback.log 2020-10-11 17:52:04,351 DEBUG signature checked 2020-10-11 17:52:04,352 DEBUG Start checking 2020-10-11 17:52:50,028 DEBUG last read 0 2020-10-11 17:52:50,250 INFO The sha3 sum check is OK and the sum is signed 2020-10-11 17:52:51,038 DEBUG Start doing persistence partition 2020-10-11 17:52:51,038 DEBUG Persistence thread started 2020-10-11 17:52:52,194 DEBUG New partition created 2020-10-11 17:52:56,005 INFO Persistent partition done 2020-10-11 17:52:56,513 DEBUG Finished 2020-10-11 17:52:56,516 DEBUG Starting unmounting 2020-10-11 17:52:56,516 INFO Inga partitioner är monterade. From # journalctl -fko short-iso-precise : 2020-10-11T17:52:51.216184+0200 svarten.tribun kernel: sdc: sdc1 sdc2 2020-10-11T17:52:51.506131+0200 svarten.tribun kernel: sdc: sdc1 sdc2 sdc3 2020-10-11T17:52:51.509131+0200 svarten.tribun kernel: sdc: sdc1 sdc2 sdc3 2020-10-11T17:52:56.534536+0200 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#14 device offline or changed 2020-10-11T17:52:56.534915+0200 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 2020-10-11T17:52:56.534983+0200 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 2020-10-11T17:52:56.535032+0200 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#15 device offline or changed 2020-10-11T17:52:56.535331+0200 svarten.tribun kernel: blk_update_request: I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 2020-10-11T17:52:56.535367+0200 svarten.tribun kernel: Buffer I/O error on dev sdc, logical block 0, async page read 2020-10-11T17:52:56.535399+0200 svarten.tribun kernel: sd 10:0:0:0: [sdc] tag#16 device offline or changed ...Etc, etc, like earlier.
Hello, There is now a 1.24 release. I have re-factored the unmounting phase, now using udisks. Previously, encrypted partition couldn't be unmounted. I have also added waiting step after each operation. The modification of unmounting phase is also for format operation.
Thank you 1) Again no translation, all is in english like comment#11. 2) For the same stick like before, from a quick view I still see same errors in journal, except i do not see the "sdc: sdc1 sdc2 sdc3" lines. Also tried a USB SSD disk, Seagate M3 Portable 1TB, same errors but the lines "sd 10:0:0:0: [sdc] tag#19 device offline or changed" are about three times more repeated. I have verified the result is bootable and persistence working. I wonder if kernel or something else treats some storage media differently than others, and tries to exercise them some way after isodumper have unmounted them? Some "aggressive" automounting? Based on storage capacity, or operating mode details maybe?
There is nothing more that I can do for the moment. I can reproduce the generation of I/O errors. As the feature of persistence is restored, I suggest not to block this update.
I agree, except we have to get the translation working again.
(In reply to Morgan Leijström from comment #30) > I agree, except we have to get the translation working again. Also, french translation is not applied - tested on Cauldron and M7.
CC: (none) => ouaurelien
See Comment 31. Missing translations.
Keywords: validated_update => feedback
/usr/share/locale/fr/LC_MESSAGES/isodumper.mo missing from isodumper package Also: /usr/share/locale/sv/LC_MESSAGES/isodumper.mo
@Yuri: it seems several locales are not included for our tool isodumper in final package. Can you took a look about it if not synced with svn? It seems fr locale is OK on transifex.
CC: (none) => yurchor
(In reply to Aurelien Oudelet from comment #34) > @Yuri: it seems several locales are not included for our tool isodumper in > final package. Can you took a look about it if not synced with svn? > > It seems fr locale is OK on transifex. There is no significant difference: https://www.mageia.org/langs/report_tx_git.php?c=Cauldron&r=isodumper Just package the git/master branch and you get all the translations.
Removing the OK until the above "final package" has been tested.
Whiteboard: MGA7-64-OK => (none)CC: (none) => andrewsfarm
We forgot to send back to dev. We need to get a version with translation restored, and then QA retest, and this is good to go.
Hello, Isodumper 1.25 is now in testings. Advisory: ==================================== Isodumper 1.25 The program crashed when making the persistence partition. The update fix this problem. The persistence creation process has now a step of waiting for the partition table to be known by the kernel. Avoid race conditions to detect other instance Now use udisk2 to unmount partitions. =====================================
Assignee: yves.brungard_mageia => qa-bugsSource RPM: isodumper-1.22 => isodumper-1.25
Troubles here, testing this under Cauldron. $ rpm -qa --last | grep isodumper isodumper-qt-1.25-1.mga8.noarch dim. 15 nov. 2020 09:51:56 isodumper-1.25-1.mga8.noarch dim. 15 nov. 2020 09:51:56 after update. Executing isodumper from CLI and from Plasma Application Menu: both as user and as root it always says: "An other instance of isodumper is already running." But there is no other process named isodumper in KSysGuard.
Same problem on mga7, plasma. On the positive side, the message is in Swedish :)
This is also fixed in Cauldron. And for M7 with Plasma, this is also OK with isodumper-1.26 Suggested Advisory: ==================================== Updated isodumper packages fix translations and some bugs. The isodumper program crashed when making the persistence partition on slow USB devices. The isodumper-1.26-1.mga7 packages fixes that by adding a step of waiting for the partition table to be known by the kernel when creating the persistence partition. Now, it also uses udisk2 to unmount partitions. We added other checks to avoid race conditions to detect other instance of isodumper to be launched. Also, several translations have been added. ===================================== in updates_testing: isodumper-1.26-1.mga7 isodumper-qt-1.26-1.mga7 isodumper-gtk-1.26-1.mga7 from SRPM isodumper-1.26-1.mga7.src.rpm
Source RPM: isodumper-1.25 => isodumper-1.21-1.mga7.noarch.rpmWhiteboard: (none) => MGA7-64-OKKeywords: feedback => advisory, validated_updateAssignee: yves.brungard_mageia => qa-bugs
Failing when it should start writing [morgan@svarten ~]$ isodumper <bound method YCheckBox.value of <yui.YCheckBox; proxy of <Swig Object of type 'YCheckBox *' at 0x7f8cef4ccb70> >> Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 847, in run self.handleevent() File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 810, in handleevent self.do_write() File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 374, in do_write success, message = self.u.do_unmount(target) File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 132, in do_unmount fs = iface[self.SERVICE + '.Filesystem'] File "/usr/lib/python3.7/site-packages/pydbus/proxy.py", line 102, in __getitem__ raise KeyError(iface) KeyError: 'org.freedesktop.UDisks2.Filesystem' [morgan@svarten ~]$ Same if started by root.
Source RPM: isodumper-1.21-1.mga7.noarch.rpm => isodumper-1.26-1.mga7.noarch.rpmAssignee: qa-bugs => yves.brungard_mageiaWhiteboard: MGA7-64-OK => (none)Keywords: validated_update => (none)
(In reply to Morgan Leijström from comment #42) > Failing when it should start writing > > [morgan@svarten ~]$ isodumper > <bound method YCheckBox.value of <yui.YCheckBox; proxy of <Swig Object of > type 'YCheckBox *' at 0x7f8cef4ccb70> >> > Traceback (most recent call last): > File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 847, > in run > self.handleevent() > File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 810, > in handleevent > self.do_write() > File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 374, > in do_write > success, message = self.u.do_unmount(target) > File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 132, > in do_unmount > fs = iface[self.SERVICE + '.Filesystem'] > File "/usr/lib/python3.7/site-packages/pydbus/proxy.py", line 102, in > __getitem__ > raise KeyError(iface) > KeyError: 'org.freedesktop.UDisks2.Filesystem' > [morgan@svarten ~]$ > > Same if started by root. Thanks Morgan. Can you provide a udisksctl dump with the same devices connected?
Just fyi, ok on my Mageia 7 x86_64 system under kde plasma. $ rpm -qa|grep isodumper isodumper-gtk-1.26-1.mga7 isodumper-1.26-1.mga7 isodumper-qt-1.26-1.mga7 $ cat .isodumper/isodumper.log Target Device: Kingston DT 101 G2 (/dev/sde) 14.912 GiB Image : /s3/m8/Mageia-8-beta1-Live-Xfce-x86_64/Mageia-8-beta1-Live-Xfce-x86_64.iso Executing copy from /s3/m8/Mageia-8-beta1-Live-Xfce-x86_64/Mageia-8-beta1-Live-Xfce-x86_64.iso to /dev/sde Image Mageia-8-beta1-Live-Xfce-x86_64.iso successfully written to /dev/sde Bytes written: 2810071040 The sha3 sum check is OK and the sum is signed Added persistent partition
# udisksctl dump, Relevant part: /org/freedesktop/UDisks2/drives/SanDisk_Extreme_AA011224122232524201: org.freedesktop.UDisks2.Drive: CanPowerOff: true Configuration: {} ConnectionBus: usb Ejectable: true Id: SanDisk-Extreme-AA011224122232524201 Media: MediaAvailable: true MediaChangeDetected: true MediaCompatibility: MediaRemovable: true Model: Extreme Optical: false OpticalBlank: false OpticalNumAudioTracks: 0 OpticalNumDataTracks: 0 OpticalNumSessions: 0 OpticalNumTracks: 0 Removable: true Revision: 0001 RotationRate: -1 Seat: seat0 Serial: AA011224122232524201 SiblingId: /sys/devices/pci0000:00/0000:00:1c.4/0000:05:00.0/usb6/6-1/6-1:1.0 Size: 32017047552 SortKey: 01hotplug/1605444234266812 TimeDetected: 1605444234266812 TimeMediaDetected: 1605444234266812 Vendor: SanDisk WWN:
Ah, there is a UDisks2 part too: And yes the stick content is from previous Live test a year ago /org/freedesktop/UDisks2/block_devices/sdc: org.freedesktop.UDisks2.Block: Configuration: [] CryptoBackingDevice: '/' Device: /dev/sdc DeviceNumber: 2080 Drive: '/org/freedesktop/UDisks2/drives/SanDisk_Extreme_AA011224122232524201' HintAuto: true HintIconName: HintIgnore: false HintName: HintPartitionable: true HintSymbolicIconName: HintSystem: false Id: by-uuid-2019-07-12-19-46-31-00 IdLabel: Mageia-7.1-Live-Xfce-x86_64 IdType: iso9660 IdUUID: 2019-07-12-19-46-31-00 IdUsage: filesystem IdVersion: Joliet Extension MDRaid: '/' MDRaidMember: '/' PreferredDevice: /dev/sdc ReadOnly: false Size: 32017047552 Symlinks: /dev/disk/by-id/usb-SanDisk_Extreme_AA011224122232524201-0:0 /dev/disk/by-label/Mageia-7.1-Live-Xfce-x86_64 /dev/disk/by-path/pci-0000:05:00.0-usb-0:1:1.0-scsi-0:0:0:0 /dev/disk/by-uuid/2019-07-12-19-46-31-00 UserspaceMountOptions: org.freedesktop.UDisks2.PartitionTable: Partitions: ['/org/freedesktop/UDisks2/block_devices/sdc1', '/org/freedesktop/UDisks2/block_devices/sdc5', '/org/freedesktop/UDisks2/block_devices/sdc2', '/org/freedesktop/UDisks2/block_devices/sdc6', '/org/freedesktop/UDisks2/block_devices/sdc7', '/org/freedesktop/UDisks2/block_devices/sdc8'] Type: dos
Tried with two other sticks, one of them the Corsair used earlier in this bug: Isodumper writes and verifies, then crashes and no isodumper.log. Using same sticks again, Isodumper crashes already when it is about to start write. Then i used gparted to make new msdos partition table: isodumper then writes and verifies, then crashes. Generally, why not make it write to isodumper.log continuously while it progresses? Also no output in journal. Why? The system i use for testing is same as usual, mga7 64 bit plasma, fully updated to updates testing.
What is the output of "urpmq --not-available"?
Created attachment 11996 [details] urpmq --not-available | sort
Created attachment 11997 [details] Commands to restore packages to available versions As I suspected, the testing repos were used to install all updates that were available, including those that were later removed for various reasons rather then being validated and pushed to updates. Save the attached file, then run it as root to downgrade the affected packages to available versions plus remove the one kernel devel package that was also removed rather then being pushed. Then please test isodumper again.
@Dave There is really a bug in Isodumper that I should prevent. For the moment, I don't know how. @Morgan Thanks for your output. I need the complete output of usdisksctl dump. The program goes all drives and blocks to find the partitions to unmount. I have to understand why and where it get something not expected.
Thank you Dave for helping me clean my computer :) Did no visible change at all though. I am cleaning even more at the moment. Then I will try again, just in case while I am at it... The system have swap, /home / in an LVM which is LUKS encrypted, residing on a SSD, and a separate ext4 /boot, and a vfat /boot/EFI. And another drive with large plain ext4.
Created attachment 11998 [details] udiskctl dump
Rephrasing: On a SSD there is: ext4 /boot vfat /boot/EFI LUKS encrypted partition, containing an LVM , containing / /home swap Then a spinning rust drive with one large plain ext4. And at the time of udiskctl dump, i had the live USB plugged in.
Created attachment 11999 [details] Additional downgrades Anyway, as I had started already: Tried additional downgrades. No change in result.
Created attachment 12000 [details] Fix for unmounting @Morgan Can you try this patch. It is to apply against /usr/lib/python3.7/site-packages/isodumper/isodumper.py for example /bin/su - cd /usr/lib/python3.7/site-packages/isodumper patch -p1 <0001-Fix-for-unmounting.patch enter isodumper.py because patch is done against lib/isodumper.py
YES! That works on my system. Note: For a slow USB stick i first got the impression it had hung during creation of persistent partition. I guess it was formatting it. (this time i had selected encrypted persistent) Not until I tried a fast stick i noticed the percent bar moving. Could you add text note "Formatting persistent partition..." as a visible text in the log window? The fast stick i used now was the one giving lots of errors in journal before, and that is same now. But that seem to not matter.
Created attachment 12001 [details] rdsosreport.txt.xz of failed 8b1xfce i586 encr persistent The log looked OK: --- cat .isodumper/isodumper.log Målenhet: Corsair Voyager GTX (/dev/sdc) 119.243 GiB Avbild: /home/morgan/Hämtningar/Torrent/Mageia-8-beta1-Live-Xfce-i586/Mageia-8-beta1-Live-Xfce-i586.iso Kör kopia från/home/morgan/Hämtningar/Torrent/Mageia-8-beta1-Live-Xfce-i586/Mageia-8-beta1-Live-Xfce-i586.iso till /dev/sdc Avbilden Mageia-8-beta1-Live-Xfce-i586.iso skrevs framgångsrikt till /dev/sdc Bytes skrivna:2664734720 The sha3 sum check is OK and the sum is signed Added encrypted persistent partition --- But the resulting stick failed to boot... I entered passphrase, and after a couple seconds i am at dracut:/# prompt. Tested on both 32 bit Thinkpad T43, and 64 bit T400. Same problem. I also tried booting with "rd.debug" added to command line, but then i got kernel panic unable to mount root fs. I am pretty sure i have tried a encrypted live persistent successfully before but several isodumper versions ago, probably another stick and other Mageia iso... Any other tester?
I tried the same ISO with creation of the mgalive-persist. When booting, I'm asked for the password. But this one can't unlock the partition. After that, the session starts.
Strange you get a different error. Maybe because the two laptops i tested on have no other disk installed, but probably yours do? Question is: problem in the iso itself, or how isodumper prepares the stick. Calling in Martin for the booting analysis.
CC: (none) => mageia
The booting problem appears to be due to a change in the kernel or udev behaviour. It only affects the 32-bit ISO, and only when persistence is enabled. Please open a new bug for that.
CC: mageia => (none)
Thank you Martin. Created: Bug 27629 - Mageia 8b1 Live i586 fail booting if persistence is used isodumper with patch test OK on same stick, same system and isodumper writing it, but used the x86_64 version of 8b1 live xfce: Works fully OK on same T400 machine: encryption, persistence verified. So, a now build of isodumper rpm to check, please :)
Created attachment 12002 [details] Isodumper signature false and OK Testing an Mageia 7.1 iso, I notice it say in the window and log file that signature is false. Tried the same on two different sticks that show no problem with the cauldron images. Also tried again with a 8b1 iso on same stick, and it say no problem. Problem 1: The popup at the end tells the user it succeeded! It should warn the read back data do not match. Problem 2: I do not think there was error. Maybe the checking method should be different for elder isos? How/Against what does it check? The sticks works (except bug that may be other issues...) First fix problem 1 so we see a warning is issued.
Created attachment 12003 [details] Isodumper signature false - mga4.1 Live As for Mageia 7.1, also the messages for 4.1 is contradictory. Strange thing is that is use a different message in the log (the line beginning with "/!\" (nice warning notice) Maybe the difference is because i di dnot order it to make persistent partition. But logically it should be the same message, so strange!
Created attachment 12004 [details] magiback.log mga7.1x86xfceLive no persistence magiback.log after having written mga7.1 x86 xfce Live without persistence; warning in log but message say it succeeded.
isodumper-1.27-1.mga7 is now in testings I have also added messages for the different steps in creating the persistence partition.
Tested writing a couple sticks, works for me on mga7, 64 bit, Plasma, Qt. Problem per comment #63 is not fixed, we should open a new bug for that. I would like at least one more tester to give OK.
(In reply to Morgan Leijström from comment #67) > Problem per comment #63 is not fixed, we should open a new bug for that. -> Bug 27667 - Log and window warn user checksum error, but dialogue say success > I would like at least one more tester to give OK. Please, someone :) - Because earlier versions of this program have worked for some, but not someone else.
Mageia7, x86_64 Updated from isodumper-gtk 1.21 to 1.27. Dumped the classic beta1 iso to an 8GB flash drive and added a persistent partition. The report: <could not cut and paste> This will take a while: Target Device: Lexar USB Flash Drive (/dev/sdg) 7.469 GiB Image: /data/isos/mageia8/Mageia-8-beta1-x86_64/Mageia-8-beta1-x86_64.iso to /dev/svg Image Mageia-8-beta1-x86_64.iso successfully written to /dev/sdg Bytes written: 4429676544 Invalid signature for /data/isos/mageia8/Mageia-8-beta1-x86_64/Mageia-8-beta1-x86_64.iso.sha3 The signature of the sum is false ! The gui takes forever to quit - in fact it times out. It cannot be removed by window manager. The command line shows: $ isodumper <bound method YCheckBox.value of <yui.YCheckBox; proxy of <Swig Object of type 'YCheckBox *' at 0x7fe3c2aa3fc0> >> <sigint here to kill it> ^CTraceback (most recent call last): File "/usr/bin/isodumper", line 21, in <module> app.run() File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 849, in run self.handleevent() File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 812, in handleevent self.do_write() File "/usr/lib/python3.7/site-packages/isodumper/isodumper.py", line 429, in do_write while not iface.done: File "/usr/lib/python3.7/site-packages/pydbus/proxy_property.py", line 22, in __get__ return instance._object["org.freedesktop.DBus.Properties"].Get(self._iface_name, self.__name__) File "/usr/lib/python3.7/site-packages/pydbus/proxy_method.py", line 97, in __call__ result = instance._bus.con.call_sync(*call_args) KeyboardInterrupt $ cat .isodumper/isodumper.log $
CC: (none) => tarazed25
Thanks Len I almost feel someone is playing tricks on us... Yes unfortunately ~/.isodumper/isodumper.log is only written at exit, so not when it crash. Can you check /var/log/magiback.log and paste/attach relevant part?
Source RPM: isodumper-1.26-1.mga7.noarch.rpm => isodumper-1.27-1.mga7.noarch.rpmAssignee: qa-bugs => yves.brungard_mageia
... and you have checked isodumper (without -gtk) got updated?
Created attachment 12017 [details] magiback log for isodumper session mageia8beta1 classic iso with persistence partition.
Great, thank you. Last line: 2020-11-25 09:28:26,087 ERROR Error while doing persistent partition: Re-reading the partition table failed.: Device or resource busy Not relevant for this bug, but it is the Live isos that can utilise the persistent partition. I am working on describing it: https://wiki.mageia.org/en/User:Morgano/Persistent_live_systems
So, do we need to test this more thoroughly, e.g. with qt and gtk and classic iso then live with persistence partition? That is four to start with.
For you it fail while making the persistent partition, so should not depend on what ISO. Just to rule out bad port, connector, or stick, could you try another USB stick, and at the same time try a Live iso? Have you checked $ isodumper --version
$ isodumper --version 1.27 Yes, I will have a go at a Live with another drive. Pretty sure the device is OK. Installation ran OK. Later.
OK. Done that. $ cat isodumper.log Target Device: USB2.0 Flash Disk (/dev/sdg) 31.902 GiB Image : /data/isos/mageia8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso Executing copy from /data/isos/mageia8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso to /dev/sdg Image Mageia-8-beta1-Live-Plasma-x86_64.iso successfully written to /dev/sdg Bytes written: 2842601472 /!\The computed and stored sums don't match Added persistent partition $ rpm -qa | grep isodumper isodumper-qt-1.27-1.mga7 isodumper-1.27-1.mga7 # udisksctl dump [....] /org/freedesktop/UDisks2/drives/USB2_2e0_Flash_Disk_2008082813350981: org.freedesktop.UDisks2.Drive: CanPowerOff: true Configuration: {} ConnectionBus: usb Ejectable: true Id: USB2.0-Flash-Disk-2008082813350981 Media: MediaAvailable: false MediaChangeDetected: true MediaCompatibility: MediaRemovable: true Model: Flash Disk Optical: false OpticalBlank: false OpticalNumAudioTracks: 0 OpticalNumDataTracks: 0 OpticalNumSessions: 0 OpticalNumTracks: 0 Removable: true Revision: 2.20 RotationRate: -1 Seat: seat0 Serial: 2008082813350981 SiblingId: /sys/devices/pci0000:00/0000:00:14.0/usb2/2-10/2-10:1:.0 Size: 0 SortKey: 01hotplug/1606309126878353 TimeDetected: 1606309126878353 TimeMediaDetected: 0 Vendor: USB2.0 WWN: About to check the Live boot.
Tried three times to boot Live from the USB key on the machine where it was produced and it failed to "take". Gave up and moved to my old Dell X51. The boot started, with a choice of many languages. Opted for fast boot with free graphics driver and it sat forever blowing bubbles. The console shows two lines, both referring to sdb and the USB key LED flickers continuously. Does not look good. About to retry using a Lexar drive. If that fails I shall cut out the persistent partition.
The Lexar drive is 20 times as fast as the "bad" drive just abandoned. The SHA3 checksum was computed without comment. Note, no extra partition. $ cat .isodumper/isodumper.log Target Device: Lexar USB Flash Drive (/dev/sdf) 7.469 GiB Image : /data/isos/mageia8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso Executing copy from /data/isos/mageia8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso to /dev/sdf Image Mageia-8-beta1-Live-Plasma-x86_64.iso successfully written to /dev/sdf Bytes written: 2842601472 SHA3 sum: 47F8F802288AB3787C1FB48F76FF557D57407FB64BB60B4C2D2C3948E16D2DEB394E9FA1083A99D2E5804864A9F0A98A682DDF7E39B4520C0D8A64245BE58CEE
Detection failed on "this" machine. Moved to the X51. First attempt at Live boot with free graphics drivers collapsed in a heap. The Plasma desktop was completely unstable. Broken images flashing and disappearing and jumping about. Restarted with nvidia proprietary driver and all went well. Signed in and was able to operate. Connected to wifi on the router. All looks normal.
Re comment 70. Beginning to feel like you Morgan - gremlins on my shoulder. Trying Xfce Live next with persistent partition.
Moved to Intel Core i9 machine and installed the update there. Dumped Xfce Live to a Lexar drive with a persistent partition. No problem until the sumcheck, which failed. Then the persistent partition was added. One odd thing happened there - "Which file manager do you want to use?" - chose Thunar, but that is the first time that has ever come up. Looked like there was an attempt to mount the key and the new partition. When remounted by plugging in again the persistent partition shows up as /run/media/lcl/mgalive-persist, containing the usual lost+found directory, free space 4.8 GB. Transferred to another machine and managed to boot from it, rebuilding nvidia driver on the fly. Xfce Live OK.
Thought I'd add my $0.02: Updated Isodumper on a non-EFI Plasma machine with an AMD Phenom X4 processor and AMD HD8490 graphics, to keep nVidia out of the mix. Dumped the M7.1 Live Plasma iso to a no-name 8GB usb stick (never before used), and told it to add a non-encrypted persistent partition. Got the bad signature notice, but at this point I'm not quite sure if this was the Official iso or one of the late test isos, so the notice might be valid. Process seemed to complete normally, but... Booting from the resulting stick failed. Several points of failure, all of which seem to be "fs" related. I'd guess the added partition is bad. Booted back into M7 Plasma, and dumped the iso again, this time without the added partition. Same hardware, same usb stick, even the same usb port. Got the same bad signature notice, but this time booting from the stick into Live mode worked perfectly.
Tried plasma iso from m7 plasma host with encrypted persistence ... Target Device: SanDisk Ultra USB 3.0 (/dev/sdf) 114.609 GiB Image : /s3/m8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso Executing copy from /s3/m8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso to /dev/sdf Image Mageia-8-beta1-Live-Plasma-x86_64.iso successfully written to /dev/sdf Bytes written: 3468244992 The sha3 sum check is OK and the sum is signed # journalctl -b --no-hostname |grep magiback Nov 25 16:53:52 dbus-daemon[1125]: [system] Activating via systemd: service name='org.mageia.Magiback' unit='magiback.service' requested by ':1.3900' (uid=500 pid=7094 comm="/usr/bin/python3 /usr/bin/isodumper --qt") Nov 25 16:58:44 magiback[7133]: Welcome to fdisk (util-linux 2.33.2). Nov 25 16:58:44 magiback[7133]: Changes will remain in memory only, until you decide to write them. Nov 25 16:58:44 magiback[7133]: Be careful before using the write command. Nov 25 16:58:44 magiback[7133]: Command (m for help): Partition type Nov 25 16:58:44 magiback[7133]: p primary (2 primary, 0 extended, 2 free) Nov 25 16:58:44 magiback[7133]: e extended (container for logical partitions) Nov 25 16:58:44 magiback[7133]: Select (default p): Partition number (3,4, default 3): First sector (6773316-240353279, default 6774784): Last sector, +/-sectors or +/-size{K,M,G,T,P} (6774784-240353279, default 240353279): Nov 25 16:58:44 magiback[7133]: Created a new partition 3 of type 'Linux' and of size 111.4 GiB. Nov 25 16:58:44 magiback[7133]: Command (m for help): The partition table has been altered. Nov 25 16:58:44 magiback[7133]: Calling ioctl() to re-read partition table. Nov 25 16:58:44 magiback[7133]: The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8). At this point isodumper-qt appears to be waiting for something. htop shows no related activity. Booted from the stick on another system, boot to plasma working. Persistence partition was not created.
Repeated test from comment 84 without persistent partition and it completed properly.
I also had isodumper hanging now, it did not update its window after screensaver, had to force close, going to bed now.
(In reply to Morgan Leijström from comment #73) > Great, thank you. > > Last line: > 2020-11-25 09:28:26,087 ERROR Error while doing persistent partition: > Re-reading the partition table failed.: Device or resource busy > Hello, Thanks for your tests. This seems to be the problem. 1. There is an error. 2. This error then hangs the interface. Stop the tests, I will fix that.
(In reply to Morgan Leijström from comment #73) > > Last line: > 2020-11-25 09:28:26,087 ERROR Error while doing persistent partition: > Re-reading the partition table failed.: Device or resource busy (In reply to Dave Hodgins from comment #84) > Nov 25 16:58:44 magiback[7133]: Command (m for help): The partition table > has been altered. > Nov 25 16:58:44 magiback[7133]: Calling ioctl() to re-read partition table. > Nov 25 16:58:44 magiback[7133]: The kernel still uses the old table. The new > table will be used at the next reboot or after you run partprobe(8) or > kpartx(8). I have seen this error one time, but I can't reproduce it again. I don't understand what triggers this error. It seems that this returns an error code, which I didn't catch nor know. Can you provide information about the state of the device (partition table) before operation and which partitions where mounted? I will include the return code in logs for the next time. I found also why the interface was stuck after this error.
(In reply to papoteur from comment #88) > I have seen this error one time, but I can't reproduce it again. > I don't understand what triggers this error. So *possible* to reproduce for you too - good ! > It seems that this returns an error code, which I didn't catch nor know. > Can you provide information about the state of the device (partition table) > before operation I was reusing an earlier stick, with or without persistense, and encryption i do not know. There may have been a second USB memory still attached when i started isodumper, that i do other work on. I dont remember the work order i did... > and which partitions where mounted? None from stick was mounted (at least manually) > I will include the return code in logs for the next time. So i suggest to release a version now just to testing so we can get the code, and maybe even more problems show up... we seem to dig up new problem every release... > I found also why the interface was stuck after this error. Good :) ----- To testers, for next round: § The journal logs USB connection and some errors. So before starting isodumper and inserting the stick start a terminal window and as root issue "journalctl -f" § /var/log/magiback.log is where isodumper continously logs, so start another terminal and issue as normal user "tail -f /var/log/magiback.log" § If you plan to use the stick, not all Live ISOs works with persistence: all 64 bit mga8beta1 works, including encryption. But 8b1 32 bit fail booting with any persistence. mag7.1 all Live works with persistence without encryption (32 bit need a bit help after updating)
(In reply to Morgan Leijström from comment #89) > So i suggest to release a version now just to testing so we can get the > code, and maybe even more problems show up... we seem to dig up new problem > every release... > I have pushed modifications for getting more debug information. Thus lib/isodumper.py and backend/raw_write.py are affected. To get more debug info, launch isodummper with --debug option. Logs will be written in ~/.isodumper/isodumper2.log I know this is not clean, but a complete change would be too long to do just now. About the bug, > Re-reading the partition table failed.: Device or resource busy a frequent cause of such error message is that a partition is busy. Partitions of the stick are unmounted before writing, thus if there are mounted, this is because of some automount feature, which depends on the desktop environment and some settings. I didn't find a clear explanation on why the error occurs. I found only some tricks to workaround, like manual intervention or here: https://unix.stackexchange.com/questions/264056/re-reading-the-partition-table-failed-with-error-16-device-or-resource-busy
Yes some automounting may be playing us tricks. Not my cup of tea. Can isodumper tell kernel to reserve the device for it? More Gremlins: $ isodumper --debug usage: isodumper [options] isodumper: error: unrecognized arguments: --debug So i try without: $ isodumper OK until it is about to write - it simply stop. In journal: nov 27 17:38:08 svarten.tribun dbus-daemon[1527]: [system] Activating via systemd: service name='org.mageia.Magiback' unit='magiback.service' requested by ':1.855' (uid=10702 pid=30240 comm="/usr/bin/python3 /usr/bin/isodumper") nov 27 17:38:08 svarten.tribun systemd[1]: Starting Magiback backend... nov 27 17:38:08 svarten.tribun magiback[7673]: Traceback (most recent call last): nov 27 17:38:08 svarten.tribun magiback[7673]: File "/usr/bin/magiback", line 5, in <module> nov 27 17:38:08 svarten.tribun magiback[7673]: import isodumper.raw_write as raw_write nov 27 17:38:08 svarten.tribun magiback[7673]: File "/usr/lib/python3.7/site-packages/isodumper/raw_write.py", line 249 nov 27 17:38:08 svarten.tribun magiback[7673]: logging.error(self.return_message) nov 27 17:38:08 svarten.tribun magiback[7673]: ^ nov 27 17:38:08 svarten.tribun magiback[7673]: SyntaxError: invalid syntax nov 27 17:38:08 svarten.tribun systemd[1]: magiback.service: Main process exited, code=exited, status=1/FAILURE nov 27 17:38:08 svarten.tribun systemd[1]: magiback.service: Failed with result 'exit-code'. nov 27 17:38:08 svarten.tribun systemd[1]: Failed to start Magiback backend. Maybe i did something wrong when i replaced the files?: ___Patching to testing version _Workspace (whatever...) cd ~ mkdir ~/Isod cd ~/Isod _Get wget http://gitweb.mageia.org/software/isodumper/plain/backend/raw_write.py wget http://gitweb.mageia.org/software/isodumper/plain/lib/isodumper.py _Backup sudo mv /usr/lib/python3.7/site-packages/isodumper/raw_write.py /usr/lib/python3.7/site-packages/isodumper/raw_write.py_1.27 sudo mv /usr/lib/python3.7/site-packages/isodumper/isodumper.py /usr/lib/python3.7/site-packages/isodumper/isodumper.py_1.27 _Place sudo chown root:root raw_write.py isodumper.py sudo mv raw_write.py /usr/lib/python3.7/site-packages/isodumper/ sudo mv isodumper.py /usr/lib/python3.7/site-packages/isodumper/ _Stop so it restarts sudo systemctl stop magiback.service
Am I a little nervous? OK. A new commit fix the syntax error in backend/raw_write.py Furthermore, the debug option should also be added in launcher isodumper (in root of the repository) landing in /usr/bin. I will have a look in locking devices.
With the version of isodumper.py and raw_write.py copied about 40 minutes ago, (no --debug option available), it worked on this try. Target Device: SanDisk Ultra USB 3.0 (/dev/sdf) 114.609 GiB Image : /s3/m8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso Executing copy from /s3/m8/Mageia-8-beta1-Live-Plasma-x86_64/Mageia-8-beta1-Live-Plasma-x86_64.iso to /dev/sdf Image Mageia-8-beta1-Live-Plasma-x86_64.iso successfully written to /dev/sdf Bytes written: 3468244992 The sha3 sum check is OK and the sum is signed Added encrypted persistent partition Booted the stick on another system, which worked, but it failed to mount the encrypted persistent partition. I'll try to debug and report that later. Will also continue testing with other iso images
I was wrong about the persistent partition not being mounted. While it doesn't show up in df or mount output, I created a file, rebooted, and the file was present, so encrypted persistent partition is working. Only quibble is that the passphrase dialog just has a box for the passphrase, with no indication of what is being requested, but that is another issue.
Created attachment 12023 [details] Log from failed to create persistent partition. This time it failed ... Target Device: SanDisk Ultra USB 3.0 (/dev/sdf) 114.609 GiB Image : /s3/m8/Mageia-8-beta1-Live-Xfce-x86_64/Mageia-8-beta1-Live-Xfce-x86_64.iso Executing copy from /s3/m8/Mageia-8-beta1-Live-Xfce-x86_64/Mageia-8-beta1-Live-Xfce-x86_64.iso to /dev/sdf Image Mageia-8-beta1-Live-Xfce-x86_64.iso successfully written to /dev/sdf Bytes written: 2810071040 The sha3 sum check is OK and the sum is signed Error 1 while doing persistent partition: Re-reading the partition table failed.: Device or resource busy I ran "> /var/log/magiback.log" as root prior to this test. I noticed from the magiback.log that it's running gpg with --keyserver pool.sks-keyservers.net --recv-k eys EDCA7A90 every time isodumper is run. It should only do that if the key has not previously been imported.
Regarding comment 95, I do not have any form of auto mounting enabled. The device notifier does notify me that the partitions are available for mounting, but the notification is all that is enabled. No auto-mount.
Here are the journal lines from that run of isodumper ... Nov 27 14:36:57 magiback[20949]: Welcome to fdisk (util-linux 2.33.2). Nov 27 14:36:57 magiback[20949]: Changes will remain in memory only, until you decide to write them. Nov 27 14:36:57 magiback[20949]: Be careful before using the write command. Nov 27 14:36:57 magiback[20949]: Command (m for help): Partition type Nov 27 14:36:57 magiback[20949]: p primary (2 primary, 0 extended, 2 free) Nov 27 14:36:57 magiback[20949]: e extended (container for logical partitions) Nov 27 14:36:57 kernel: sdf: sdf1 sdf2 Nov 27 14:36:57 magiback[20949]: Select (default p): Partition number (3,4, default 3): First sector (5487820-240353279, default 5488640): Last sector, +/-sectors > Nov 27 14:36:57 magiback[20949]: Created a new partition 3 of type 'Linux' and of size 112 GiB. Nov 27 14:36:57 magiback[20949]: Command (m for help): The partition table has been altered. Nov 27 14:36:57 magiback[20949]: Calling ioctl() to re-read partition table. Nov 27 14:36:57 magiback[20949]: The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8). Nov 27 14:36:57 myusb[23617]: drive inserted sdf1 Nov 27 14:36:58 kernel: sdf: detected capacity change from 123060879360 to 0
The line Nov 27 14:36:57 myusb[23617]: drive inserted sdf1 is from a rule I created ... ]$ cat /etc/udev/rules.d/00-myusb.rules # udev rules file for my usb drive ACTION!="add", GOTO="myusb_rules_end" SUBSYSTEM!="block", GOTO="myusb_rules_end" ATTR{partition}=="1", RUN+="/bin/logger -t myusb drive inserted %k" LABEL="myusb_rules_end
Thanks Dave for your tests. Was the interface still running well after the error occurred? Previously, it was stuck in this case. I presume also that magiback.log contents a line: Error ... while doing persistent partition: ... I hope to get the number reported here.
(In reply to Dave Hodgins from comment #95) > > I noticed from the magiback.log that it's running gpg with --keyserver > pool.sks-keyservers.net --recv-k > eys EDCA7A90 every time isodumper is run. It should only do that if the key > has not previously been imported. I can I know that the key has previously been imported?
(In reply to Dave Hodgins from comment #94) > Only quibble is that the passphrase dialog just has a box for the passphrase, > with no indication of what is being requested, but that is another issue. That is looking exactly like for a conventionally installed system with LUKS. I agree. So now I issued: Bug 27673 - LUKS key dialog at boot should say what it asks for.
(In reply to papoteur from comment #99) > Thanks Dave for your tests. > Was the interface still running well after the error occurred? Previously, > it was stuck in this case. Correct > I presume also that magiback.log contents a line: > Error ... while doing persistent partition: ... > I hope to get the number reported here. Error 1 while doing persistent partition: Re-reading the partition table failed.: Device or resource busy
(In reply to papoteur from comment #100) > (In reply to Dave Hodgins from comment #95) > > > > I noticed from the magiback.log that it's running gpg with --keyserver > > pool.sks-keyservers.net --recv-k > > eys EDCA7A90 every time isodumper is run. It should only do that if the key > > has not previously been imported. > I can I know that the key has previously been imported? gpg --list-key EDCA7A90 2>/dev/null | grep release@mageia.org >/dev/null || gpg --keyserver pool.sks-keyservers.net --recv-keys EDCA7A90
Is the Mageia key: EDCA7A90 or 835E41F4EDCA7A90 ? Or is the first a shortcut for the second? (In reply to Dave Hodgins from comment #102) > Error 1 while doing persistent partition: Re-reading the partition table > failed.: Device or resource busy Fine, thanks. I will add a test on this code and add a command "partprobe" without stopping the program. Papoteur
(In reply to papoteur from comment #104) > > Fine, thanks. I will add a test on this code and add a command "partprobe" > without stopping the program. Now in git, in backend/raw_write.py Can you give your results?
Ran it three times. Once each with gnome, plasma, and xfce4 iso images. Worked perfectly. The only problem remaining is importing a key to the root user's gpg public key ring, when it's previously already been imported. That puts an unnecessary extra load on the key server.
Forgot to mention. I added an encrypted persistent partition in each test.
Also just noticed that isodumper should require sha3sum and doesn't currently.
Just tested again. Before the test ... # gpg --list-keys|grep 835E41F4EDCA7A90 B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 # > /var/log/magiback.log There is still something wrong in http://gitweb.mageia.org/software/isodumper/plain/backend/raw_write.py /usr/lib/python3.7/site-packages/isodumper/raw_write.py has ... if mageia_keyid in [entry['keyid'] for entry in keys_list]: logging.info("Mageia key already present") else: gpg.recv_keys('pool.sks-keyservers.net', mageia_keyid) So /var/log/magiback.log should have a line with "Mageia key already present". It doesn't. It has ... 2020-11-29 21:57:15,731 DEBUG 15876: gpg --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --keyserver pool.sks-keyservers.net --recv-keys EDCA7A90 2020-11-29 21:57:16,013 DEBUG [GNUPG:] IMPORT_OK 0 B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 2020-11-29 21:57:16,013 DEBUG [GNUPG:] KEY_CONSIDERED B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 0 2020-11-29 21:57:16,013 DEBUG gpg: key 835E41F4EDCA7A90: "Mageia Release <release@mageia.org>" not changed So it's still trying to import the key instead it's still trying to re-import the key that's already present in /root/.gnupg/pubring.gpg I do not see what is wrong with the line if mageia_keyid in [entry['keyid'] for entry in keys_list]: but it isn't working.
To test how isodumper looks like, and better feedback, i tried to make it use English. For other Mageia tools like drakrpm, i can prepend the command with LC_ALL=C. But that does not work for Isodumper. I think optimally that should work? ----------- What I wanted to see is the formatting of the text at left right above the log area. In Swedish locale there is no space before the drive i.e "sdc". I believe there should be no space in the word translations, so there should be one generated by isodumper?
(In reply to Morgan Leijström from comment #110) > To test how isodumper looks like, and better feedback, i tried to make it > use English. For other Mageia tools like drakrpm, i can prepend the command > with LC_ALL=C. But that does not work for Isodumper. I think optimally > that should work? > > ----------- > > What I wanted to see is the formatting of the text at left right above the > log area. In Swedish locale there is no space before the drive i.e "sdc". I > believe there should be no space in the word translations, so there should > be one generated by isodumper? It should work with LANGUAGE=C isodumper For key loading, I will have a look later.
(In reply to papoteur from comment #111) > (In reply to Morgan Leijström from comment #110) > > What I wanted to see is the formatting of the text at left right above the > > log area. In Swedish locale there is no space before the drive i.e Checking sdc In Swedish there is no space: Kontrollerarsdc Whow/Who to ping for that? > > It should work with > LANGUAGE=C isodumper Works :)
(In reply to Morgan Leijström from comment #112) > (In reply to papoteur from comment #111) > > (In reply to Morgan Leijström from comment #110) > > > What I wanted to see is the formatting of the text at left right above the > > > log area. In Swedish locale there is no space before the drive i.e > > Checking sdc > > In Swedish there is no space: > > Kontrollerarsdc > > Whow/Who to ping for that? > Fixed in git
Created attachment 12040 [details] Test scripts for gpg signature keys @Dave, Here is a small script to check what your system get, to help me to know which value to test. You can use python3 test_keys.py as root and as user. On my side, I didn't find any difference. I can use perhaps the complete fingerprint.
Created attachment 12041 [details] Output of python3 test_keys.py run as normal user I've attached the output when run as a regular user as it's 47042 bytes. When run as root ... # python3 /usr/local/bin/test_keys.py ['A5E8DBC98C0108B4', '6458D3D47D00B581', 'D881B0141B678A63', '2067001B1B678A63', '8F0BFA96DA10B483', '29FC7FA641BCD9E7', 'B742FA8B80420F66', '3798E3D757E37087', 'EA103BA357E37087', '835E41F4EDCA7A90'] ['916A770823A7B27AADE01565A5E8DBC98C0108B4', '0868460B3FB1CF6B585F2DDC6458D3D47D00B581', 'F2AB58AD27B35E8EB8C959CED881B0141B678A63', 'C9B8CAC3318B9A9E488359612067001B1B678A63', 'BAF34745AD6CE28442BF4BF58F0BFA96DA10B483', '26446DA60B03CC3C8B43BD5229FC7FA641BCD9E7', '00EDB89585B012A8916F0DF8B742FA8B80420F66', '0E573DA0F13969EF1DD5ACAA3798E3D757E37087', '0D1CC5C42C9888284DFC9730EA103BA357E37087', 'B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90'] I don't normally run gpg as root as that's rarely needed.
The Mageia Release key is present in both root and user gpg public keyrings. [dave@x3 ~]$ gpg --list-key EDCA7A90 2>/dev/null pub rsa4096 2012-04-18 [SCEA] [expires: 2020-12-30] B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 uid [ full ] Mageia Release <release@mageia.org> [dave@x3 ~]$ gpg --list-keys 2>/dev/null|grep EDCA7A90 B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 [dave@x3 ~]$ su - Password: [root@x3 ~]# gpg --list-key EDCA7A90 2>/dev/null pub rsa4096 2012-04-18 [SCEA] [expires: 2020-12-30] B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 uid [ unknown] Mageia Release <release@mageia.org> [root@x3 ~]# gpg --list-keys 2>/dev/null|grep EDCA7A90 B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90
Hello, I don't see any problem in my test about the key already present.
I just ran it again running isodumper as a regular user rather then as root. While the writing to the usb stick is done as root, I can't tell from the magiback.log contents whether gpg is being run as my id, so using /home/dave/.gnupg/pubring.gpg or as root using /root/.gnupg/pubring.gpg Both of those files have key 835E41F4EDCA7A90 present before running isodumper. magiback.log has ... 2020-12-04 05:23:34,177 DEBUG 24735: gpg --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --keyserver pool.sks-keyservers.net --recv-k eys EDCA7A90 2020-12-04 05:23:34,178 DEBUG data copier: <Thread(Thread-113, initial daemon)>, <_io.BytesIO object at 0x7f93a4acc9b0>, <_io.BufferedWriter name=13> 2020-12-04 05:23:34,178 DEBUG closed output, 0 bytes sent 2020-12-04 05:23:34,178 DEBUG stderr reader: <Thread(Thread-114, initial daemon)> 2020-12-04 05:23:34,179 DEBUG stdout reader: <Thread(Thread-115, initial daemon)> 2020-12-04 05:23:34,498 DEBUG [GNUPG:] IMPORT_OK 0 B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 2020-12-04 05:23:34,498 DEBUG [GNUPG:] KEY_CONSIDERED B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 0 2020-12-04 05:23:34,498 DEBUG gpg: key 835E41F4EDCA7A90: "Mageia Release <release@mageia.org>" not changed So while the verification of the iso sum files is working, it's repeating the importation of a previously already imported key, every time isodumper is run. That's bad etiquette for pgp or gpg key management, as it puts an unnecessary extra load on the key server. Are you not seeing the key that's already present being imported every time isodumper is run?
As you can see here, test recognize that the key is present. Did you restarted your box or magiback since you imported the new code ? magiback run as a service and start when called for the first time, but still running after that. You can use systemctl restart magiback Key getting is done by magiback which run as root. 2020-11-29 15:05:10,350 DEBUG Writing thread started 2020-11-29 15:05:10,351 DEBUG Starting getting sum 2020-11-29 15:05:10,353 DEBUG 7045: gpg --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --version 2020-11-29 15:05:10,353 DEBUG stderr reader: <Thread(Thread-3, initial daemon)> 2020-11-29 15:05:10,354 DEBUG stdout reader: <Thread(Thread-4, initial daemon)> 2020-11-29 15:05:10,358 DEBUG chunk: b'gpg (GnuPG) 2.2.19\nlibgcrypt 1.8.5\nCopyright (C) 2019 Free Software Foundation, Inc.\nLicense GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to ' 2020-11-29 15:05:10,360 DEBUG 7048: gpg --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --list-keys --fingerprint --fingerprint 2020-11-29 15:05:10,360 DEBUG stderr reader: <Thread(Thread-5, initial daemon)> 2020-11-29 15:05:10,360 DEBUG stdout reader: <Thread(Thread-6, initial daemon)> ... 2020-11-29 15:05:10,365 DEBUG [GNUPG:] KEYEXPIRED 1580388440 2020-11-29 15:05:10,366 INFO Mageia key already present 2020-11-29 15:05:10,366 INFO N'a pas trouvé de fichier de signature /home/yves/dev/isodumper/lib/Mageia-6-netinstall-x86_64.iso.sha3.gpg 2020-11-29 15:05:10,366 INFO N'a pas trouvé de fichier de somme /home/yves/dev/isodumper/lib/Mageia-6-netinstall-x86_64.iso.sha3 2020-11-29 15:05:18,285 DEBUG Fin de l'écriture 2020-11-29 15:05:18,288 DEBUG Start checking 2020-11-29 15:05:20,541 DEBUG last read 0 2020-11-29 15:05:20,548 INFO Somme SHA3 : E40A490D5310808ABC00ECC92B6893F1BDD5611A397D54FCCC2A871802669FBE3631357D62F455AF4C07B602E60AE89089733225ABFFFA6E8569ADDCB73FB6AF 2020-11-29 15:05:21,306 DEBUG Start doing persistence partition 2020-11-29 15:05:21,306 DEBUG Persistence thread started 2020-11-29 15:05:22,439 DEBUG New partition created 2020-11-29 15:05:41,365 INFO Persistent partition done 2020-11-29 15:05:41,874 DEBUG Finished
$ uptime 06:48:27 up 17 days, 12:11, 4 users, load average: 0.05, 0.10, 0.09 The above would be from the latest kernel update as this system is normally running 24x7. # systemctl restart magiback.service # >/var/log/magiback.log # isodumper After copying the iso image and verifying it, magiback.log shows it no longer retrieving the all ready available public key. I didn't realize magiback ran as a service. That was the problem. I'm willing to ok the update now, but have another comment to discuss first. The gui is somewhat messy. The button to write to device should be on a line by itself after the lines for adding a persistent partition and option encryption selection with an encryption key to make it clear those options should be selected, if desired, before selecting the write to device button. Having the option to format the device with different file system types is confusing as it seems to imply that the persistent partition can be different file system types. I'd remove that button completely and let the user use disk drake if they want to overwrite an iso image on a device with a normal partition. What do you think? Approve the update as is or make more changes?
> Having the option to format the device with different file system types is > confusing as it seems to imply that the persistent partition can be different > file system types. I'd remove that button completely and let the user use > disk drake if they want to overwrite an iso image on a device with a normal > partition. The big advantage of Isodumper is that it protects the user from formatting his running system, home etc. It is convenient it is the same tool. It even can backup and restore previous content. Analogous: If he could use diskdrake to format stick, he could as well use dd instead of isodumper. > What do you think? Approve the update as is or make more changes? I think we should release this - it have been held long enough... And discuss the interface in a new issue. I too find it a bit messy. Also, when it is finished it resets the controls immediately - would be nice if the check boxes remained set so user see he did not forget them - IMO they should remain as they were used, analogous to the log is visible and i believe the encryption key is preserved already (dots remain).
Validating the update. I'll open a new bug report to discuss the possible layout changes later today. Thanks for your patience on this one.
Keywords: (none) => validated_updateWhiteboard: (none) => MGA7-64-OK
BTW - for later fix IMO, I just let it flash a mga7.1 iso, and saw it still complain "The signature of the sum is false".
(In reply to Dave Hodgins from comment #122) > Validating the update. I'll open a new bug report to discuss the possible > layout changes later today. > > Thanks for your patience on this one. Please CC me on that bug. I have a few comments on the subject.
(In reply to Morgan Leijström from comment #123) > BTW - for later fix IMO, I just let it flash a mga7.1 iso, and saw it still > complain "The signature of the sum is false". Did you try to check the signature, like: gpg --verify Mageia-7.1-Live-Xfce-x86_64.iso.sha3.gpg Mageia-7.1-Live-Xfce-x86_64.iso.sha3 I agree that the GUI is to improve. I will work on that later. Morgan suggested already some improvements, but I don't find them anymore.
(In reply to papoteur from comment #125) > Did you try to check the signature, like: > gpg --verify Mageia-7.1-Live-Xfce-x86_64.iso.sha3.gpg [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ LC_ALL=C gpg --verify Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg Mageia-7.1-Live-Xfce-i586.iso.sha3 gpg: Signature made Sun Jul 14 23:11:16 2019 CEST gpg: using RSA key 835E41F4EDCA7A90 gpg: Good signature from "Mageia Release <release@mageia.org>" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: B210 76A0 CBE4 D93D 66A9 D08D 835E 41F4 EDCA 7A90
(In reply to Morgan Leijström from comment #126) > (In reply to papoteur from comment #125) > > Did you try to check the signature, like: > > gpg --verify Mageia-7.1-Live-Xfce-x86_64.iso.sha3.gpg > > [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ LC_ALL=C gpg --verify > Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg Mageia-7.1-Live-Xfce-i586.iso.sha3 > gpg: Signature made Sun Jul 14 23:11:16 2019 CEST > gpg: using RSA key 835E41F4EDCA7A90 > gpg: Good signature from "Mageia Release <release@mageia.org>" [unknown] How about ... $ LC_ALL=C sha3-512sum -c Mageia-7.1-Live-Xfce-i586.iso.sha3
bug 27744 created for change suggestions.
See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=27744
[morgan@svarten Mageia-7.1-Live-Xfce-i586]$ LC_ALL=C sha3-512sum -c Mageia-7.1-Live-Xfce-i586.iso.sha3 Mageia-7.1-Live-Xfce-i586.iso: OK
(In reply to Morgan Leijström from comment #129) > [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ LC_ALL=C sha3-512sum -c > Mageia-7.1-Live-Xfce-i586.iso.sha3 > > Mageia-7.1-Live-Xfce-i586.iso: OK When you run isodumper are you running it as root or as a regular user and letting isodumper ask for the root password when required for writing to the device? If you are running it as root, then as root does LC_ALL=C gpg --verify Mageia-7.1-Live-Xfce-x86_64.iso.sha3.gpg work ok?
(In reply to Dave Hodgins from comment #122) > Validating the update. I'll open a new bug report to discuss the possible > layout changes later today. > > Thanks for your patience on this one. Assigning to QA. Advisory: ==================================== Updated isodumper packages fix some bugs. The isodumper program crashed when making the persistence partition on slow USB devices. The isodumper-1.28-1.mga7 packages fixes that by adding a step of waiting for the partition table to be known by the kernel when creating the persistence partition. Now, it also uses udisk2 to unmount partitions. We added other checks to avoid race conditions to detect other instance of isodumper to be launched. Also, several translations have been added. ===================================== in updates_testing: isodumper-1.28-1.mga7 isodumper-qt-1.28-1.mga7 isodumper-gtk-1.28-1.mga7 from SRPM isodumper-1.28-1.mga7.src.rpm Advisory pushed to SVN.
Source RPM: isodumper-1.27-1.mga7.noarch.rpm => isodumper-1.21-1.mga7.src.rpm
(In reply to Dave Hodgins from comment #130) > When you run isodumper are you running it as root or as a regular user and > letting isodumper ask for the root password when required for writing to > the device? Tried both, no difference. I also tried to copy the folder from the large storage where i have large files, to a folder under /home, and let isodumper use the iso from there, no difference. > If you are running it as root, then as root does > LC_ALL=C gpg --verify Mageia-7.1-Live-Xfce-x86_64.iso.sha3.gpg > work ok? No differences: [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ LC_ALL=C gpg --verify Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg gpg: no signed data gpg: can't hash datafile: No data [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ sudo LC_ALL=C gpg --verify Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg [sudo] lösenord för morgan: gpg: no signed data gpg: can't hash datafile: No data [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ pwd /home/morgan/Ut/isotest/Mageia-7.1-Live-Xfce-i586 [morgan@svarten Mageia-7.1-Live-Xfce-i586]$ su - Lösenord: [root@svarten ~]# cd /home/morgan/Ut/isotest/Mageia-7.1-Live-Xfce-i586 [root@svarten Mageia-7.1-Live-Xfce-i586]# LC_ALL=C gpg --verify Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg gpg: no signed data gpg: can't hash datafile: No data [root@svarten Mageia-7.1-Live-Xfce-i586]# ls -l totalt 2301360 -rw-r--r-- 1 morgan kamo 2356486144 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso -rw-r--r-- 1 morgan kamo 543 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.gpg -rw-r--r-- 1 morgan kamo 64 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.md5 -rw-r--r-- 1 morgan kamo 543 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.md5.gpg -rw-r--r-- 1 morgan kamo 160 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.sha3 -rw-r--r-- 1 morgan kamo 543 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg -rw-r--r-- 1 morgan kamo 160 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.sha512 -rw-r--r-- 1 morgan kamo 543 jul 16 2019 Mageia-7.1-Live-Xfce-i586.iso.sha512.gpg -rw-r--r-- 1 morgan kamo 70570 jul 16 2019 Mageia-7.1-Live-Xfce-i586.lst Maybe there is something that have gone wrong and need a reset ? I dont know how these things work. About the messages and behaviour: Isodumper say "The signature of the sum is false !" But if i understand correctly that is wrong, it should say if fail to find the signature? Thought: Isodumper is supposed to write general images, i.e also them it creates backing up. To at lease verify the written, it could create a checksum during write, and verify it get the same checksum when reading back. Extra good if it could create a sha checksum file when it is backing up, for separate use.
CC: (none) => yves.brungard_mageia
An update for this issue has been pushed to the Mageia Updates repository. https://advisories.mageia.org/MGAA-2020-0241.html
Resolution: (none) => FIXEDStatus: ASSIGNED => RESOLVED
I'd been using Mageia 1 beta 1 iso images in my tests. In Mageia 7 and 7.1 the .gpg files are detached signatures so the command to verify the iso sha3 sum file is # gpg --verify Mageia-7.1-Live-Xfce-i586.iso.sha3.gpg Mageia-7.1-Live-Xfce-i586.iso.sha3 gpg: Signature made 2019-07-14T17:11:16 EDT gpg: using RSA key 835E41F4EDCA7A90 gpg: Good signature from "Mageia Release <release@mageia.org>" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: B210 76A0 CBE4 D93D 66A9 D08D 835E 41F4 EDCA 7A90 In Mageia 8 beta 1 the .gpg files contain signed copies of the sum files instead of detached signatures so the command is ... gpg --verify Mageia-8-beta1-Live-Xfce-i586.iso.sha3.gpg gpg: Signature made 2020-08-02T04:57:02 EDT gpg: using RSA key B21076A0CBE4D93D66A9D08D835E41F4EDCA7A90 gpg: Good signature from "Mageia Release <release@mageia.org>" [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: B210 76A0 CBE4 D93D 66A9 D08D 835E 41F4 EDCA 7A90 As is, isodumper works for m8 beta 1, but not for m7 or m7.1 gpg signatures. I'll add that to bug 27744