Bug 32035 - Isodumper various issues
Summary: Isodumper various issues
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: High major
Target Milestone: ---
Assignee: QA Team
QA Contact:
URL:
Whiteboard:
Keywords: validated_update
Depends on:
Blocks:
 
Reported: 2023-06-20 19:19 CEST by Morgan Leijström
Modified: 2023-08-19 21:42 CEST (History)
3 users (show)

See Also:
Source RPM: isodumper-1.52-1.mga9.src.rpm
CVE:
Status comment:


Attachments

Description Morgan Leijström 2023-06-20 19:19:44 CEST
__Description of problem:
After it backed up a stick, two problems (maybe same cause)
1 it pause and pops up a message about backup done (should instead just continue with next step)
2 forgets source image for writing to stick and complain about it


__Version
1.48-1.mga9


__Steps to Reproduce
1. Check all boxes and fill in fields.
2. After the backup, a popup saying backup is finished.
3. OK that popup, and another popup "No image to write is selected"
And looking in the main window i see *all* text input fields are cleared!

The first quirk is that it pops up saying backup is done - it should just go on with next step.


__Workaround
run it again, without the backup step.


__Additional quirk
In the log window the line 
  sdd sucessfully written to ....
is duplicated.
Comment 1 papoteur 2023-06-25 19:19:57 CEST
Thanks Morgan for the report.
isodumper-1.49-1.mga9 is in testings to solve that.
Something that is not cool is that saved image is owned by root.
Comment 2 Morgan Leijström 2023-06-25 23:49:09 CEST
OK better.

It now after backup phase stops and asks if i am sure i want to overwrite the content on the stick.

Also, then an pop up asking for my sudo password.

It all feels a bit strange - better if it asks all it need in the beginning (such as sudo), and inform that content on the stick will be overwritten.

- so it can then execute all steps without user waiting babysitting and responding to two popups in the middle of the process.

(instead execute all in one go)


BTW, I presume that if problem with making the backup, that the whole process is then aborted with a message?


---

Minor note for backup progress bar at backup
(saw it on 1.48, not tested on 1.49):

How I got there: There is something wrong with one of my old 4GB stick, backup slows to a crawl after a while reading, and i aborted by Ctrl-C in the terminal from where i launced isodumper.
(btw it would be good if there were better way to abort)

After launching isodumper again and started backing up another stick, the progress bar for a couple seconds initially showed 80% !  - the value when i aborted isodumper previous run.  It should not have remembered that.

---

The image ownership:
is it wrong for both old version in Mageia 8 and this new in 9?

Do you want to work on it now or should we note it at
https://wiki.mageia.org/en/IsoDumper_Writing_ISO_images_on_USB_sticks
Comment 3 Morgan Leijström 2023-06-25 23:56:58 CEST
BTW,

 /var/log/magiback.log seem to not note making backup

 ~/.config/isodumper.log does not exist.
Is it now in another place, or do it need to be enabled?
Comment 4 papoteur 2023-07-01 10:12:26 CEST
(In reply to Morgan Leijström from comment #3)
> BTW,
>  ~/.config/isodumper.log does not exist.
> Is it now in another place, or do it need to be enabled?
Yes, message says that it is in ~/.isodumper/isodumper.log
Comment 5 papoteur 2023-07-01 10:24:33 CEST
New 1.50 is coming in testing.
          - New order in checking before operation, to check all at start
          - Keep authorization during 10 min
          - Give user ownership to the backup image
The latest is done by passing uid/gid from GUI application to apply them to the file at end of writing.
The 10 minutes are hardcoded. I think it is a good duration.
Comment 6 Morgan Leijström 2023-07-01 18:42:33 CEST
(In reply to papoteur from comment #5)
> New 1.50 is coming in testing.
>           - New order in checking before operation, to check all at start

Now it performed a full run in one go :)

>           - Keep authorization during 10 min
>           - Give user ownership to the backup image

User set correctly, but
* It set Wrong group! *

It set same group ID as my user ID.  That may be correct for some users with private group, but I use a different group.


> The latest is done by passing uid/gid from GUI application to apply them to
> the file at end of writing.
> The 10 minutes are hardcoded. I think it is a good duration.

Backup itself took about 6 minutes on this my main machine, USB2, a three year old cheap 16MB stick.  I also have a 128MB stick, will probably not work...

================================================

(In reply to Morgan Leijström from comment #0)
>   sdd sucessfully written to ....
> is duplicated.

This is fixed OK.

================================================

(In reply to papoteur from comment #4)
> Yes, message says that it is in ~/.isodumper/isodumper.log

Right. Added now to wiki page in the section of this version

There is very little loggin into that file.
I only got one row per run, correctly saying a GPG file was not found 

Seems wrong.

================================================

(In reply to Morgan Leijström from comment #3)
> 
>  /var/log/magiback.log seem to not note making backup

It would be great to add that so all main phases are seen in the log

================================================

The text block is not translated (in Swedish at least):

 The selected operations will be executed in order from top to bottom.
 If both write image and create partition are selected, the partition
 will be created in the free space after the image.

================================================

Major problems!

I set it to write an ISO and create ext4, at the popup for rights i cancel.
It then immediately and wrong in log say it wrote the image, tells many bytes written, and performs verify
Even more wrong is that the popup then tell it suceeded, but i know it did not write and i afterwards also verify old content is there, and no extra partiton made.

What it did right was tha tit did not write when i did not give it right.

Then two bugs:

1) It should have aborted all execution.

2) The read back verification obviously is broken.

Source RPM: isodumper-1.48-1.mga9.src.rpm => isodumper-1.50-1.mga9.src.rpm

Comment 7 papoteur 2023-07-02 15:51:03 CEST
(In reply to Morgan Leijström from comment #6)
> (In reply to papoteur from comment #5)
> User set correctly, but
> * It set Wrong group! *
> 
> It set same group ID as my user ID.  That may be correct for some users with
> private group, but I use a different group.
UID and GID are those that the application knows and there are set to the file. I don't know how this can't be correct. 
> 
> > The latest is done by passing uid/gid from GUI application to apply them to
> > the file at end of writing.
UID and GID are those that the application knows and there are set to the file. I don't know how this can't be correct. 
> > The 10 minutes are hardcoded. I think it is a good duration.
> 
> Backup itself took about 6 minutes on this my main machine, USB2, a three
> year old cheap 16MB stick.  I also have a 128MB stick, will probably not
> work...
Which duration do you suggest ?

> There is very little loggin into that file.
> I only got one row per run, correctly saying a GPG file was not found 
> 
> Seems wrong.
No, launch isodumper with -d (--debug) option to have more information
> 
> ================================================
> 
> (In reply to Morgan Leijström from comment #3)
> > 
> >  /var/log/magiback.log seem to not note making backup
> 
> It would be great to add that so all main phases are seen in the log
It uses the same function for backup and backup. Thus messages are the same. I will add a preamble for backup mode.
> 
> ================================================
> 
> The text block is not translated (in Swedish at least):
> 
>  The selected operations will be executed in order from top to bottom.
>  If both write image and create partition are selected, the partition
>  will be created in the free space after the image.
Same for me, I have to check more deeply
> Major problems!
> 
> I set it to write an ISO and create ext4, at the popup for rights i cancel.
> It then immediately and wrong in log say it wrote the image, tells many
> bytes written, and performs verify
> Even more wrong is that the popup then tell it suceeded, but i know it did
> not write and i afterwards also verify old content is there, and no extra
> partiton made.
> 
> What it did right was tha tit did not write when i did not give it right.
> 
> Then two bugs:
> 
> 1) It should have aborted all execution.
> 
> 2) The read back verification obviously is broken.

I found a fix.
Comment 8 Morgan Leijström 2023-07-02 18:01:10 CEST
(In reply to papoteur from comment #7)
> (In reply to Morgan Leijström from comment #6)
> > (In reply to papoteur from comment #5)
> > User set correctly, but
> > * It set Wrong group! *
> > 
> > It set same group ID as my user ID.  That may be correct for some users with
> > private group, but I use a different group.
> UID and GID are those that the application knows and there are set to the
> file. I don't know how this can't be correct. 
> > 
> > > The latest is done by passing uid/gid from GUI application to apply them to
> > > the file at end of writing.
> UID and GID are those that the application knows and there are set to the
> file. I don't know how this can't be correct. 

All other applications set correctly my primary group.

Please try yourself running as a user with different user id number than primary group.

================================================

> > > The 10 minutes are hardcoded. I think it is a good duration.
> > 
> > Backup itself took about 6 minutes on this my main machine, USB2, a three
> > year old cheap 16MB stick.  I also have a 128MB stick, will probably not
> > work...
> Which duration do you suggest ?

Is it possible to make it valid for the duration of the run?
i.e set 120 minutes if there need to be a time out, then invalidate it when job is completed?

Maybe someone want to make a backup of 1TB external SSD disk and make a live and have old USB2 computer like me...

================================================

> launch isodumper with -d (--debug) option to have more information

 -d option is not recognised.

 --debug added to Wiki.


OK now there is much debug in ~/.isodumper/isodumper.log

Weird: One line really stand out as i was writing mga9 live ISO and it list details of qelectrotech org  how is that related?!  (numbers obfuscated)

2023-07-02 17:11:38,602 DEBUG    line: 'uid:-::::16452365668::DD3134567387585782346235634500::Laurent Trinques (qelectrotech.org.gpg) <scorpio@qelectrotech.org>::::::::::0:'


================================================

I find it strange that some steps like formatting is not logged in /var/log/magiback.log

IMO, each main step as selected bu the checkbox should have *something*, as well as start and finish of whole run.
Comment 9 Morgan Leijström 2023-07-02 18:13:24 CEST
BTW,
https://wiki.mageia.org/en/IsoDumper_Writing_ISO_images_on_USB_sticks#Debugging
state that all outputs from last session is in isodumper.log
(and have said so since it was created)

I see that in Mageia 9 all sessions outputs are appended.
Is this a bug or should i edit wiki?
Do the version in Mageia 8 behave the same?
Comment 10 Morgan Leijström 2023-07-05 00:30:57 CEST
I note version 1.51 in testing so took it for a quick run.
Ready to test? - What fixes to look for?

---

Another quirk noted:   Log both in /var/log/magiback.log and in GUI say it create a persistent partition, even when it is not for Live persistence.

- It should only be called persistent when it is labeled "mgalive-persist"!
Comment 11 papoteur 2023-07-05 08:45:46 CEST
Hi Morgan,
The modifications are:
IsoDumper 1.51
--------------
          - Close magiback when closing isodumper
          - Keep the authorization one hour
          - Add the -d option for debug in addition of --debug
          - Log only in /var/log/magiback
Comment 12 Morgan Leijström 2023-07-07 09:43:44 CEST
Updated wiki regarding -d and not logging in ~/.isodumper/isodumper.log
Comment 13 Morgan Leijström 2023-07-07 10:17:15 CEST
Problems remaining v1.51
  non translated text, comment 6
  backup file group, comment 8
  Logging can be improved, i.e comment 10

Summary: Isodumper after making backup forgets image to write => Isodumper various issues

Comment 14 papoteur 2023-07-11 18:04:00 CEST
I prepare an update for comment 6 and comment 10. Comment 8 was already managed with 1.50.
I will wait one or two days, as translations have to be updated.
Comment 15 Morgan Leijström 2023-07-11 23:23:25 CEST
In 1.51 still group is set wrong per comment 8.
Have you tried yourself with user id <> primary group?
Comment 16 papoteur 2023-07-12 11:56:35 CEST
(In reply to Morgan Leijström from comment #15)
> In 1.51 still group is set wrong per comment 8.
> Have you tried yourself with user id <> primary group?

No, I don't such a setting.
What do you get in a Python console with:
import os
os.getuid(), os.getgid()

On my side, I get:
(1000, 1000)

These are used to set the ownership of the backup file.
And the command in bash console:
id

Here:
uid=1000(yves) gid=1000(yves) groupes=1000(yves),7(lp),81(audio),...
Comment 17 Morgan Leijström 2023-07-12 19:23:38 CEST
[morgan@svarten ~]$ python3
Python 3.10.11 (main, Apr 16 2023, 03:21:15) [GCC 12.2.1 20230415] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getuid(), os.getgid()
(10702, 10790)

And they are correct.

But somehow isodumper set both user and group to 10702 on that backup file.
Repeated just now:

$ ls -la test*
-rw-r--r-- 1 morgan 10702 8032092160 jul 12 19:22 testbak.img
Comment 18 papoteur 2023-07-13 09:51:52 CEST
Hi Morgan,
I found the problem with group id and fixed it.
1.52 is now in testing.
Translation for your language is not yet updated, you won't see the fix for comment 6.
Comment 19 Morgan Leijström 2023-07-13 15:25:38 CEST
1.52 still set same wrong group here.

Please create a new user, i.e uid=1010 and make a group i.e 1111 and set that as users primary group, log in and test...

Another thing to test: 
I set it to backup, write ISO, and create a FAT32 label "files"
-> that partition do not get created, and no sign in log.

If I set it to only make partition, it works.
Comment 20 papoteur 2023-07-13 17:02:38 CEST
(In reply to Morgan Leijström from comment #19)
> 1.52 still set same wrong group here.
> 
> Please create a new user, i.e uid=1010 and make a group i.e 1111 and set
> that as users primary group, log in and test...
I have done the test:
isodumper --version
1.52
[test@localhost ~]$ ll /documents/aeffacer.img 
-rw-r--r-- 1 test isotest 16106127360 juil. 13 16:48 /documents/aeffacer.img

This is OK in this side.
What could happen for you is that the service magiback wasn't stopped from the previous run.
Comment 21 Morgan Leijström 2023-07-13 19:16:29 CEST
Ah yes.
Closed magiback, and now in new run it sets correct group :)

I see again that magiback is running even after i close isodumper.
Why is not magiback closed?

---

Still the problem that it forgets to make a partition in a combined run;

It works if i select persistent partition, but if i tell it to backup, write an ISO , make a FAT32 and label it fat32usb, then there is no partition made and no note of it in the log except that it shows i have chosen FAT32.
Stephen Germany 2023-07-27 17:26:31 CEST

CC: (none) => stephengermany

Comment 22 Morgan Leijström 2023-07-27 19:11:51 CEST
Meanwhile, 1.52 should get moved to release as 1.48 in release have severe problem

https://ml.mageia.org/l/arc/dev/2023-07/msg00197.html

Then continue fixes from comment 21

Severity: normal => major
Source RPM: isodumper-1.50-1.mga9.src.rpm => isodumper-1.52-1.mga9.src.rpm
Priority: Normal => High

Comment 23 papoteur 2023-08-03 21:17:03 CEST
There is now version 1.53 in testing.
This release allows to create additional partition of any type after writing the ISO.
It fixes also a bug not allowing the magiback closing.
Comment 24 papoteur 2023-08-06 15:03:18 CEST
And now 1.54 with latest translations.
Comment 25 David GEIGER 2023-08-10 06:40:57 CEST
latest 1.55 fixes some missing translations in the package due to some typo in some translations.

Assigning to QA,

Packages in 9/Core/Updates_testing:
======================
isodumper-gtk-1.55-1.mga9.noarch.rpm
isodumper-qt-1.55-1.mga9.noarch.rpm
isodumper-1.55-1.mga9.noarch.rpm

From SRPMS:
isodumper-1.55-1.mga9.src.rpm

Please test and validated it, thanks!

Assignee: yvesbrungard => qa-bugs
CC: (none) => geiger.david68210

Comment 26 Morgan Leijström 2023-08-10 16:06:25 CEST
Works for me:

§ Full run OK:
Backup file OK, mga9 Live OK, encrypted persistence OK.
  - and the resulting Live system works

§ Partial run OK:
an ISO without signature OK, unencrypted FAT32 OK

§ Swedish translation.


--------------------------------------

To polish later:


§ Non translated text in window:
" The selected operations will be executed in order from top to bottom.
 If both write image and create partition are selected, the partition
 will be created in the free space after the image. "


§ Decide if it should output progress or not... At prompt from where it started it write what partition type is chosen. No more no less.  Either do not do that, like it do not tell anything else, - or output all choices and run progress. 


§ Internal naming of partitioning step: I see in log it talk about persistence even when the label is not "mgalive-persist".
IMO, it should generically say "partition", "partitioning".

2023-08-10 14:56:00,393 DEBUG    Start doing persistence partition
2023-08-10 14:56:00,393 INFO     Persistence thread started
2023-08-10 14:56:01,093 DEBUG    New partition created
2023-08-10 14:56:01,093 INFO     Executing mkdosfs -F 32 -n F32 /dev/sdc3
2023-08-10 14:56:04,613 INFO     Persistent partition done
2023-08-10 14:56:05,119 DEBUG    Finished
2023-08-10 14:56:05,123 WARNING  Added partition

CC: (none) => sysadmin-bugs
Keywords: (none) => validated_update

Comment 27 Thomas Backlund 2023-08-19 21:42:35 CEST
moved to release

Status: NEW => RESOLVED
Resolution: (none) => FIXED


Note You need to log in before you can comment on or make changes to this bug.