Bug 29364 - rpmdb not converted on update to sqlite
Summary: rpmdb not converted on update to sqlite
Status: RESOLVED FIXED
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: Cauldron
Hardware: All Linux
Priority: release_blocker critical
Target Milestone: Mageia 9
Assignee: RPM stack maintainers
QA Contact:
URL:
Whiteboard:
Keywords: IN_RELEASENOTES9
Depends on:
Blocks:
 
Reported: 2021-08-12 12:31 CEST by Marc Krämer
Modified: 2023-07-19 11:34 CEST (History)
12 users (show)

See Also:
Source RPM: rpm.rpm
CVE:
Status comment:


Attachments

Description Marc Krämer 2021-08-12 12:31:21 CEST
On updating older cauldron to latest state shows the following error:

Found bdb_ro Packages database while attempting sqlite backend: using bdb_ro backend.

After this, install fails until rpmdb --rebuilddb is called.

I assume this has to be triggered in post install section...
Comment 1 Jani Välimaa 2021-08-12 16:35:32 CEST
Converting should be triggered automatic on next reboot.

I don't know if converting can be done during the pkg installation and between transactions.
Comment 2 Marja Van Waes 2021-08-12 17:06:37 CEST
(In reply to Jani Välimaa from comment #1)
> Converting should be triggered automatic on next reboot.
> 
> I don't know if converting can be done during the pkg installation and
> between transactions.

Maybe someone of the rpm stack maintainers can tell, assigning to them.

Assignee: bugsquad => rpmstack
CC: (none) => marja11

Comment 3 Neal Gompa 2021-08-12 17:58:07 CEST
It's not safe to do it in between transactions either. If doing online upgrade, please reboot immediately after or run "rpm --rebuilddb".

CC: (none) => ngompa13

Comment 4 Dave Hodgins 2021-08-12 20:49:11 CEST
How will upgrades from Mageia 8 to 9 be handled?

CC: (none) => davidwhodgins

Comment 5 Pascal Terjan 2021-08-12 22:33:15 CEST
My understanding is that recent urpmi do it on startup (http://gitweb.mageia.org/software/rpm/urpmi/commit/?id=c338043e44c14fc023e6386cf62c0acf97da51be) so after rpm + urpmi etc are updated in the priority upgrade, on restarting the new urpmi would do it

CC: (none) => pterjan

Comment 6 Pascal Terjan 2021-08-12 22:35:27 CEST
Ah sorry, that code is only when using a chroot.
Comment 7 Marc Krämer 2021-08-13 13:36:20 CEST
what about using the same mechanism, that restarts urpmi after initial change of basic packages.
The current situation for an update is, that after installing basic updates (rpm, perl, ...) urpmi restarts and afterwards urpmi fails to install anything due to readonly rpmdb. So, if rpmdb --rebuilddb is triggered afterwards, everything should be fine.
Comment 8 Neal Gompa 2021-08-13 20:08:53 CEST
(In reply to Dave Hodgins from comment #4)
> How will upgrades from Mageia 8 to 9 be handled?

We should adjust our documentation so that people know to do system upgrades in a single transaction for URPMI and to reboot *immediately* afterward. That's the safest way to handle an online system upgrade.

For DNF users, we already recommend the "system-upgrade" method, which transparently handles this correctly.

The issue here is Cauldron-to-Cauldron. People don't generally reboot immediately with stuff that feels like small updates, even if they're fairly major.
Comment 9 Dave Hodgins 2021-08-13 22:50:58 CEST
The "least experienced" users will upgrade using mgaapplet when it prompts them
that a new release is available. It does not use a single transaction.
Comment 10 Manuel Hiebel 2021-08-14 09:40:34 CEST
could mageia-prepare-upgrade (used in mga3 for the usr move) help ?
Comment 11 Neal Gompa 2021-08-14 16:40:02 CEST
(In reply to Dave Hodgins from comment #9)
> The "least experienced" users will upgrade using mgaapplet when it prompts
> them
> that a new release is available. It does not use a single transaction.

That sounds like something we should fix, then.
Comment 12 Dave Hodgins 2021-08-14 22:31:31 CEST
Using a single transaction requires a lot more free space. There has to be
a better way.
Comment 13 Neal Gompa 2021-08-15 15:52:30 CEST
(In reply to Dave Hodgins from comment #12)
> Using a single transaction requires a lot more free space. There has to be
> a better way.

No. A single transaction ensures everything is properly replaced in one go, which eliminates large classes of issues when doing upgrades. For system upgrades, there's a lot less value in the split transactions because the dependency web forces a lot of it to be upgraded together anyway. But because inter-application dependencies are rarely specified at the RPM level, split upgrades tend to subtlety break things.

We already tell people to download everything in advance, which forces the free space to be allocated anyway. Going to a single transaction is not worse.
Comment 14 Thierry Vignaud 2021-10-31 04:49:25 CET
(In reply to Pascal Terjan from comment #6)
> Ah sorry, that code is only when using a chroot.

That's now fixed in git, it works in all case, even when not using --(urpmi-)-root.

Status: NEW => RESOLVED
Resolution: (none) => FIXED
CC: (none) => thierry.vignaud

Comment 15 Dave Hodgins 2022-10-25 16:29:38 CEST
Reopening due to bug 31024

Status: RESOLVED => REOPENED
Resolution: FIXED => (none)

Dave Hodgins 2022-10-25 16:30:47 CEST

Priority: Normal => release_blocker

Comment 16 Dave Hodgins 2022-10-25 20:13:52 CEST
Currently rpmdb --rebuilddb --define "_db_backend sqlite" has to be run
manually during upgrades from m8 to m9.
Comment 17 Dave Hodgins 2023-01-19 02:00:24 CET
Still an issue in today's test of an upgrade using my aarch64 rpi 4b.
Comment 18 Marc Krämer 2023-04-25 12:47:38 CEST
fixed in my latest updates.
Comment 19 Dave Hodgins 2023-06-02 14:57:56 CEST
Please see attachment 13862 [details] from bug 31982

If I'm reading it correctly, the rpm db gets converted ok, but not until
it caused the first 3132 packages to be skipped leaving an upgrade hanging.

CC: (none) => tarazed25

Morgan Leijström 2023-06-02 15:01:55 CEST

Severity: normal => critical
CC: (none) => fri
Target Milestone: --- => Mageia 9

Comment 20 Filip Komar 2023-06-02 16:50:10 CEST
(In reply to Dave Hodgins from comment #19)
> Please see attachment 13862 [details] from bug 31982
> 
> If I'm reading it correctly, the rpm db gets converted ok, but not until
> it caused the first 3132 packages to be skipped leaving an upgrade hanging.

It also seems that preparing kernel symbols failed (bad exit status: 130) but I guess that it's totally unrelated. OTOH it might show another bug?

CC: (none) => filip.komar

Dave Hodgins 2023-06-02 17:39:38 CEST

CC: (none) => tmb

Comment 21 Nicolas Lécureuil 2023-06-19 01:13:15 CEST
Thierry, can you look for this blocker bug ?

CC: (none) => mageia

Comment 22 Martin Whitaker 2023-06-24 22:08:01 CEST
(In reply to Dave Hodgins from comment #19)
> Please see attachment 13862 [details] from bug 31982
> 
> If I'm reading it correctly, the rpm db gets converted ok, but not until
> it caused the first 3132 packages to be skipped leaving an upgrade hanging.

I don't think you are reading it correctly. Len chose the option to download all packages before installing. On the first pass, all priority packages, including urpmi and gurpmi, were downloaded and installed. urpmi then restarted, at which point the mga9 version of urpmi is being used. That downloaded all the files, repeatedly and unnecessarily warning about the bdb_ro database for every package downloaded, then converted the database to sqlite before starting to install packages. It then stalled, which is the subject of bug 31982, but, as far as I can see, is unrelated to this bug.

As per comment 18, I believe this bug is fixed.

CC: (none) => mageia

Comment 23 Marc Krämer 2023-06-26 12:01:43 CEST
It is fixed, had a bunch of updates, everything ok.
Comment 24 Morgan Leijström 2023-06-26 16:45:41 CEST
Soo closing.

Status: REOPENED => RESOLVED
Resolution: (none) => FIXED

Comment 25 Bit Twister 2023-07-19 07:49:20 CEST
Change media from mga8 to cauldron and did a
urpmi.removemedia -y Debug Backport Testing 32bit 
drakrpm-edit-media --expert to enable update remaing media.
urpmi  --downloader wget --wait-lock --replacefiles --auto-update --auto --download-all
and saw numerous warning: Found bdb_ro lines

     grep bdb_ro  install_updates.log | wc -l
    2325

Just one example
  http://mirror.math.princeton.edu/pub/mageia/distrib/cauldron/x86_64/media/core/release/perl-Sub-Quote-2.6.8-1.mga9.noarch.rpm
 warning: Found bdb_ro Packages database while attempting sqlite backend: using bdb_ro backend.

Posted to [qa-discuss] Ready to release the rc iso images?
and David W. Hodgins suggested I modify this bug report.

Resolution: FIXED => (none)
CC: (none) => bittwister2
Status: RESOLVED => REOPENED

Comment 26 Morgan Leijström 2023-07-19 07:59:34 CEST
BTW was this supposed to be fixed for DNF too?
problem: https://forums.mageia.org/en/viewtopic.php?t=14988
Comment 27 Martin Whitaker 2023-07-19 10:39:27 CEST
See comment 22. If you use --download-all, you will get lots of unnecessary warnings during the download phase, but the database is converted before the packages are installed.

Status: REOPENED => RESOLVED
Resolution: (none) => FIXED

Comment 28 Morgan Leijström 2023-07-19 11:34:23 CEST
Ah good.

And DNF is good, using our method in release notes; Answering myself:

(Morgan Leijström from comment #26)
> BTW was this supposed to be fixed for DNF too?
> problem: https://forums.mageia.org/en/viewtopic.php?t=14988

(Neal Gompa from comment #8)
> For DNF users, we already recommend the "system-upgrade" method, which
> transparently handles this correctly.

Keywords: (none) => IN_RELEASENOTES9


Note You need to log in before you can comment on or make changes to this bug.