I manage multiple Almalinux 8 VPS and dedicated servers, and recently I have been experiencing a lot of RPM Database corruption specifically this one:
error: rpmdb: BDB0113 Thread/process 401048/139672503925632 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 - (-30973)
error: cannot open Packages database in /var/lib/rpm
Error: Error: rpmdb open failed
And yes, I have run this MANY times:
mkdir -p /tmp/rpm-backup
cp -vf /var/lib/rpm/__db.* /tmp/rpm-backup
rm -vf /var/lib/rpm/__db.*
rpm --quiet -qa
rpm --rebuilddb
dnf clean all
This does solve the issue, but temporarily, and I do mean temporarily. I have one specific instance, as soon as I run dnf update -y after this, it downloads packages and tries to update it fine and as it installs, it fails with rpmdb error.
This issue is also concerning why I experienced issues like this: https://forums.almalinux.org/t/unable-to-detect-release-version/4070
I do not know what to do, there’s an underlying issue. Servers were rebooted multiple times and I do not have the experience or expertise to find out why this keeps happening and the main issue, it seems to happen with many instances I have, and not just one.
And yes, I tried following this: https://access.redhat.com/solutions/3330211 it's so bad that as I try to install these packages, the database goes corrupt. I can't install these packages without running into the issue constantly.
I would appreciate any assistance in specifically how to debug this exactly and provide you guys more details for any sort of assistance.
