[BlueOnyx:20897] Re: PKG Autoinstall defect - Summary

[Felix Kaegi]

Hi Michael

After running the script I still have the following PKG that I'm not sure if
I can remove them, or if they are somehow system relevant. I don't recall
installing them.

atsphp
contrexx
dalbum
eramba
opencart
owncloud
phpservermon
wordpress-japanese	I don't need it in Japanese
wordpress-spanish	I don't need it in Spanish

Best regards
Felix


-----Original Message-----
From: Blueonyx [mailto:blueonyx-bounces at mail.blueonyx.it] On Behalf Of
Michael Stauber
Sent: Tuesday, April 11, 2017 01:20
To: blueonyx at mail.blueonyx.it
Subject: [BlueOnyx:20894] PKG Autoinstall defect - Summary

Hi all,

As you have seen today: We did something incredibly stupid and I sincerely
apologize to all of you.

If you are still affected by this, here is the quick summary for how to fix
it:

============================================================
1.) As "root" from SSH run:

yum clean all
yum update

That makes sure you're fully update and have the fix.

2.) Removal of unwanted PKG's:

cd ~
wget http://devel.blueonyx.it/pub/BlueOnyx/.scripts/pkgRemoval.pl.txt
mv pkgRemoval.pl.txt pkgRemoval.pl
chmod 755 pkgRemoval.pl

Then edit pkgRemoval.pl and in the list of unwanted PKGs delete all lines of
PKGs that you want to keep. Once satisfied with this list, run this command:

./pkgRemoval.pl
============================================================

What caused the problem? A stupid beginners mistake in coding. In a for/next
loop a variable was conditionally re-used without resetting it between runs.

If you then happened to have the "Active Monitor" component "Software
Updates" enabled, then the daily cronjob would poll the list of PKGs
available to you on NewLinQ to keep you informed of PKGs that are installed
and for which updates are available.

However: If the faulty code loop tripped over itself while updating CODB
with the new data from NewLinQ, then it would start to mark all PKGs for
install. Depending on when it tripped, it might mark all PKGs, in other
cases it just marked everything past a certain point.

The next run of "Active Monitor" then installed all PKGs that had been
marked for install.

During testing of the code changes on Friday prior to release everything
seemed to be fine and the problem didn't occur. On Saturday I spotted it on
a client box under maintenance contract. However, in that case just two
"unwanted" PKGs had been installed and that client also had the "All
Packages Bundle" of which he used only half a dozen elements.

This had allowed me to immediately identify the problem and to publish a
fix.

But this is actually where I had screwed up again: I made the assumption
that the problem was minor and that it perhaps might not affect anyone else
in even more drastic ways. But it did due to the delayed nature with which
the "Active Monitor" component "Software Updates" works.

Among the first casualties of the problem were the support ticket relay
system, the list servers, the YUM repository MySQL server backend and the
Solarspeed email server. They all had gotten "Dfix" installed, which
collided with the already present "Dfix2". Two MariaDB servers also didn't
survive the unmonitored updates to MariaDB-10.1, as some of them were also
still running with passwords in the old format.

So I only became aware of the problem when the phone started to ring itself
off the nightstand.

I spent the last 10 hours fixing our own broken stuff while simultaneously
fixing the boxes of everyone that contacted me in any way. If you need help
with the fix or still have pending issues on your servers, then please let
me know and I'll get to it ASAP.

--
With best regards

Michael Stauber
_______________________________________________
Blueonyx mailing list
Blueonyx at mail.blueonyx.it
http://mail.blueonyx.it/mailman/listinfo/blueonyx