[BlueOnyx:24560] Re: Yum update error 5209R - solved

[Storer, Darren]

Hi Michael,

Thanks for the summary - you haven't had much fun this weekend - sounds
awful! :-(

Everything is okay now - thanks again for everything you continue to do for
the project.

MfG

Darren




On Mon, 7 Dec 2020 at 08:13, Michael Stauber <mstauber at blueonyx.it> wrote:

> Hi Darren,
>
> > The repo problem seems to be back again - is there a workaround that we
> > can use during outages?
> >
> > Could not retrieve mirrorlist
> > http://updates.blueonyx.it/index.php?release=7&arch=x86_64 error was
> >
> > 14: curl#56 - "Callback aborted"
>
> That should be working again by now.
>
> We had a little hickup with three of our four virtualization nodes
> restarting. This took down ~30 VPS's including primary and secondary DNS
> and two mirrors. A third mirror and the tertiary DNS is in another
> location. And it took down the list, most of the blueonyx.it webpages
> and a couple of other odds and sods as well.
>
> After the nodes had rebooted and all VPS's that were supposed to be
> running were running. So everything seemed to be fine again.
>
> But that wasn't really the case.
>
> Within the hour all DNS records dropped due to short TTL and the primary
> and secondary DNS no longer being connected to the internet. The
> tertiary DNS isn't really redundant, so it couldn't take over either.
>
> Why did all VPS's drop off the net despite them running? During a
> Container restart on OpenVZ 7 the CT's network disposition is
> ARP-broadcasted and it looks like the switch lost these sometime after
> the VPS's initially had come back. So any IP packets destined for the
> VPS's ended up in nirvana as the last hop wasn't routed to where it
> should go. Restarting all VPS's *again* and triggering new ARP seeding
> solved that.
>
> I'm in the process of setting up new mirrors anyway (they need a more up
> to date base OS than they currently have) and while I do so I'll make
> sure that our tertiary DNS can work independently to prevent complete
> mirror outages such as this in the future.
>
> --
> With best regards
>
> Michael Stauber
> _______________________________________________
> Blueonyx mailing list
> Blueonyx at mail.blueonyx.it
> http://mail.blueonyx.it/mailman/listinfo/blueonyx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.blueonyx.it/pipermail/blueonyx/attachments/20201207/918e8b8d/attachment.html>