[BlueOnyx:13005] Re: Problem: user names disappearsatJapanesecontrol page

[Michael Stauber]

Hi Eiji,

> Before I prepare the server,
> I must get the answers from the contact to DTI.
> 
> a.  Do you in trouble update of the mirror stopped
> b. What do you want to mirror site of BlueOnyx future?

I don't really care much if DTI continues to mirror us or not. If they
don't read this list or there is nobody that we can talk to in case of
mirror problems, then they are not of much real use to us.

> c.  How do you handle the modules that are delivered BlueOnyx future?

Is this a question about the BlueOnyx 5207R that's currently in
development? Or for both the "new" and the "old" BlueOnyx? In any case
it'll be pretty much the same as before. Content like that will be
published to the YUM repositories. Selected add-ons that go beyond
what's currently included in BlueOnyx might be served out of NewLinQ to
make installation easier for people that are less inclined to use the
command line or YUM.

> 1.  I'm not sure why however you delete ftp.dti.ad.jp from the mirror list

YUM for the BlueOnyx YUM repository works like this:

[root at kosh /]# cat /etc/yum.repos.d/BlueOnyx.repo

[BlueOnyx]
name=BlueOnyx - $basearch
#baseurl=http://updates.blueonyx.it/pub/BlueOnyx/5106R/CentOS6/blueonyx/$basearch/
mirrorlist=http://updates.blueonyx.it/mirror.php?release=$releasever&arch=$basearch
gpgcheck=1
enabled=1
gpgkey=http://updates.blueonyx.it/pub/BlueOnyx/RPM-GPG-KEY-NUSOL-5106R

So your YUM client will connect to the URL specified in "mirrorlist".
Which is this:

http://updates.blueonyx.it/mirror.php?release=$releasever&arch=$basearch

The variables will automatically be substituted with your OS version
(like 6.4 for CentOS or SL 6.4) and $basearch will get replaced with
"i386" or "x86_64" for either 5107R or 5108R. YUM handles that all by
itself.

Depending on your BlueOnyx version, your YUM client therefore will
access one of these URLs:

5106R:
http://updates.blueonyx.it/mirror.php?release=5.9&arch=i386

5107R:
http://updates.blueonyx.it/mirror.php?release=6.4&arch=i386

5108R:
http://updates.blueonyx.it/mirror.php?release=6.4&arch=x86_64


Depending on which mirrors are currently active, the PHP script will
return a list of URLs where the respective YUM updates can be found.
That list looks like this for 5108R:

http://updates.blueonyx.it/pub/BlueOnyx/5106R/CentOS6/blueonyx/x86_64/
http://mirror.blueonyx.de/pub/BlueOnyx/5106R/CentOS6/blueonyx/x86_64/
http://bb-one.blueonyx.it/pub/BlueOnyx/5106R/CentOS6/blueonyx/x86_64/
http://www.blueonyx.nl/pub/BlueOnyx/5106R/CentOS6/blueonyx/x86_64/
http://blueonyx.precisionweb.net/BlueOnyx/5106R/CentOS6/blueonyx/x86_64/

Your YUM client will then use the fastest server it can find to check
for and to download updates. Should it be unable to connect to the
fastest one, it will use another one from that list.

Now for the last five months the Japanese Mirror was ON the list of
mirrors. So all Japanese BlueOnyx servers connected first to
updates.blueonyx.it to poll the mirror list. They got the full list of
mirrors. That included the URL for the Japanese mirror as well. So they
connected to the fastest mirror (the Japanese one, as it was regionally
the closest). That mirror then told them: Sorry, no new updates.

Anyone who happened to be directed to any other BlueOnyx mirror could
get updates. But those who relied on the Japanese one got let down.

This is why I was so upset.

Yes, various mirrors have been removed temporarily from our list of
mirrors. For various problems. Server movements, temporary outages,
Apache crashing on those boxes, whatever. But those removals were always
temporary and eventually these servers were added back once the problems
had been solved. But that also meant that we could talk to someone in
case of problems. Or the operator of the server informed us in advance.

> 2.  You have removed the ftp.dti.ad.jp mirror almost five months before.
>      Did you thought what you try BlueOnyx machine future in Japan?  

I did not remove ftp.dti.ad.jp five months ago. They simply stopped
updating the mirror from our main mirror. So whatever data they have in
the mirror: It is from January. I realized this only when you didn't
receive the latest updates I released for the character problem. At that
point I manually compared the file listings of ftp.dti.ad.jp with the
one from the other mirrors and noticed the discrepancies.

> 6.  The site http://devel.blueonyx.it/trac/  is not telling only 
>      about 5106R and 5107R.  It's same as 5108R?

The source code for 5107R and 5108R is identical. So that code is kept
in the same SVN. The RPMs that are built for 5107R are i386 or i686
RPMs. The RPMs for 5108R are x86_64 RPMs. They are served out of
different subdirectories in the 5107R YUM repository.

> 7.  The amount of transfer month or become the gigabytes - how much?

I cannot predict that with any reliability. I don't have the traffic
figures that a Japanese mirror would get. I assume it would be a greater
than what the three US based mirrors combined have in traffic.

The actual RPMs served off the YUM repository are usually between 20KB
and 200KB and usually when we release updates there is only half a dozen
or a dozen new RPMs get released at one time. And even that doesn't
happen every day.

So usually a mirror operator will see this kind of traffic:

- Daily connections when BlueOnyx servers check if there are new
updates. In that case only the XML files of the repository will be
downloaded. These are small.

- ISO downloads: People might use the mirror to manually download
BlueOnyx ISO images, the Tarball installer or the OS templates. Those
are around 250MB - 700 MB large and consume the most traffic. However,
the number of daily downloads of ISO's fluctuates a lot. Whenever a new
ISO is released, the number of ISO downloads naturally is higher than on
the average.

- RSYNC related traffic: Whenever the mirror connects to
devel.blueonyx.it it will generate a little traffic. However, this will
be the smallest amount as only changes are copied. It doesn't copy the
whole repository every time. Instead it only pulls those files that are
new or that have changed since the last update.

Attached is a graphic of the monthly traffic statistics from one of the
three US based mirrors.

There are certain peaks that go up to 5MB/sec on days that updates are
released. The two US based mirrors I monitor have an average daily
traffic of around 20-80kb/sec and per month I *guess* the total monthly
traffic of the three US mirrors is somewhere in between 300-400GB.
Possibly Chris Gebhardt could back those figures up with some more
detailed statistics, as he is so kind as to host the US based mirrors.

I hope this answers your questions. I can see that there perhaps was
another misunderstanding and I hope my explanations solved that. No, we
did not intentionally cut off the Japanese users from updates. They
didn't get any because ftp.dti.ad.jp didn't do their job. We trusted
them to provide reliable regional mirroring services for our Japanese
friends, but they let you down and they let us down. Most certainly not
by intent or malice and it's rather negligence or a technical fault on
their end. Either way around: It is not nice that the important DNS
update didn't get to the Japanese users in time.

-- 
With best regards

Michael Stauber
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vps_111-month.png
Type: image/png
Size: 17685 bytes
Desc: not available
URL: <http://mail.blueonyx.it/pipermail/blueonyx/attachments/20130515/14a686c1/attachment.png>