[BlueOnyx:03700] Re: Blueonyx Digest, Vol 14, Issue 43
Chris Dowland
chris at aleadingedge.com
Fri Feb 26 12:30:36 -05 2010
Sent from my HTC Tilt™ 2, a Windows® phone from AT&T
-----Original Message-----
From: blueonyx-request at blueonyx.it
Sent: Friday, February 26, 2010 7:05 AM
To: blueonyx at blueonyx.it
Subject: Blueonyx Digest, Vol 14, Issue 43
Send Blueonyx mailing list submissions to
blueonyx at blueonyx.it
To subscribe or unsubscribe via the World Wide Web, visit
http://www.blueonyx.it/mailman/listinfo/blueonyx
or, via email, send a message with subject or body 'help' to
blueonyx-request at blueonyx.it
You can reach the person managing the list at
blueonyx-owner at blueonyx.it
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Blueonyx digest..."
Today's Topics:
1. [BlueOnyx:03691] Re: Unable to create new sites (Steve Howes)
2. [BlueOnyx:03692] Re: Unable to create new sites (David Booth)
3. [BlueOnyx:03693] Re: Unable to create new sites (Steve Howes)
4. [BlueOnyx:03694] Re: Unable to create new sites (Steve Howes)
5. [BlueOnyx:03695] Re: ftp service status issue (Bret Hughes)
6. [BlueOnyx:03696] Re: Unable to create new sites
(Christoph Schneeberger)
7. [BlueOnyx:03697] Re: Unable to create new sites
(Christoph Schneeberger)
8. [BlueOnyx:03698] Re: Unable to create new sites (Steve Howes)
----------------------------------------------------------------------
Message: 1
Date: Thu, 25 Feb 2010 22:42:06 +0000
From: Steve Howes <steve-lists at geekinter.net>
Subject: [BlueOnyx:03691] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <6BA551E7-9B8B-4F80-8EAE-7DA8C96B5C99 at geekinter.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
On 25 Feb 2010, at 20:26, David Booth wrote:
> I used a script (with a few false starts) for a while but found
> codb.oids got overwritten anyway after a few days - often incorrectly.
> I haven't had any ill effects from just using X. After I do it and
> carry on with creating my new site/user/whatever I see it increment to
> X+1.
> I even did a few tests - delete an object and it decrements to X-1.
> Come back in a few hours and I see it all rebuilt with 1-A,B-C, etc -
> sometimes right, sometimes wrong.
> I have had paid help on this and it has escalated somewhere for a 'big
> fix of the underlying code one day'
It is normal to see gaps. Gaps are not 'broken'. If an underlying fix
is needed it'll be done. Thats why we exist! The BlueOnyx team is
there to fix and improve stuff.
> Seems to me there's a universal unknown about missing ID numbers.
> Nobody knows if it causes problems. Who would create a file based
> object indexing engine that depends on an index being absent for
> consistency? Are we going to run out of numbers one day?
It does cause problems. It does not depend on an index being absent,
it depends on the index being *accurate*. If you have a book index
referring to a page you've torn out it isn't much use. Much as if you
added a page and didn't put it in the index. If an ID is not in the
index it is 'free for use'. If there is an object with that ID that is
not in the index you then end up with the object being overwritten/the
new object cant be made.
It is like having a car park full of invisible cars. If you have an
inaccurate record of who is parked where all hell is going to break
loose when you park inside another car. Rest assured, we are not going
to run out of numbers though. You can get big numbers in maths. I hear
some even have *five* figures! ;)
> There are gaps in the numbers anyway - and no-one knows why.
I think you mis-understand. See above, its because you create objects,
then delete one from 'the middle'. You can't re-number the entire
database you you use the indexes to keep track of what ID numbers are
free for future use. It's the equivalent of filesystem fragmentation.
You get 'gaps'. It is not a problem in the case of this DB because
objects don't have to 'fit' a gap.
> Right now my codb.oids says
>
> 1-2014,2012-2013
>
> which is ridiculous. The second interval is contained in the first.
Agreed. Probably where these ranges have been changed to inaccurately
reflect the ID numbers in use (i.e. no gap listed where there is one).
This format may well not be parsable by the codb engine, or will parse
in an unexpected manner meaning the problem comes back.
> and the last integer (maximum) of
>
> ls objects | sort -n
>
> is 9084
>
> So if I need to add a site/user/dns today I'll have to fix it first.
> Whether I run a script or just make it 1-9084 doesn't make a
> difference.
Then there is something broken that needs debugging collecting to work
out what is wrong. The 'big underlying fix' clearly needs doing if
there is a problem, but if we're re-writing large chunks of stuff then
we need to know why the current implementation is not up to the job.
If we don't we're going to end up with the same thing again. The
problem needs understanding before it can be fixed.
In summary:
1. We need examples to track down the problem
2. Gaps are normal! They are *NOT* the problem
3. A 'missing' number *IS* a problem if the ID is actually in use
4. A 'non gap' *IS* a problem if the object ID is not in use (what if
something uses that index to iterate through objects?)
S
------------------------------
Message: 2
Date: Fri, 26 Feb 2010 10:36:30 +1100
From: David Booth <md at goulburn.net.au>
Subject: [BlueOnyx:03692] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <630854397f3dc044c75c15ddd200958c at goulburn.net.au>
Content-Type: text/plain; charset=US-ASCII; format=flowed
> <snip>
Roger to all that. Thanks.
> There are gaps in the numbers anyway - and no-one knows why.
>
> I think you mis-understand. See above, its because you create objects,
> then delete one from 'the middle'. You can't re-number the entire
> database you you use the indexes to keep track of what ID numbers are
> free for future use. It's the equivalent of filesystem fragmentation.
> You get 'gaps'. It is not a problem in the case of this DB because
> objects don't have to 'fit' a gap.
It's not that I mis-understand. As you say, it's not a problem for this
DB unless:
> <snip>
> 4. A 'non gap' *IS* a problem if the object ID is not in use (what if
> something uses that index to iterate through objects?)
What something might that be? Why would it be? And if it must be, why
would it not check in ~/objects
These are questions this mortal can only wonder about. To me, the
cobalt engine is ineffable.
------------------------------
Message: 3
Date: Thu, 25 Feb 2010 23:53:29 +0000
From: Steve Howes <steve-lists at geekinter.net>
Subject: [BlueOnyx:03693] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <78267AB1-FF4B-4611-86D6-1DD6AE17C24F at geekinter.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
On 25 Feb 2010, at 23:36, David Booth wrote:
> These are questions this mortal can only wonder about. To me, the
> cobalt engine is ineffable.
The Cobalt database is very simple, and remarkable complex at the same
time. Replacing it would be a mammoth task. There are thousands of
BlueOnyx servers out there with data already in the database,
'upgrading' it is not a simple task. If we can gather evidence as to
what specific events cause the index to get out of sync. If we can
work that out, and prove it is a bug and not disk corruption/crashes
etc then we can probably fix the current database (if indeed it is at
fault). When it is working its a perfectly viable system.
Unfortunately getting specific data rather than people saying 'i got
that a few times' is rather tricky. The plural of 'anecdote' is not
'data'. If we can get the exact log entries for when it *breaks*,
rather than an error caused by it being *broken* then that gives us a
damned good base to start looking at. If anyone gets that a related
error message, by all means send me your entire /var/log/messages back
for a week and I'll read it line by line if i have to!
S
------------------------------
Message: 4
Date: Fri, 26 Feb 2010 00:20:27 +0000
From: Steve Howes <steve-lists at geekinter.net>
Subject: [BlueOnyx:03694] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <8D6A35F3-4183-48FF-B7A1-229397CEA26C at geekinter.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
On 25 Feb 2010, at 20:26, David Booth wrote:
> Are we going to run out of numbers one day?
Hehe, sorry to go off-topic but that reminded me of a funny spoof
science show I saw once..
http://www.youtube.com/watch?v=Pj2NOTanzWI#t=1m17s
Made me chuckle..
S
------------------------------
Message: 5
Date: Thu, 25 Feb 2010 20:29:06 -0600
From: Bret Hughes <bhughes at elevating.com>
Subject: [BlueOnyx:03695] Re: ftp service status issue
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <4B8731F2.4090807 at elevating.com>
Content-Type: text/plain; charset="iso-8859-1"
webmaster wrote:
> Had this problem a while back on my 550.
>
> Turned out to be DNS issues.
>
>
>
>
>
>> I have a BX server that shows the ftp service is not running from
>> the status page of the UI. Investigation shows that proftpd is run
>>
> >from xinetd and while it seems to take a while to respond it does
>
>> allow me to connect. The client has just made some firewall changes
>> and is convinced that the reason they cannot connect from the
>> internal network is because ftp is not running.
>>
>> Any ideas why the status shows that ftp is not running and what I
>> can do to fix it?
>>
>> Thanks,
>>
>> Bret
Yep. Outgoing DNS queries blocked by over-zealous router dude.
Turns out you can render a site dog ass slow by blocking outgoing http
traffic if they are importing news and or weather too :)
Thanks for the help.
Bret
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.blueonyx.it/pipermail/blueonyx/attachments/20100225/231cf70a/attachment-0001.html
------------------------------
Message: 6
Date: Fri, 26 Feb 2010 10:05:41 +0100
From: Christoph Schneeberger <cschnee at box.telemedia.ch>
Subject: [BlueOnyx:03696] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <4B878EE5.60207 at box.telemedia.ch>
Content-Type: text/plain; charset=ISO-8859-1
Steve Howes wrote:
> On 25 Feb 2010, at 17:00, Steve Howes wrote:
>
>>> Would extracting the ranges and adding that to the oid file in the
>>> form
>>> a-b,c-d,x-y etc help eventually ?
>>>
>> I'll try and write a script to do that. last time i tried doing it by
>> hand i went a bit crazy.
>>
>
> Hi,
>
> I've made a script to do this. It's not my nicest bit of code by far,
> but it should work! To 'cover my arse' I will say the following:
>
> 1. Backup your /usr/sausalito/codb/ folder in it's entirety
> 2. Back up what you can with raqbackup/cmuexport
> 3. If its a VM and you can copy the entire disk, do that to
> 4. This script is untested in a live environment and I cannot accept
> any responsibility if it is wrong. It is supplied on a best efforts
> basis
> 5. Once you've run the script and copied over the file, try and do
> something simple like rename a person, change an alias. If that
> doesn't work, copy the old codb.oids back and let me know.
>
> To install do this:
>
> cd ~
> wget http://www.geekinter.net/factoids.txt -Ofactoids.pl
> chmod 700 factoids.pl
> ./factoids.pl
>
> This will give you a file 'myoids' in your home directory which
> *should* be an accurate list, but please have a quick look in it with
> cat to make sure it is sensible.
>
> By putting this in place of /usr/sausalito/codb/codb.oids it will mean
> the CODB does not try and overwrite objects that exist. BUT - and its
> a big BUT, if an object has been created successfully (one which the
> GUI thinks wasn't) then it could end up trying to create a duplicate
> next (i.e. you will get 'address in use' or 'site already exists' sort
> of errors). There could be further 'tidying' left to do. We'll have to
> cross this if/when we come to it. Doing a tail-f /var/log/messages
> whilst doing the tests would be good, if anything bad happens post the
> relevant sections to the list.
>
> Hope that all made sense, it's been a long day ;)
>
>
Thanks a lot, worked out pretty fine.
The cmuExport went through without any problems, just took half the
night to create a 50GB dump. I then also created a vzdump to be really
on the safe side.
The script created a perfectly valid codb.oids.
So I restarted cced, added an alias: worked.
I then went ahead and created www.xytest.ch: worked.
I then again deleted the site www.xytext.ch: worked.
Recreated another test site: worked again.
I have created a vip account for you on our coffeemachine, you can login
whenever you want and download free coffee without any limits ;-)
Cheers and thanks,
Christoph
------------------------------
Message: 7
Date: Fri, 26 Feb 2010 10:13:52 +0100
From: Christoph Schneeberger <cschnee at box.telemedia.ch>
Subject: [BlueOnyx:03697] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <4B8790D0.4010403 at box.telemedia.ch>
Content-Type: text/plain; charset=ISO-8859-1
Steve Howes wrote:
> On 25 Feb 2010, at 23:36, David Booth wrote:
>
>> These are questions this mortal can only wonder about. To me, the
>> cobalt engine is ineffable.
>>
>
> The Cobalt database is very simple, and remarkable complex at the same
> time. Replacing it would be a mammoth task. There are thousands of
> BlueOnyx servers out there with data already in the database,
> 'upgrading' it is not a simple task. If we can gather evidence as to
> what specific events cause the index to get out of sync. If we can
> work that out, and prove it is a bug and not disk corruption/crashes
> etc then we can probably fix the current database (if indeed it is at
> fault). When it is working its a perfectly viable system.
> Unfortunately getting specific data rather than people saying 'i got
> that a few times' is rather tricky. The plural of 'anecdote' is not
> 'data'. If we can get the exact log entries for when it *breaks*,
> rather than an error caused by it being *broken* then that gives us a
> damned good base to start looking at. If anyone gets that a related
> error message, by all means send me your entire /var/log/messages back
> for a week and I'll read it line by line if i have to!
>
>
For the sake of completeness: The BlueOnyx was created from the vz
template sometime in Q3/2009 and then I imported all sites that are
currently on it from a strongbolt installation running on a RaQ4. Since
then nothing probably has changed, so I assume it messed up during the
imports of the existing sites. Unfortunately this is pretty long ago and
I have no backup showing the exact messages when it messed up. Should it
happen again, I'll happily supply you with all information needed.
So far I haven't seen this problem on any of the other BlueOnyx servers
we run (quite a few).
Cheers,
Christoph
------------------------------
Message: 8
Date: Fri, 26 Feb 2010 13:04:42 +0000
From: Steve Howes <steve-lists at geekinter.net>
Subject: [BlueOnyx:03698] Re: Unable to create new sites
To: BlueOnyx General Mailing List <blueonyx at blueonyx.it>
Message-ID: <1028E4CF-5F20-4B30-A854-57C60E1E0B47 at geekinter.net>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
On 26 Feb 2010, at 09:05, Christoph Schneeberger wrote:
> The script created a perfectly valid codb.oids.
> So I restarted cced, added an alias: worked.
> I then went ahead and created www.xytest.ch: worked.
> I then again deleted the site www.xytext.ch: worked.
> Recreated another test site: worked again.
>
> I have created a vip account for you on our coffeemachine, you can
> login
> whenever you want and download free coffee without any limits ;-)
Really glad that worked. I'll add it to my toolkit of 'handy scripts'
for future use. If you get any ill effects further down the line let
me know, but I'm hoping it was something that isn't likely to happen
again.
> For the sake of completeness: The BlueOnyx was created from the vz
> template sometime in Q3/2009 and then I imported all sites that are
> currently on it from a strongbolt installation running on a RaQ4.
> Since
> then nothing probably has changed, so I assume it messed up during the
> imports of the existing sites. Unfortunately this is pretty long ago
> and
> I have no backup showing the exact messages when it messed up.
> Should it
> happen again, I'll happily supply you with all information needed.
> So far I haven't seen this problem on any of the other BlueOnyx
> servers
> we run (quite a few).
That is re-assuring to know. The only other time I've had to fix this
fault was on an old RaQ 550. I'll keep an ear out for anyone else
getting this and do any fixes that are needed.
Hopefully it'll all be ok now. Have a good weekend, and if I am ever
nearby I'll come and have some of that coffee ;)
Cheers.
S
------------------------------
_______________________________________________
Blueonyx mailing list
Blueonyx at blueonyx.it
http://www.blueonyx.it/mailman/listinfo/blueonyx
End of Blueonyx Digest, Vol 14, Issue 43
****************************************
More information about the Blueonyx
mailing list