[BlueOnyx:03691] Re: Unable to create new sites

[Steve Howes]

On 25 Feb 2010, at 20:26, David Booth wrote:
> I used a script (with a few false starts) for a while but found
> codb.oids got overwritten anyway after a few days - often incorrectly.
> I haven't had any ill effects from just using X. After I do it and
> carry on with creating my new site/user/whatever I see it increment to
> X+1.
> I even did a few tests - delete an object and it decrements to X-1.
> Come back in a few hours and I see it all rebuilt with 1-A,B-C, etc -
> sometimes right, sometimes wrong.
> I have had paid help on this and it has escalated somewhere for a 'big
> fix of the underlying code one day'

It is normal to see gaps. Gaps are not 'broken'. If an underlying fix  
is needed it'll be done. Thats why we exist! The BlueOnyx team is  
there to fix and improve stuff.

> Seems to me there's a universal unknown about missing ID numbers.
> Nobody knows if it causes problems. Who would create a file based
> object indexing engine that depends on an index being absent for
> consistency? Are we going to run out of numbers one day?

It does cause problems. It does not depend on an index being absent,  
it depends on the index being *accurate*. If you have a book index  
referring to a page you've torn out it isn't much use. Much as if you  
added a page and didn't put it in the index. If an ID is not in the  
index it is 'free for use'. If there is an object with that ID that is  
not in the index you then end up with the object being overwritten/the  
new object cant be made.

It is like having a car park full of invisible cars. If you have an  
inaccurate record of who is parked where all hell is going to break  
loose when you park inside another car. Rest assured, we are not going  
to run out of numbers though. You can get big numbers in maths. I hear  
some even have *five* figures! ;)

> There are gaps in the numbers anyway - and no-one knows why.

I think you mis-understand. See above, its because you create objects,  
then delete one from 'the middle'. You can't re-number the entire  
database you you use the indexes to keep track of what ID numbers are  
free for future use. It's the equivalent of filesystem fragmentation.  
You get 'gaps'. It is not a problem in the case of this DB because  
objects don't have to 'fit' a gap.

> Right now my codb.oids says
>
> 1-2014,2012-2013
>
> which is ridiculous. The second interval is contained in the first.

Agreed. Probably where these ranges have been changed to inaccurately  
reflect the ID numbers in use (i.e. no gap listed where there is one).  
This format may well not be parsable by the codb engine, or will parse  
in an unexpected manner meaning the problem comes back.

> and the last integer (maximum) of
>
> ls objects | sort -n
>
> is 9084
>
> So if I need to add a site/user/dns today I'll have to fix it first.
> Whether I run a script or just make it 1-9084 doesn't make a
> difference.

Then there is something broken that needs debugging collecting to work  
out what is wrong. The 'big underlying fix' clearly needs doing if  
there is a problem, but if we're re-writing large chunks of stuff then  
we need to know why the current implementation is not up to the job.  
If we don't we're going to end up with the same thing again. The  
problem needs understanding before it can be fixed.

In summary:

1. We need examples to track down the problem
2. Gaps are normal! They are *NOT* the problem
3. A 'missing' number *IS* a problem if the ID is actually in use
4. A 'non gap' *IS* a problem if the object ID is not in use (what if  
something uses that index to iterate through objects?)

S