[BlueOnyx:18953] Re: 5208R login problem - fixed. YUM updates available

[Michael Stauber]

Hi Carl,

> Something seems wrong with cced.init. 
> Message: Undefined index: isLicenseAccepted

Yes. The field "isLicenseAccepted" is part of the CODB Object "System".
Your CCEd wasn't able to find the "System" Object and that caused this
error.

I suspect this problem is related to a memcache corruption. Memcache was
recently introduced to speed up CCEd and Hisao Shibuya coded it for us.
It roughly speeds up the usual CCEd communication between GUI and CODB
by 50%.

The thing here is as follows: CCEd uses an object oriented database,
which we call CODB. Short for "Cobalt Object Database". Everything
within is an Object with or without namespaces.

To read or modify data of an existing Object you need to know the Object
ID ("OID" in short). So for every GET or SET transaction the GUI does it
needs to do a FIND transaction first to get the Object ID of the desired
Object.

That means that half of our CODB transactions are FIND transactions.
Sadly, CODB itself is not indexed sufficiently enough. So if there are
many Objects, the FIND transactions may take a while to complete.

The usage of "memcached" is supposed to speed this up and it carries the
index of Objects and their OIDs. Instead of iterating over all the
Objects there are to find the one we're looking for, "memcached" just
looks at it's cache and instantly tells us what we're looking for.

Hence: It's a lot faster.

Between Christmas and New Year we published base-memcache (and related
updates) that enabled the "memcached" for 5207R, 5208R and 5209R.

Enabling it by default might not have been the smartest idea. We thought
it was good enough for production usage, but apparently there is a kink
that we need to fix first. So I just published a YUM update that will
turn it off again.

The problem seems to be this: The cache size is currently set to 64MB by
default. This "one size fits all" approach is bad. Because if you have
many CODB objects, then 64MB might not be enough. This might then cause
errors when CCE tries to use the cache. Such as not returning the
"System" object. Or other objects that it's looking for. Which caused
the error that you reported.

Like said: The updated "base-memcache-*" RPMs in YUM for
5207R/5208R/5209R turn "memcached" off again to prevent this. Likewise
the default cache size is bumped from 64MB to 512MB for those that want
to play with it and want to enabled it again via the GUI.

Which can be done under "Server Management" / "System Settings" / "Cache".

> How many processes (cced) are supposed to be running?

The number of cced processes entirely depends on current GUI usage.
Usually you have just one /usr/sausalito/sbin/cced around and waiting
for transactions.

Whenever CCEd is contacted (via "cceclient", GUI related Constructors,
Handlers or the GUI's PHP pages it will fire an additional cced child to
handle the communication between that script and CCEd.

A "good" script will always tell CCEd a "goodbye!" when all its
transactions are concluded. Which makes that cced-child go away.

But sometimes scripts die prematurely or are aborted. In which case the
cced-child will remain around until either a CCEd restart (or rehash)
happens, or until that cced-child times out. That timeout process is a
lengthy one. I think 5 minutes or thereabouts.

So it's not unusual to have 5-10 cced processes around. If you're not
using the GUI at the moment and the number of cced processes doesn't
drop back to 1-3, then you can simply restart or rehash CCEd itself to
make them go back to normal:

The best way (which works on 5207R, 5208R and 5209R) is this one:

/usr/sausalito/sbin/cced.init rehash
... or ...
/usr/sausalito/sbin/cced.init restart

The difference between "rehash" and "restart"? A "rehash" is a lot
faster, as it doesn't run the constructors. A "restart" will stop CCEd
and will then fire it up again running all Constructors, which might
take a while.

Just for the sake of killing it quickly and bringing it up again a
"rehash" usually suffices.

-- 
With best regards

Michael Stauber