[BlueOnyx:25579] Re: Large number of users - 5210R - fix available

[ppimentel]

Already tested in another instalation.
Works fine.
Much better.

Thank you so much

P. Pimemntel


A 2022-08-03 00:13, Michael Stauber escreveu:
> Hi all,
> 
> Pedro wrote:
>>> I have a site with +/- 5000 users runnig on a 5209R. With 5210R its 
>>> almost impossible to manage the site (with 5000 users).
>>> It takes several minutes just to open vsiteList in "Site Management" 
>>> of the GUI;
> 
> Pedro was so kind to allow me access to the server in question and I
> could run some diagnostics on it.
> 
> It turns out that with 5000 Users the GUI doesn't only make 5000 GET
> Requests on the User Objects, but also also *another* 3-4 GET Requests
> against each User Object to get some NameSpace data. For the
> configuration of Shell, Email, Subdomains and (if installed) OpenVPN.
> 
> That means: If someone has 5000 Users in a single Vsite, then the GUI
> needs to make at least +20.000 GET requests just to populate the User
> list and show in one glance what features which user has enabled.
> 
> Better "database-design" (the structuring of the data) could have
> prevented this, but that ship has sailed and we can't fix that w/o
> fundamental changes.
> 
> With the slow as hell CCE PHP Class of the GUI those +20000 CCE GET
> requests indeed took way too long to be comfortable. In fact whenever
> I tried the GUI timed out after a few minutes. So all I usually got
> after minutes of waiting was a blank page.
> 
> I spent this afternoon hacking together a solution, which is
> currently published into the "BlueOnyx 5210R Testing" YUM repository.
> 
> The fix to base-user-* published there externalizes the GUI based CCE
> GET requests for the User Objects (and their NameSpaces) into a Perl
> script, which the GUI calls via a secure CCEWrapper. And that
> Perl-Script spits out a single JSON-encoded output that contains all
> requested Objects and all their NameSpaces in one go.
> 
> The GUI parses this output and uses it instead of making its own
> individual GET requests. If this round-trip via Perl fails for
> whatever reason, then the GUI automatically falls back to using the
> old method and fetches the data itself.
> 
> Perl's access to CCE is faster than the one we have in PHP. That's
> where the speed increase comes from and in Pedro's case the load time
> for the UserList GUI page with 5235 Users went down to 22-23 seconds.
> 
> That is still too slow for my taste, but it should be closer to what
> it used to be on the 5209R, where the Zend API CCE module is still
> used instead of the CCE PHP Class.
> 
> Without throwing a cache at this problem I may not be able to speed it
> up any further. And caching the user list will create a heap of other
> problems that I need to carefully consider before making any further
> code changes.
> 
> Code changes in SVN:
> https://devel.blueonyx.it/trac/changeset?reponame=&new=4281%40%2F&old=4278%40%2F