[BlueOnyx:26873] Re: Log rotation

[Michael Stauber]

Hi Aaron,

> I do see the web.log file I was looking for in ~/var/logs, but it starts 
> at midnight on March 1, 2024. There is no data prior to that. I thought 
> it used to store until reaching a file size limit at which point it 
> would compress everything.
> 
> Looking at /etc/logrotate.d/site[virtual site number], I see:
> 
> /home/.sites/site30/var/logs/web.log {
>     su SITE30-logs site30
>     monthly
>     rotate 0
>     shred
>     maxage 1827
>     missingok
>     minsize 10000
> }
> 
> Is this a file that the UI edits or that I’m supposed to edit? 

The GUI creates it on Vsite creation, but I believe once it's present, 
the GUI doesn't edit it any further.

> It seems like the vsite defaults are configured to not rotate and delete data 
> after a month, which is not good.
Yes and no. The logrotate config for this makes sure that 
/home/sites/<FQDN>/var/logs/web.log is created. But beyond that point it 
doesn't manage anything in /home/sites/5211r1.smd.net/var/logs/ 
directory anymore, because other scripts manage the expiry.

The web.log in each Vsite's /var/logs directory only has the recent data 
from the last run, so that Monitorix, Webalizer and possibly AWStatshave 
something original to chew on.

Our module base-sitestats has a script that then parses the web.log and 
extracts only the useful data and stores that in data-files in numbered 
directories for Year, Month and Day in web.cache and web.json files.

We don't keep the actual logfile snippets as they would use up a lot of 
space. Only the condensed extracts that were aggregated are kept in 
those web.cache and web.json files.

How long this data is stored is configured in "Server Management" / 
"System Settings" / "Data Retention" under "Vsite Usage Information".

And under "Usage Information" / "Web" on the server-level or the 
Vsite-Level you can then use Monitorix to visualize the information. It 
uses the web.json files to display all the relevant information.

-- 
With best regards

Michael Stauber