[BlueOnyx:12015] Re: Varnish Cache

[Dirk Estenfeld]

Michael,

only one short mind about your explanation which might cut some of the problems you are pointing out.
For an SSL certificate you also have to use a single ip. Why not to setup varnish in BO in that way, that a site which want to use varnish have to be assigned with a dedicated Ip address. 
But you are right. Some other problems will still exist.
Maybe it is possible to do a base line implementation which will run with an easy setup and the ability to change config file settings within the gui for professionals or more complicated settings.

Best regards,
Dirk


-----------------------------------------------
Black Point Arts Internet Solutions GmbH - Hanauer Landstrasse 423a - 60314 Frankfurt

-----------------------------------------------

-----Ursprüngliche Nachricht-----
Von: blueonyx-bounces at mail.blueonyx.it [mailto:blueonyx-bounces at mail.blueonyx.it] Im Auftrag von Michael Stauber
Gesendet: Mittwoch, 16. Januar 2013 21:22
An: BlueOnyx General Mailing List
Betreff: [BlueOnyx:12014] Re: Varnish Cache

Hi Frank,

> What I don't understand, that's if it is complicated to consider this 
> tool in Blueonyx. In fact, in its simpliest usage (caching sites), the 
> only thing to do is to put Apache on another port, and Varnish 
> listening to port 80.
> Is there terribly difficult to enhanced the GUI to permit default port 
> modification from 80 to, e.g. 8080 ? In a first time this may be done 
> perhaps even in a conf file, not GUI.
> 
> Does it involves only to put "8080" instead of "80" in the siteX 
> files, or very much more ?

I wish it was that simple.

Yes, the GUI side of things to put Apache on another port aren't that terribly complicated to do. It gets complicated in a scenario where you have more than one Vsite on your BlueOnyx. Which is typically the case.

Say 20 Vsites run on one IP. You want to switch one Vsite to
127.0.0.1:80 instead, so that you can put a Varnish in front of it.

Well, you can't.

You'd have to switch all other 19 Vsites as well, or you'd have to assign a different IP to them. You also need to instruct Apache not to bind itself to all available IP's, which is done somewhere else (directly in httpd.conf).

Now if you switch all 20 Vsites to 127.0.0.1:80, you could then put Varnish in front of them.

But fun and games: Your Varnish configuration now needs rewrite rules to handle all 20 Vsites - including *all* their web server aliases. Because if Varnish listens to the incoming port 80 traffic on that single external IP, it needs to set the "req.http.host" variable to the FQDN that was requested, or the underlying webserver wouldn't know which of the 20 Vsites on that IP it ought to serve.

It may get worse: One of the 20 sites might have SSL enabled. So we'd also bind Apache's port 443 to localhost and need Varnish rules to deal with port 443 traffic.

Now it gets worse: The client of site #15 needs a special Varnish rule as he doesn't want to cache images out of his /news/ directory. But the client of site #2 also has a /news/ directory that he wants cached totally. Which is possible, but then we'd need separate Varnish rules for for those two Vhosts (and perhaps the rest of them as well).

Eventually, after weeks of coding we might end up with a more or less useful Varnish Cache implementation in BlueOnyx that's well enough integrated into the GUI. Only to have you realize that the client with that particular demand would have been better off using a dedicated server (or VPS) of his own. Because without a lot of tinkering none of his PHP scripts was cached by Varnish, because all the scripts used
session_start() in every request, which made caching impossible. What the client might have needed was a lot of ESI coding in his site template to make the pages more cache-friendly (and a highly customized Varnish config), but he didn't know enough about it to ponder these possibilities.

Even if we finally might have a GUI to tweak all the Varnish related
options: Whenever someone applies the changes, the cache needs to be restarted, which will affect all cached Vsites.

All in all I'd say: The Varnish cache *might* be useful in a shared hosting environment, but that usefulness will be quite limited. It's not a "one size fits all" solution. The usefulness depends on the served content for which you might need a specifically tweaked configuration.
This required configuration adds a layer of complexity that will make troubleshooting and/or servicing by non Linux experts even more of a challenge.

My bottom line: It's possible and it is considered for the re-themed next version BlueOnyx. However, if a client requests a Varnish cache, then the likeliness that he needs a dedicated server to best cater his very individual needs is very high. Or he might not know what he's asking for.

Related to this topic: The BlueOnyx team is using Varnish cache for the site "updates.blueonyx.it", which is the main YUM repository. This move was necessary for adding a bit more resiliency and robustness. We do cache a few things such as the XML files, but mostly the Varnish cache acts as frontend to round-robin the HTTP requests to those HTTP servers that have replied positively to periodically run availability checks. We also use Varnish to filter requests. People who accidentally have set "updates.blueonyx.it" as SWupdate server are redirected to "newlinq.blueonyx.it", the highly intrusive robot-crawler of the search engine Baidu is sent off to index their own domain and we also do a few other funky things that slow down or divert unwanted traffic.

At the same time it's used as a bit of a learning experience to get more up to date on all the useful things that Varnish can and can't do.

--
With best regards

Michael Stauber
_______________________________________________
Blueonyx mailing list
Blueonyx at mail.blueonyx.it
http://mail.blueonyx.it/mailman/listinfo/blueonyx