[BlueOnyx:15270] Re: AOL and DKIM, SPF & Re: SSL change after updates?
Matt James
matt at rainstorminc.com
Thu Apr 24 13:57:54 -05 2014
Hi Michael,
That sounds great! Can't wait to try it out. Thanks!
--
Matt James
RainStorm, Inc
(207) 866-3908 x54
On Apr 23, 2014, at 6:13 PM, Michael Stauber <mstauber at blueonyx.it> wrote:
> Hi Eric, hi Matt,
>
> I'm combining my reply to your two separate topics regarding wildcards
> and SPF. It's a slightly longer rant. :p
>
> SPF:
> ====
>
> Let me start with SPF, because it's a bit easier to answer. As you know,
> SPF records are simply TXT DNS records that are "tacked on". If you know
> the right Syntax, you can already go ahead and can use SPF records on
> 5106R/5107R/5108R as is. Just create a TXT record with the proper
> payload via the GUI and you will be fine.
>
> Pages such as http://www.spfwizard.net/ sure are a great help finding
> out how the Syntax for your SPF record(s) should look.
>
> However, this usually overlooks some fundamental stuff. What about
> emails from localhost? Which would mean system generated emails, mails
> from certain scripts and so on. So you *will* need to think your SPF
> record layout through thoroughly to really match all usage cases.
>
> Sure, having pages in the BlueOnyx GUI that *assist* with creating
> proper SPF records would be nice. But such a wizard is quite complex and
> at the end of the day there will be usage cases that it won't and can't
> cover. That's where your own brain juice is again needed.
>
> If I wanted to do such a wizard, it would take me four weeks. Minimum.
> And once I release it, people will come forward and point out all the
> juicy usage cases where it doesn't work right. So we're looking at
> another two or three weeks of tweaking it beyond that.
>
> I'm like *THIS* close (imagine me holding thumb and index finger 3mm
> apart) of having the new BlueOnyx 520XR GUI done. I *really* don't want
> to delay the release (or complicate it unnecessarily) with an SPF
> implementation that will be riddled with bugs and limited in capability.
>
> Asking me to implement that now is inviting failure. The new GUI will
> already have sufficient bugs and deficiencies that need to be weeded out
> through thorough testing before it can be considered stable.
>
> So long reply for a short answer: Yes, one of these days BlueOnyx will
> have a basic Wizard to do SPF records. In the meantime (and until that
> happens) you can create SPF records by doing normal TXT records in the
> DNS GUI.
>
> Wildcards for hostnames:
> ========================
>
> I just shot my development box for 520XR six ways 'til Sunday *trying*
> to get this work. I'll actually have to restore it from a backup before
> I can continue development. Aside from loosing time with the restore,
> I'll also have to merge changes back into the backup that I did since
> the last backup run. Fun and games - half a day wasted.
>
> Let me explain some of the basics of Sausalito here. Not much, but
> little enough to understand the problem I'm facing there:
>
> CODB database fields can only take data that matches a certain scheme.
> We can define these types of data that a field can take with regular
> expressions.
>
> For example:
>
> username:
>
> That field only takes alphanumerical characters plus a limited subset of
> other characters. Such as ".", "_" and "-". That's defined in a schema
> file like this:
>
> <typedef
> name="alphanum_plus"
> type="re"
> data="^[A-Za-z0-9\\._-]+$"
> />
>
> You can see, the regular expression for this is: ^[A-Za-z0-9\\._-]+$
>
> In a CODB Schema file for a data-field that should contain the username,
> we can then simply define that field like this:
>
> <property
> name="username" type="alphanum_plus" default=""
> writeacl="ruleCapable(modifyUserDefaults)"
> />
>
> And CODB will then make sure that only data is stored into that field
> that matched the regular expression we defined for "alphanum_plus".
> Likewise the new GUI will also check input with this *while* you type it
> into the respective form field. If the entered data doesn't match
> validating against "alphanum_plus", you get an error message.
>
> Which is all nice and comfy. It's a real time saver when coding GUI
> pages, as you don't have to worry about the user submitting data that
> doesn't belong into the respective fields.
>
> Now let me come to "hostname", "domainname" and "fqdn". They are also
> defined in Schema files and the regular expressions for that are already
> horribly complex.
>
> So I *can* tweak the "hostname" field to accept a wildcard instead.
>
> At the moment the definition for "hostname" looks like this:
>
> <typedef name="hostname" type="re"
> data="^[A-Za-z0-9][A-Za-z0-9\-]*(\\.[A-Za-z0-9][A-Za-z0-9\-]*)*$"
> />
>
> If I change it to also accept a wildcard, it'll look like this:
>
> <typedef name="hostname" type="re"
>
> data="(^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$)|(^\\*$)"
> />
>
> Yes, that is a hell of a lot more complicated, but it does the trick. It
> probably can be simplified a bit further, but ... I'm in a hurry.
>
> Ok, so far so good. Now *all* "hostname" fields in *each* and *every*
> part of the GUI accept wildcards. Let us think for a moment if that's a
> smart idea. Where are we using these?
>
> - Vsite Add: We can create Vsites with a hostname of "*".
> - Vsite Mod: We can change the hostname of Vsites to "*".
> - DNS Management: We can create DNS records with "*" for the hostname.
> - Email aliases: Email server aliases with "*" for hostname.
> - Web aliases: Web server aliases with "*" for hostname.
> - SSL certificates: Can now contain a wildcard in hostname, too.
>
> So with DNS we're good. We want wildcard DNS. We might also want
> wildcard web server aliases.
>
> What we do NOT want is email server aliases *and* Vsites themselves that
> have a wildcard in their name.
>
> Which now immediately creates a problem: ALL our "hostname" input fields
> in the entire GUI now accept wildcards. Because every GUI page and every
> Schema file that deals with hostnames is now hot-wired to accept
> wildcards. Even for the parts where we don't want them.
>
> In a lot of places the FQDN is also assembled out of "hostname" and
> "domainname", which have separate definitions. And FQDN has a separate
> definition of what data is allowed and which not. So if the hostname
> *has* a wildcard is merged with the domainname into an FQDN, then the
> storage of the data will fail. Because FQDN can't take wildcards in the
> hostname part according to our CODB data-type definitions.
>
> But hey, it gets worse: We have Perl handlers which update the email and
> webserver configuration. Also proftpd.conf. If I feed those Perl
> handlers wildcard hostnames, they blow up. Because they're not designed
> to have a wildcard in the data. Wildcards, which could or could not be
> (depends on the context) be interpreted as a regular expression itself.
>
> Just three examples:
>
> a.) FQDN contains a wildcard:
>
> Handler base_unique.pl blows up while verifying if another site exists
> that already might contain that name. The handler to create the email
> server mapfiles triggers after it and blows up, too. Instant hang up of
> CODB and further transactions via the GUI aren't possible until the
> haning processes are killed and CCEd is restarted.
>
> b.) Email server alias contains a wildcard:
>
> See above. One of the handlers for creating the email server mapfiles
> doesn't handle this well.
>
> c.) Web server alias contains a wildcard:
>
> Handlers work fine and the changes go through as intended. Except that
> Apache won't restart because the /etc/httpd/conf/vhosts/site* file now
> contains a syntax error:
>
> # owned by VirtualHost
> NameVirtualHost 208.77.221.197:80
>
> # ServerRoot needs to be set. Otherwise all the vhosts
> # need to go in httpd.conf, which could get very large
> # since there could be thousands of vhosts:
>
> <VirtualHost 208.77.221.197:80>
> ServerName www.basura2.net
> ServerAlias basura2.net *.basura2.net
> ServerAdmin admin
> DocumentRoot /home/.sites/70/site4/web
> ErrorDocument 401 /error/401-authorization.html
> ErrorDocument 403 /error/403-forbidden.html
> ErrorDocument 404 /error/404-file-not-found.html
> ErrorDocument 500 /error/500-internal-server-error.html
> RewriteEngine on
> RewriteCond %{HTTP_HOST} !^208.77.221.197(:80)?$
> RewriteCond %{HTTP_HOST} !^www.basura2.net(:80)?$ [NC]
> RewriteCond %{HTTP_HOST} !^basura2.net(:80)?$ [NC]
> RewriteCond %{HTTP_HOST} !^*.basura2.net(:80)?$ [NC]
> [...]
>
> In the last line above you see the Rewrite-Condition containing the
> wildcard. The * in it needs to be escaped like this:
>
> RewriteCond %{HTTP_HOST} !^\*.basura2.net(:80)?$ [NC]
>
> So that handler needs an extra IF-clause to check web server aliases for
> wildcards. And if present, it needs to escape them. Great! \o/
>
> Long story short:
>
> To allow wildcard hostnames we need:
>
> - Updated basetypes.schema with EXTRA definitions for:
> - hostname with and without wildcards
> - FQDN with and without wildcards in the hostname part
>
> - Updated base-dns.mod:
> - DNS.schema needs an update to change the datatype for
> "hostname" to "hostnamePlusWildcard".
> - GUI pages for DNS management (server + Vsite) needs
> changes to change input-validation for "hostname" to
> "hostnamePlusWildcard". That's like 9 GUI pages that
> need changes.
>
> - Updated base-vsite.mod:
> The following GUI pages need input validation for "hostname"
> changed to "hostnamePlusWildcard":
> - "Site Management" / "Web"
> - "Site Management" / "Add Site"
> - "Site Management" / "Site Template"
> - Vsite.schema needs to be updated.
> - Various handlers need to be updated.
>
> - base-ssl.mod:
> Don't get me started on this one. Changes needed?
> - In 5 GUI pages.
> - Probably two schema files
> - Various handlers and constructors
> - PLUS the script that dynamically creates the SSL-containers.
>
> - base-email.mod:
> I'd be damned if I knew. Can't even tell where and where not
> I'd have to make changes. Probably various handlers and
> constructors. Maybe a schema file or two.
>
>
> My bottom line:
> ===============
>
> Yesterday I thought I'd be able to finish the new BlueOnyx GUI this
> week. With *that* dumped into my lap we're looking at a 4-6 week holdup
> plus a lot more potential bugs in critical subsystems.
>
> So I'm offering a compromise here: The new GUI will allow wildcard DNS
> records. It will also allow that "web server alias" contains a wildcard
> in the hostname (see attached screenshots).
>
> But that's it. Neither FQDN's (for Vsite names) or email server aliases
> will take wildcards.
>
> It'll still require a few days of tweaking and testing before I can be
> comfortable with it. And this will *NOT* make it into 5106R, 5107R or
> 5108R. It'll only be in BlueOnyx models with the new GUI (5207R/5208R -
> or better).
>
> Good enough?
>
> --
> With best regards
>
> Michael Stauber
> <Bildschirmfoto vom 2014-04-23 15:51:50.png><Bildschirmfoto vom 2014-04-23 15:51:43.png>_______________________________________________
> Blueonyx mailing list
> Blueonyx at mail.blueonyx.it
> http://mail.blueonyx.it/mailman/listinfo/blueonyx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.blueonyx.it/pipermail/blueonyx/attachments/20140424/78abe048/attachment.html>
More information about the Blueonyx
mailing list