[BlueOnyx:00984] Re: What to store on a separate disk

[Julien Buratto]

Hi,

first of all let me thank Michael with the listing of folders which I
understand my not to be final..
Scott, what I'm trying to achive is a EC2+EBS machine, not a
standalone (only EC2) BlueOnyx.

About Amazon, Scott is right, it's possible to take "snapshots" of VMs
which are basically the current virtual machine "in full" (included
updates), while EBS ensure a SAN storage for data which is always
updated and subjected to RAID logics for data integrity.
So, separation could be:
- OS + applications => snapshot (to storage)
- User data => SAN

Infact Michael, VMs use virtual volumes on physical disks, so when you
backup a system you backup all the virtual volume (with a lot of
stuff), but what about if the virtual filesystem gets somehow broken ?
Well infact we add a level of uncertainess related to the
virtualization storage software (Xen, Vmware, etc) which creates the
virtual volume filesystem. (a virtual filesystem on top of another
file system)

Wrap up:
1) If we choose to virtualize the volumes we have: (physical +
filesystem) points of failures
2) If we choose to virtualize the virtual machines we have: (physical
+ filesystem + VM specific filesystem) points of failures

>From Amazon point of view we could have:

1)
- Create an EC2 to run BlueOnyx
- Create EBS (external volume) to store:
/etc
/home
/usr/sausalito/codb/
/var/lib/mysql/
/var/named/
- For each "yum update" (do you really do updates so often on a
production server ?) store a snapshot to S3 (Storage)

2)
- create a EC2 (virtual machine) which acts as a VM Server
- create a EBS (san disk) and use it to store VMs virtual filesystems
- create a EBS to store the VM Server OS config (and yes, also VM
Servers need updates)
- snapshot the VM Server periodically

In my opinion, the two approaces are not so different infact, but why
wrap a virtual environment in another virtual environment ?
What about performances, system administration and networking ?

A part from that, I perfectly see the complexity of partially backup an OS.

Julien

PS: If some other folders to externalize popups in your mind, let me know .)


2009/4/1 Scott Hayes <shayes at officetracker.com>:
> If I'm not mistaken you can use the snap shot feature to capture the
> current state of your instance so you can bring it backup with all
> the updates or you can use the snap shot for a new virtual server so
> it doesn't require a massive yum update. The updates are persistent
> between restarts. This doesn't seem to be a show stopper since Amazon
> is very reliable and the snap shot feature is your friend.
>
> Julien, I would love to see BlueOnyx running in the cloud. Please
> keep us updated with your progress.
>
> Just my 2 cents.
>
> We currently have a Centos5 and a Windows server running in the cloud
> and have been very happy so far. 1 year later
>
> Scott,
>
> At 12:54 PM 4/1/2009, you wrote:
>>Hi Julien,
>>
>> > as I'm running BX on Amazon Cloud (where persistence of data is
>> > guaranteed only if using external volumes/partitions), I have already
>> > set my virtual machine to attach and mount a partition /home
>> > externally so that, in case the virtual machine gets broken, I don't
>> > loose all sites/mails and so on.
>> >
>> > What I was wondering is:
>> > - What other directory are important ?
>> >
>> > As when you shutdown (terminate) a virtual machine in Amazon Cloud all
>> > data stored in the virtual machine after that it has been created gets
>> > lost, I was wondering what other dirs/folders are importart to be
>> > saved.
>>
>>I'd say this pretty much eliminates BlueOnyx from being used on an Amazon
>>Cloud and would generally advise against using it there.
>>
>>There is quite a bit of data outside the /home partition that is important -
>>and not just MySQL.
>>
>> >From /etc/ alone you need a ton of stuff, including /etc/mail/,
>> sendmail.cf,
>>sendmail.mc, proftpd.conf, webalizer.conf, named.conf, the /etc/named/
>>symlink, some stuff from /etc/sysconfig/ and what not. If you run
>>any accounts
>>other than admin and/or run sites with or without users, then include
>>/etc/passwd, /etc/shadow, /etc/group, /etc/hosts
>>
>>Ah bloody hell. You almost need half of /etc/ when I think about it.
>>
>>Then you need /usr/sausalito/codb/ or your BlueOnyx GUI will loose
>>it's brain.
>>Assuming you install third party software that brings extra GUI pages aboard
>>you will even need more than just /usr/sausalito/codb/
>>
>>You'll also need /var/lib/mysql/ and if you're running DNS you need the zone
>>files from /var/named/chroot/var/named and assuming you run a mailserver you
>>also need /var/spool
>>
>>But this all brings us to the next point: If the OS or the data in the cloud
>>is not persistent over reboots: What about YUM updates? If you run a YUM
>>update and whatever that brings aboard is not persistant, then what's the
>>point in using a cloud?
>>
>>Even saving the data on a remote volume will you not let get past
>>the obstacle
>>that OS updates or patches will sooner or later cause inconsistencies there.
>>
>>I think you may be better off if you use another approach: Install something
>>in the cloud that allows you to virtualize BlueOnyx (VMware, VirtualBox, Xen
>>... whatever!) and store the virtualized BlueOnyx on the remotely mounted
>>disk. Voila! Problem solved. Because then your BlueOnyx is "stock" again,
>>persistance is ensured and it's even "portable" as you wouldn't really depend
>>on being stuck with Amazon's cloud. Because if that (for whatever reason) is
>>no longer an option, you could simply install your virtualization stuff
>>somewhere else and again mount the drive where the virtual server is stored.
>>
>>--
>>With best regards
>>
>>Michael Stauber
>>
>>
>>_______________________________________________
>>Blueonyx mailing list
>>Blueonyx at blueonyx.it
>>http://www.blueonyx.it/mailman/listinfo/blueonyx
>
>
> _______________________________________________
> Blueonyx mailing list
> Blueonyx at blueonyx.it
> http://www.blueonyx.it/mailman/listinfo/blueonyx
>



-- 
Julien Buratto