[ale] School Project to Create Distributed Filesystem

[Omar Chanouha]

Hey Greg,

To be honest, I had to wikipedia pretty much all the buzzwords in your
email, so let me see if I understand your proposal.

Have a cluster of servers, each running an iscsi server.
Then to connect to all these server's at one time, use mdraid on a client.
Finally, use linux-ha in order to keep the client up-to-date in terms
of what iscsi servers are connected.

Is that right? Where does linux-ha fit into place, server or client
side or both?

Thanks,

-OFosho, Miami Dolphin and Confused Noob

On Tue, Feb 24, 2009 at 4:04 PM, Greg Freemyer <greg.freemyer at gmail.com> wrote:
> How many man-hours do you think is reasonable and what sort of skills
> are you trying to show.
>
> If programming, then yes you need to do something simple from scratch.
>
> If higher-level, then pulling together multiple existing projects into
> one, configuring them to work together, and writing failover scripts
> to keep the whole thing robust seems like a good project.
>
> I setup iSCSI in a few hours a couple years ago.  Should be even easier today.
>
> Once you have that, layering mdraid on top of multiple iSCSI devices
> should be a simple matter of following a simple howto.  I don't think
> iSCSI would be an issue at all for mdraid.
>
> Then you need to put a filesystem on it.  (mkfs.ext3 /dev/md3)
>
> I would hope anyone on your team could get all that done in a 8-hour
> day.  (Assuming you have PCs with linux already installed and network
> connectivity between them as a starting point.)
>
> Now comes the fun part in my opinion, installing heartbeat (linux-ha)
> on all the nodes.  You will need to configure it to have it control
> mdraid, mounting the filesystem, and launching nfs to export the
> filesystem.
>
> Their are recipes for the filesystem and nfs part, so that should go
> fast.  Starting and stopping mdraid I would throw into the filesystem
> control script.
>
> One issue is the nfs is not great at the actual failover.  At least it
> used to have issues when I was involved a few years ago.
>
> The "cool" part for me would be create scripts that cause mdraid to
> grow the array as you add machines.  mdadm has a lot of options for
> that already, but you could use heartbeat to invoke the reshapes based
> on machines coming and going.
>
> Greg
>
> On Tue, Feb 24, 2009 at 2:54 PM, Omar Chanouha <ofosho at gatech.edu> wrote:
>> I appreciate all the ideas. I think the bottom line is that the
>> aggressive timeline we have been given, piggybacking over nfs it not
>> going to be possible. I think we will just write a short proof of
>> concept user space app that connects via our simplistic protocol, to
>> our simplistic protocol. Not as "noble" (I like that comment btw), but
>> when the professor wants the whole thing working well, 3 weeks before
>> the end of the semester, and this is only one of 5 classes, nobility
>> must take a back seat.
>>
>> Thanks,
>>
>> -OFosho, Miami Dolphin and Former NFS Kernel Hacker
>>
>> On Tue, Feb 24, 2009 at 12:34 PM, Ed Cashin <ecashin at noserose.net> wrote:
>>> On Tue, Feb 24, 2009 at 11:57 AM, Greg Freemyer <greg.freemyer at gmail.com> wrote:
>>>> Just thought of another approach.
>>>>
>>>> Layer one = iSCSI or NBD to export raw block devices out of each server.
>>>>
>>>> Layer two = mdraid to create a fault tolerant block device. ie. use
>>>> Raid 10 or Raid 6 etc. to integrate the raw devices into a whole.
>>>>
>>>> Layer three = LinuxHA to control the whole mess.
>>>>
>>>> Then you just throw in a filesystem and NFS support and you're done.
>>>> LinuxHA already has filesystem and nfs control scripts I think.
>>>
>>> There's a HOWTO that might be interesting to folks who are
>>> considering this architecture.  It's directed towards a specific
>>> platform, but it is short and generalizes well, since the platform
>>> is based on debian lenny.
>>>
>>>  http://support.coraid.com/support/cln/ft/failover-kit.html
>>>
>>> The use of md isn't covered, but it's been tested with the
>>> configuration in the HOWTO and works fine as long as the
>>> admin understands clearly that heartbeat must control md
>>> just like it controls filesystem mounting.  Specifically, the
>>> standby must never have md running with its own independent
>>> ideas about the state of the Software RAID.
>>>
>>> --
>>>  Ed Cashin <ecashin at noserose.net>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>>
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>>
>
>
>
> --
> Greg Freemyer
> Litigation Triage Solutions Specialist
> http://www.linkedin.com/in/gregfreemyer
> First 99 Days Litigation White Paper -
> http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
>
> The Norcross Group
> The Intersection of Evidence & Technology
> http://www.norcrossgroup.com
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
>