[ale] School Project to Create Distributed Filesystem

[Greg Freemyer]

How many man-hours do you think is reasonable and what sort of skills
are you trying to show.

If programming, then yes you need to do something simple from scratch.

If higher-level, then pulling together multiple existing projects into
one, configuring them to work together, and writing failover scripts
to keep the whole thing robust seems like a good project.

I setup iSCSI in a few hours a couple years ago.  Should be even easier today.

Once you have that, layering mdraid on top of multiple iSCSI devices
should be a simple matter of following a simple howto.  I don't think
iSCSI would be an issue at all for mdraid.

Then you need to put a filesystem on it.  (mkfs.ext3 /dev/md3)

I would hope anyone on your team could get all that done in a 8-hour
day.  (Assuming you have PCs with linux already installed and network
connectivity between them as a starting point.)

Now comes the fun part in my opinion, installing heartbeat (linux-ha)
on all the nodes.  You will need to configure it to have it control
mdraid, mounting the filesystem, and launching nfs to export the
filesystem.

Their are recipes for the filesystem and nfs part, so that should go
fast.  Starting and stopping mdraid I would throw into the filesystem
control script.

One issue is the nfs is not great at the actual failover.  At least it
used to have issues when I was involved a few years ago.

The "cool" part for me would be create scripts that cause mdraid to
grow the array as you add machines.  mdadm has a lot of options for
that already, but you could use heartbeat to invoke the reshapes based
on machines coming and going.

Greg

On Tue, Feb 24, 2009 at 2:54 PM, Omar Chanouha <ofosho at gatech.edu> wrote:
> I appreciate all the ideas. I think the bottom line is that the
> aggressive timeline we have been given, piggybacking over nfs it not
> going to be possible. I think we will just write a short proof of
> concept user space app that connects via our simplistic protocol, to
> our simplistic protocol. Not as "noble" (I like that comment btw), but
> when the professor wants the whole thing working well, 3 weeks before
> the end of the semester, and this is only one of 5 classes, nobility
> must take a back seat.
>
> Thanks,
>
> -OFosho, Miami Dolphin and Former NFS Kernel Hacker
>
> On Tue, Feb 24, 2009 at 12:34 PM, Ed Cashin <ecashin at noserose.net> wrote:
>> On Tue, Feb 24, 2009 at 11:57 AM, Greg Freemyer <greg.freemyer at gmail.com> wrote:
>>> Just thought of another approach.
>>>
>>> Layer one = iSCSI or NBD to export raw block devices out of each server.
>>>
>>> Layer two = mdraid to create a fault tolerant block device. ie. use
>>> Raid 10 or Raid 6 etc. to integrate the raw devices into a whole.
>>>
>>> Layer three = LinuxHA to control the whole mess.
>>>
>>> Then you just throw in a filesystem and NFS support and you're done.
>>> LinuxHA already has filesystem and nfs control scripts I think.
>>
>> There's a HOWTO that might be interesting to folks who are
>> considering this architecture.  It's directed towards a specific
>> platform, but it is short and generalizes well, since the platform
>> is based on debian lenny.
>>
>>  http://support.coraid.com/support/cln/ft/failover-kit.html
>>
>> The use of md isn't covered, but it's been tested with the
>> configuration in the HOWTO and works fine as long as the
>> admin understands clearly that heartbeat must control md
>> just like it controls filesystem mounting.  Specifically, the
>> standby must never have md running with its own independent
>> ideas about the state of the Software RAID.
>>
>> --
>>  Ed Cashin <ecashin at noserose.net>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
>



-- 
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com