[ale] School Project to Create Distributed Filesystem

[Greg Freemyer]

Very close, but you use heartbeat / linux-ha to run (mdraid, the
filesystem and nfx export) on only one server at a time.  Otherwise
you could have data consistency issues between the nodes.  That is why
true cluster / distributed filesystems are a killer to write.

heartbeat is effectively a cluster controller.

For instance, you could configure it to know about 8 nodes and that it
needs at least 6 of the nodes available for there to an operational
cluster.  (Raid -6 is operation even if 2 nodes have failed.)

Then you have a 9th IP that is called the cluster IP that floats
between the nodes.  All of the clients connect exclusively to the
cluster IP.

Once the cluster is formed (6 or more nodes), heartbeat controls which
of the nodes is the master.

On the master it will use IP aliasing to respond to any and all tcp/ip
connections to the cluster IP.  Further heartbeat you will need to
configure to launch mdraid, form the raid array, mount the filesystem,
and export it via nfs.

If you are familiar with init scripts (/etc/init/*) then you are
familiar with the scripts heartbeat uses.   Heartbeat is actually a
superset of those scripts in that heartbeat scripts also support a
"monitor" command.  So you just have heartbeat invoke:

rcmdraid start
rcfilesystem start
rcnfs start

on the master, and on all the slaves it calls the above with stop.

Fortunately the vast majority of this already exists, but I'm not
aware of anyone putting all the pieces together in this way.

FYI: Heartbeat also provides a reliable comm channel if you need it.
I don't think you will, but you might.

Greg

On Wed, Feb 25, 2009 at 3:09 PM, Omar Chanouha <ofosho at gatech.edu> wrote:
> Greg/All, Let me get one more shot at the belt.
>
> Have a cluster of servers each sharing their files via iSCSI. Each
> server then keeps an array of all the other servers via mdraid.
> Linux-HA is used on the servers to maintain an up-to-date array of
> iSCSI drives. Each server shares the combined RAID via an nfs server.
> When a client wants to connect, all it needs to do is connect to one
> of the servers over nfs and it will have all the files.
>
> As you said, the robustness comes into play with the mdraid update
> scripts/making nfs work properly.
>
> Is that right?
>
> I will have to ask the teacher if he will allow that. He may not.
>
> Thanks,
>
> -OFosho, Miami Dolphin and Open Source Aficionado
>
> On Wed, Feb 25, 2009 at 7:26 AM, Jim Kinney <jim.kinney at gmail.com> wrote:
>> 2009/2/24 Ken Ratliff <forsaken at targaryen.us>:
>>>
>>> On Feb 24, 2009, at 8:25 PM, Brian Pitts wrote:
>>>
>>> Another one worth checking out is gluster. It uses FUSE.
>>>
>>> http://www.gluster.org/
>>>
>>>
>>> Gluster is.... finicky. We have it deployed in a few production clusters and
>>> it works well when it's working, but when it breaks, it tends to break hard.
>>
>> Sounds like a good project for students to work on :-) It needs help.
>>>
>>>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>>
>>>
>>
>>
>>
>> --
>> --
>> James P. Kinney III
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
>



-- 
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com