[ale] distributed network file system

[John Heim]

From: "Derek Atkins" <warlord at MIT.EDU>
To: "Atlanta Linux Enthusiasts" <ale at ale.org>
Sent: Tuesday, June 26, 2012 8:04 AM
Subject: Re: [ale] distributed network file system


> Hi,
>
> "John Heim" <john at johnheim.net> writes:
>
>> I would like to set up a distributed network file system in my 
>> department.
>> There is a dizzying array of possibilities, gfarmfs, ceph, glusterfs, 
>> just
>> to name a few.
>>
>> Needs:
>> 1. Should work on a large number of small nodes, 100Gb each.
>> 2. Parallelism & striping.
>> 3. Prefer debian package, GPL.
>> 4. Meta data in mysql would be nice.
>>
>> Any experience and/or recommendations?
>
> What are your requirements for usage of the space?  Are you trying to
> get a distributed SAN array?  Or are you just trying to get a
> distributed file space?
>
> If the latter you might also want to look at OpenAFS.  It is F/OSS,
> although it's not GPL.  Oh, and the metadata isn't stored in MySQL.

We have a 2Tb SAN for users to use for files space. But we have about 300 
users so each gets only 6Gb.  That's just not enough for some users. Mostly, 
its enough on a long term basis but sometimes they need to generate 50Gb - 
100Gb of data. We have all kinds of disk space on each users workstation but 
they can't get to it. This is deliberate. We don't want them saving files 
where they won't be backed up.  And we want to be able to re-image a machine 
at a moment's notice w/o having to have the user back up his stuff.

I got the brilliant idea of using the 100Gb (or so) of free space on each 
workstation for a distributed network file system.  So we'd need to be able 
to wipe out a node w/o losing anything.  I could make sure we copy the data 
off before we re-image a workstation. But an end-user might simply turn 
their workstation off. Whatever we use would have to deal with that.

The mysql thing was just a preference (over postgres). I have nothing 
against other DBMSes. Its just that we already have mysql.