[ale] XFS on Linux - Is it ready for prime time?

Fri Apr 23 10:43:54 EDT 2010

On Thu, Apr 22, 2010 at 9:13 PM, Doug McNash <dmcnash at charter.net> wrote:

>
> So, you think we shouldn't be using Raid5, huh. I asked why Raid5 and they
> said they wanted the reliability. XFS was chosen because of its reputation
> for handling large (140M) video files and associated metadata. And yes it is
> NFS.
>

Not having to calculate the checksum for a write is a small speed boost. But
for R5, the loss of two drives is total loss of data. That checksum
calculation is a major slowdown on recovery. During recovery performance is
order of magnitude slower.

>
> The big remaining problem is that periodically the system stalls and may
> take a few seconds to send back the write acks. At that point the writer
> assumes the file is not being written and starts tossing data.
>

That's most likely NFS being slow and cranky.

>
> Is this stalling the nature of a Raid5?, XFS? and would it be improved by
> different choices?
>

It _could_ be the bit rate to the filesystem is faster than the filesystem
can handle. You will need to look at the drive configuration,write speeds
and bus speeds and compare to the network IO speeds. A dual Gbit bonded
connection feeding a pci slot sata II with 2 drives will have the slot as
the bottleneck (I think - don't have my speed sheets up).

> --
> doug mcnash
>
> ---- scott mcbrien <smcbrien at gmail.com> wrote:
> > A lot of the XFS guys from SGI now work for RH.  But that's an aside.
> > More to the point is how many machines are simultaneously accessing
> > said NAS?  If it's designed for a single system to access, then just
> > use something like ext3.  If you're needing multiple simultaneous
> > accesses with file locking to avoid the problems that occur with
> > multiple machines opening and closing files, you might try GFS.  A
> > couple of other things
> >
> > 1)  You probably don't want to use RAID 5.  RAID 5 has data throughput
> > issues, especially for large stripe units and/or small file changes.
> > I know the alternatives aren't that attractive, the most likely being
> > RAID10 or RAID0+1 because they require 2x as many discs.  But for the
> > additional expense, you'll get much more throughput.
> >
> > 2) One might want a different file system because of the data stored
> > on the filesystem.  reiserfs for example is really good at storing
> > copious amounts of small files, where GFS is good for multiple machine
> > accesses, while ext3 is just solid for average people and process
> > access on a single machine.
> >
> > 3) RAID 5 isn't high performance.
> >
> > 4)  I'm guessing that they're sharing the filesystem via NFS, you
> > might want to make sure the NFS server is properly tuned and the
> > clients aren't doing anything insane to corrupt your data
> >
> > 5)  You really need to move off of RAID5
> >
> > -Scott
> >
> > On Wed, Apr 21, 2010 at 10:15 PM, Jim Kinney <jim.kinney at gmail.com>
> wrote:
> > > How odd. I started using xfs before it was a native thing in redhat
> (pre
> > > RHEL stuff, pre ext3 days). It seemed to always be solid and reliable.
> It
> > > was provided by SGI ( and all the port was provided by SGI as well) and
> it
> > > had a solid track record as the file system that was suitable for huge
> > > amounts of data (moving video files was common use). It worked on all
> of my
> > > stuff for all RAID I threw at it. It was imperative to install the
> xfs-tools
> > > to work with it but it sounds like you already have it. If xfs-check is
> > > dying due to ram issues, I would be more suspicious of bad hard drives
> than
> > > the xfs code. If there has been a ton of write/delete/write cycles on
> the
> > > drives then the journalling may be corrupted. I'm not sure how to fix
> that.
> > >
> > > On Wed, Apr 21, 2010 at 9:34 PM, Doug McNash <dmcnash at charter.net>
> wrote:
> > >>
> > >> I'm consulting at a company that wants to turn their Linux based NAS
> in to
> > >> a reliable product.  They initially chose XFS because they were under
> the
> > >> impression that it was high performance but what they got was
> something of
> > >> questionable reliability. I have identified and patched several
> serious bugs
> > >> (2.6.29) and I have a feeling there are more unidentified ones out
> there.
> > >> Furthermore, xfs_check craps out of memory every time so we have to do
> an
> > >> xfs_repair at boot and it takes forever.  But today we got into a
> situation
> > >> where xfs_repair can't repair the disk (a raid5 array btw).
> > >>
> > >> Does anyone out there use xfs? How about a suggestion for a stable
> > >> replacement.
> > >> --
> > >> doug mcnash
> > >> _______________________________________________
> > >> Ale mailing list
> > >> Ale at ale.org
> > >> http://mail.ale.org/mailman/listinfo/ale
> > >> See JOBS, ANNOUNCE and SCHOOLS lists at
> > >> http://mail.ale.org/mailman/listinfo
> > >
> > >
> > >
> > > --
> > > --
> > > James P. Kinney III
> > > Actively in pursuit of Life, Liberty and Happiness
> > >
> > >
> > > _______________________________________________
> > > Ale mailing list
> > > Ale at ale.org
> > > http://mail.ale.org/mailman/listinfo/ale
> > > See JOBS, ANNOUNCE and SCHOOLS lists at
> > > http://mail.ale.org/mailman/listinfo
> > >
> > >
> >
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://mail.ale.org/mailman/listinfo/ale
> > See JOBS, ANNOUNCE and SCHOOLS lists at
> > http://mail.ale.org/mailman/listinfo
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>

-- 
-- 
James P. Kinney III
Actively in pursuit of Life, Liberty and Happiness
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20100423/b9351856/attachment-0001.html 