<br><br><div class="gmail_quote">On Thu, Apr 22, 2010 at 9:13 PM, Doug McNash <span dir="ltr"><<a href="mailto:dmcnash@charter.net">dmcnash@charter.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
So, you think we shouldn't be using Raid5, huh. I asked why Raid5 and they said they wanted the reliability. XFS was chosen because of its reputation for handling large (140M) video files and associated metadata. And yes it is NFS.<br>
</blockquote><div><br>Not having to calculate the checksum for a write is a small speed boost. But for R5, the loss of two drives is total loss of data. That checksum calculation is a major slowdown on recovery. During recovery performance is order of magnitude slower.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
The big remaining problem is that periodically the system stalls and may take a few seconds to send back the write acks. At that point the writer assumes the file is not being written and starts tossing data.<br></blockquote>
<div><br>That's most likely NFS being slow and cranky. <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Is this stalling the nature of a Raid5?, XFS? and would it be improved by different choices?<br></blockquote><div><br>It _could_ be the bit rate to the filesystem is faster than the filesystem can handle. You will need to look at the drive configuration,write speeds and bus speeds and compare to the network IO speeds. A dual Gbit bonded connection feeding a pci slot sata II with 2 drives will have the slot as the bottleneck (I think - don't have my speed sheets up).<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
--<br>
<font color="#888888">doug mcnash<br>
</font><div><div></div><div class="h5"><br>
---- scott mcbrien <<a href="mailto:smcbrien@gmail.com">smcbrien@gmail.com</a>> wrote:<br>
> A lot of the XFS guys from SGI now work for RH. But that's an aside.<br>
> More to the point is how many machines are simultaneously accessing<br>
> said NAS? If it's designed for a single system to access, then just<br>
> use something like ext3. If you're needing multiple simultaneous<br>
> accesses with file locking to avoid the problems that occur with<br>
> multiple machines opening and closing files, you might try GFS. A<br>
> couple of other things<br>
><br>
> 1) You probably don't want to use RAID 5. RAID 5 has data throughput<br>
> issues, especially for large stripe units and/or small file changes.<br>
> I know the alternatives aren't that attractive, the most likely being<br>
> RAID10 or RAID0+1 because they require 2x as many discs. But for the<br>
> additional expense, you'll get much more throughput.<br>
><br>
> 2) One might want a different file system because of the data stored<br>
> on the filesystem. reiserfs for example is really good at storing<br>
> copious amounts of small files, where GFS is good for multiple machine<br>
> accesses, while ext3 is just solid for average people and process<br>
> access on a single machine.<br>
><br>
> 3) RAID 5 isn't high performance.<br>
><br>
> 4) I'm guessing that they're sharing the filesystem via NFS, you<br>
> might want to make sure the NFS server is properly tuned and the<br>
> clients aren't doing anything insane to corrupt your data<br>
><br>
> 5) You really need to move off of RAID5<br>
><br>
> -Scott<br>
><br>
> On Wed, Apr 21, 2010 at 10:15 PM, Jim Kinney <<a href="mailto:jim.kinney@gmail.com">jim.kinney@gmail.com</a>> wrote:<br>
> > How odd. I started using xfs before it was a native thing in redhat (pre<br>
> > RHEL stuff, pre ext3 days). It seemed to always be solid and reliable. It<br>
> > was provided by SGI ( and all the port was provided by SGI as well) and it<br>
> > had a solid track record as the file system that was suitable for huge<br>
> > amounts of data (moving video files was common use). It worked on all of my<br>
> > stuff for all RAID I threw at it. It was imperative to install the xfs-tools<br>
> > to work with it but it sounds like you already have it. If xfs-check is<br>
> > dying due to ram issues, I would be more suspicious of bad hard drives than<br>
> > the xfs code. If there has been a ton of write/delete/write cycles on the<br>
> > drives then the journalling may be corrupted. I'm not sure how to fix that.<br>
> ><br>
> > On Wed, Apr 21, 2010 at 9:34 PM, Doug McNash <<a href="mailto:dmcnash@charter.net">dmcnash@charter.net</a>> wrote:<br>
> >><br>
> >> I'm consulting at a company that wants to turn their Linux based NAS in to<br>
> >> a reliable product. They initially chose XFS because they were under the<br>
> >> impression that it was high performance but what they got was something of<br>
> >> questionable reliability. I have identified and patched several serious bugs<br>
> >> (2.6.29) and I have a feeling there are more unidentified ones out there.<br>
> >> Furthermore, xfs_check craps out of memory every time so we have to do an<br>
> >> xfs_repair at boot and it takes forever. But today we got into a situation<br>
> >> where xfs_repair can't repair the disk (a raid5 array btw).<br>
> >><br>
> >> Does anyone out there use xfs? How about a suggestion for a stable<br>
> >> replacement.<br>
> >> --<br>
> >> doug mcnash<br>
> >> _______________________________________________<br>
> >> Ale mailing list<br>
> >> <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> >> <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> >> See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> >> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > --<br>
> > James P. Kinney III<br>
> > Actively in pursuit of Life, Liberty and Happiness<br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Ale mailing list<br>
> > <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> > <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> > See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> > <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
> ><br>
> ><br>
><br>
> _______________________________________________<br>
> Ale mailing list<br>
> <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>-- <br>James P. Kinney III<br>Actively in pursuit of Life, Liberty and Happiness <br><br>