<div dir="ltr">starting from scratch on this is my preference but it's not in the budget. Parking 70TB "elsewhere" for a 4 array reformat while NOT buying new drives is a challenge.<br></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Mon, Apr 22, 2013 at 1:24 PM, Derek Atkins <span dir="ltr"><<a href="mailto:warlord@mit.edu" target="_blank">warlord@mit.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">Jim Kinney <<a href="mailto:jim.kinney@gmail.com">jim.kinney@gmail.com</a>> writes:<br>
<br>
> given what the drive arrays are used for, dedup is a big plus (people make<br>
> copies and copies of research files like drive space is $0!). And, yeah, even<br>
> with no dedup, 32G is not going to cut it anyway.<br>
<br>
</div>You can get mobos that support 256MB for not too much money.<br>
<br>
For my case I cannot imagine that dedup would significantly help me; my<br>
plan is to store audio and video, and perhaps *some* VM data, but most<br>
likely I don't imagine that dedup would do *me* any signigicant<br>
advantage.<br>
<div class="im"><br>
> The checksums for RAID6 are the same as in raidz2.<br>
<br>
</div>Are they? I though ZFS/RAID-Z(2) had additional checksums above and<br>
beyond what RAID5(6) provided, such that it could detect and correct a<br>
write-hole (or other) disk/storage failure.<br>
<div class="im"><br>
> I found a 250G SSD but can't use it. I have one other system I can put it on<br>
> but it has not enough PCIe slots for the need.<br>
><br>
> the solution to all of this is $$$$$ for new system :-(<br>
<br>
</div>Or just replace the mobo ;)<br>
<span class="HOEnZb"><font color="#888888"><br>
-derek<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> On Fri, Apr 19, 2013 at 11:58 AM, Derek Atkins <<a href="mailto:warlord@mit.edu">warlord@mit.edu</a>> wrote:<br>
><br>
> Jim,<br>
><br>
> Jim Kinney <<a href="mailto:jim.kinney@gmail.com">jim.kinney@gmail.com</a>> writes:<br>
><br>
> > After much research, I have decided to NOT implement ZFS. The main<br>
> factors of<br>
> > interest were the raidz2 and data deduplication. In order to support the<br>
> > deduplication,the server needs 5G RAM per 1TB storage. That's a deal<br>
> breaker<br>
> > right there as the system to be used will only support 32G and has 50TB.<br>
> > Raidz2 is basically software RAID6 which I can already do.<br>
> ><br>
> > Looked at getting an SSD for the deduplication data and using l2arc but<br>
> I have<br>
> > no space to install it on that server. Grr.<br>
><br>
> Do you really feel you need ZFS deduplication? According to<br>
> <a href="http://doc.freenas.org/index.php/Hardware_Recommendations" target="_blank">http://doc.freenas.org/index.php/Hardware_Recommendations</a> you only need<br>
> 5GB/TB if you're using dedup. If you're just using ZFS then they<br>
> recommend 1GB/TB. Granted, if you've got 50TB of data then a 32GB<br>
> memory limit wont be useful to you.<br>
><br>
> However, thank you for making me look at that; I was considering a<br>
> 32GB-limit Mobo for my system, which I was expecting to start at 12TB<br>
> usable but potentially expanding to 96TB. I don't think a 96TB ZFS<br>
> would work well with 32GB memory.<br>
><br>
> OTOH, is ZFS really only just "RAID6"? I thought that the ZFS checksums<br>
> would provide improved data integrity (scrubbing) over pure RAID6?<br>
><br>
> -derek<br>
><br>
> > On Fri, Mar 29, 2013 at 3:10 PM, Ed Cashin <<a href="mailto:ecashin@noserose.net">ecashin@noserose.net</a>> wrote:<br>
> ><br>
> > Thanks in advance for the notes you take. I'm interested to hear<br>
> what you<br>
> > find.<br>
> ><br>
> > By the way, work sent me to a Nexenta course in 2010 where I learned<br>
> ZFS<br>
> > stuff from Richard Elling:<br>
> ><br>
> > <a href="http://www.richardelling.com/" target="_blank">http://www.richardelling.com/</a><br>
> ><br>
> > ... and one thing that kept coming up is how users usually reach for<br>
> the<br>
> > RAID-Z options when stripes of mirrors would perform better and<br>
> offer more<br>
> > flexibility. I thought that was notable.<br>
> ><br>
> > On Fri, Mar 29, 2013 at 2:44 PM, Jim Kinney <<a href="mailto:jim.kinney@gmail.com">jim.kinney@gmail.com</a>><br>
> wrote:<br>
> ><br>
> > There was an update to this that just hit /.<br>
> ><br>
> > WooHoo!<br>
> ><br>
> > I have not used ZFS (minimal solaris experience) before but I<br>
> have a<br>
> > situation where raidz looks like a very good choice. An external<br>
> 16<br>
> > bay JBOD box hanging off a SAS HBA looks like a prime candidate.<br>
> The<br>
> > big issue for me is the support for native NFSv4 ACLs which<br>
> solves<br>
> > some issues.<br>
> ><br>
> > As I go through this exercise, I'll keep some notes and maybe<br>
> look at<br>
> > a presentation on doing this.<br>
> ><br>
> > hints, gotcha's, are very welcome.<br>
> ><br>
> > I did find a pdf of the zfsadmin.pdf <a href="http://pdfhome.org/" target="_blank">http://pdfhome.org/</a><br>
> zfsadmin.pdf<br>
> ><br>
> > --<br>
> > --<br>
> > James P. Kinney III<br>
> ><br>
> > Every time you stop a school, you will have to build a jail.<br>
> What you<br>
> > gain at one end you lose at the other. It's like feeding a dog<br>
> on his<br>
> > own tail. It won't fatten the dog.<br>
> > - Speech 11/23/1900 Mark Twain<br>
> ><br>
> > <a href="http://electjimkinney.org" target="_blank">http://electjimkinney.org</a><br>
> > <a href="http://heretothereideas.blogspot.com/" target="_blank">http://heretothereideas.blogspot.com/</a><br>
> ><br>
> > _______________________________________________<br>
> > Ale mailing list<br>
> > <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> > <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> > See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> > <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
> ><br>
> > --<br>
> > Ed Cashin <<a href="mailto:ecashin@noserose.net">ecashin@noserose.net</a>><br>
> > <a href="http://noserose.net/e/" target="_blank">http://noserose.net/e/</a><br>
> > <a href="http://www.coraid.com/" target="_blank">http://www.coraid.com/</a><br>
> ><br>
> > _______________________________________________<br>
> > Ale mailing list<br>
> > <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> > <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> > See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> > <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
> ><br>
> > --<br>
> > --<br>
> > James P. Kinney III<br>
> ><br>
> > Every time you stop a school, you will have to build a jail. What you<br>
> gain at<br>
> > one end you lose at the other. It's like feeding a dog on his own tail.<br>
> It<br>
> > won't fatten the dog.<br>
> > - Speech 11/23/1900 Mark Twain<br>
> ><br>
> > <a href="http://electjimkinney.org" target="_blank">http://electjimkinney.org</a><br>
> > <a href="http://heretothereideas.blogspot.com/" target="_blank">http://heretothereideas.blogspot.com/</a><br>
> ><br>
> > _______________________________________________<br>
> > Ale mailing list<br>
> > <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> > <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> > See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> > <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
><br>
> --<br>
> Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory<br>
> Member, MIT Student Information Processing Board (SIPB)<br>
> URL: <a href="http://web.mit.edu/warlord/" target="_blank">http://web.mit.edu/warlord/</a> PP-ASEL-IA N1NWH<br>
> <a href="mailto:warlord@MIT.EDU">warlord@MIT.EDU</a> PGP key available<br>
><br>
> _______________________________________________<br>
> Ale mailing list<br>
> <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
><br>
> --<br>
> --<br>
> James P. Kinney III<br>
><br>
> Every time you stop a school, you will have to build a jail. What you gain at<br>
> one end you lose at the other. It's like feeding a dog on his own tail. It<br>
> won't fatten the dog.<br>
> - Speech 11/23/1900 Mark Twain<br>
><br>
> <a href="http://electjimkinney.org" target="_blank">http://electjimkinney.org</a><br>
> <a href="http://heretothereideas.blogspot.com/" target="_blank">http://heretothereideas.blogspot.com/</a><br>
><br>
> _______________________________________________<br>
> Ale mailing list<br>
> <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br>
--<br>
Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory<br>
Member, MIT Student Information Processing Board (SIPB)<br>
URL: <a href="http://web.mit.edu/warlord/" target="_blank">http://web.mit.edu/warlord/</a> PP-ASEL-IA N1NWH<br>
<a href="mailto:warlord@MIT.EDU">warlord@MIT.EDU</a> PGP key available<br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>-- <br>James P. Kinney III<br><i><i><i><i><br></i></i></i></i>Every time you stop a school, you will have to build a jail. What you
gain at one end you lose at the other. It's like feeding a dog on his
own tail. It won't fatten the dog.<br>
- Speech 11/23/1900 Mark Twain<br><i><i><i><i><br><a href="http://electjimkinney.org" target="_blank">http://electjimkinney.org</a><br><a href="http://heretothereideas.blogspot.com/" target="_blank">http://heretothereideas.blogspot.com/</a><br>
</i></i></i></i>
</div>