[ale] ZFS on Linux

Mon Apr 1 08:38:37 EDT 2013

That sucks!

I will need to look closely at the filesystem tool chain. Maybe there's a
new flag to disable the cache to unstick a file delete.

On Mon, Apr 1, 2013 at 7:14 AM, Wolf Halton <wolf.halton at gmail.com> wrote:

> A gotcha on ZFS:
> If it ever fills up a drive array, for instance the backup process starts
> writing recursively, you can't remove anything, since it uses disk space to
> write a cache of what you delete.  You have to copy the files to a larger
> array and reformat your stuffed ZFS array.  Ask me how I know?
>
>
> On Sat, Mar 30, 2013 at 1:49 AM, gcs8 <gcsviii at gmail.com> wrote:
>
>> David, my whole rack takes under 6 amps to power, I am pretty good at
>> hardware design..
>>
>>
>> On Sat, Mar 30, 2013 at 12:11 AM, Jim Kinney <jim.kinney at gmail.com>wrote:
>>
>>> Heh, heh.
>>>
>>> Informatics research eats drive space like it's free.
>>>
>>> You should see the nifty new tape system that UPS delivered today.
>>>
>>> dual LTO6 with 48 slots. It will be managed by a head end connected over
>>> 10G ethernet to second server with 2 arrays (1 has 15 2TB drive and the
>>> other is 12 2TB drives.)
>>>
>>> The 16 4TB drive system actually has dual 2.5 SAS drives for OS drives
>>> plus the main 16 and a honkin' LTO6 tape system with 24 slots. This
>>> connected over a dedicated 10G ethernet to the main number cruncher with
>>> serious CPU and RAM and x16 2TB drives. There's an option to add an
>>> additional 12 drives in the back of the server.
>>>
>>> http://www.supermicro.com/products/chassis/3U/837/SC837E16-RJBOD1.cfm
>>>
>>>
>>> On Fri, Mar 29, 2013 at 10:41 PM, dev null zero two <
>>> dev.null.02 at gmail.com> wrote:
>>>
>>>> rainbow tables ofc :-)
>>>>
>>>> what sort of case are you using for that many hdds?
>>>>
>>>> Sent from my mobile. Please excuse the brevity, spelling, and
>>>> punctuation.
>>>> On Mar 29, 2013 10:36 PM, "David Tomaschik" <david at systemoverlord.com>
>>>> wrote:
>>>>
>>>>> I can only imagine the power bill for 20-30 hard drives & all the
>>>>> controllers and CPU needed to spin that all up.  (And unfortunately, I have
>>>>> good guesses as to what its used for.)
>>>>>
>>>>>
>>>>> On Fri, Mar 29, 2013 at 3:13 PM, gcs8 <gcsviii at gmail.com> wrote:
>>>>>
>>>>>> i run raidz2, if i could chang anything it would be adding SSD
>>>>>> caching and dual SSD ZIL.
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 29, 2013 at 5:07 PM, Jim Kinney <jim.kinney at gmail.com>wrote:
>>>>>>
>>>>>>> That sounds encouraging. I'll have probably 2 arrays. One with 15
>>>>>>> 1TB data and a 2nd with 16 4TB SAS2. thinking raidz with a spare drive for
>>>>>>> each.
>>>>>>>  On Mar 29, 2013 4:43 PM, "gcs8" <gcsviii at gmail.com> wrote:
>>>>>>>
>>>>>>>> I run a 20 2TB disk array at home and have rather fond of ZFS, its
>>>>>>>> been stable for me for ~3 years now.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 29, 2013 at 3:10 PM, Ed Cashin <ecashin at noserose.net>wrote:
>>>>>>>>
>>>>>>>>> Thanks in advance for the notes you take.  I'm interested to hear
>>>>>>>>> what you find.
>>>>>>>>>
>>>>>>>>> By the way, work sent me to a Nexenta course in 2010 where I
>>>>>>>>> learned ZFS stuff from Richard Elling:
>>>>>>>>>
>>>>>>>>>   http://www.richardelling.com/
>>>>>>>>>
>>>>>>>>> ... and one thing that kept coming up is how users usually reach
>>>>>>>>> for the RAID-Z options when stripes of mirrors would perform better and
>>>>>>>>> offer more flexibility.  I thought that was notable.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Mar 29, 2013 at 2:44 PM, Jim Kinney <jim.kinney at gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> There was an update to this that just hit /.
>>>>>>>>>>
>>>>>>>>>> WooHoo!
>>>>>>>>>>
>>>>>>>>>> I have not used ZFS (minimal solaris experience) before but I
>>>>>>>>>> have a situation where raidz looks like a very good choice. An external 16
>>>>>>>>>> bay JBOD box hanging off a SAS HBA looks like a prime candidate. The big
>>>>>>>>>> issue for me is the support for native NFSv4 ACLs which solves some issues.
>>>>>>>>>>
>>>>>>>>>> As I go through this exercise, I'll keep some notes and maybe
>>>>>>>>>> look at a presentation on doing this.
>>>>>>>>>>
>>>>>>>>>> hints, gotcha's, are very welcome.
>>>>>>>>>>
>>>>>>>>>> I did find a pdf of the zfsadmin.pdf
>>>>>>>>>> http://pdfhome.org/zfsadmin.pdf
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> --
>>>>>>>>>> James P. Kinney III
>>>>>>>>>> *
>>>>>>>>>> *Every time you stop a school, you will have to build a jail.
>>>>>>>>>> What you gain at one end you lose at the other. It's like feeding a dog on
>>>>>>>>>> his own tail. It won't fatten the dog.
>>>>>>>>>> - Speech 11/23/1900 Mark Twain
>>>>>>>>>> *
>>>>>>>>>> http://electjimkinney.org
>>>>>>>>>> http://heretothereideas.blogspot.com/
>>>>>>>>>> *
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Ale mailing list
>>>>>>>>>> Ale at ale.org
>>>>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>   Ed Cashin <ecashin at noserose.net>
>>>>>>>>>   http://noserose.net/e/
>>>>>>>>>   http://www.coraid.com/
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Ale mailing list
>>>>>>>>> Ale at ale.org
>>>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Charles Selfridge
>>>>>>>>
>>>>>>>> PBYC  IT director
>>>>>>>>
>>>>>>>> (404) 910-3409
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Ale mailing list
>>>>>>>> Ale at ale.org
>>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ale mailing list
>>>>>>> Ale at ale.org
>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Charles Selfridge
>>>>>>
>>>>>> PBYC  IT director
>>>>>>
>>>>>> (404) 910-3409
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ale mailing list
>>>>>> Ale at ale.org
>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> David Tomaschik
>>>>> OpenPGP: 0x5DEA789B
>>>>> http://systemoverlord.com
>>>>> david at systemoverlord.com
>>>>>
>>>>> _______________________________________________
>>>>> Ale mailing list
>>>>> Ale at ale.org
>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>> http://mail.ale.org/mailman/listinfo
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Ale mailing list
>>>> Ale at ale.org
>>>> http://mail.ale.org/mailman/listinfo/ale
>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>> http://mail.ale.org/mailman/listinfo
>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> James P. Kinney III
>>> *
>>> *Every time you stop a school, you will have to build a jail. What you
>>> gain at one end you lose at the other. It's like feeding a dog on his own
>>> tail. It won't fatten the dog.
>>> - Speech 11/23/1900 Mark Twain
>>> *
>>> http://electjimkinney.org
>>> http://heretothereideas.blogspot.com/
>>> *
>>>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>>>
>>>
>>
>>
>> --
>> Charles Selfridge
>>
>> PBYC  IT director
>>
>> (404) 910-3409
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>>
>
>
> --
> Wolf Halton
> This Apt Has Super Cow Powers - http://sourcefreedom.com
> Open-Source Software in Libraries - http://FOSS4Lib.org
> Advancing Libraries Together - http://LYRASIS.org
> Apache Open Office Developer wolfhalton at apache.org
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>

-- 
-- 
James P. Kinney III
*
*Every time you stop a school, you will have to build a jail. What you gain
at one end you lose at the other. It's like feeding a dog on his own tail.
It won't fatten the dog.
- Speech 11/23/1900 Mark Twain
*
http://electjimkinney.org
http://heretothereideas.blogspot.com/
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130401/82ad710c/attachment-0001.html>