[ale] Giant storage system suggestions

[Ron Frazier (ALE)]

Just for kicks, I did some googling on large scale data. These looked interesting.

http://arstechnica.com/business/2012/01/the-big-disk-drive-in-the-sky-how-the-giants-of-the-web-store-big-data/
http://highscalability.com/google-architecture/
http://datasafestorage.wordpress.com/2011/06/10/5-reasons-why-google-uses-tape-for-disaster-recovery/

Sincerely,

Ron





--

Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9 Mail.
Please excuse my potential brevity.

(To whom it may concern. My email address has changed. Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address. Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, you might want to
call on the phone. I get about 300 emails per day from alternate energy
mailing lists and such. I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O) Leave a message.
linuxdude AT techstarship.com


Alex Carver <agcarver+ale at acarver.net> wrote:

I suppose you're right. :) Perhaps if I write it as (Giant Storage) 
System it might be more descriptive of what I'm looking for. It doesn't 
have to be a full-size rack of drives, it just has to have a huge amount 
of disk space available that can grow over time (though eventually it 
would likely fill a full-size rack depending on space consumption).

RAID 10 might be a bit overkill but some type of RAID arrangement was 
certainly in the idea only with a virtual file system spread over the 
whole thing.

On 7/13/2012 17:18, Scott McBrien wrote:
> I feel like a single machine with a pile of drives doesn't qualify as a 'giant storage system' to most of us. Though I suppose if I change the emphasis on what words I pay attention to, I could read this as 'a giant storage machine' ;-)
>
> Please use something like RAID10 on these disks, it'll halve your usable space, but provide a little resiliency.
>
> -Scott
>
> On Jul 13, 2012, at 5:40 PM, Alex Carver <agcarver+ale at acarver.net> wrote:
>
>> On 7/12/2012 00:20, gcs8 wrote:
>>> I built a large zfs pool for my personal use hear at the house, (
>>> gcs8.org/san ) iscsi has good throughput and you can have a small server
>>> take care of sharing it out from there. I use freenas and it has served me
>>> pretty well. I have ~35.6tb after raid z2. The theory for my design was
>>> that if I lose any hardware I can replace it with what ever sense freenas
>>> it taking care of my disks not the hardware. If I could change 2 things
>>> about my setup I would try and get infinaban or at least 10gb eth, and use
>>> a ssd for cacheing.
>>>
>>> Now I can't afford to keep a second one to rsync to but I do use crash
>>> plain to back it up, works fine I have 8.7tb backed up with them right now.
>>> Just my. 02 cents.
>>
>> This is pretty much what I was thinking about (I had even looked at that
>> very Norco enclosure) except for adding the SATA/SAS expander to allow
>> more drives to hang off a single SATA/SAS card. A second or third
>> enclosure plus SATA/SAS cables pulled from the first enclosure to the
>> others (what's the longest cable available? I haven't seen external SAS
>> connectors.) would give me more physical expansion (as needed) but still
>> have only one machine/OS running.
>>_____________________________________________

>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>
>_____________________________________________

> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>


_____________________________________________

Ale mailing list
Ale at ale.org
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20120714/acf1df9e/attachment.html