[ale] Performance issue

[Jim Kinney]

Yep. 6Gbps is the interface. But even at a paltry 100Mbps actual IO to the
rust layer, the 12 disk raid 6 array should _easily_ be able to hit 1Gbps
of data IO plus control bits. The 38 disk array should hit nearly 4Gbps.

The drives are Toshiba, Seagate and HGST. They all are rated for rw in the
230-260 MBps sustained (SATA can only do bursts at those rates) so 1.8 Gbps
actual data to the platters.

I'm expecting a sustained 15Gbps on the smaller array and 48Gbps on the
larger. My hardware limits are at the PCIe bus. All interconnects are rated
for 24Gbps for each quad-channel connector. It really looks like a kernel
issue as there seems to be waits between rw ops.

Yeah. I work in a currently non-standard Linux field. Except that Linux
_is_ what's always used in the HPC, big-data arena. Fun!  ;-)

I don't buy brand name storage arrays due to budget. I've been able to
build out storage for under 50% of their cost (including my time) and get
matching performance (until now).

On Aug 21, 2016 10:04 AM, "DJ-Pfulio" <DJPfulio at jdpfu.com> wrote:

> On 08/20/2016 10:00 PM, Jim Kinney wrote:
> > 6Gbps SAS. 12 in one array and 38 in another. It should saturate the bus.
>
> 6Gbps is the interface speed. No spinning disks can push that much data
> to my knowledge - even SAS - without SSD caching/hybrids. Even then,
> 2Gbps would be my highest guess at the real-world performance (probably
> much lower in reality).
>
> http://www.tomsitpro.com/articles/best-enterprise-hard-drives,2-981.html
>
> You work in a highly specialized area, but most places would avoid
> striping more than 8 devices for maintainability considerations. Larger
> stripes don't provide much more throughput and greatly increase issues
> when something bad happens.  In most companies I've worked, 4 disk
> stripes were used as the default since it provides 80% of the
> theoretical performance gains that any striping can offer.  That was the
> theory at the time.
>
> Plus many non-cheap arrays will have RAM for caching which can limit
> actual disks being touched. Since you didn't mention EMC/Netapp/HDS, I
> assumed those weren't being used.
>
> Of course, enterprise SSDs changed all this, but would be cost
> prohibitive at the sizes you've described (for most projects).  I do
> know a few companies which run all their internal VMs on RAID10 SSDs and
> would never go back. They aren't doing "big data."
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20160821/c4f973d3/attachment.html>