<p dir="ltr">I need to check on both of the expanders. I recall LSI soc but that was a while ago.</p>
<p dir="ltr">Sector and block are complicated. Wound up using 512k at the raid level and multiples for lvm and xfs. In the mix is Luks and it seems to be agnostic.</p>
<p dir="ltr">I do have a partition I let the system decide block size on and it also is crap.</p>
<p dir="ltr">What the systems (yes. There are two) appear to do is read a block, wait, write a block, wait, read, wait, write, wait... If there was a bad sector alignment clash, I would expect constant but slow rw.</p>
<div class="gmail_extra"><br><div class="gmail_quote">On Aug 21, 2016 8:50 PM, "LnxGnome" <<a href="mailto:lnxgnome@hopnet.net">lnxgnome@hopnet.net</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Math says you should get your 12Gbps+ out of your 40TB array, and
35Gpbs+ from the 104TB array, assuming 7.2K drives maxing at 170MB/s
internal transfer.<br>
<br>
What kind of expanders are you using?<br>
Sector/block alignment? IRQ conflicts?<br>
<br>
What's the single drive performance like? other RAID configs?<br>
<br>
<div>On 8/20/16 5:16 PM, Jim Kinney wrote:<br>
</div>
<blockquote type="cite">All,<br>
<br>
Getting DISMAL performance from new hardware.<br>
<br>
Twin 6-core Xeon E5-2630L v.2 @2.40 GHz<br>
128GB DDR4 RAM<br>
LSI megaraid 2108 in a PCIe v3 slot<br>
Onboard LSI 3108<br>
Drives are all 6Gbps SAS2 4TB<br>
2 arrays, RAID 6, one is 40Tb, other 104TB<br>
<br>
Everything hardware says I should expect a minimum through put of
12Gbps on either array.<br>
<br>
The max I'm getting from iotop is 784 Mbps.<br>
<br>
W. T. F!!!<br>
<br>
According to top, the system is basically idle. Ditto from iostat
and every other tool I check. I'm doing a 'cp -a' from/to same
array.<br>
<br>
Yes. New location is Luks encrypted. That process is totally
asleep it's getting so little work. The rng is busy (dev/random is
not running out (watch -n 1 cat
/proc/sys/kernel/random/<wbr>entropy_avail was always above 2048)) but
not overloaded or asleep.<br>
<br>
Tested going unencrypted to unencrypted with similar performance.<br>
<br>
Changed tuned-adm to latency-performance from balanced and the
read portion of the cp went to 0 bps for 20 seconds until I
switched it back to balanced mode. So that was a fail. It improved
greatly using throughout-performance mode (bursts of 250MB/s) but
was still showing long periods of 0 writes. Load went up from a
paltry 8 to 22 so it's working more.<br>
<br>
But top shows wa values across multiple cores hitting 100 so it's
hitting a wall somewhere.<br>
<br>
Any ideas of more places to look?<br>
<br>
<br>
<br>
<br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my
brevity.
<br>
<fieldset></fieldset>
<br>
<pre>______________________________<wbr>_________________
Ale mailing list
<a href="mailto:Ale@ale.org" target="_blank">Ale@ale.org</a>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/<wbr>listinfo/ale</a>
See JOBS, ANNOUNCE and SCHOOLS lists at
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/<wbr>listinfo</a>
</pre>
</blockquote>
<br>
</div>
<br>______________________________<wbr>_________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" rel="noreferrer" target="_blank">http://mail.ale.org/mailman/<wbr>listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" rel="noreferrer" target="_blank">http://mail.ale.org/mailman/<wbr>listinfo</a><br>
<br></blockquote></div></div>