Nice write-up. Thanks!<br><br><div class="gmail_quote">On Thu, Jan 3, 2013 at 9:00 PM, Ron Frazier (ALE) <span dir="ltr"><<a href="mailto:atllinuxenthinfo@techstarship.com" target="_blank">atllinuxenthinfo@techstarship.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div text="#000000" bgcolor="#ffffff">
<div style="font-size:17px" lang="x-western">Hi
all,<br>
<br>
I recently posted a message about misaligned partitions, to which a
couple of people replied with info about 4k sectors for HDD's. I
decided to post this message with a different subject because this can
have a much broader impact than many people probably know. Some of you
may already know this. Others may not. I haven't purchased any new
drives lately, so I'm just learning about it.<br>
<br>
To reiterate what happened, I replaced a 500 GB hybrid SSD HDD with a
new 750 GB one of newer vintage. I cloned the hard drive using a 3
year old version of Acronis True Image, which had always worked
before. When I brought up the new drive in Ubuntu 11.10 Disk Utility,
for each partition I clicked on, it said the partition boundary was
misaligned, and performance would suffer. Based on the responses to
the other threads, I started doing research.<br>
<br>
I'm not the best at explaining this, so bear with me. See also the
links and quotes below.<br>
<br>
Essentially, all HDD's made after January 2011 are now employing 4096
byte or 4 KB sectors instead of the traditional 512 byte sectors. The
firmware in the drive reports sectors as though they were 512 so legacy
software will work. Also, traditionally, the first partition of an MBR
formatted HDD started on sector 63, after the MBR. So, if the data
from a 512 byte sector drive is cloned onto a 4 KB sector drive, the
partition boundaries will be offset and the data will not align
properly with the new physical media. You might think, so what, the
firmware is translating, no problem. Wrong! As it turns out,
misaligned partitions can lead to write performance which is 10 times
slower.<br>
<br>
There are the equivalent of 8 - 512 byte sectors in 1 - 4 KB sector.<br>
<br>
Let's say you want to read 512 bytes, what your software thinks is 1
sector. It sends the request to the HDD. The firmware finds the
actual 4 KB sector with what you want in it. It then extracts the 512
bytes you want and sends that to you. That works pretty well. Writing
is a whole other matter.<br>
<br>
Here's a quote from the Seagate article below:<br>
<br>
"A write process can be more complicated, particularly when data the
host computer attempts to write is a subset of a physical 4K sector. In
these cases, the hard drive must first read the entire 4K sector
containing the targeted location of the host write request, merge the
existing data with the new data and then rewrite the entire 4K sector.
(On the next revolution.)"<br>
<br>
So, if you're writing 512 bytes, the firmware finds and reads the 4 KB
sector that contains the area you want to write, replaces the 512 bytes
you're writing while retaining the rest, and writes the sector back to
the disk. It has to wait for the platter to spin around again before
it can rewrite the sector. At 7200 RPM, this takes an additional 8.3
mS. Lets say you're writing 1024 bytes and that bridges two 4 KB
sectors. The system has to read both sectors, merge 512 bytes into
one, then 512 bytes into the other, then write them back on the next
revolution. You get the idea.<br>
<br>
The IBM article below indicates that this can lead to a 10 TIMES speed
degradation if doing small writes.<br>
<br>
You fix at least part of the problem by realigning both the start and
end locations of each partition on the disk to 4 KB boundaries. The
standard industry practice seems to be to align them on on 1 MiB
boundaries, which is a multiple of 4 KB.<br>
<br>
So, you want your first partition to start at 1 MiB, which is sector
number 2048 rather than 63. I would leave 1 MiB between partitions,
and make the next partition start and end on 1 MiB boundaries.<br>
<br>
I should note that you don't want to do this with Windows XP from what
I've read. It expects the 1st partition to be at 63, period.<br>
<br>
In my case, I cloned a disk with 512 byte sectors to a disk with 4 K
sectors, but I kept the partition arrangement exactly the same, which
had always worked before. So, everything on the clone ended up being
misaligned.<br>
<br>
Here's why I worded the subject the way I did.<br>
<br>
If you have an SSD, you MUST align the partitions properly for optimum
performance. They have 4K sectors.<br>
<br>
If you have a HDD with 4K sectors, you MUST align it properly for
optimum performance.<br>
<br>
If you have a hybrid SSD HDD with 4K sectors, you MUST align it
properly for optimum performance.<br>
<br>
If you have a hybrid SSD HDD with 512 byte sectors on the platters, you
MOST LIKELY need to align it properly for optimum performance. Very
hard to find any data on this.<br>
<br>
AND, if you have a lowly old HDD with 512 byte sectors, and you EVER
MIGHT clone it to any of the others, you probably need to align it
properly for optimum performance of the drive that you clone it to.<br>
<br>
As was pointed out in the other thread, some cloning software
understands the 4K sectors. The older stuff, including my 3 year old
Acronis True Image, does not. However, I'm pretty sure that if I align
the partitions of an old HDD to 1 MiB boundaries, then do an identical
clone, that should be preserved. If I let it expand the partitions on
the
new HDD, I don't know. Even with new software that understands, you
might have to turn on an option such as align to 4K, align to 1 MiB,
align to target, etc.<br>
<br>
I looked at the manual for Terabyte Unlimited's cloning software and I
may replace Acronis with that. The product looks pretty impressive,
and the manual mentions 4K sectors. I'm not afraid to pay for
something
this complex and critical if I have to. I've heard good things about
Paragon software but don't know anything about it.<br>
<br>
You can check to see if you have an alignment conflict with Disk
Utility (in Ubuntu). I don't know what program that actually runs. It
will only show a message if the drive is partitioned the old way and it
has 4K sectors, apparently. If the drive is partitioned the old way
and you have 512 byte sectors, it won't complain. The version of Disk
Utility in Ubuntu 11.10 does not partition the drive properly by
default.<br>
<br>
Because of the potential problems of having ANY drives around with 512
byte sector alignment (also called cylinder alignment, I think), I've
decided to
repartition all my boot drives and am considering doing the archival
drives. I'm putting everything on 1 MiB boundaries whether the actual
drive has 512 byte or 4K sectors.<br>
<br>
The only way I could get into trouble with that is if I partition a
drive with old software or if I clone a drive with old software and
resize the partitions.<br>
<br>
You can use the latest version of Gparted from <a href="http://gparted.org" target="_blank">gparted.org</a> to
repartition the drive non destructively with data on it. You should
backup the data first. Don't use the one in Ubuntu, for example. It's
old and buggy. Download the ISO for the Gparted live CD and use that
so you have the newest version.<br>
<br>
This lifehacker article tells how to do it.<br>
<br>
<a href="http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance" target="_blank">http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance</a><br>
<br>
It's a little dated. The UI for Gparted has changed slightly. There
is a drop down box for alignment. Set this to MiB. There is no longer
an align to cylinders check box.<br>
<br>
So, here's what I did to all my boot drives.<br>
<br>
For the first partition, I simultaneously reduce the size and move the
boundaries to allow 2 MiB before and after.<br>
<br>
Then, I move the partition to the left by 1 MiB as the article
indicates.<br>
<br>
I then move each other partition that has at least a 2 MiB boundary to
the left to within 1 MiB. If any partition doesn't have at least 2 MiB
to it's left, I first move it to the right up to 2 MiB, then back to 1
MiB. This is due to an anomaly in the way Gparted calculates space, as
mentioned in the article.<br>
<br>
When, I'm done, there is 1 MiB before the first partition, although you
cannot see it on the Gparted drive map. I have 1 MiB unused spaces
between
partitions. And, every partition begins and ends on a 1 MiB boundary.<br>
<br>
This will optimize performance if the drive has 4K sectors, or if it is
ever cloned to a drive that does.<br>
<br>
Doing this, particularly moving the boot partition away from sector 63
will probably break your boot loader. The article talks about how you
can fix a Windows installation with a Windows install disk. If you
don't have one handy, you may not want to try this.<br>
<br>
See the Gparted faq (below) for info on fixing a Linux boot loader.<br>
<br>
Here's how to check if you've finished the job properly using Gparted.<br>
<br>
Click on the first partition, right click, and select information.<br>
<br>
The first sector should be at 2048.<br>
Look at the total sectors. In one case, for me, it's 307,183,616.<br>
Get out an engineering calculator that can handle big numbers. Enter
this in and divide by 2048.<br>
In my case, the answer is 149,992. If this is a whole number, with no
decimal, you're good.<br>
<br>
The unused space between each partition should have 2048 total sectors,
or a multiple of that.<br>
<br>
For each other partition, check that the <first sector> / 2048 is
a whole number<br>
and that the <total sectors> / 2048 is a whole number.<br>
<br>
If all the numbers check out, you have optimized your partition
structure for drives with 4K sectors.<br>
<br>
Below are a number of links and some quotes that I found helpful.<br>
<br>
I hope this info is helpful to you.<br>
<br>
Sincerely,<br>
<br>
Ron<br>
<br>
---------------------------------------<br>
<br>
<a href="http://en.wikipedia.org/wiki/Advanced_Format" target="_blank">http://en.wikipedia.org/wiki/Advanced_Format</a>
<br>
<a href="http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731" target="_blank">http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731</a>
<br>
<a href="http://wdc.custhelp.com/app/answers/detail/a_id/5655" target="_blank">http://wdc.custhelp.com/app/answers/detail/a_id/5655</a>
<br>
<a href="https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues" target="_blank">https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues</a>
(geeky)
<br>
<a href="http://gparted.org/faq.php" target="_blank">http://gparted.org/faq.php</a>
<br>
<a href="http://gparted.org/display-doc.php?name=help-manual" target="_blank">http://gparted.org/display-doc.php?name=help-manual</a>
<br>
<a href="http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem" target="_blank">http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem</a>
<br>
<br>
---------------------------------
<br>
<br>
<a href="http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/" target="_blank">http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/</a>
(good)
<br>
<br>
"As all hard drive manufacturers have agreed to transition to the
Advanced Format sector design by January 2011, the industry must adapt
to and embrace this change to minimize potential negative side effects.
"<br>
<br>
"A write process can be more complicated, particularly when data the
host computer attempts to write is a subset of a physical 4K sector. In
these cases, the hard drive must first read the entire 4K sector
containing the targeted location of the host write request, merge the
existing data with the new data and then rewrite the entire 4K sector.
(On the next revolution.)
"<br>
<br>
"Fdisk: GNU Fdisk is a command line utility that partitions hard
drives.
Versions 1.2.3 and above support Advanced Format drives.
"<br>
<br>
"Parted: GNU Parted is a graphical utility for partitioning hard
drives.
Versions 2.1 and above support Advanced Format drives.
"<br>
<br>
------------------------------
<br>
<br>
<a href="http://www.terabyteunlimited.com/index.htm" target="_blank">http://www.terabyteunlimited.com/index.htm</a>
- cloning software looks cool<br>
<a href="http://goinglinux.com/" target="_blank">http://goinglinux.com/</a>
- Just did a podcast, # 195, on SSD's.<br>
<br>
from prior thread - <a href="http://jrfom.com/2010/05/03/4kb-sectors/" target="_blank">http://jrfom.com/2010/05/03/4kb-sectors/</a>
<br>
from prior thread - <a href="http://lwn.net/Articles/377895/" target="_blank">http://lwn.net/Articles/377895/</a>
<br>
<br>
-----------------------
<br>
<br>
from prior thread - <a href="http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX" target="_blank">http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX</a>
(good) (dated)
<br>
<br>
"Most modern file systems use data structures that are 4096 bytes or
larger in size.
"<br>
<br>
"Writes of misaligned data structures, on the other hand, require the
disk's firmware to first read two sectors, modify portions of both
sectors, and then write two sectors.
<br>
The biggest write performance effects occurred with small file creation
(extracting the kernel tarball). Effects on tarball extraction ranged
from 1.04 (for ext4fs) to 25.53 (for ReiserFS), with a mean of 10.9.
The second-best performer in this test was XFS, with a figure of 1.82.
Because these figures are ratios of unaligned to aligned performance, a
value of 10.9 means that a tarball extraction that takes 10 seconds on
a properly aligned partition takes 109 seconds on an improperly aligned
partition—a huge difference! The XFS figure of 1.82 means that this
10-second operation takes 18.2 seconds on an improperly aligned
partition.
"<br>
<br>
</div>
<pre cols="72">--
(To whom it may concern. My email address has changed. Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address. Please send all personal correspondence to the new address.)
(PS - If you email me and don't get a quick response, you might want to
call on the phone. I get about 300 emails per day from alternate energy
mailing lists and such. I don't always see new email messages very quickly.)
Ron Frazier
<a href="tel:770-205-9422" value="+17702059422" target="_blank">770-205-9422</a> (O) Leave a message.
linuxdude AT <a href="http://techstarship.com" target="_blank">techstarship.com</a>
</pre>
</div>
<br>_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>-- <br>James P. Kinney III<br><i><i><i><i><br></i></i></i></i>Every time you stop a school, you will have to build a jail. What you
gain at one end you lose at the other. It's like feeding a dog on his
own tail. It won't fatten the dog.<br>
- Speech 11/23/1900 Mark Twain<br><i><i><i><i><br><a href="http://electjimkinney.org" target="_blank">http://electjimkinney.org</a><br><a href="http://heretothereideas.blogspot.com/" target="_blank">http://heretothereideas.blogspot.com/</a><br>
</i></i></i></i>