[ale] If you own (almost) ANY HDD, repartition with 4k boundaries

Thu Jan 3 21:25:54 EST 2013

Nice write-up. Thanks!

On Thu, Jan 3, 2013 at 9:00 PM, Ron Frazier (ALE) <
atllinuxenthinfo at techstarship.com> wrote:

> **
> Hi all,
>
> I recently posted a message about misaligned partitions, to which a couple
> of people replied with info about 4k sectors for HDD's.  I decided to post
> this message with a different subject because this can have a much broader
> impact than many people probably know.  Some of you may already know this.
> Others may not.  I haven't purchased any new drives lately, so I'm just
> learning about it.
>
> To reiterate what happened, I replaced a 500 GB hybrid SSD HDD with a new
> 750 GB one of newer vintage.  I cloned the hard drive using a 3 year old
> version of Acronis True Image, which had always worked before.  When I
> brought up the new drive in Ubuntu 11.10 Disk Utility, for each partition I
> clicked on, it said the partition boundary was misaligned, and performance
> would suffer.  Based on the responses to the other threads, I started doing
> research.
>
> I'm not the best at explaining this, so bear with me.  See also the links
> and quotes below.
>
> Essentially, all HDD's made after January 2011 are now employing 4096 byte
> or 4 KB sectors instead of the traditional 512 byte sectors.  The firmware
> in the drive reports sectors as though they were 512 so legacy software
> will work.  Also, traditionally, the first partition of an MBR formatted
> HDD started on sector 63, after the MBR.  So, if the data from a 512 byte
> sector drive is cloned onto a 4 KB sector drive, the partition boundaries
> will be offset and the data will not align properly with the new physical
> media.  You might think, so what, the firmware is translating, no problem.
> Wrong!  As it turns out, misaligned partitions can lead to write
> performance which is 10 times slower.
>
> There are the equivalent of 8 - 512 byte sectors in 1 - 4 KB sector.
>
> Let's say you want to read 512 bytes, what your software thinks is 1
> sector.  It sends the request to the HDD.  The firmware finds the actual 4
> KB sector with what you want in it.  It then extracts the 512 bytes you
> want and sends that to you.  That works pretty well.  Writing is a whole
> other matter.
>
> Here's a quote from the Seagate article below:
>
> "A write process can be more complicated, particularly when data the host
> computer attempts to write is a subset of a physical 4K sector. In these
> cases, the hard drive must first read the entire 4K sector containing the
> targeted location of the host write request, merge the existing data with
> the new data and then rewrite the entire 4K sector. (On the next
> revolution.)"
>
> So, if you're writing 512 bytes, the firmware finds and reads the 4 KB
> sector that contains the area you want to write, replaces the 512 bytes
> you're writing while retaining the rest, and writes the sector back to the
> disk.  It has to wait for the platter to spin around again before it can
> rewrite the sector.  At 7200 RPM, this takes an additional 8.3 mS.  Lets
> say you're writing 1024 bytes and that bridges two 4 KB sectors.  The
> system has to read both sectors, merge 512 bytes into one, then 512 bytes
> into the other, then write them back on the next revolution.  You get the
> idea.
>
> The IBM article below indicates that this can lead to a 10 TIMES speed
> degradation if doing small writes.
>
> You fix at least part of the problem by realigning both the start and end
> locations of each partition on the disk to 4 KB boundaries.  The standard
> industry practice seems to be to align them on on 1 MiB boundaries, which
> is a multiple of 4 KB.
>
> So, you want your first partition to start at 1 MiB, which is sector
> number 2048 rather than 63.  I would leave 1 MiB between partitions, and
> make the next partition start and end on 1 MiB boundaries.
>
> I should note that you don't want to do this with Windows XP from what
> I've read.  It expects the 1st partition to be at 63, period.
>
> In my case, I cloned a disk with 512 byte sectors to a disk with 4 K
> sectors, but I kept the partition arrangement exactly the same, which had
> always worked before.  So, everything on the clone ended up being
> misaligned.
>
> Here's why I worded the subject the way I did.
>
> If you have an SSD, you MUST align the partitions properly for optimum
> performance.  They have 4K sectors.
>
> If you have a HDD with 4K sectors, you MUST align it properly for optimum
> performance.
>
> If you have a hybrid SSD HDD with 4K sectors, you MUST align it properly
> for optimum performance.
>
> If you have a hybrid SSD HDD with 512 byte sectors on the platters, you
> MOST LIKELY need to align it properly for optimum performance.  Very hard
> to find any data on this.
>
> AND, if you have a lowly old HDD with 512 byte sectors, and you EVER MIGHT
> clone it to any of the others, you probably need to align it properly for
> optimum performance of the drive that you clone it to.
>
> As was pointed out in the other thread, some cloning software understands
> the 4K sectors.  The older stuff, including my 3 year old Acronis True
> Image, does not.  However, I'm pretty sure that if I align the partitions
> of an old HDD to 1 MiB boundaries, then do an identical clone, that should
> be preserved.  If I let it expand the partitions on the new HDD, I don't
> know.  Even with new software that understands, you might have to turn on
> an option such as align to 4K, align to 1 MiB, align to target, etc.
>
> I looked at the manual for Terabyte Unlimited's cloning software and I may
> replace Acronis with that.  The product looks pretty impressive, and the
> manual mentions 4K sectors.  I'm not afraid to pay for something this
> complex and critical if I have to.  I've heard good things about Paragon
> software but don't know anything about it.
>
> You can check to see if you have an alignment conflict with Disk Utility
> (in Ubuntu).  I don't know what program that actually runs.  It will only
> show a message if the drive is partitioned the old way and it has 4K
> sectors, apparently.  If the drive is partitioned the old way and you have
> 512 byte sectors, it won't complain.  The version of Disk Utility in Ubuntu
> 11.10 does not partition the drive properly by default.
>
> Because of the potential problems of having ANY drives around with 512
> byte sector alignment (also called cylinder alignment, I think), I've
> decided to repartition all my boot drives and am considering doing the
> archival drives.  I'm putting everything on 1 MiB boundaries whether the
> actual drive has 512 byte or 4K sectors.
>
> The only way I could get into trouble with that is if I partition a drive
> with old software or if I clone a drive with old software and resize the
> partitions.
>
> You can use the latest version of Gparted from gparted.org to repartition
> the drive non destructively with data on it.  You should backup the data
> first.  Don't use the one in Ubuntu, for example.  It's old and buggy.
> Download the ISO for the Gparted live CD and use that so you have the
> newest version.
>
> This lifehacker article tells how to do it.
>
>
> http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance
>
> It's a little dated.  The UI for Gparted has changed slightly.  There is a
> drop down box for alignment.  Set this to MiB.  There is no longer an align
> to cylinders check box.
>
> So, here's what I did to all my boot drives.
>
> For the first partition, I simultaneously reduce the size and move the
> boundaries to allow 2 MiB before and after.
>
> Then, I move the partition to the left by 1 MiB as the article indicates.
>
> I then move each other partition that has at least a 2 MiB boundary to the
> left to within 1 MiB.  If any partition doesn't have at least 2 MiB to it's
> left, I first move it to the right up to 2 MiB, then back to 1 MiB.  This
> is due to an anomaly in the way Gparted calculates space, as mentioned in
> the article.
>
> When, I'm done, there is 1 MiB before the first partition, although you
> cannot see it on the Gparted drive map.  I have 1 MiB unused spaces between
> partitions.  And, every partition begins and ends on a 1 MiB boundary.
>
> This will optimize performance if the drive has 4K sectors, or if it is
> ever cloned to a drive that does.
>
> Doing this, particularly moving the boot partition away from sector 63
> will probably break your boot loader.  The article talks about how you can
> fix a Windows installation with a Windows install disk.  If you don't have
> one handy, you may not want to try this.
>
> See the Gparted faq (below) for info on fixing a Linux boot loader.
>
> Here's how to check if you've finished the job properly using Gparted.
>
> Click on the first partition, right click, and select information.
>
> The first sector should be at 2048.
> Look at the total sectors.  In one case, for me, it's 307,183,616.
> Get out an engineering calculator that can handle big numbers.  Enter this
> in and divide by 2048.
> In my case, the answer is 149,992.  If this is a whole number, with no
> decimal, you're good.
>
> The unused space between each partition should have 2048 total sectors, or
> a multiple of that.
>
> For each other partition, check that the <first sector> / 2048 is a whole
> number
> and that the <total sectors> / 2048 is a whole number.
>
> If all the numbers check out, you have optimized your partition structure
> for drives with 4K sectors.
>
> Below are a number of links and some quotes that I found helpful.
>
> I hope this info is helpful to you.
>
> Sincerely,
>
> Ron
>
> ---------------------------------------
>
> http://en.wikipedia.org/wiki/Advanced_Format
> http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731
> http://wdc.custhelp.com/app/answers/detail/a_id/5655
> https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues (geeky)
> http://gparted.org/faq.php
> http://gparted.org/display-doc.php?name=help-manual
>
> http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem
>
> ---------------------------------
>
>
> http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/(good)
>
> "As all hard drive manufacturers have agreed to transition to the Advanced
> Format sector design by January 2011, the industry must adapt to and
> embrace this change to minimize potential negative side effects. "
>
> "A write process can be more complicated, particularly when data the host
> computer attempts to write is a subset of a physical 4K sector. In these
> cases, the hard drive must first read the entire 4K sector containing the
> targeted location of the host write request, merge the existing data with
> the new data and then rewrite the entire 4K sector. (On the next
> revolution.) "
>
> "Fdisk: GNU Fdisk is a command line utility that partitions hard drives.
> Versions 1.2.3 and above support Advanced Format drives. "
>
> "Parted: GNU Parted is a graphical utility for partitioning hard drives.
> Versions 2.1 and above support Advanced Format drives. "
>
> ------------------------------
>
> http://www.terabyteunlimited.com/index.htm - cloning software looks cool
> http://goinglinux.com/ - Just did a podcast, # 195, on SSD's.
>
> from prior thread - http://jrfom.com/2010/05/03/4kb-sectors/
> from prior thread - http://lwn.net/Articles/377895/
>
> -----------------------
>
> from prior thread -
> http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX(good) (dated)
>
> "Most modern file systems use data structures that are 4096 bytes or
> larger in size. "
>
> "Writes of misaligned data structures, on the other hand, require the
> disk's firmware to first read two sectors, modify portions of both sectors,
> and then write two sectors.
> The biggest write performance effects occurred with small file creation
> (extracting the kernel tarball). Effects on tarball extraction ranged from
> 1.04 (for ext4fs) to 25.53 (for ReiserFS), with a mean of 10.9. The
> second-best performer in this test was XFS, with a figure of 1.82. Because
> these figures are ratios of unaligned to aligned performance, a value of
> 10.9 means that a tarball extraction that takes 10 seconds on a properly
> aligned partition takes 109 seconds on an improperly aligned partition—a
> huge difference! The XFS figure of 1.82 means that this 10-second operation
> takes 18.2 seconds on an improperly aligned partition. "
>
>  --
>
> (To whom it may concern.  My email address has changed.  Replying to former
> messages prior to 03/31/12 with my personal address will go to the wrong
> address.  Please send all personal correspondence to the new address.)
>
> (PS - If you email me and don't get a quick response, you might want to
> call on the phone.  I get about 300 emails per day from alternate energy
> mailing lists and such.  I don't always see new email messages very quickly.)
>
> Ron Frazier770-205-9422 (O)   Leave a message.
> linuxdude AT techstarship.com
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>

-- 
-- 
James P. Kinney III
*
*Every time you stop a school, you will have to build a jail. What you gain
at one end you lose at the other. It's like feeding a dog on his own tail.
It won't fatten the dog.
- Speech 11/23/1900 Mark Twain
*
http://electjimkinney.org
http://heretothereideas.blogspot.com/
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130103/5b92c090/attachment-0001.html>