<p dir="ltr">Interesting. Unfortunately, btrfs isn't suitable for my workload due to performance issues, and so I tend to only use it when I need to treat files in ways which btrfs uniquely boosts performance. Sadly, it doesn't work (well) for anything that depends on SQLite or heavy fsync usage unless you disable the fsync call (such as by using libeatmydata, which converts fsync into a no-op or by using a similar approach but rate-limiting calls to fsync to be less frequent).</p>
<p dir="ltr">Dpkg is particularly bad about that; a system using it can take tens of times longer on btrfs vs. ext4. Mozilla apps, too, but not as bad. </p>
<p dir="ltr">I think I remember reading something about a recent commit to change that, but the kernel I am presently using is pretty old son won't try again probably until F18,depending on what kernel it has. </p>
<div class="gmail_quote">On Nov 4, 2012 1:48 PM, "Phil Turmel" <<a href="mailto:philip@turmel.org">philip@turmel.org</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 11/04/2012 12:38 PM, Michael Trausch wrote:<br>
> So I had an interesting few days... Aside from the fact that I have been<br>
> sick, it turns out I have had an interesting problem appear.<br>
><br>
> I changed motherboards recently, to test UEFI and so forth out. When I did<br>
> so I started having some problems that traditionally scream "memory<br>
> errors", except my RAM was just fine.<br>
><br>
> I hadn't immediately thought to check the drive's SMART log because I am<br>
> used to distributions signaling via the UI when such events happen. Well,<br>
> it turns out that Fedora doesn't do smart monitoring by default!<br>
><br>
> I had an apparently bad SATA cable (am running tests now to see if the new<br>
> cable is actually the solution here). The symptom was UDMA CRC error counts<br>
> through the roof, which the drive detected and then aborted the<br>
> corresponding command.<br>
><br>
> I mention this as we recently had a thread on silent corruption.<br>
><br>
> So, to the question part: even with smartctl and friends not installed and<br>
> running, shouldn't modern file systems be storing checksums to catch this<br>
> sort of thing without obscure errors? I thought that ext4 had such support,<br>
> but I would appear to be incorrect there.<br>
<br>
Btrfs has content checksums. Ext4 has experimental journal checksums,<br>
but that has been the subject of recent bugs, and is not yet recommended<br>
for production.<br>
<br>
The key issue is that much of the efficiency gains in modern I/O systems<br>
is based upon buffering / implicit write-back cacheing, where multiple<br>
small writes to the same sector of a file coalesce into a single, later,<br>
actual write. Since many applications depend on this for performance,<br>
it cannot be disabled by default. Filesystems that attempt to generate<br>
checksums between those writes must either abort them when a subsequent<br>
write comes, or keep multiple versions of the sector in memory. Either<br>
way, the checksum must then be written to the inodes in a way that<br>
synchronizes with the actual sector write itself.<br>
<br>
Btrfs can maintain synchronization because it doesn't rewrite in<br>
place--it always allocates new space for rewritten sectors, eventually<br>
garbage-collecting the superceded ones. For filesystems that rewrite<br>
file contents in place, I haven't yet seen a solution. I'm not entirely<br>
sure there is one, at a usable level of performance.<br>
<br>
Note that btrfs has a mount option, "nodatacow", to disable data<br>
copy-on-write for performance reasons on certain applications, like<br>
large database files. This also disables checksums, as the FS can no<br>
longer ensure synchronization.<br>
<br>
HTH,<br>
<br>
Phil<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</blockquote></div>