<html><head></head><body>Have you checked the raid array health? A raid 5 with a bad drive will CRAWL.<br><br><div class="gmail_quote">On April 22, 2015 3:08:39 PM EDT, Edward Holcroft <eholcroft@mkainc.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default">This is what I got local drive to local drive test (no USB involved). This seems impossibly slow.</div><div class="gmail_default"><br /></div><div class="gmail_default">StartTime 1429713308.46 (Wed Apr 22 10:35:08 2015)</div><div class="gmail_default">EndTime 1429728281.54 (Wed Apr 22 14:44:41 2015)</div><div class="gmail_default">ElapsedTime 14973.08 (4 hours 9 minutes 33.08 seconds)</div><div class="gmail_default">SourceFiles 52736</div><div class="gmail_default">SourceFileSize 98274663358 (91.5 GB)</div><div class="gmail_default">NewFiles 52736</div><div class="gmail_default">NewFileSize 98274663358 (91.5 GB)</div><div class="gmail_default">DeletedFiles 0</div><div class="gmail_default">ChangedFiles 0</div><div class="gmail_default">ChangedFileSize 0 (0 bytes)</div><div class="gmail_default">ChangedDeltaSize 0 (0 bytes)</div><div class="gmail_default">DeltaEntries 52736</div><div
class="gmail_default">RawDeltaSize 98251115454 (91.5 GB)</div><div class="gmail_default">TotalDestinationSizeChange 92760516730 (86.4 GB)</div><div class="gmail_default">Errors 0</div><div><br /></div></div></div><div class="gmail_extra"><br /><div class="gmail_quote">On Wed, Apr 22, 2015 at 9:53 AM, James Sumners <span dir="ltr"><<a href="mailto:james.sumners@gmail.com" target="_blank">james.sumners@gmail.com</a>></span> wrote:<br /><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Try it to another internal volume or across your network. After discovering my oversight yesterday, I ran a backup to pickup the missed data. This was the result:<div><br /></div><div>````</div><div><div>Import of duplicity.backends.dpbxbackend Failed: No module named dropbox</div><div>Reading globbing filelist ./excludes</div><div>Local and Remote metadata are synchronized, no sync needed.</div><div>Last full backup date: We
d Apr
15 15:18:59 2015</div><div>--------------[ Backup Statistics ]--------------</div><div>StartTime 1429648992.82 (Tue Apr 21 16:43:12 2015)</div><div>EndTime 1429649857.73 (Tue Apr 21 16:57:37 2015)</div><div>ElapsedTime 864.91 (14 minutes 24.91 seconds)</div><div>SourceFiles 192730</div><div>SourceFileSize 9120190895 (8.49 GB)</div><div>NewFiles 112321</div><div>NewFileSize <a href="tel:6508225863" value="+16508225863" target="_blank">6508225863</a> (6.06 GB)</div><div>DeletedFiles 4</div><div>ChangedFiles 8</div><div>ChangedFileSize 13665 (13.3 KB)</div><div>ChangedDeltaSize 0 (0 bytes)</div><div>DeltaEntries 112333</div><div>RawDeltaSize 6410062775 (5.97 GB)</div><div>TotalDestinationSizeChange 4983679911 (4.64 GB)</div><div>Errors 0</div><div>-------------------------------------------------</div></div><div>````</div><div><br /></div><div>Notice that I added almost 6GB to my backup set and it only took 14.5 minutes to run the backup. This is from a VM to a physical server a
cross
my network via CIFS.</div><div><br /></div><div>Duplicity will use the bandwidth available to it. Unless you renice it, duplicity runs full tilt and will impact performance until it is done. There's something else going on with your setup.<br /><div class="gmail_extra"><span class=""><br /><div class="gmail_quote">On Wed, Apr 22, 2015 at 8:36 AM, Edward Holcroft <span dir="ltr"><<a href="mailto:eholcroft@mkainc.com" target="_blank">eholcroft@mkainc.com</a>></span> wrote:<br /><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div style="font-size:small">I tested this on my production box with all the identical pieces of kit. A full rsync of the 1.5 TB or so of data on this server takes maybe 10 -11 hours. A full backup using Duplicity to the same USB drive on the same server was less than half that - I gave up on 26% after some 16 hours. I&
#39;d
expect some added overhead for the encrypting and tarring that's going on, but this is nuts. I have to be doign something wrong or nobody would be using duplicity. Here's the command I use, which is pretty stock as far as I know:</div><div style="font-size:small"><br /></div><div>duplicity /data --exclude /data/ret --exclude /data/rsynclogs file:///mnt/daily/MONDAY >> /data/rsynclogs/daily-`date +%Y%m%d_%H%M%S`.log<br /></div><div><br /></div><div>I also tried with --no-encyrption and dropped the logging with no noticeable difference.</div><div><br /></div><div>ed</div></div><div class="gmail_extra"><br /></div></blockquote></div></span><span class="">-- <br /><div><div dir="ltr"><div><div dir="ltr"><div>James Sumners<br /><a href="http://james.sumners.info/" target="_blank">http://james.sumners.info/</a> (technical profile)</div><div><a href="http://jrfom.com/" target="_blank">http://jrfom.com/</a> (personal site)</div><div><a href="http://haplo.bandcamp.com/"
target="_blank">http://haplo.bandcamp.com/</a> (band page)</div></div></div></div></div>
</span></div></div></div>
<br />_______________________________________________<br />
Ale mailing list<br />
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br />
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br />
See JOBS, ANNOUNCE and SCHOOLS lists at<br />
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br />
<br /></blockquote></div><br /><br clear="all" /><div><br /></div>-- <br /><div class="gmail_signature"><div dir="ltr">Edward Holcroft | Madsen Kneppers & Associates Inc.<br />11695 Johns Creek Parkway, Suite 250 | Johns Creek, GA 30097<br />O (770) 446-9606 | M (770) 630-0949</div></div>
</div>
<br />
<span style="font-family:arial"><font size="2">MADSEN, KNEPPERS & ASSOCIATES USA, MKA Canada Inc. WARNING/CONFIDENTIALITY NOTICE: This message may be confidential and/or privileged. If you are not the intended recipient, please notify the sender immediately then delete it - you should not copy or use it for any purpose or disclose its content to any other person. Internet communications are not secure. You should scan this message and any attachments for viruses. Any unauthorized use or interception of this e-mail is illegal.</font></span><p style="margin-top: 2.5em; margin-bottom: 1em; border-bottom: 1px solid #000"></p><pre class="k9mail"><hr /><br />Ale mailing list<br />Ale@ale.org<br /><a href="http://mail.ale.org/mailman/listinfo/ale">http://mail.ale.org/mailman/listinfo/ale</a><br />See JOBS, ANNOUNCE and SCHOOLS lists at<br /><a href="http://mail.ale.org/mailman/listinfo">http://mail.ale.org/mailman/listinfo</a><br /></pre></blockquote></div><br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>