[ale] rsync

[Lightner, Jeff]

On your point 2.

An NFS mount is still transferring data over the network so bandwidth is still a concern.   True enough if you do rsync from local physical drive mount to NFS mount locally you’d get little benefit (and probably some performance loss) from doing compression in rsync.   However, it seems it would be more to the point to suggest doing the rsync to the host that is sharing out the NFS filesystem in such a case.    (Of course point 1 obviates point 2 anyway but I thought it worth mentioning for complete detail.)

From: ale-bounces at ale.org [mailto:ale-bounces at ale.org] On Behalf Of Brian Mathis
Sent: Wednesday, August 28, 2013 12:00 PM
To: Atlanta Linux Enthusiasts
Subject: Re: [ale] rsync

Compression won't help for 2 reasons:
1) Image data is already compressed
2) It sounds like you're syncing between local mounts, in which case the data would be uncompressed before writing to the destination, which is the local mount, which then sends it over the network.  I hope rsync is smart enough to ignore the compression option when syncing between local sources.
Compression is only going to help if you're syncing between 2 separate machines and the data is being transferred over the network by rsync itself -- not when using a network file system.


❧ Brian Mathis

On Wed, Aug 28, 2013 at 9:40 AM, Edward Holcroft <eholcroft at mkainc.com<mailto:eholcroft at mkainc.com>> wrote:
I am rsyncing a many files, mostly photos from an EBS volume to an S3 bucket (locally mounted using FUSE) on AWS.

Given that it's primarily photographic data (jpg), is it still a good idea to use compression (z- option), or will I not gain any real benefit by trying to compress jpegs?

Alternately, if anyone has a better way of getting 600GB of data from EBS to S3, please let me know.

ed

--
Edward Holcroft | Madsen Kneppers & Associates Inc.
3020 Holcomb Bridge Rd. NW | Norcross, GA 30071
O (770) 446-9606<tel:%28770%29%20446-9606> | M (770) 630-0949<tel:%28770%29%20630-0949>

MADSEN, KNEPPERS & ASSOCIATES USA, MKA Canada Inc. WARNING/CONFIDENTIALITY NOTICE: This message may be confidential and/or privileged. If you are not the intended recipient, please notify the sender immediately then delete it - you should not copy or use it for any purpose or disclose its content to any other person. Internet communications are not secure. You should scan this message and any attachments for viruses. Any unauthorized use or interception of this e-mail is illegal.




_______________________________________________
Ale mailing list
Ale at ale.org<mailto:Ale at ale.org>
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo






Athena®, Created for the Cause™

Making a Difference in the Fight Against Breast Cancer





How and Why I Should Support Bottled Water!
Do not relinquish your right to choose bottled water as a healthy alternative to beverages that contain sugar, calories, etc. Your support of bottled water will make a difference! Your signatures count! Go to http://www.bottledwatermatters.org/luv-bottledwater-iframe/dswaters and sign a petition to support your right to always choose bottled water. Help fight federal and state issues, such as bottle deposits (or taxes) and organizations that want to ban the sale of bottled water. Support community curbside recycling programs. Support bottled water as a healthy way to maintain proper hydration. Our goal is 50,000 signatures. Share this petition with your friends and family today!



---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130828/39401b16/attachment.html>