[ale] Parallel data transfers (NFS).

[Jeff Layton]

On 11/06/2014 08:29 AM, JD wrote:
> On 11/06/2014 07:47 AM, Raj Wurttemberg wrote:
>>> The slowdown is likely to be the storage on the dev server or the network.
>>> You have to look at each point in the chain to determine if the current
>>> solution is already flooding any part therein.
>>>
>>> A simple way to cause XX concurrent copies is to spawn them based on first
>>> character in the filename.
>>>
>>> I suspect you were looking for a -j10 switch? Something like GNU
>> "parallel"
>>> perhaps? https://www.gnu.org/software/parallel/
>>>
>>> https://superuser.com/questions/327659/whats-the-best-way-to-perform-
>>> a-parallel-copy-on-unix
>>> has more discussions on this.

Not tooting my own horn but this link might help:

http://www.admin-magazine.com/HPC/Articles/Moving-Your-Data-It-s-Not-Always-Pleasant

It's focused more on HPC and data migration from an old
storage system to a new one, but it might help with some
alternative ideas.

Jeff