Extremely slow copy from source to target

MDusek · Guest

Hello,
I am comparing two large folders at RAID6 storage under Windows 10, using the latest Viceversa demo. Size of each folder is about 3TB and both contain about 4,000,000 files. The comparison works well. Then I start augmentation of the source in order to copy about 60,000 new files from the source to the target. Several first files are copied at normal speed, then the speed drops to about 150 kbps and time estimation is 150 hours.
I would be grateful for any hint of what might be wrong.
Best wishes,
Michal

TGRMN Software · Site Admin Joined: 10 Jan 2005 Posts: 8684

Hi,

Is ViceVersa copying very small files when the speed drops?

If you have not yet, try the option Use Windows native file copy in profile settings -> performance
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

Guest · Posted: Thu Feb 27, 2020 7:57 am Post subject:

I tried the native copy already and it did not help. The files are between 0.25 and 1.0 MB, and copy through file manager (Altap Salamander) goes by normal speed.

Alice · Joined: 29 Jan 2015 Posts: 262

My experiences here, which might trigger a thought.

1) is there a backup/snapshot running on your Destination server at the time of transfer that will cripple the transfer speed.

2) where I have a very large number of small files to copy/move its much quicker and more efficient to .ZIP them at Source (if you have the disk space use 'Compression Level : Store' for speed), then VV that .ZIP file to the Destination and unzip. Finally run your VV Augment Job at Destination.

3) try running multiple VV Jobs simultaneously, see here :

Guest · Posted: Thu Feb 27, 2020 11:34 am Post subject:

Thank you very much for your replies.
ad 1) There is no backup running on the destination
ad 2) The source and destination are on the same computer, RAID6 storage. Copying the files by file manager takes 15 minutes. The same operation through ViceVersa estimates 150 hours, one 0.25 MB files is copied several seconds!! This is why I think the problem is caused by a bug in the VV, maybe because of an extremely high (4,000,000) number of files in both folders.
ad 3) In light what I have written above, this is not relevant.

I will try the same job with different folders to see if the problem can be repeated.

Guest · Posted: Thu Feb 27, 2020 8:29 pm Post subject:

OK, I have tried again and everything was running at normal speed. The only difference that I have registered the software before the test but this should not be the reason. So I move the case to unresolved mysteries ...