Forum Index  ViceVersa HOME         FAQ and Knowledge Base

 FAQForum FAQ   SearchSearch Forum  RegisterRegister 
 ProfileProfile   Log inLog in 

Multi-thread comparing
Goto page 1, 2  Next
 
Post new topic   Reply to topic     Forum Index -> Wish List
Author Message
Guest






PostPosted: Wed Dec 12, 2007 10:54 pm    Post subject: Multi-thread comparing Reply with quote

Start the 'indexing process' for the target and the source at the same time, probably using multiple threads. Now it takes too long to fetch the file structure of the first hard disk, and only when it's finished from the second.
Back to top
TGRMN Software
Site Admin


Joined: 10 Jan 2005
Posts: 8672

PostPosted: Thu Dec 13, 2007 8:42 am    Post subject: Reply with quote

This is a very good point. We'll check feasibility for one of the future releases, thanks for the input!
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com
Back to top
Adequat



Joined: 05 Jul 2006
Posts: 65

PostPosted: Fri Apr 11, 2008 8:19 am    Post subject: Reply with quote

This espescially makes sense if source and targets are on different physical drives, then we can expect double speed.
Back to top
Guest






PostPosted: Fri Oct 17, 2008 11:30 am    Post subject: Reply with quote

The option for multi-threaded file-copy operations would also be of much use.

Especially when copying large files to a target network share over CIFS over gigabit ethernet for instance.

Doing a single-threaded copy usually results in speeds up between 200-300Mbit while the disks at the serving and receiving end can do alot more.

Doing multiple copy threads simultaneously usually gives greater overall network utilization resulting in a smaller timeframe needed to restore large backups.
Back to top
Guest






PostPosted: Mon Apr 27, 2009 3:51 pm    Post subject: Reply with quote

It's been a while since I proposed this feature. I am still a grateful customer of your product, but every time I click the compare button it annoys me that it could have finished in half the time.

Are you making any headway on this?
Back to top
malega



Joined: 11 Mar 2006
Posts: 64

PostPosted: Thu Jun 11, 2009 12:56 am    Post subject: Speed Reply with quote

I've also been a long time user and like the product, but I am now getting frustrated with the failure to address this speed issue. It's been years since a major update of the software.

My files have increased significantly in size and it's getting to the point that VV2 is too slow in comparing. They have been promising for at least 2 years to address this issue, but they have not done so.

I am beginning to look for similar software that does not have this speed issue.
Back to top
TGRMN Software
Site Admin


Joined: 10 Jan 2005
Posts: 8672

PostPosted: Fri Jun 12, 2009 6:23 am    Post subject: Reply with quote

Hi, are you comparing files over the internet / VPN ? Is that causing the delay or actually copying the file? thanks
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com
Back to top
tedge
Guest





PostPosted: Fri Jan 15, 2010 3:04 pm    Post subject: comparison and CRC check Reply with quote

I understand what is meant by the time it take to compare. I use ViceVersa to move large amounts of data across a network as my storage capacity increases. As we speak, I am moving a 760gb directory between to NAS, it takes several minutes to index the source, then several to index the target if it is similar sized when it could be doing them at the same time...todays processors have multiple cores and handle multiple threads easily, a single threaded function comparing multiple targets is not staying with the times. I understand this issue but it is not the largest for me, as I said, I am moving very large directories regularly, when you have 760gb worth of data that must be moved, and moved correctly which means using CRC to check the data. Doing a CRC check on a 8gb file does not take much processor power but it does take a lot of time, stopping the transfer after each file to do the CRC check adds TONS of time to the transfer. There is plenty of processor to spare to continue the transfer while the CRC check completes and your software could re-que the transfer if the CRC checked invalid. That would greatly improve transfer times.
Back to top
gleep



Joined: 17 Feb 2010
Posts: 1

PostPosted: Wed Feb 17, 2010 2:34 pm    Post subject: Reply with quote

It is not so much about comparing as it is about the indexing that is done before the actual comparing begins. Either on the network or on a hard disk it takes time to retrieve the directory structure and file attributes, which are fed to the comparing algorithm. When you press 'Compare' it starts doing this, i.e. when it says 'Collecting File Information...'

If it would collect the file information from the two sources at the same time, this should cut the time needed for the whole process almost in half, as the file collection usually takes the most time when the files are small in contrast to comparing the attributes which is efficiently programmed. Processor speed would not be a problem, fetching the data is more a physical bottleneck of the source and the target device.

I don't quite understand why the first reply by the developers (second post) deems this as a 'very good point', only to give a new reply a year and a half later on Jun 12, 2009 which gives away that they don't really get the problem. Of course it must be difficult to fix, as I proposed this more than two years ago.
Back to top
onlyasking



Joined: 12 Dec 2008
Posts: 49

PostPosted: Wed May 12, 2010 7:29 pm    Post subject: Reply with quote

i wonder why they wont address this?
Back to top
TGRMN Software
Site Admin


Joined: 10 Jan 2005
Posts: 8672

PostPosted: Thu May 13, 2010 3:00 am    Post subject: Reply with quote

Hello

yes, collecting source and target info in parallel could be very beneficial in some cases. It would not really help to speed up collecting remote data over VPN.
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com
Back to top
JuggalotusHeat



Joined: 29 Oct 2011
Posts: 2

PostPosted: Wed Nov 02, 2011 4:37 am    Post subject: Reply with quote

So is this going to be worked in or not? If multi-connections/multi-threading was implemented it would speed up everything. Right now it takes forever and a day for my email backups to run because it does it on a per-file basis. If it gave me the option to have 10 connections at once I could max my pipe and get them pushed ASAP. Will this ever be implemented?
Back to top
wmfoster2001



Joined: 23 Jun 2012
Posts: 1

PostPosted: Sat Jun 23, 2012 12:48 pm    Post subject: This is why we didn't buy the program at work. Reply with quote

Comparing two 2Tb of small files on shares on different SANs is just far too slow.

If the ability to scan both trees at the same time had been included we would have purchased the software as the visual comparison feature is quite useful too us.

Regards,
Michael.
Back to top
Guest






PostPosted: Wed Jun 09, 2021 6:18 pm    Post subject: Reply with quote

This still seems like a good idea so I thought I'd give it a nudge now there's some work going into version 3.

If the source and target are on different drives, please can you start the source and target scans in different threads? Sometimes a full CRC compare of a pair of drives can take me more than a day, so it would be nice to have a 2x speed improvement reading both drives at the same time rather than alternating between them. The directory read is fine for me, it's the CRC that takes the time.

Thanks!
Back to top
TGRMN Software
Site Admin


Joined: 10 Jan 2005
Posts: 8672

PostPosted: Mon Jun 14, 2021 10:11 am    Post subject: Reply with quote

Hi, thank you for the feedback, we are looking into this, to see if the target can be prefetched and if CRC comparison can be done at the same time for multiple files during the comparison phase.
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com
Back to top
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> Wish List All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © phpBB Group
Copyright © TGRMN Software. TGRMN Software products: