The Turnstone's Bill

DropSync Gets Some PXListView Awesomeness

One of the DropSync’s nicest features is it’s Activity window. The idea is that by opening this window you can look up a complete history of what changes have been made for any of the folders or files under DropSync’s control. In the shipping version of DropSync this mostly works, except when you start to get lots and lots of items in your history. Actually, the number of items where this performance hit kicks in is embarrassingly small.

For quite some time now, this has been a problem that’s nagged me. Recently though, I started implementing automated syncing for DropSync, which of course makes it easy to generate thousands of history entries. This mean’t that the performance limits of my Activity window went from being annoying to critical.

Originally I had implemented the Activity window as an NSCollectionView. I did this, partly because it was easy to do, but also because NSCollectionView provides really nice animations for free. Unfortunately though, it doesn’t scale well to large numbers views, and it also doesn’t play that nicely with core-data backed NSArrayControllers.

Eventually I discovered a much better solution, PXListView by Alex Rozanski which works much like UITableView on iOS. Alex explains the motivation behind PXListView really nicely in this blog post. In summary, the key to the performance gain comes from re-using views and keeping only the minimum number required for display in memory at any one time.

In my case, switching to PXListView has meant that I can now realistically handle hundreds of thousands of history entries instead of just a few hundred. I was able to keep all the benefits of having a view based list, including binding the controls in my view items to corresponding properties of objects in an NSArrayController.

Thanks very much Alex Rozanski for contributing this great bit of code to the community.

Fixing the Ghost File Problem

During my last round of beta testing with DropSync I got a bug report where error messages were being returned by rsync about non existent files. An example error (with filenames anonymised) was as follows;

Error: mkstemp "/Volumes/Folder/..afilename.mBhoxf" failed: No such file or directory (2)

After a bit of googling I came across this discussion thread in which the essence of the problem was laid bare. Basically, it was not a bug in rsync at all, but a bug from Apple. Under certain circumstances (drives mounted via a Time Capsule, or some other afp network shares) any attempt to create a file with two leading dots (ie “..”) would fail, eventually leading to the creation of a Ghost file. In addition to the fact that this would prevent rsync from working, these Ghost files are particularly annoying because they are tricky to delete.

Even though this was not an rsync bug, rsync is one of the few programs that will actually create files with a leading “..”. This is because it creates temporary files by prepending a dot to the name of its original file, and appending a hash. This means that any hidden files (with a single dot) will end up producing a double dot file at the destination by rsync.

I’ve filed a bug with apple rdar://9262177 but until the issue gets fixed, there is a fairly good workaround.

The workaround is to modify the way rsync creates temporary files so that is only prepends a dot if there isn’t already one. I’ve included an rsync binary with this fix in DropSync, but you’ll need to select it in the Advanced configuration tab.

To compile your own rsync with the workaround type the following commands in terminal; Download and unpack the rsync sources

1
2
    curl http://rsync.samba.org/ftp/rsync/src/rsync-3.0.8.tar.gz > rsync-3.0.8.tar.gz
    tar zxvf rsync-3.0.8.tar.gz

apply the patch

1
2
3
    curl http://www.mudflatsoftware.com/rsync/nodotdot.diff > nodotdot.diff
    cd rsync-3.0.8
    patch -p1 < ../nodotdot.diff

Build rsync

1
2
    ./configure
    make

A Simpler NSTask

Standard Cocoa provides the NSTask class to make calls out to the operating system. This is really really useful because the operating system is packed with command-line tools that can do alot more than straight Cocoa. Actually using NSTask though can be a bit of a pain. In order to look at the output of your task you’ll need to watch it’s standardOutput, and if the task takes an appreciable amount of time you will probably need to monitor this output on another thread. This isn’t all that hard, but it isn’t all that easy either. I realised there was a better way when I started using the NSURLConnection class, which is a Class whose API is designed specifically for long running jobs. Monitoring an NSURLConnection is as simple as making yourself its delegate and implementing a couple of methods.

Enter MFTask, which is a wrapper class for NSTask that provides some delegate methods for monitoring output, termination and launch. In my experience, it’s much easier to use than a raw NSTask. In DropSync I also tend to run lots of MFTask’s so I’ve also written a very basic queuing class specifically designed to handle running a queue of MFTask’s.

The source code for MFTask is available on bitbucket and comes with a small sample application which should demonstrate how to use it.

Let’s say you wanted to setup and run an NSTask, and monitor it’s output. Using MFTask you would first setup the task just like a normal NSTask. For example, here’s a task to find all the xml documents on your system. Setting up the task you would simply set its launch path and arguments as you would a normal NSTask;

Setting up Task Arguments
1
2
3
4
MFTask *taskObject = [[MFTask alloc] init];
[taskObject setLaunchPath:@"/usr/bin/find"];
NSArray *argvals = [NSArray arrayWithObjects:@"/",@"-name",@"*.xml",nil];
[taskObject setArguments:argvals];

In addition, an MFTask requires a delegate, so in order to monitor output of the task you will need a delegate which implements the following methods

Delegate Methods
1
2
3
4
5
- (void) taskDidRecieveData:(NSData*) theData fromTask:(MFTask*)task;
- (void) taskDidRecieveErrorData:(NSData*) theData fromTask:(MFTask*)task;
- (void) taskDidTerminate:(MFTask*) theTask;
- (void) taskDidRecieveInvalidate:(MFTask*) theTask;
- (void) taskDidLaunch:(MFTask*) theTask;

The methods taskDidRecieveData and taskDidReceiveErrorData provide a means of monitoring the standard output and standard error streams respectively. Simply write these methods to deal with data coming off the task. For example, here is a delegate method that simply dumps the output to NSLog (not very useful I know … but it illustrates the point).

Receiving Data in a Callback
1
2
3
4
- (void) taskDidRecieveData:(NSData*) theData fromTask:(MFTask*) task {
  NSString *stringRep = [[NSString alloc]; initWithData:theData encoding:NSASCIIStringEncoding];
  NSLog(@"%@\n",stringRep);
}

Finally, to launch the task you set its delegate to the class where you defined the delegate methods and then call launch. For example if your delegate class was called MFTaskDelegate;

Setting a Delegate
1
2
3
MFTaskDelegate *taskDelegate = [[MFTaskDelegate alloc]; init];
[taskObject setDelegate:taskDelegate];
[taskObject launch];

Now your task should be off and running and feeding data to your delegate.

If the task terminates normally your delegate will recieve a taskDidTerminate message.
If you need to terminate a task simply send it a terminate message as you would a normal NSTask object, and the task will go about terminating and will tell the delegate when it actually terminates by sending it a taskDidTerminate message.
If you want to terminate a task but don’t want to recieve any messages from you can use the invalidate method which not only terminates the task, but tells it not to send any further messages to its delegate. This is useful if your delegate might go away for some reason before the task is finished.

That’s it. I hope you find MFTask to be useful.