Today I want to talk about a time when I had to optimize some dot net code. I started using Linq. But the code didn’t perform fast enough (probably because I was a novice at Linq). I needed the fastest performance when joining data from two collections? I’m not sure Linq is the fastest…but generally it is good enough.
One time I had to join two sets together, having a one to many relationship, and had to tune the program as much as possible. I used Linq to sort the two sets. Then I did a for loop on the the set containing the data of the one side of the relationship. I also kept a pointer to the current record in the many set, starting at 0.
Inside the loop I would call a method that would process the many set, passing the pointer value where to start processing. As soon as I found a value that did not equal the current record in the outer loop it would exit passing back the current pointer value.
In this fashion the inner processing never had to restart at the beginning, and it would exit as soon as a non-match was detected. There was no need to go back to the beginning because it was already determined that the current record had a higher value than the outer set.
Linq worked well for me by sorting the two sets so I could process in this manner. The question I want to ask today is was it worth it? I will say that the particular application on which I was working was tweaking performance down to the millisecond. Even an improvement of 2 milliseconds was sought.
What I’m wondering is how often is this kind of performance tuning meaningful? Are many of you writing code that is seeking to eek a few milliseconds of performance out of your code? Is this kind of optimization useful, maybe Linq may find a more efficient method of doing the sort and the join concurrently, and can be run in parallel? (the earlier code was written before parallel Linq was released) Maybe Linq is more maleable and can adapt the method to handle small sets differently than large sets??
Share your opinion with use here, or by email to btaylor@sswug.org.
Cheers,
Ben