Measuring TestTrack API Performance (2 of 2)

In part 1 of this article, I  looked at raw performance numbers of the TestTrack SDK. Interesting data but no pretty pictures! We saw the performance differences between IIS and Apache on the Windows server. If you graph the results though, you'll see that the difference is very minor. Chart 1: Apache vs. IIS in calling getRecordListForTable, timing for entire query In chart 2 you can see that larger queries are not resulting in longer wait times per record, so the SDK is handling large requests just as well as the smaller ones. Chart 2: Apache vs. IIS in calling getRecordListForTable, timing per record fetched Based on chart 2, it looks like Apache has a higher "start-up" cost than IIS, which is then made up for as you fetch more data.

Test Runs

Not much of interest here, the numbers were virtually identical to those seen with the defect record query. One noteworthy change was that logoff takes longer if you grab both TestTrack Pro and TestTrack TCM licenses on logon. In my defect queries I was only grabbing a TestTrack Pro  license but timing increased by ~15% when I grabbed both licenses.

Overhead

How much overhead does the SDK add to the actual data, as far as size? To save time, I just did a simple csv export from TestTrack and compared that with the SOAP query results for the defect list.

	True Size (bytes)	SOAP Size (bytes)
100 records	17,533	49,740
1000 records	176,303	490,040

Table 1: Space usage of CSV vs. XML formats Almost 3 times as much space required for XML compared to a simple csv output! Not surprising, given that every piece of data requires wrapping tags but reinforces the need for good data filtering practices on queries.

Conclusion

You can fetch 1,000 records in a little over 1 second and I'm hard pressed to come up with a scenario where you would want to fetch even that much in one chunk. With proper use of column and record filtering, the SDK won't be a bottleneck in customization and integration with TestTrack.