July 7, 2009
Measuring TestTrack API Performance (2 of 2)
In part 1 of this article, I looked at raw performance numbers of the TestTrack SDK. Interesting data but no pretty pictures! We saw the performance differences between IIS and Apache on the Windows server. If you graph the results though, you'll see that the difference is very minor. Chart 1: Apache vs. IIS in calling getRecordListForTable, timing for entire query In chart 2 you can see that larger queries are not resulting in longer wait times per record, so the SDK is handling large requests just as well as the smaller ones. Chart 2: Apache vs. IIS in calling getRecordListForTable, timing per record fetched Based on chart 2, it looks like Apache has a higher "start-up" cost than IIS, which is then made up for as you fetch more data.Email sign up
Table 1: Space usage of CSV vs. XML formats
Almost 3 times as much space required for XML compared to a simple csv output! Not surprising, given that every piece of data requires wrapping tags but reinforces the need for good data filtering practices on queries.
Test RunsNot much of interest here, the numbers were virtually identical to those seen with the defect record query. One noteworthy change was that logoff takes longer if you grab both TestTrack Pro and TestTrack TCM licenses on logon. In my defect queries I was only grabbing a TestTrack Pro license but timing increased by ~15% when I grabbed both licenses.
OverheadHow much overhead does the SDK add to the actual data, as far as size? To save time, I just did a simple csv export from TestTrack and compared that with the SOAP query results for the defect list.
|True Size (bytes)||SOAP Size (bytes)|