StarTrinity.com

Measuring quality

StarTrinity SIP Tester™ - frequently asked questions (FAQ)

How to configure Cisco Unified Call Manager (CUCM) with SIP Tester?

Please follow this tutorial

How to run automated tests with SIP Tester? Is there any API?

There is a command-line interface (CLI), you can use it for automated tests. There are also "readfile", "writefile", "writecdr" CallXML elements for advanced API. Results of testing could be exported to CSV files or database. In addition to that, there is a web API which is used by our AJAX-based web interface.

How to read SIP Tester's log files?

You can read logs as text files using Far Manager. Default file format is [program folder]\Logs\[date]\[hour]_SipTester.log. Format of lines is [minute]:[second].[millisecond][threadId]~[logLevel]~ [optional module] [optional objectName] [log message text]

Can I use multiple ethernet cards with different IP adresses with SIP Tester on a single server?

Yes, you can. SIP Tester opens SIP and RTP ports on all network adapters. It selects correct adapter by target IP address according to windows IP routing table. For example if you send calls to "10.10.10.10", and there are 2 NICs: #1 "10.10.10.1/24" and #2 "10.10.2.1/24", SIP Tester automatically selects NIC#1. Initially it opens sockets on all available interfaces.

Why are RTP and MOS fields empty in CDR?

RTP fields are refreshed every few seconds. If they are empty, reason could be one of the following:

RTP packets from localhost to localhost are not detected by winpcap
Winpcap library is not installed. You can download it here
RTP media is not established yet for the call
RTP packets don't reach SIP Tester, it gets only SIP packets for sniffing
SIP Tester's packet analyser is disabled. You can enable it by setting "EnablePacketAnalyser" = "1" on settings screen and restarting SIP Tester

Which programming language is used in SIP Tester?

SIP Tester is developed with C# (GUI and high-level logic), C/C++ (SIP and RTP processing), XAML, HTML and Javascript. Some procedures like memcpy are written in ASM.

Can SIP Tester run on Linux OS?

Sorry, no. We use C#, .NET, WPF and some other Microsoft technologies which make the development more effective. We may support Linux in future.

Why is "auto update" not working?

It could be because of firewall. SIP Tester gets new files from "startrinity.com" server via HTTP protocol. If your organization uses HTTP proxy server, please set settings "HttpProxyUrl", "HttpProxyUserName", "HttpProxyPassword". Note that "HttpProxyUrl" should contain "http://" prefix (example: "http://192.168.1.1:8080")

How to test my T.38 fax server?

Question: I need to test a T.38 fax server. Can you point me to the basic instructions for setting that up? My specific tests would be SipTester originated calls, upon answer I need to send a formatted INFO message to the fax server, at which point it should respond with answer tones as G.711 RTP; then, ideally, we would respond with origination tones and that would cause the fax server to re-INVITE to T.38. How much of this is possible? My primary objective here is putting the server under load for an extended period of time.
It is possible, please set setting "T38AutoReceivePath" to some existing folder. CallXML script for outgoing calls should be something like this:
<call value="sip:xxx@my.mediaserver.com:5060" >
<on event="answer" >
<sendsiprequest>
<![CDATA[
INFO sip:xxx@my.mediaserver.com SIP/2.0
Max-Forwards: 70
Content-Type: text/xml

<an><example XML inside SIP INFO /></an>
]]>
</sendsiprequest>
<playaudio value="c:\some_fax_tone.wav" />
<wait value=”1h” /> 
</on>

How to measure delay in IVR menu?

You can measure time between sent DTMF message and received RTP audio signal (response to DTMF) using waitforsignal CallXML element

How does SIP Tester measure G.107 MOS score?

The SIP Tester measures G.107 e-model MOS score from following:

Audio codec: G711 PLC-mode codec is more robust to packet loss than G.729 or G.723
Packet loss, considering loss due to high jitter
Burst rate: is high when multiple sequential RTP packets are lost in a row

The source code contains the G.107 e-model formula:

G.107 MOS score does not take into account round-trip-time (RTT), the RTT should be controlled independently.

How to measure RTP round-trip time/delay (RTT)?

The SIP Tester measures RTCP from RTCP packets and automatically reports the measurement as CDR field. Additionally, The RTT is measured using "RTP header extensions" in following steps:

Caller party changes ‘RTP extension’ for outgoing RTP stream, saves the NIC timestamp as T1
Called party detects this change and changes its RTP extension (loopback connection before codec processing, before jitter buffer)
Caller party detects the change in RX RTP stream, saves it as T2
RTT is calculated as T2 – T1

You need to set up special scripts at both endpoints:

<call value="sip:111@devpc:5070" callerId="1111" />
<on event="answer">
<setrtpextension wg67_pttType="1" wg67_pttId="16" />
<playaudio value="music.wav" maxtime="3s" />
<setrtpextension wg67_pttType="0" wg67_pttId="16" />
<playaudio value="music.wav" maxtime="3s" />
<getwg67info ptt2squdelayvar="ptt2squdelay" ptttypevar="ptttype" pttidvar="pttid" squvar="squ" pmvar="pm" pttsvar="ptts" sctvar="sct" />
<log value="WG67 measurements: ptt2squdelay=$ptt2squdelay;ms, ptttype=$ptttype;, pttid=$pttid;, squ=$squ;, pm=$pm;, ptts=$ptts;, sct=$sct;" />
<writecdr field="RX_WG67_PTT_ID" value="$pttid;" />
<writecdr field="RX_WG67_PTT2SQU_DELAY" value="$ptt2squdelay;" numeric="true" qualityIsAscending="false" />

<exit />
</on>


<accept />
<setrtpextension wg67_pttType="1" wg67_pttId="33" wg67_squ="rxptt" wg67_pm="0" wg67_ptts="0" wg67_sct="0" />
<playaudio value="music2.wav" repeat="infinite" maxtime="60s" />
<exit />

How to change IP Type of Service Header (ToS) Field and Differentiated Services Code Point (DSCP)?

Please follow this instruction. Alternatively you can change this field in SIP Tester settings if you use "WinPCAP RTP Sender" mode.

What is algorithm of recording into WAV file?

The WAV file is created with the SIP call - when INVITE is sent or received. After 183 (Session Progress) or 200 (OK), when RTP packets are sent/received and passed via jitter buffer, audio data is written to WAV file. By default nothing is written to WAV file if no RTP packets are received (e.g. before RTP stream initiation). This is configured with setting "RecordSilenceWithoutRtp".

Why do I have DNS errors in SIP Tester?

Please use a "nslookup" tool to diagnose DNS issues. Probably you need to configure DNS server address on your server

Why can't SIP Tester send email via gmail.com?

Reason could be in gmail's auto-ban of your server's IP address. Please login from server’s IP address to gmail.com in browser

Can SIP Tester send and receive calls simultaneously?

Yes, SIP Tester can both send and receive calls, but they can affect statistics (answered calls count ratio = answered calls count / generated calls count). There are 2 independent scripts, one for outgoing calls, another for incoming. The scripts work independently.

Can I have multiple IVR test scenarios in SIP Tester?

Yes, you can. You can write multiple test scripts using <switch>-<case> syntax, each test script may have its own probability of execution. For incoming calls you can select a specific test by caller ID, called ID or randomly. For outgoing calls testing multiple IVR scenarios code may look like this:

It is possible to include test scripts from separate files:

How to send/receive audio+video calls in SIP Tester?

Please use following example CallXML scripts

How to use custom audio codecs in SIP Tester (OPUS, AMR-WB, etc.)?

Please use following example CallXML scripts

SIP Tester hangs when starts in "starting.." state - how to fix?

There can be many reasons:

Some other process uses same local SIP UDP port. Please check other processes (some softswitch on port 5060, another instance of SIP Tester?) using "netstat" tool, also check settings in SIP Tester settings XML file: "LocalSipPort", "LocalSipPortsRange"
Corrupt settings XML file (can become corrupt when no free disk space) - here is a story of one user:
Here is how I resolved my issue:
1. I found that StarTrinity.SIPTester.settings was not readable by notepad or any text editor I had at hand.
2. I renamed StarTrinity.SIPTester.settings to a .bak file
3. I found the two most recent backup settings files from backup_settings could not be opened in a text editor.
4. I located the most recent settings backup file that was readable. It contained the XML for my SIP Outgoing Calls. I was very happy : )
5. I moved the "good" backup file to the working directory for SIP Tester and renamed it with the appropriate capitalization.
6. I restarted Windows.
My SIP tester worked as before!
Post Mortem
My SIP Tester settings file was corrupted as well as at least two backups.

Performance issues

Why is CPU throttled by SIP Tester? Why is performance warning displayed? How to achieve higher call load?

CPU overload leads to high RTP jitter and to packet loss. Reason of high CPU load can be one of following:

RTP audio processing for PESQ mean opinion score (MOS) calculation. Audio processing is turned on by CallXML element "verifyaudio" with attribute "mosvar"
Text to speech generation. It is done by "say" CallXML element.
Writing to logs. You can reduce it by setting "LogLevel" = "0" at "Settings" tab
Media transcoding which may be needed for recording WAV files. You can disable it by setting "EnableLightweightMediaProcessing" = "1"
Processing too many concurrent calls. You can reduce number of concurrent calls at Outgoing calls tab
Displaying calls in CDR GUI table. You can turn off the entire GUI by using command line interface. Setting command line parameter 'EnableCallMeasurements' = 'false' will also reduce CPU load. Alternatively, you can go to some other tab (e.g. 'Reports/Statistics') which does not take too much CPU resources.
Using Win7 or WinXP instead of WinServer2008 or WinServer2012 versions of Windows OS
Using small number of media threads. You can increase it with 'MediaThreadsCount' setting
Some other heavy process running on same physical server: database backup, windows update, antivirus scanner, etc
Using default NIC drivers from Microsoft. Please install latest NIC drivers from hardware vendor, from official website
Having 100Mbps link speed instead of 1Gbps/10Gbps: bad NIC cable connection, bad IP switch, bad NIC driver

SIP Tester has one SIP thread, one GUI+CDR thread and multiple RTP media threads. To optimize CPU load you can do this:

Set setting "EnableLightweightMediaProcessing" = "1". This will turn off media transcoding and recording to WAV. Only playback will be available.
Set setting "GuiUpdateIntervalMs" = "0" and update the GUI manually with "refresh" button (bottom-left). This will turn off automatic GUI updating on timer.
Set setting "MediaTransportPoolInitialSockectsCountPerInterface" to some big value (default value is "4"). Opening a new RTP socket takes SIP thread's CPU time and causes overload when test starts
Hide unnecessary displayed CDR columns
Set "LogLevel" = "0" in case if you have logging turned on
Turn off packet analyser ("EnablePacketAnalyser" = "0") in case if you don't need to measure quality of RTP media streams
Run SIP Tester in command-line mode, so the GUI will not consume CPU resources. Also set command-line parameter "EnableCallMeasurements" = "false" to turn off CDR module
Set setting "EnableCallMeasurements" = "0" in SIP Tester's GUI
Run multiple instances of SIP Tester on the same server: copy DLL, EXE, XML program files into some other directory, set "LocalSIPPort" to some different value in settings XML file
Turn off RTP media if you need only SIP packets: set "send SDP in INVITE" = "off"
Use WinServer instead of Win7 and Win8. NIC drivers and winsocket library have better performance in Windows Server
Use real server instead of virtual machine
Use multiple multi-port NICs: for example 2x Intel I-210 4-port NICs
Tune up your NIC. Install latest NIC drivers. Increase RX and TX buffer size. Set "Interrupt Moderation" = "Adaptive". Increase number of RSS queues. Configure base core for RSS queues and settings "Affinity*"
Enable Intel Turbo Boost
If core#0 is overloaded by NIC interrupts, set processor affinity mask using command line (e.g. start /wait /affinity 0xFA StarTrinity.SIPTester.CLI.exe [params])
Turn off hyper-threading to get higher CPU frequency per single core
Disable Intel C-STATE in BIOS
Disabled Intel Speedstep on BIOS
Disable all onboard devices in the BIOS like COM ports, Parallel ports, WWAN, WLAN, Bluetooth - they can generate unneeded interrupts
Set Power Plan to 'Best Performance' on Control Panel
If the software displays performance warnings incorrectly, try to modify global settings SipThreadMaxQueueCountToAbortCalls, SipThreadMaxQueueDelaymsToAbortCalls, SipThreadMaxQueueCountToRejectNewCall, SipThreadMaxQueueDelayMsToRejectNewCall
If you don't need to generate RTP packets: set attribute fakeSdp="true" in CallXML elements "call" and "accept", this is used for high-CPS tests without RTP
Randomize call duration in CallXML script to reduce CPS when calls are ended

How to achieve low transmitted RTP jitter (less than 1ms)?

By default the software uses Sleep() WinAPI function which can introduce jitter of 5-15ms. To reduce the jitter, please set following settings

MediaClockUseSpinWait = 1
MediaClockPeriodMs = 20
MediaThreadsCount = 2
EnablePacketAnalyser = 1
EnableWinpcapRtpSender = 1
GuiUpdateIntervalMs = 0

Why is CPS lower than I set in call generation parameters? How to achieve highest CPS?

Reason can be one of following:

Number of concurrent calls limit is exceeded
High CPU load which can cause big delays in SIP thread. You can check SIP thread delays on Reports/Statistics tab
CPU load caused by GUI thread. You may consider running SIP Tester in command line mode
CPU load caused by RTp media threads. You may consider making calls without RTP media

What does SIP Tester do if there are not enough CPU resources?

When CPU's performance is not enough, it rejects calls with 503, or skips generation of outgoing calls. Current calls are not terminated

Why do I hear chopped audio in WAV recordings?

Reason can be in slow hard disk which is used for recording. You can switch to SSD or RAM disk, or increase setting "RecorderQueueMaxCount". The software has an internal queue in memory which stores audio frames while HDD writer is accessing disk. If queue becomes overloaded, audio frames become dropped and following is written to log:
~wrn~ [unm] cannot enqueue write data for recorder xx.proxy.recorder, file 'xx.wav': queue overflow (count = 50049)

Why do I have memory leaks in SIP Tester?

SIP Tester can take up to 10GB of RAM if it is configured to save RTP packets to memory

How to turn off HDD operations?

You can turn off extra HDD operations by following settings: LogLevel = 0 (Error), DebugMedia = 0, DebugMediaTX = 0, SaveSipAndRtpPacketsToDisk = 0, SaveSipPacketsToDisk = 0, WriteCdrToCsv = 0

SIP protocol issues

Why doesn’t SIP Tester send INVITE or REGISTER?

It could be an ARP issue. You can debug it with wireshark, looking at ARP packets. You can also use "ping" tool, it also does not send ICMP packets in such case

Why doesn’t SIP Tester receive incoming calls (INVITE's)?

If you don't see incoming calls in CDR, there can be one of following reasons:

Windows Firewall blocking incoming packets: please add rule in Windows Firewall to accept UDP packets on ports 5000...30000
It is firewall of server hosting provider (e.g. Windows Azure)
Calls are sent to wrong SIP port (default SIP port for SIP Tester is 5070)
Calls are sent to wrong IP
Calls are blocked by router(s) or internet providers. Please use wireshark tool to trace where INVITE packets get lost
There are issues in NAT

How to test availability of SIP devices using SIP OPTIONS?

You can use sendsipmessage in CallXML script, see example in the CallXML docs

Why is a new generated call in a "Null" state?

Most common reason is that SIP call is made over TCP protocol, and TCP connection is not established yet

Why doesn’t SIP Tester send BYE?

Because SIP call is not answered (there is no 200 OK), and response status code is 404, 408, or any other unsuccessful response code. If your destination responds with 404, probably you have to configure extension number at your PBX or check destination number in SIP Tester.

Why does SIP tester abort a long-duration call?

One of the following:

By default if SIP Tester considers SIP call as a "dead call" and aborts the call, if it does not get RTP packets for a long time. If you have discontinous transmission (DTX) enabled on your server and you don't want the call to be aborted, please set "IdleMediaTimeToEndSession" = "1000000" [seconds] on settings screen
Call is aborted on a "lifetime" timeout, please edit setting "MaxCallLifeTimeInHours" (default 48 hours)
Call is aborted if session keep-alive REINVITE fails. Please edit "SessionRefreshIntervalSeconds" setting (default 1 hour)
SIP Tester ends executing CallXML script in normal way

Why don’t I get simulated DTMF on my server? Why does RTP go from SIP Tester to some bad IP address)?

Please check packet with 200 OK response from your server. Your server MAY send 200 OK response with wrong IP address in Contact header and SDP, so SIP Tester sends RTP to this IP address. We had similar issue with FreeSwitch. You can configure SIP Tester to ignore these fields in 200 OK response by setting "SymmetricSIP" = "1" and "SymmetricRTP" = "1" in settings

Why does SIP Tester send ACK to 5060 port while it is configured to send INVITEs to 5063 port?

Please check packet with 200 OK response from your server. Reason could be in Contact header without "5063" port number. SIP Tester sends ACK to IP address and port which is specified in this Contact header. If no port is specified, default value (5060) is used.

How to split signaling from media (use 2 separate NICs for SIP and RTP)?

Set setting "ForcedLocalAddress" to IP address of separate NIC which will be used for RTP

Why doesn't SIP Tester listen for UDP packets on port which is indicated in "Via" header?

We had such issue with one of our customers who used SIP tester to generate calls to his server over TCP transport. Initial INVITE message from SIP Tester contained following "Via" header:
Via=SIP/2.0/tcp 10.6.0.5:55658;rport;branch=z9hG4bKPj912a82c1407d4dad91d1c40992b5931c
Here "rport" and "tcp" indicate that provisional responses (100 Trying) should be sent over TCP, not over UDP. That's why SIP tester did not listen for UDP packets. Solution for this issue was to fix parsing in customer's server and send provisional response over TCP in such case.

How to set RTP ports?

You can set RTP port numbers which are used for SIP calls explicitly by "localRtpPort" parameter of "call" and "accept" CallXML elements. Without this parameter, ports are selected from a range using settings "MediaTransportPoolMinPort" and "MediaTransportPoolMaxSocketsCountPerInterface".

How to put P-Asserted-Identity or any other custom SIP header (PAI) into generated call?

You can put any custom SIP header into SIP call by modifying CallXML script:

How to enable TLS v1.2 in SIP Tester?

Go to SIP Tester - settings - set setting "TlsMethod" = "SSLv23", restart SIP Tester

Audio verification and PESQ MOS

How to measure PESQ MOS?

If you are using SBC or PBX which accepts REGISTERs and INVITEs:

Set up UAC registrations
Select some reference WAV file for audio verification
Use this CallXML script for incoming calls:
<callxml>
<accept />
<playaudio value="C:\your_reference_file.wav" />
</callxml>
Use this CallXML script for outgoing calls:
<callxml>
<call value="$random_least_busy_uac_registration();" />
<on event="answer">
<verifyaudio reference="C:\your_reference_file.wav" maxtime="10s" mosvar="pesq_mos" />
<writecdr header="PESQ_MOS" value="$pesq_mos;" numeric="true" qualityIsAscending="true" />
<exit />
</on>
</callxml>

You can have 2 instances of SIP Tester installed on different PCs, so RTP packets will go across your network.

Why doesn’t SIP Tester record pcap files, why does it show empty RTP statistics fields in CDR?

The SIP Tester uses WinPCAP or NPCAP libraries to capture packets and exact packets' timestamps directly from network adapter, to measure delays and RTP jitter with best accuracy. Sometimes the packet capturing does not work for following reasons:

"Packet analyser" module is disabled (setting "EnablePacketAnalyser") - please set it to "1" and restart the software
Winpcap is not installed
Winpcap does not work in your environment (we have observed issues with various virtual adapters and also related to non-administrator rights)

To diagnose the issue:

Make the call using SIP Tester or a SIP softphone
Record the call using Wireshark
Analyse recorded packets in Wireshark (menu - Telephony - VoIP calls). See if the packets are really captured from the network adapter(s).
If you see that the packets are captured and displayed correctly in Wireshark, but not captured by SIP Tester - please contact us with your recorded pcap file

How is PESQ MOS calculated?

"verifyaudio" CallXML element analyses content of RTP packets and compares it with reference WAV file. The algorithm is based on ITU-T recommendation P.862 (PESQ).

How to test audio connection in a conference?

Please use this sample script

Why do I have low PESQ MOS results in LAN environment?

Even with no jitter and no packets loss you can get low PESQ MOS score, reason could be in audio codec or your media server. Please turn on audio recording and compare recorded file with reference file manually using audio editor. With G.729 codec reason of low PESQ MOS could be in disturbed "ssshh" sounds. You can try a different reference file for audio verification, it can help.

Can it match with less than full length of reference wav file?

Yes. The algorithm chooses minimal length of recorded and reference signals, it uses this minimal length for comparison.

Can your SIP tester test echo cancelation in SIP intercom device?

Yes, it could be done using "verifyaudio" CallXML element. SIP Tester can verify audio in RTP stream by comparing it to reference file. If echo is too loud, it would disturb audio signal and it will result in reduction of audio quality (MOS indicator). You should run a test where some physical audio simulator near your intercom device plays an audio message with reference file for every incoming call.

Licensing and support

How do you provide technical support?

We usually provide support in following way:

Ask for description of an issue, CallXML scripts, system .log files with LogLevel="Debug", .pcap traces
Analyze system logs, CallXML scripts, .pcap traces
Try to reproduce the issue in our local environment
Connect to customer's desktop remotely if is allowed in your company
Extend logging if it is needed
Talk to the customer by email, skype or phone
Fix source code, release new version. Usually it takes few days to fix an issue or develop a new feature

Where can I find the limits of the free version?

You can find detailed information about limits of your current license in License information window. At this time (March 2015) it is 150 attempted and received calls, 50 concurrent calls for freeware version.

Are there any differences between the paid and the free version?

There is no difference other than higer number of concurrent calls and total number of attempted+received calls.

After reinstalling license does not work, how to fix it?

Your license key has been deleted by uninstaller. Please re-enter your license key on License information tab and restart SIP Tester to apply it.

What is "FastSpring", why do you use them?

We use FastSpring as e-commerce provider and exclusive reseller because we decided not to implement payment processing ourselves. They accept credit cards, paypal, wire transfer payments, mail checks, handle VAT and do great job for us.

Can you receive a PO (purchase order)?

Yes, please send the PO to sales [at] startrinity [dot] com. Based on the PO we create an invoice, we pass payments via US-based reseller "FastSpring"

Can we (as reseller) purchase SIP Tester to our customer, will you issue license to a different company name?

Yes, please

Make order(s) via FastSpring
Send us email to sales [at] startrinity [dot] com, tell order ID (s) and name of end customer, we will update license(s) in database, and end-user wil lreceive SIP Tester under their own company name.

Known issues (not bugs) in StarTrinity SIP Tester

WinPCAP 4.1.3 does not work on some network adapters. If it is a virtual machine, try to change type of virtual network adapter
"Caller/called SDP-RTP delay" can be negative, and it is not an error, if RTP packets are detected BEFORE SDP negotiation packet (183 or 200 response packet). It is normal for incoming calls, when SIP Tester replies with RTP and 183/200 packet to remote caller.