StarTrinity.com

VoIP software

StarTrinity SIP Testerâ„¢ - performance and hardware requirements

Performance of the StarTrinity SIP Tester depends its configuration and hardware where it is installed. Load capacity greatly depends on the CPU - clock frequence and number of cores. 1GB of RAM is enough for the tests. This page contains performance reports of SIP Tester with various hardware and operating modes. Based on these reports, you can estimate hardware requirements for your own testing, as CPU load linearly depends on number of channels. We tested first instance of SIP Tester (UAS, call receiver) by another instance of SIP Tester (UAC, call generator) installed on a more powerful server. Servers and laptops were connected directly with a patch cable or via 1GBit Cisco switch.

Server with 4x3.9GHz Intel Core i7-3770 CPU, 1GBit ethernet - 5400 G.729 channels (ptime=50)
SIP Tester was installed on 2 servers - call generator (UAC) and call receiver (UAS) - with 1GBit network cards connected via Cisco SG200 switch. Configuration was following:
  • EnableLightweightMediaProcessing = 1
  • EnableLightweightMediaProcessingPlayback = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G729
  • RtpTxPacketTime = 50ms
  • LogLevel = 0
  • Call duration = random from 80 to 90 seconds, configured at UAC instance

Caller (UAC) side measurements

Distribution of RX RFC3550 jitter and number of current calls:
performance chart
For 5400 concurrent calls, 1M simulated calls we have got 0% lost packets, max RX RFC3550 jitter = 28.61ms and max answer delay = 1048ms (view detailed report).

RX RFC3550 jitter history and histogram:
history and histogram chart
CPU and network load: 91MBps, 28% of CPU
CPU and network load

Called (UAS) side measurements

For 5400 concurrent calls, 1M simulated calls we have got 0% lost packets, max RX RFC3550 jitter = 28.61ms (view detailed report). Note that all 1000092 INVITE packets reached the called side.

RX RFC3550 jitter history and histogram:
history and histogram chart
CPU and network load: 91MBps, 31% of CPU
CPU and network load

Server with 4x3.9GHz Intel Core i7-3770 CPU, 1GBit ethernet - 2600 G.711 channels (ptime=20)
SIP Tester was installed on 2 servers - call generator (UAC) and call receiver (UAS) - with 1GBit network cards connected via Cisco SG200 switch. Configuration was following:
  • EnableLightweightMediaProcessing = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G711a
  • RtpTxPacketTime = 20ms
  • LogLevel = 0
  • Call duration = random from 80 to 90 seconds, configured at UAC instance

Caller (UAC) side measurements

Distribution of RX RFC3550 jitter and number of current calls:
performance chart For 2600 concurrent calls, 1M simulated calls we have got max RX RFC3550 jitter = 16.37ms and max answer delay = 473 (view detailed report).

RX RFC3550 jitter history and histogram:
history and histogram chart
RX lost packets history and histogram: there was one gap during 10 hours of test - 0.03% dropped packets per single call
history and histogram chart
CPU and network load: 224MBps, 34% of CPU
CPU and network load

Called (UAS) side measurements

For 2600 concurrent calls, 1M simulated calls we have got max RX RFC3550 jitter = 12.88ms (view detailed report).

RX RFC3550 jitter history and histogram:
history and histogram chart
RX lost packets history and histogram: there was one gap during 10 hours of test - 0.13% dropped packets per single call
history and histogram chart
CPU and network load: 224MBps, 37% of CPU
CPU and network load

Server with 4x3.9GHz Intel Core i7-3770 CPU, 5x1GBit - 8000 G.711 channel, SIP+RTP+DTMF (WinPCAP RTP sender mode, WAV playback, ptime=20)
SIP Tester was installed on 2 servers - call generator (UAC) and call receiver (UAS) - with 5 1GBit network cards connected via Cisco SG200 switch. On both servers 4 adapters (Intel I350-4) were used for RTP transmission, 1 more adapter (TP-link) - for RTP transmission + measurement and for SIP packets. RTP jitter and packet loss measurement was enabled only for TP-link adapter for 3% of calls to reduce CPU load. Configuration was following:
  • EnableWinpcapRtpSender = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G711U
  • RtpTxPacketTime = 20ms
  • LogLevel = 0
  • DisablePacketAnalysisOnIpAddresses = "10.10.10.41;10.10.10.42;10.10.10.43;10.10.10.44" (call generator) and "10.10.10.61;10.10.10.62;10.10.10.63;10.10.10.64" (call receiver)
  • Call duration = 220 seconds, random, logarithmic probability distribution
  • Max time to wait answer at UAC: 30 seconds, random, logarithmic probability distribution
  • Delay before answer at UAS: 10 seconds, random, logarithmic probability distribution
  • Audio playback: music.wav, looped
  • Number of DTMF digits sent: 1
  • Target number of channels (concurrent calls): 8000
  • Target calls per second: 100
Script for call generator:
<callxml>
 <block probability="0.03">
  <assign var="localRtpAddress" values="10.10.10.4" />
  <call maxringtime="$rand(30000);ms" value="sip:111@10.10.10.6:5070" codec="G711U" localRtpAddress="$localRtpAddress;">
   <on event="answer">
    <playaudio value="music.wav" repeat="infinite" maxtime="$rand(2000);ms" />
    <playdtmf value="4" />
    <playaudio value="music.wav" repeat="infinite" maxtime="$rand(200000);ms" />
    <exit />
   </on>
  </call>
  <exit />
 </block>
 <assign var="localRtpAddress" values="10.10.10.41;10.10.10.42;10.10.10.43;10.10.10.44" />
 <call maxringtime="$rand(30000);ms" value="sip:111@10.10.10.6:5070" codec="G711U" localRtpAddress="$localRtpAddress;">
  <on event="answer">
   <playaudio value="music.wav" repeat="infinite" maxtime="$rand(2000);ms" />
   <playdtmf value="4" />
   <playaudio value="music.wav" repeat="infinite" maxtime="$rand(200000);ms" />
   <exit />
  </on>
 </call> </callxml>
Script for call receiver:
<callxml>
 <wait value="$rand(10000);ms" />
 <block probability="0.03">
  <assign var="localRtpAddress" values="10.10.10.6" />
  <accept localRtpAddress="$localRtpAddress;" />
  <playaudio value="music.wav" repeat="infinite" maxtime="$rand(2000);ms" />
  <playdtmf value="3" />
  <playaudio value="music.wav" repeat="infinite" maxtime="$rand(200000);ms" />
  <exit />
 </block>
 <assign var="localRtpAddress" values="10.10.10.61;10.10.10.62;10.10.10.63;10.10.10.64" />
 <accept localRtpAddress="$localRtpAddress;" />
 <playaudio value="music.wav" repeat="infinite" maxtime="$rand(2000);ms" />
 <playdtmf value="3" />
 <playaudio value="music.wav" repeat="infinite" maxtime="$rand(200000);ms" />
 <exit /> </callxml>

Screen recordings



Caller (UAC) side measurements

Distribution of RX RFC3550 jitter and number of current calls:
performance chart For 8000 concurrent calls, 100CPS, 1M simulated calls we have got max RX RFC3550 jitter = 21ms and max 100 (Trying) delay = 781ms (view detailed report + full configuration). No RTP packets have been lost (0% packet loss)

CPU and network load: 150MBps x 4 (RTP 97%) + 17.4Mbps (SIP+RTP 3%), 48% of CPU
CPU and network load

Called (UAS) side measurements

For 8000 concurrent calls, 1M simulated calls we have got max RX RFC3550 jitter = 14.7ms (view detailed report + full configuration). No RTP packets have been lost (0% packet loss).

CPU and network load: 150MBps x 4 (RTP) + 22.5Mbps (SIP), 56% of CPU
CPU and network load

Server with 4x3.9GHz Intel Core i7-3770 CPU, 1GBit - 1000 calls per second (WinPCAP RTP sender mode)
SIP Tester was installed on 2 servers - call generator (UAC) and call receiver (UAS) - with 1GBit network card connected via Cisco SG200 switch. Configuration was following:
  • EnableWinpcapRtpSender = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G711a
  • RtpTxPacketTime = 20ms
  • LogLevel = 0
  • Call duration = 1 second
  • Max calls per second = 1000
  • Max concurrent calls = 1200

Caller (UAC) side measurements

For 9.85M simulated calls we have got max answer delay = 1000.41ms (view detailed report + full configuration). No RTP packets have been lost (0% packet loss).

Answer delay history and histogram:
history and histogram chart
CPU and network load: 82.3Mbps (RTP) + 13.7Mbps (SIP), 38% of CPU. Core #4 is taken by SIP thread
CPU and network load

Called (UAS) side measurements

For 9.85M simulated calls we have got max answer delay = 1000.41ms, same as measured at UAC side - no network delays (view detailed report + full configuration). No RTP packets have been lost (0% packet loss).

CPU and network load: 52.5MBps (RTP) + 23.7Mbps (SIP), 32% of CPU
CPU and network load

Laptop with 2x2.4GHz Intel Pentium 2020M CPU, 100Mbit ethernet - 1200 G.729 channels (ptime=50)
SIP Tester was configured with following settings:
  • EnableLightweightMediaProcessing = 1
  • EnableLightweightMediaProcessingPlayback = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G729
  • RtpTxPacketTime = 50ms
  • LogLevel = 0
  • Call duration = random from 80 to 90 seconds, configured at UAC instance
For 1200 concurrent calls, 152128 simulated calls we have got max RFC3550 jitter = 22.51ms and max answer delay = 1500ms (view detailed report).

This chart shows history of measured RFC3550 jitter over time and histogram for 1200 concurrent calls:
history and histogram chart Following chart shows distribution of measured RFC3550 jitter and number of current calls:
performance chart The charts were generated by UAC (call generator) instance of SIP Tester.

Laptop with 2x2.4GHz Intel Pentium 2020M CPU, 100Mbit ethernet - 550 calls per second (no RTP, SIP only)
We tested 2 instances of SIP Tester installed on a laptop and a more powerful server connected directly via 100MBit ethernet interface. During 1 hour 5 minutes it generated 2.16M SIP calls at call rate 554 calls per second. Max answer delay was 194ms, average - 0.48ms. Max delf-tested delay in SIP Tester's signaling thread was 75ms. view detailed report

Laptop with 4x1.6GHz AMD A8 CPU, 1GBE - 1200 G.729 channels (ptime=50)
SIP Tester was configured with following settings:
  • EnableLightweightMediaProcessing = 1
  • EnableLightweightMediaProcessingPlayback = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G729
  • RtpTxPacketTime = 50ms
  • LogLevel = 0
  • Call duration = random from 40 to 50 seconds, configured at UAS instance
For 1200 concurrent calls we have got max RFC3550 jitter = 26.91ms and max answer delay = 1581ms (view detailed report). After increasing call load up to 1500 calls we have got max RFC3550 jitter = 164ms and max answer delay = 20193ms (view detailed report), this is not acceptable.

Following chart shows distribution of measured RFC3550 jitter and number of current calls:
performance chart The chart was generated by UAC (call generator) instance of SIP Tester.

Laptop with 4x1.6GHz AMD A8 CPU, 1GBE - 600 G.711 channels (ptime=20)
SIP Tester was configured with following settings:
  • EnableLightweightMediaProcessing = 1
  • EnableLightweightMediaProcessingPlayback = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G711a
  • RtpTxPacketTime = 20ms
  • LogLevel = 0
  • Call duration = random from 80 to 90 seconds, configured at UAC instance
For 600 concurrent calls we have got max RFC3550 jitter = 25.12ms and max answer delay = 1052ms (view detailed report). After increasing call load up to 800 calls we have got max RFC3550 jitter = 171ms and max answer delay = 20130ms (view detailed report), this is not acceptable.

Following chart shows distribution of measured RFC3550 jitter and number of current calls:
performance chart The chart was generated by UAC (call generator) instance of SIP Tester.

Laptop with 4x1.6GHz AMD A8 CPU, 1GBE - 150 G.711 channels (ptime=20, debug media recording)
SIP Tester was configured with following settings:
  • EnableLightweightMediaProcessing = 0
  • DebugMedia = 1
  • EnablePacketAnalyser = 1
  • ForcedAudioCodec = G711a
  • RtpTxPacketTime = 20ms
  • LogLevel = 0
  • Call duration = random from 80 to 90 seconds, configured at UAC instance
For 150 concurrent calls we have got max RFC3550 jitter = 38ms and max answer delay = 1126ms (view detailed report). After increasing call load up to 200 calls we have got max RFC3550 jitter = 47ms and max answer delay = 3816ms (view detailed report), this is not acceptable.

Following chart shows distribution of measured RFC3550 jitter and number of current calls:
performance chart The chart was generated by UAC (call generator) instance of SIP Tester.

Virtual machines
Note: SIP Tester shows bad performance on XEN platform (as reported by one of our customers). It works well on VMWare and VirtualBox platforms.
Host hardware Host software VM configuration Attempted calls Concurrent calls Max CPS Call duration Codec configuration SIP Tester configuration Max call jitter (avg/90-percentile/max, ms) Max call delta (avg/90-percentile/max, ms) Lost packets
Intel Core i7-3770@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 4 cores 10000 450 5 90sec G711 ptime=20ms 16 media threads, LWMP,PA on 12,8/13/28,2 75/53,4/345 0%
Intel Core i7-3770@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 4 cores 10000 520 8 90sec G729 ptime=20ms 16 media threads, LWMP,PA on 13,1/12,2/75,6 73,2/48/519 0%
Intel Core i7-3770@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 4 cores 260000 450 5 90sec G711 ptime=20ms 16 media threads, LWMP,PA on 11,9/12,9/19,1 60,7/54,4/377 0%
Intel Core i7-2660@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 4 cores 165000 450 5 90sec G711 ptime=20ms 16 media threads, LWMP,PA on 14.5/15.3/28.7 111/154/315 0%
Intel Core i7-3770@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 2 cores 5000 180 2 90sec G711 ptime=20ms 16 media threads, LWMP,PA on 24/40/77 204/199/1133 0%
Intel Core i7-3770@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 2 cores 2000 100 2 90sec G711 ptime=20ms 16 media threads, LWMP,PA on 8,7/10,44/14,45 40/40/121 0%
Intel Core i7-3770@3.4GHz 8 cores WinServer2012 64bit, VirtualBox 4.3.16 2 cores 2000 130 2 90sec G711 ptime=20ms 16 media threads, LWMP,PA on 10/12,5/26,6 43,8/49,9/111 0%

Copyright 2011-2017 StarTrinity.com | Blog | Contact lead developer via LinkedIn | Support tool