VoIP software

VoIP community

VoIP marketplace

logged in as
log out

VoIP. How it works in detail. Troubleshooting, fraud cases. Theory and practice. Free e-book

The book is designed for people who want to learn how to fix issues with VoIP - SIP, RTP, RTCP, UDP, TCP, IP protocol stack. It includes small tasks for homework to check your new learned skills with practice. The e-book is dynamically updated with new content. Last modification time: 2018-07-01. Please contact us if you want us to extend or clarify any part of the book.
Warning: the e-book is protected by copyright, making copies is not allowed

What is VoIP. VoIP business types, market players.
What is VoIP call
Encapsulation and structure of packets
Protocols used for VoIP
Types of VoIP issues
VoIP troubleshooting methods

What is VoIP. VoIP business types, market players.

VoIP stands for "voice over IP protocol". Today it is used not only for voice: it transfers video data, IM (instant messaging, text chat)/presence, other data like SS7 protocol. Major benefit of the VoIP is low price of data transfer: internet is very cheap. Another benefit is low price of both client-side and server-side hardware needed for voice/video calls: VoIP client software can run on cheap Android or Raspberry Pi devices, on already existing Windows laptops and PCs; VoIP server software can run on Linux or Windows servers; no special hardware is needed (if we compare VoIP with SS7/TDM). The VoIP has helped to reduce price of international calls dramatically, prices still go down (it is good for customers). Major drawback of the VoIP is easiness of fraud and untransparency in market: fraud in IP network is much easier than fraud in PSTN network.
Here is list of popular VoIP business types, VoIP market players. The VoIP market is complex and some business types are confidential, so this list is not full.
  • OTT (over the top) app. developers provide apps for free voice/video calls over internet; Skype, WhatsApp, Viber, etc. The OTT apps allow free calls between people having same application installed on 2 remote devices. They also allow making calls to real phone numbers for some payment (in this way generating VoIP retail traffic), and in case of international OTT bypass allow making VoIP calls to real phone number which is linked to customer's account, via internet and for lower price (compared to termination prices of MNOs).
  • Mobile network operators (MNOs) / tier-1 operators offer official ("white") VoIP routes where calls are terminated to real phones over GSM networks. Such VoIP routes are direct, they deliver Caller ID, and they have great quality and high price.
  • VoIP retail - B2C business when customers make mostly long-distance (international) calls to real phone numbers at cheaper price (compared to dialing from mobile phones). Such business aggregate voice calls from many people (typically immigrants) and send the voice traffic to VoIP termination providers.
  • Wholesale VoIP termination providers (tier-2, tier-3 operators) aggregate VoIP traffic and resell VoIP routes. Having many interconnections, wholesalers provide A-Z VoIP routes - VoIP routes to all countries in the world. Also, the wholesalers are able to redirect calls to multiple redundant routes: if one route is down, call gets redirected to another one which is up. Some wholesalers specialize on particular country or region and particular VoIP traffic type.
  • Grey VoIP termination providers (are considered by MNOs as fraudsters) provide a way to send international (long-distance) voice calls to real phone numbers via GSM gateways (SIM boxes), reducing price by using local tariffs. Together with OTT termination providers, the GSM termination providers play important role in the market, reducing prices for calls. Although the GSM termination is not considered as "legal" in destination countries, we see that it is done by many people including corrupt employees, right inside MNO.
  • VoIP marketplaces where wholesalers and direct providers meet and interconnect: various websites and global telecom industry events/conferences
  • Fraud management and revenue assurance (FM&RA) companies help MNOs to fight against grey termination - block SIM cards that are being used for grey international traffic. This business does not only include blocking SIM cards, there are many other ways of revenue leakage; it is very complex and very confidential. FM&RA use many interconnections to send test calls to block SIM cards which are used by grey termination providers
  • Call centers receive calls on real phone number(s) and transfer the calls to agents. The call centers may also generate outgoing calls in case of telemarketing, this type of VoIP traffic (CC) is unloved by VoIP termination providers. Incoming calls go to call centers via PSTN-SIP gateway or via DID providers
  • DID (direct inward dialing) providers route calls from real phone number to call center or IP PBX via VoIP protocol
  • IP PBX / UC (unified communications) platforms connect employees of a company with free voice/video calls; also connect employees using real phone number(s). The IP PBX systems generate retail VoIP traffic and send it to VoIP providers
  • Software/hardware developers - the complex VoIP market needs complex software and hardware for every component of VoIP system, also test tools and services. Some companies like Speedflow and China Skyline also do VoIP wholesale business
  • Looped generated traffic providers send VoIP calls via a chain of VoIP wholesalers for various confidential purposes. The generated calls usually get disconnected if generated traffic gets terminated by some other provider or IVR, or if generation side detects mismatch in billing.
  • IPRN (international premium rate number) providers are same as DID providers except that the calls are used for payments from caller to service provider. Unlike a normal call, premium rate calls are more expensive, and part of the call charge is paid to the service provider, in this way enabling businesses to be funded via the calls. Adult chat lines (phone sex) and tech support are a very common use of premium-rate numbers. Other services include directory enquiries, weather forecasts, competitions and voting (especially relating to TV shows).
  • Various fraudsters and hackers - see types of VoIP fraud and attacks here
For advanced readers - diagram of international VoIP market is here.

What is VoIP call

The VoIP stands for Voice over Internet Protocol, and VoIP call is a series of IP packets with data inside the packets, sent between caller and callee over IP network. Structure (format) of the data inside the packets and way of communication between caller and callee are standartized with a number of protocols. In 2018 most typically used protocols for VoIP are IP, UDP, SIP and RTP. Here is diagram of VoIP call using above-listed protocols.
SIP call flow diagram
Arrows mean transmitted packets; "SIP/UDP/IP" means SIP-protocol data encapsulated into UDP and IP packet, in other words SIP packet transmitted by means of UDP and IP protocol. "RTP/UDP/IP" bold bidirectional arrow means multiple audio packets with voice data transmitted between caller and callee in two ways. The voice audio data is encoded with some codec, encapsulated into RTP and transmitted by means of UDP and IP protocol.

Encapsulation and structure of packets

Encapsulation in IP packets is a way how the data inside packets is organised: one upper-lever protocol is nested inside another lower-level protocol.
SIP packet. Encapsulation

The next picture displays captured packet (sniffed, recorded packet data) in wireshark tool.

SIP packet. Encapsulation in wireshark

Protocols used for VoIP

Network protocol analyser tool "Wireshark"

Before you continue with the protocols, please download and install wireshark tool - free and opensource software. It is necessary for VoIP troubleshooting and understanding network protocols. The wireshark is able to record (sniff, capture) packets from network adapter, save them to file (.pcap, .pcapng), parse captured packets and display them in user-friendly way. It also includes tools to parse and display VoIP calls, RTP media streams. Most important things for you to do with wireshark:
  • Select network adapter(s), start capture, stop capture
  • Apply a display filter. Examples: "sip", "sip || rtp", "http", "tcp", "udp"
  • Look into captured packets and see the sequence, timestamp and contents of packets
  • Analyse captured VoIP call(s) (Menu - Telephony - VoIP calls)
    VoIP calls in Wireshark
  • Analyse captured RTP streams (Menu - Telephony - RTP - RTP Streams)
    RTP streams in Wireshark
Task: please do all above things yourself: record any network activity (e.g. you can visit HTTP or HTTPS website and record it in wireshark). You can use sample pcap file with SIP and RTP packets, single VoIP call.

IP protocol (IP)

The Internet protocol (IP) is designed to send data packets across IP networks from source to destination. The source and destination both have IP address. There are 2 most popular versions of the IP protocol: version 4 and version 6. In 2018 most widely used version is still version 4; in the IP v.4 the address includes 32 bits and is written as 4 numbers (bytes) from 0 to 255 separated by dot, example: Here is a task to check your new skills: please download this pcap file, open it in wireshark and see what is source IP address and what is destination IP address of the single packet inside the file. If you are unable to do it, please contact us and we will clarify this chapter. As you can see in wireshark, IP protocol data in the packet (also called IP header) has many fields, most importand are "Source", "Destination". The "Differentiated Services Field" (includes DSCP bits) is used to prioritise VoIP traffic in IP routers.
IP packet header in wireshark
Tasks with this file:
  • See what are source and destination IP addresses of the first packet in file (SIP INVITE packet)
  • See what is "Differentiated Services Field" of the first INVITE packet
  • See what are IP addresses for RTP packets
  • See what is "Differentiated Services Field" of the all RTP packets, is there any pattern
  • See what is average time period (in milliseconds) between consequent RTP packets going from one source IP address to another
  • See what is size of RTP packet, what is size of largest and smallest SIP packet
  • See what is size of IP header

Transport protocols: User Datagram Protocol (UDP)

The UDP is designed to send data over IP protocol and UDP header contains fields "Source port" and "Destination port". The port is a 16-bit integer having values from 0 to 65535. Multiple applications running on source and destination at the same time use different port numbers; the port number identifies application which is linked to the port. Standard port number for SIP protocol is 5060; for RTP protocol port numbers are assigned dynamically. The UDP protocol has no means to control delivery of packet from source to destination, so the packets could be lost in IP network. Small packet loss (less than 2%) is normally handled by SIP and RTP.
Tasks with this file:
  • See what is source and destination UDP port number for both SIP and RTP packets in the file

Transport protocols: Transmission Control Protocol (TCP)

In some cases VoIP calls go over Transmission Control Protocol (TCP). In other words same SIP packet data can be encapsulated into either TCP or UDP packets. The TCP protocol header contains source and destination port numbers as UDP, it also contains fields used to control guaranteed delivery of packets. TCP protocol provides guaranteed delivery of packets in same order from source to destination, it retransmits lost packets unlike UDP protocol. On the other hand TCP adds extra delay for setting up connection, increasing Post-dial delay (PDD). Besides the guaranteed delivery of data, use of TCP instead of UDP for VoIP means
  • Better passing through IP routers (NAT, firewalls): TCP is used for HTTP and HTTPS web browsing and it works everywhere
  • Better security: source IP address in TCP can not be spoofed
  • SIPS: encryption of SIP data using TLS method (similar to HTTPS encryption)
Task: open this file to see which TCP ports are used by caller and called endpoints.
Note: wireshark does not automatically recognize TCP packets in this file as SIP. Please select TCP packet, click "Decode As" - enter "SIP" in table field "Current" - OK

Session Initiation Protocol (SIP)

The SIP is most widely used protocol for VoIP calls. It defines specific format of messages (SIP packets) and their sequence to make calls over IP networks. The SIP messages are carried by means of UDP, TCP, or TLS/TCP (it is called SIPS). A typical sequence of SIP messages during a VoIP call (SIP call flow diagram) is displayed in beginning of the book, here we show it one more time:
SIP call flow diagram, SIP messages sequence, typical SIP call
  • INVITE packet (request from endpoint A to endpoint B) means initiation of call: "A wants to make new call to B"
  • 100 Trying packet (response from B to A to INVITE) means confirmation of INVITE packet delivery: "B received the INVITE". 100 means response code
  • 180 Ringing packet (response from B to A to INVITE) means indication of ringing state: "Phone B started to make rings". At ths time caller (A) starts to hear ringback tone. 180 means response code
  • 200 OK packet (response from B to A to INVITE) means indication of answer (connection, start of billing): "Phone B answered to the call". 200 means response code
  • ACK packet (from B to A) means confirmation of 200 OK packet delivery: "A received the 200 OK"
  • BYE packet (request from A to B) means termination of call: "Phone A ends the call"
  • 200 OK packet (response from B to A to BYE) means confirmation of BYE packet delivery: "B received the BYE"
Task: open this file in wireshark, go to "Telephony - VoIP calls" in menu and see SIP call flow diagram for the recorded VoIP call.
Other possible scenarios of VoIP calls:
  • Call is ended by B after answer
    SIP call flow diagram, SIP messages sequence, typical SIP call
    • Difference from previous flow is initiator of BYE packet
    Task: open this file in wireshark, see which party (A or B, caller or callee) ended the call with BYE.
  • Call is ended by A before answer
    SIP call flow diagram, SIP messages sequence, typical SIP call, CANCEL before answer
    • Caller party (endpoint A) sends SIP packet "CANCEL" before connection - means "A ends the call before answer"
    • Called party (endpoint B) responds to CANCEL with "200 OK" - means "B confirms receiving of CANCEL"
    • Called party (endpoint B) responds to initial INVITE with "487 Request Terminated" - means "B confirms ending of call". 487 means response code
    • Caller party (endpoint A) sends ACK to 487 packet - means "A confirms receiving of 487"
  • Call is rejected by B (ended before answer)
    SIP call flow diagram, SIP messages sequence, typical SIP call, rejected
    • Called party (endpoint B) responds to initial INVITE with "486 Busy Here" - means "B rejects the call with status = busy". 486 means response code
    • Caller party (endpoint A) sends ACK to 486 packet - means "A confirms receiving of 486"
  • Normal call flow (same as on first diagram) but with audio ringtone coming from B (can be ringback tone or music)
    SIP call flow diagram, SIP messages sequence, typical SIP call, with early media
    • Called party (endpoint B) responds to INVITE with "183 Session Progress" which means initiation of audio channel between A and B
    • RTP media packets (audio signal) are sent between endpoints after "183", before "200 OK" (answer signal), it is called early media. Note that it is bidirectional audio stream, but it does not mean that SIP servers always pass early media from A to B. It depends on every particular softswitch. From our experience we see that most softswitches really pass early media from caller to called party in early media, before start of billing. It is safe for caller-side software because caller does not pay for early media RTP traffic.
    • Does this file contain early media (RTP packets before 200 OK)?
    • Does this file contain early media?
Caller and called parties are also called SIP endpoints, A and B, user agents, UAC (User Agent Client) and UAS (User Agent Server). Full technical documentation is available in RFC3261 standard document published by IETF. Also, you can find many articles and tutorials published on our website.
SIP response codes
In this book we publish only most needed information about SIP, here is list of most popular response codes. Full list of SIP response codes is in RFC3261,
  • 100 Trying - confirmation about receiving INVITE request
  • 180 Ringing - indication of ringing phone and ringback tone
  • 183 Session Progress - set up of early media (sound coming from destination with custom ringback tone before answer, during call setup)
  • 200 OK - answer of the call, connection between A and B endpoints
  • 302 Moved Temporarily - redirection of call to another destiantion
  • 400 Bad Request - indication of incorrect format (malformed request packet). Usually the malformed packets are sent by hackers. Normally the 400 response is received if endpoints have different understanding of SIP packet format and syntax (e.g. if made by different vendors)
  • 401 Unauthorized - it is actually request of another INVITE with encrypted user/password credentials
  • 403 Forbidden - means that the caller has no right to make call (invalid user name or password, invalid source IP address)
  • 404 Not Found - usually means wrong number format or call to non-existent extension in PBX
  • 401 Proxy Authentication Required - it is actually request of another INVITE with encrypted user/password credentials. Same as 401, but sent by SIP servers
  • 408 Request Timeout - means that UAC did not receive any response from UAS after series of INVITE retransmissions. Usually happens when destination server is down, when sending call to wrong IP address and port, when INVITE is blocked by firewall
  • 486 Busy Here - destination is busy (usually busy by another call)
  • 487 Request Terminated - usually follows after CANCEL SIP packet, when call is cancelled, ended before answer
  • 488 Not Acceptable Here - usually means incompatible codecs between UAC and UAS
  • 500 Internal Server Error - some internal error happened in the UAS, usually indicates a bug in VoIP server, or malformed INVITE request
  • 503 Service Not Available - usually means temporarily overload: UAS can not accept more calls at this time
  • 603 Decline - means generic declining for various reasons
Task: see SIP response code in this file
SIP trace

SIP trace is a series of captured/recorded SIP packets, SIP packet data represented as text. The SIP trace is necessary element for VoIP troubleshooting. When something goes wrong with your phone system, the first thing to look at is SIP trace. SIP trace and pcap recordings provide evidence of VoIP issues.

SIP messages contain request/status line (red), headers (blue) and body (green) (format of body is defined by protocol SDP). Here is sample SIP trace:
SIP messages for Call-ID 524c3ea2fdd1401ab4a90c940b073359

17:42:22.442 [+0,00ms] [TX] INVITE to
Via: SIP/2.0/UDP;rport;branch=z9hG4bKPjefcc1928b9d44bf68b202f5674f15b9c
Max-Forwards: 70
Contact: <sip:unknown@>
Call-ID: 524c3ea2fdd1401ab4a90c940b073359
CSeq: 17970 INVITE
Supported: 100rel, timer
User-Agent: StarTrinity.SIP 2018-06-14 19.29 UTC
Session-Expires: 72000;refresher=uas
Content-Type: application/sdp
Content-Length: 291

o=- 3739369344 3739369344 IN IP4
c=IN IP4
t=0 0
m=audio 21504 RTP/AVP 8 0 4 18 101
a=rtpmap:8 PCMA/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:4 G723/8000
a=rtpmap:18 G729/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15

============================================================end of message=================
17:42:22.532 [+90,26ms] [RX] Trying from
SIP/2.0 100 Trying
Via: SIP/2.0/UDP;rport=5090;received=;branch=z9hG4bKPjefcc1928b9d44bf68b202f5674f15b9c
Call-ID: 524c3ea2fdd1401ab4a90c940b073359
From: <>;tag=d03ca0a253134ecc82c17a89e4b31e45
To: <>
CSeq: 17970 INVITE
Supported: 100rel, timer
Content-Length: 0

============================================================end of message=================
17:42:22.542 [+100,16ms] [RX] Session Progress from
SIP/2.0 183 Session Progress
Via: SIP/2.0/UDP;rport=5090;received=;branch=z9hG4bKPjefcc1928b9d44bf68b202f5674f15b9c
Call-ID: 524c3ea2fdd1401ab4a90c940b073359
From: <>;tag=d03ca0a253134ecc82c17a89e4b31e45
To: <>;tag=e5316920762f4d01b4618882a3fbcad4
CSeq: 17970 INVITE
Supported: 100rel, timer
Contact: <sip:123456@>
Content-Type: application/sdp
Content-Length: 251

o=- 3739358524 3739358524 IN IP4
c=IN IP4
t=0 0
m=audio 26000 RTP/AVP 8 101
a=rtcp:26001 IN IP4
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15

============================================================end of message=================
17:42:25.584 [+3 142,67ms] [RX] Service Unavailable from
SIP/2.0 503 Service Unavailable
Via: SIP/2.0/UDP;rport=5090;received=;branch=z9hG4bKPjefcc1928b9d44bf68b202f5674f15b9c
Call-ID: 524c3ea2fdd1401ab4a90c940b073359
From: <>;tag=d03ca0a253134ecc82c17a89e4b31e45
To: <>;tag=e5316920762f4d01b4618882a3fbcad4
CSeq: 17970 INVITE
Supported: 100rel, timer
Content-Length: 0

============================================================end of message=================
17:42:25.584 [+3 142,70ms] [TX] ACK to
Via: SIP/2.0/UDP;rport;branch=z9hG4bKPjefcc1928b9d44bf68b202f5674f15b9c
Max-Forwards: 70
Call-ID: 524c3ea2fdd1401ab4a90c940b073359
CSeq: 17970 ACK
Content-Length: 0

============================================================end of message=================
===============================saved by StarTrinity SIP Tester at 2018-06-30 18:27:45======
You can see INVITE request sent by UAC, multiple responses sent by UAS (100 Trying, 183 Session Progress, 503 Service Unavailable). SIP Request Line INVITE SIP/2.0 means request type = "INVITE" (making a VoIP call), SIP protocol version = "2.0" (the only one version used, as we see in 2018), and destination SIP URI = "". The URI in SIP is simular to URL in HTTP, it specifies target for the request. "sip:" or "sips:" here define protocol type, "123456" is called number (telephone number of B endpoint).
In next chapters we will describe the SIP trace in detail.
Types of SIP packets
SIP Requests also known as SIP methods, SIP request methods. Most popular SIP request types:
  • INVITE - most widely used SIP request, means request to make new VoIP call from UAC to UAS. When the INVITE is sent after connection (after "200 OK response") it usually means change of audio codec, putting call on hold, or sending/receiving fax
  • ACK - confirmation that response was delivered to UAS
  • BYE - ends current VoIP call after answer, sent by UAS or UAC
  • CANCEL - ends a call before answer, sent by UAC
  • REGISTER - sent from SIP phone user-agent to SIP server to register the phone to specific extension (i.e. internal phone number)
  • REFER - starts call transfer procedure: e.g. A makes call, B answers, understands that call needs to be passed to C and sends request to B: "please transfer to C"
  • We have listed only most popular SIP request types, see others in RFC3261
SIP Responses The SIP response packets are is sent back to requesting endpoint and contain SIP response code and optional error details in SIP headers:
SIP/2.0 500 Internal Server Error
Via: SIP/2.0/UDP;branch=z9hG4bK-10854-1-0
From: sipp <sip:111@>;tag=10854SIPpTag001
To: sut <sip:222@>;tag=38404-23F2
Date: Tue, 05 Feb 2013 05:49:37 GMT
Call-ID: 1-10854@
Allow-Events: telephone-event
Warning: 399 "Transcoder Not Configured"
Server: Cisco-SIPGateway/IOS-15.3.20130202.123643.
Reason: Q.850;cause=16
Content-Length: 0
In this example "Warning" header gives details about the error and "Reason" header gives information about PSTN release code.
If a packet is lost somewhere between the endpoints A or B (UDP protocol does not gaurantee delivery of packets), the SIP packets are retransmitted.
Task: see types of SIP packets (requests, responses), and number of retransmitted packets in file #1, file #2, file #3. See what is destination IP address where packets are retransmitted to, do you think it is correct or not?
SIP headers
A header in SIP packet has a simple format: [name]: [value]. SIP headers are separated by line breaks. Most widely used SIP headers are
  • From header indicates identity of caller, usually contains Caller ID (ANI, CLI, A number). Also contains unique "tag" set by UAC
  • To header identifies recipient of the call, usually contains Called ID (called number, B number). Also contains unique "tag" set by UAS
  • Via header saves information about proxy servers - server IP address and port - to pass response from UAS back to UAC via same set of proxy servers
  • Record-Route header is optionally inserted by proxy servers to route subsequent SIP requests for the same SIP call
  • Call-ID header identifies SIP call along with "From" and "To" headers
  • Contact header specifies IP address and port to be used by peer SIP endpoints for subsequent SIP requests for the same SIP call
  • CSeq header contains a decimal number which increases by 1 with every subsequent SIP request within the same SIP call, with the exception of CANCEL and ACK requests which use the CSeq number of corresponding INVITE request
  • Server header indicates model, software/firmware version of UAS
  • User-Agent header indicates model, software/firmware version of UAC
  • Content-Length header as you will read later, INVITE and "200 OK" packets usually contain encapsulated SDP protocol data with information about RTP audio streams. This header specifies size of encapsulated SDP (or other) data
  • Content-Type header specifies type of encapsulated data. Equals to "application/sdp" in case of encapsulated SDP
  • Allow header lists set of SIP requests supported by user agent which generated the SIP message
  • Supported header specifies set of supported SIP extensions by user agent which generated the SIP message
  • Require header specifies set of required SIP extensions by user agent which generated the SIP message
  • Session‑Expires header - part of RFC 4028 (Session Timers) SIP extension - specifies max. interval to refresh SIP call with keep-alive packet (the RFC3261 standard does not define way to abort hang calls)
  • Expires header specifies lifetime of corresponding SIP request
  • WWW-Authenticate, Authorization headers are used for digest authentication
There are many more headers; adding custom headers into requests and responses is a way to extend SIP protocol. Extensions of SIP protocol are defined by additional RFC standard documents (beyond RFC3261), you can find them here.
  • See SIP headers in this file: what software is used for UAC and UAS? What is SIP Call ID?
  • See how to fix Contact header sent by UAS in this file
Registration in SIP means passing information about user agent to server, it is implemented with REGISTER. It means "Hello SIP server, here are my contact details for extension number 456: IP address, SIP port 5060". Without registration SIP phones must know IP addresses of every destination phone; with registration the IP addresses are stored by SIP server, also known as SIP registrar server. Her is example SIP trace for the registration process:
SIP messages for Call-ID 64b10daa51dd4c5a9d6038a43163b22d

14:32:00.599 [+0,00ms] [TX] REGISTER to
Via: SIP/2.0/UDP;rport;branch=z9hG4bKPj609d6772ba754dcab9cba92bcb34be69
Max-Forwards: 70
From: <>;tag=aef6297c37914f69832efc2c44c980aa
To: <>
Call-ID: 64b10daa51dd4c5a9d6038a43163b22d
CSeq: 22151 REGISTER
Contact: <sip:100@>
User-Agent: StarTrinity.SIP 2018-06-14 19.29 UTC
Expires: 3600
Content-Length: 0

============================================================end of message=================
14:32:00.674 [+74,84ms] [RX] Unauthorized from
SIP/2.0 401 Unauthorized
Via: SIP/2.0/UDP;rport=5090;received=;branch=z9hG4bKPj609d6772ba754dcab9cba92bcb34be69
Call-ID: 64b10daa51dd4c5a9d6038a43163b22d
From: <>;tag=aef6297c37914f69832efc2c44c980aa
To: <>
CSeq: 22151 REGISTER
WWW-Authenticate: Digest realm="*",nonce="6ab702ae1b742592",opaque="66982f5173f64308",algorithm=md5
Content-Length: 0

============================================================end of message=================
14:32:00.674 [+74,88ms] [TX] REGISTER to
Via: SIP/2.0/UDP;rport;branch=z9hG4bKPjcfb02f366c7f435ba7991cbb13b9497c
Max-Forwards: 70
From: <>;tag=aef6297c37914f69832efc2c44c980aa
To: <>
Call-ID: 64b10daa51dd4c5a9d6038a43163b22d
CSeq: 22152 REGISTER
Contact: <sip:100@>
User-Agent: StarTrinity.SIP 2018-06-14 19.29 UTC
Expires: 3600
Authorization: Digest username="100", realm="*", nonce="6ab702ae1b742592", uri="", response="abd79c866205325d0511a95b1e5f5087", algorithm=md5, opaque="66982f5173f64308"
Content-Length: 0

============================================================end of message=================
14:32:00.758 [+158,50ms] [RX] OK from
SIP/2.0 200 OK
Via: SIP/2.0/UDP;rport=5090;received=;branch=z9hG4bKPjcfb02f366c7f435ba7991cbb13b9497c
Call-ID: 64b10daa51dd4c5a9d6038a43163b22d
From: <>;tag=aef6297c37914f69832efc2c44c980aa
To: <>
CSeq: 22152 REGISTER
Contact: sip:100@;expires=3600
Content-Length: 0

============================================================end of message=================
===============================saved by StarTrinity SIP Tester at 2018-07-02 14:32:12======
As you can see in the SIP trace, first REGISTER request sent by UAC to UAS means resitration of extension 100 at SIP server on SIP port 6000. The UAC sends its IP address in Contact header: Contact: <sip:100@>. "Expires" SIP header equals to 3600 means that the registration entry should be valid for 3600 seconds - in 60 minutes the registrar server should clear the registration entry if not received new REGISTER requests for this extension. REGISTER request with "Expires: 0" means unregistration.
Digest authentication
As you can see in previous SIP trace (with REGISTER), first request is rejected with status "401 Unauthorized" and WWW-Authenticate header, it is called digest challenge. The WWW-Authenticate SIP header contains nonce and opaque values which are used by UAC along with user name and password to send second INVITE request. The second INVITE request contains Authorization header with digest response, it is used by UAS to verify user name and password. If user name and password are correct, UAS sends "200 OK" response, otherwise it sends "403 Unauthorized" response.

Session Description Protocol (SDP)

The content is not ready yet

RTP protocol

The content is not ready yet

Encrypted RTP: SRTP

The content is not ready yet

RTCP protocol

The content is not ready yet

Types of VoIP issues

Types of VoIP issues by symptom
  • Failed calls
  • Calls dropped after connection
    • no response to 200 OK - firewall on UAC side [sample sip trace]
    • 200 OK sent to wrong destination. UAC behind NAT
    • failed re-invite or UPDATE (ping) request (done by session timer)
  • Audio quality issues
    • No audio
    • One-way audio
    • Bad audio quality
    • High round-trip delay
Types of VoIP issues by root cause
  • Firewall misconfiguration
    • Router or server may require reboot to apply new firewall settings

VoIP quality indicators

Multiple-calls quality indicators

Indicators applicable for VoIP route, VoIP traffic, VoIP provider, VoIP originator.
  • Answer seizure ratio (ASR). Averaging interval
  • Average call duration (ACD). Averaging interval
  • Post-dial delay (PDD)

Single-call quality indicators: audio quality

  • RTP max delta
  • RTP delta
  • RTP jitter
  • RTP packet loss
  • RTP packet loss burst rate
  • RTP packet loss in jitter buffer caused by jitter
  • RTP packet loss in jitter buffer caused by RTP clock skew
  • G.107 MOS
  • Confidence score (0-100%)

VoIP troubleshooting methods

  • Understanding components of the VoIP system and interaction between them
  • Short-time experiments
  • Long-time experiments

Real VoIP troubleshooting cases

The real cases are reported with permission of our customers and readers of the book.

Case #1: Mitel 6873i SIP phone registration at 3CX IP PBX

The issue has been reported by Telin, cloud IP PBX service provider, reseller of 3CX:
We have new Mitel 6873 handsets (SIP phones) deployed at a client's location and there appears to be an issue with the phones connect back to a 3CX server at the same time when another softphone connects to the same extension. It creates a contact name mismatch and mitel phone unregisters. We have been working with mitel for a couple weeks and they are pointing their finger at 3CX and 3CX pointing finger back at them.
Troubleshooting steps:
  • Clarify the situation with Telin. What happens in terms of SIP protocol
    • At the same time both Mitel phone and 3CX softphone send REGISTER to 3CX server to same extension. Mitel phone sends via UDP. 3CX softphone sends via tunnel.
    • The Mitel phone receives 200 OK response to the REGISTER, this response contains multiple "Contact" headers.
    • The Mitel phone displays an error related to registration, and calls do not work. When 3CX softphone is not registered, everything is fine.
    Response from Mitel:
    Mitel phone analyses contact header which is received from server in response to register. It does not accept the syntax.
    Contact: <sip:003@;transport=TCP;rinstance=1-xs7ou4dgsbv7tg5.ntvyho6kmku6kv-a;ob;inst="D7FCE2">;reg-id=1;+sip.instance="<urn:uuid:00000000-0000-0000-0000-00008340d0fa>";expires=74

    Contact: <sip:003@;transport=TCP>;rinstance=1-xs7ou4dgsbv7tg5.ntvyho6kmku6kv-a;ob;inst="D7FCE2";reg-id=1;+sip.instance="<urn:uuid:00000000-0000-0000-0000-00008340d0fa>";expires=74

    The problem seems to be in incorrect contact header in the 200 OK for the softphones. Below is one of the sipphone contact headers in NotWorking trace. In this contact header the > sign is added in an incorrect place which takes some of the feature parameters like rinstance inside the sip: part. I have shown the incorrect and correct way (based on how Mitel phone processes the contact header).
  • Clarify the situation ourselves by making the experiment one more time, sniffing working and non-working REGISTER cases.
    • Working case: one mitel phone registers and receives "200 OK" response to REGISTER with following contact header:
      Contact: "Mitel Phone"<sip:701@>;+sip.instance="<urn:uuid:00000000-0000-1000-8000-00085D5C0866>";expires=120
    • Not working case: one 3CX softphone is registered, at the same time one more (mitel) phone send REGISTER with contact header
      Contact: "Mitel Phone" <sip:701@>;+sip.instance="<urn:uuid:00000000-0000-1000-8000-00085D5C0866>";expires=0
      and receives "200 OK" response to REGISTER with following 3 contact headers:
      Contact: "Mitel Phone"<sip:701@>;+sip.instance="<urn:uuid:00000000-0000-1000-8000-00085D5C0866>";expires=120
      Contact: "Mitel Phone"<sip:701@;rinstance=1-hkg2btlqkdv6ydfaixt62uwrre1zz4sd;inst="b98a8bbf">;expires=62
      Contact: <sip:701@;rinstance=f27cf6aeccdab97f>;expires=55
      We see that the response is not correct. It muct contain only 2 contact headers, not 3, since only 2 phones are registered to one extension
  • At this point we know that we have a multi-component system and an issue related to one or multiple components, or to combination of components. It is helpful to understand which component causes the issue. So we make a new experiement with a new system containing different components: 3CX PBX, Mitel phone and Polycom phone: we replace 3CX softphone by Polycom phone to see what happens. This is what we observe:
    • In this combination it works. All phones registered successfully
    • Contact headers in "200 OK" response to REGISTER:
      Contact: "Mitel Phone"<sip:701@>;+sip.instance="<urn:uuid:00000000-0000-1000-8000-00085D5C0866>";expires=120
      Contact: <sip:701@;rinstance=f27cf6aeccdab97f>;expires=72
      We see that the 3CX PBX server responded correctly with 2 contact headers
    One more experiment with Polycom phone and Mitel phone. Polycom sends contact header
    and receives
    Contact: <sip:007@;rinstance=8d481b0693d20404>;expires=106
    - this is also correct, 2 contact headers in response, and it works in this combination too.
  • Based on the experiments we make conclusion and suggest possible ways towards solution
    • 3CX PBX is wrong in sending 3 Contact headers. Further action: submit bug report to 3CX team including SIP trace
    • Mitel phone could have a configurable setting whether to verify Contact headers in response to REGISTER or not. Further actions: try to downgrade to old firmware version which does not verify contact headers in response, or request a new version with the configurable setting
Submit my case

Chapters to be written in future

  • components of voip and telephone systems
    • jitter buffer
    • codec: encoder, decoder
    • PLC
    • SIP endpoints (User Agents)
      • pc phones
      • mobile dialers
      • hardware phones
      • VoIP-PSTN gateways: SIP-PRI, SIP-SIGTRAN-SS7, SIP-GSM, SIP-bluetooth
    • IP Network, other traffic running along with VoIP traffic. Congestions causing packet loss
    • IP routers, IP switches
      • Network Address Translation (NAT). port triggering. port forwarding
      • SIP going through NAT. Symmetric SIP behaviour on SIP servers
      • RTP going through NAT. Symmetric RTP behaviour on SIP servers
    • firewalls
    • VPN and other tunnels
    • Synchronous Bandwidth Optimizers (SBOs)
    • Internet Service Providers. Static IP, dynamic IP. NAT, firewall
    • Hosting providers. Amazon, Azure, NAT, firewall
    • SIP servers
      • Routing-billing engine
      • Proxy servers
      • Softswitches
      • Session Border Controllers (SBCs)
        • Heterogenous SIP implementations
        • SIP attacks
        • Fraudulent VoIP traffic
      • IP PBX servers
    • voip route providers, sip trunks. direct routes. PDD, ASR, ACD. FAS, international bypass
    • Call generators and receivers. Looped generated traffic.
    • Public Switched Telephone Network (PSTN)
  • network impacts. jitter, packet loss, rtt. report via rtcp within 1 hop. non-visibility after sbc.
  • cases of troubleshooting. 1 works in one case, doesnt work in another case. comparison method.
  • intermittent bugs.
  • software tools and services for VoIP tests.
    • active testing, passive monitoring - startrinity, sipp, gl
    • continuous monitoring, one-time tests - voipmonitor, startrinity
    • test automation. unit tests, unit test frameworks.
    • voip routes testing., chekmyroutes. cli delivery
    • penetration test tools
    • IVR audio verification / 2-way audio verification test tools
  • test cases by object of testing
    • ip network. nat. bandwidth, dscp
    • softswitch, pbx,cucm
    • gateway sip-pstn
    • call center
    • PSTN
    • conference servers
    • ivr servers. ivr testing audio verification
    • voip routes. cli delivery. uptime. fas, overbilling. voip-OTT routes
  • test cases by symptom
    • always happens or intermittent issue?
    • ways to increase probability of intermittent issue. stress testing. complexity of test cases. difference between test case and real voip calls.
    • audio issues: chopped sound, one way audio, no audio. nat issues. jitter, loss, dscp flag
    • call failure:408, 503,other codes
    • server memory leak
    • server intermittent crashes
    • server sip and rtp stack software upgrade, stability test
  • test scenarios (scripts)
  • regular dialing: same number, params from csv
  • monitoring, email notifications
  • SIP Security, SIP Trunking
  • list of RFC documents related to SIP. general overview for every RFC. RFC3261, RFC 5359
  • VoIP fraud types
    • FAS: types: IVR, LDFAS, etc. generation, detection. 200OK-ACK delay
    • International bypass: OTT, GSM. Blending. CLI delivery testing. CDR profiling, filtering
    • Sending dialer traffic
    • Brute force attack
    • Man-in-the-middle attacks
    • Sending malformed SIP packets
    • DOS attacks (REGISTER, INVITE). DDOS attacks.
    • UDP flooding: SIP, RTP. TCP SYN attack. Role of SBC's
    • Weak/default passwords - brute force attacks
    • UDP source address spoofing
    • Spam over IP telephone
    • Toll fraud
    • Vishing - Phishing over VoIP
    • Missed calls with CLI = IPRN
    • Traffic to IPRN while route provider is not aware of IPRN ranges
  • Fraud tools
    • SIPVicious
    • Mr.SIP
Copyright 2011-2018 | Blog | Contact lead developer via LinkedIn | TeamViewer link