Overview

To determine the probable cause, it’s recommended to use the troubleshooting view along with the aggregate information derived from the Webex Calling Analytics dashboard. Use the Webex Calling Analytics dashboard to view issues impacting multiple users with a common cause that is related to location, network, and so on; while the Webex Calling troubleshooting view can identify issues with individual calls.

The media quality troubleshooting feature doesn’t require any specific configuration and is available by default in the Control Hub. It’s available to full customer, read-only, support, partner, and partner read-only administrators.

Administrators can search using the following criteria to get a list of calls where a media session was used with at least one Webex Calling registered endpoint:

  • Email IDs

  • Phone numbers (exact string match)

  • MAC address

  • Call IDs

The media quality troubleshooting allows administrators to:

  • View the end-to-end experience of the participants of the call.

  • View a hop detail of the call.

  • View if the media traverses through the Webex Calling cloud, or directly between the users (using Interactive Connectivity Establishment (ICE)).

  • View Insights, if there’s no media in the call, or when the path optimization setup was unsuccessful.

  • View calls for the past 21 days.

  • Analyze the call quality metrics that impacted the experience of the user. For example, an administrator may observe high jitter on clients over Wi-Fi networks, but packet loss and latency may be acceptable.

  • Detect if the issue is with the caller or the callee.

Media Quality Troubleshooting applies only for media sessions, and it doesn’t show call signaling sessions. For example, Alex calls Bob who has setup CFA (Call Forward All) to Christina, and Christina answered the call. In this scenario, the media troubleshooting view shows that the call occurred between Alex and Christina, as the focus is on troubleshooting media experience rather than the signaling flow or the call life cycle.

The calls using Webex Calling appear after the call ends.

The troubleshooting view helps to identify the problem area by providing all the relevant metrics and can’t necessarily provide you the root cause for a poor call. Look at these pointers to identify various factors and determine the resolution options:

  • The end-to-end experience of the user.

  • The Hop Details view.

  • Send or receive metrics from the user or the media relay point.

  • Whether the call occurred to or from an external network or between the Webex Calling registered endpoints.

Supported Call Flows

The media quality reports are collected from the caller and callee endpoints and the media relay points. This allows a segmentation of the media experience to narrow down and identify whether the issue occurred at the:

  • Caller or callee

  • Media path to or from the Webex Calling cloud.

Call legs appear if there was a media session that is established with at least one Webex Calling registered endpoint on the call. For example, for an outbound call from hunt group to eight agents, if only one agent answer, then there is no media experience to troubleshoot for the other seven agents.

There are five types of media experiences or paths for Webex Calling troubleshooting, which are:

  • On-net Optimized – Calls within the organization where ICE is successful and the media flows directly between the users. See Webex Calling Media Optimization with Interactive Connectivity Establishment (ICE) for detailed information.

  • On-net Unoptimized – Calls within the organization where Interactive Connectivity Establishment (ICE) was not possible or established. In this scenario, the media flows through Webex calling cloud.

  • On-net Cloud Hosted – Calls within an organization where media is provided by a media server that is hosted in the cloud (for example, listening to voicemail, dialing into an auto attendant).

  • Off-net Call to or from the Webex Calling registered endpoint -

    • via Cloud Connected PSTN Provider- Inbound and outbound calls of an organization where the other party is on the PSTN network. The media is relayed through a cloud connected PSTN provider (CCPP), over a high-quality interconnect.

    • via Local Gateway- Inbound and outbound calls of an organization where the other party’s media is through the enterprise. Behind the local gateway the media session can be from enterprise hosted user (for example, registered to call control in the enterprise) or from PSTN, where PSTN is provided by the enterprise.

If there are 1 or 2 Webex Calling registered users who are involved in a point-to-point on-net call, then the troubleshooting view presents metrics for one or both the parties. If the call is off net (User 1 receives a call from a PSTN user), then the troubleshooting view presents only User1's client metrics, along with the metrics taken from the media relay point.

Most of the call scenarios in the troubleshooting view show two call legs (caller and callee); however, some of the call scenarios (such as call park or retrieve) show only one call leg. In such cases, the other call leg shows up separately in the troubleshooting view. This does not hinder troubleshooting the call and detecting where the problem occurred. However, it does require the admin to manually correlate the two call legs using a common entity such as overlapping time. Future enhancements to the troubleshooting view will eliminate the need to use manual lookups.

Accessing the Webex Calling troubleshooting view

To analyze a Webex call, perform the following:

1

From the customer view in https://admin.webex.com/, go to Monitoring > Troubleshooting.

2

Select Meetings & Calls then search for the email ID of the user or device, phone number (exact string match), MAC address of the user or device, or the Call IDs of the call leg you’d like to view.

Displays the information for all calls and meetings that are associated with the search.

The listing view displays the call made using at least one Webex Calling registered endpoint and having a media session.

3

The Webex Calling calls are graded for quality. However, for the Webex meetings or Call on Webex sessions, this grading doesn’t apply. The call experience is graded as:

  • Poor – indicates that either the caller or the callee had a poor end-to-end experience (for example, choppy audio).
  • Good – indicates the end-to-end experience for the caller and the callee didn’t exceed thresholds.
  • None– applies to meetings or Call on Webex sessions.
  • Not-available– applies to meetings or Call on Webex sessions.
4

Click a specific call on the listing view, to inspect the Hop details. The Hop Detail view displays:

From the Hop details, you can:

  • View insights about the call in either or both these scenarios:

    • There was no-media for any of the hops related to the call

    • The Path optimization setup was unsuccessful.

  • Hover on the device to view the end-to-end call experience.

  • Hover on the hop between the endpoint and Webex Calling cloud to view the hop details.

The end-to-end call experience is based on the media quality data that is collected from each Webex Calling registered endpoint (Webex App or device such as 8865 or Desk Pro) at the end of the call. The call is graded as good, if it satisfies these thresholds:

  • Packet losses less than 5%

  • Latency or Round Trip Time (RTT) less than 400 ms

  • Jitter less than 150 ms

The quality of the hop is derived from data that are collected from the media relay point in the Webex Calling cloud. For PSTN calls through CCPP or local gateway, the data collection is from the Webex Calling cloud and not from the PSTN endpoint. A hop is graded as good, if it satisfies these thresholds.

  • Packet loss less than 2.5%.

  • Latency or RTT less than 200 ms.

  • Jitter less than 75 ms.

The hop metrics vary during a session depending on the sampling time and variability in the network. The values that are reported by the media relay point and the clients (end-to-end experience) may not align. However, they should be in close alignment to allow segmentation along the path.

We recommended using the individual call troubleshooting view with the aggregate information that is derived from Analytics.

Let’s analyze the call quality for the different call types using the troubleshooting view.

On-net Optimized Call Flow

Calls within the organization where ICE is successful and the media relay in the cloud is removed from the path. The media flow is directly between the user's devices.

Inference:

1

The call is graded as good as both the caller and callee have good end-to-end experience.

2

The administrator can observe that media flows directly between the two users and does not travel through Webex calling cloud.

Optimized call flows can have a poor experience, if the enterprise or local network is the source of the issue, since the media between the two users will traverse the local network. Latency or RTT is always lower on an optimized call but packet loss and jitter can still be a factor depending on the network between the two users.

On-net Unoptimized Call Flow
Calls within the organization where an ICE call isn’t successful and the media flows through the Webex Calling cloud.

Inference:

The administrator can observe the following:

1

The caller’s end-to-end experience is graded as poor.

2

There is a problem with the caller's network hop that affects both the send and the receive stream.

3

The callee’s network hop does not have an issue.

4

The callee’s end-to-end experience is graded as poor due to the issue from the caller.

On-net Cloud Hosted Call Flow
Calls within an organization where the media is provided by a media server that is hosted in the cloud.

Inference:

The call is graded as good as the caller’s end-to-end experience is within the accepted thresholds. The administrator can observe that the following:

1

The network hop for the caller is graded as poor as some of the metrics are above the acceptable threshold.

2

The send stream from the voicemail is graded as good per the metrics collected from the media relay point.

3

The media server used to collect or deposit the voicemail does not currently report metrics. However these servers are hosted and managed as part of the Webex Calling Cloud so the quality of that link segment is internal and always high quality, low latency.

4

The administrator can observe the caller’s end-to-end experience is graded as good although the hop is graded as poor. This is due to the callee's hop having a good network that compensates for the performance degradation of the caller's network.

Off-net PSTN through CCPP
Calls in and out of an organization where the other party is on the PSTN network. The media is relayed from a cloud connected PSTN provider (CCPP).

In the example, the client media is coming from a PSTN provider through the cloud.

The call is graded as poor as the callee’s end-to-end experience is not within the accepted thresholds. The administrator can observe the following:

1

The problem is with the caller’s PSTN hop specifically with the sending stream.

2

The callee’s network hop does not have an issue.

3

The callee’s end-to-end experience is graded as poor due to the issue from the caller.

4

The end-to-end experience and the receive stream metrics of the caller who is on the PSTN network are not currently available as these metrics are not transmitted to the Webex Calling Cloud.

Off-net PSTN through Local Gateway

Calls in and out of an organization where the caller's media is coming from an enterprise. The media session can be from an enterprise hosted user (for example, registered to UCM) or from PSTN, where the PSTN is provided through the enterprise.

Inference

The call is graded as poor as the caller’s end-to-end experience is not within the accepted thresholds. The administrator can observe the following:

1

There is a problem with the caller’s hop to the Webex Calling Cloud both on the send and receive stream.

2

The caller’s end-to-end experience is graded as poor either due to the issue observed in the hop or the issues at the user’s end (devices, network, and so on.)

3

The incoming traffic to the Webex Calling Cloud from the callee’s end is graded as good.

4

The end-to-end experience and the receive stream metrics of the callee who is called from the local gateway are not currently available as these metrics are not transmitted to the Webex Calling Cloud.

Troubleshoot the media quality issue

The hop-by-hop view helps you to locate where the problem occurred. Now that you’ve found where the problem is, and with the metrics (jitter, packet loss, latency) you can try the following to troubleshoot the issue.

Typical possibilities for media issues are:

  • Network/ISP/location specific issues - Due to firewall, network configuration or bandwidth there’s a pattern of poor experiences in a particular location or network subnet. Use the per call troubleshooting view (identify the location associated with the poor session) with the analytics view to review the aggregate patterns for the location.

  • User specific issues - A user or device is connected on a poor network (for example, Wi-Fi or working from home) which means their experience is impacted by the associated network capabilities. See the Use CScan to Test Webex Calling Network Quality article to identify the issue with the network.

  • Call type specific issues - A user’s poor experience is because of the quality on the far end. This is typical in PSTN scenarios where the user is talking to another user on a mobile network and the session has high packet loss on the PSTN network.

  • No-media issue- There may be no media transmission in some hops. The Insights banner displays the cause at the top of the Hop details page and the liable party in the information box of the hop detail page. Some of the possible causes for no-media in calls along with liable parties are listed here:

    • Webex not receiving media from the sender.

    • Webex not receiving media from the receiver.

    • Webex not receiving media from either direction.

    • Webex not sending media to the receiver. Webex engineering addresses this issue.

    • Webex isn’t receiving media from Cloud PSTN. Webex engineering addresses this issue.

    • Webex isn’t receiving media from cloud service. Webex engineering addresses this issue.

    • Webex isn’t receiving media from Local gateway. Customer administrator must investigate the issue.

  • Media Path Optimization Failure- Few calls cannot successfully set up media path optimization. The Insights banner displays the cause for unsuccessful ICE calls and the resolution at the top of the Hop details page.

    Some of the possible reasons are

    • ICE unsuccessful due to stun server access - see Webex Calling Port Reference Information

    • ICE unsuccessful due to connectivity check - verify connectivity between networks

    • ICE unsuccessful as the default path round trip time was similar/better than any optimized path

Legends on the troubleshooting view

See the following Call Details in the right pane of the hop-by-hop view.

TermDefinition
Calling DateThe date when the call occurred.
Calling TimeThe time of when the call started and ended, shown in the time zone that you selected in the search view.
Session Type

The type of session that is supported. For example: Webex Call

ParticipantsThe number of participants who joined the call.
Caller NameName of the caller.
Caller EmailEmail address of the caller.
Caller NumberPhone number that the caller used during the call.
AudioThe type of audio used.
VideoDisplays Yes if video is enabled by a participant. If video wasn't enabled at all, it shows No.
Path Optimization

Specifies if the call path optimization applies to the call. The accepted values are:

ICE (Interactive Connectivity Establish)

PNC (Private Network Connect).

No optimization

Calling Type

Calling Type can be one of the following:

Emergency

Enterprise

International

Mobile

National

Operator

Premium

Shortcode

Tollfree

Unknown

URI

View these call metrics in the hop-by-hop view:

TermDefinition
Endpoint

Displays one of the following:

  • Desk Phone for a physical endpoint

  • Webex App

  • Room Device for Cisco Room Series devices

Hardware

Displays one of the following:

  • Desk phone model information for a physical endpoint

  • “-“ for a Webex App

  • Room Series model information for Cisco Room Series device

Location

Webex Calling location that is configured for the user.

Local IP

The local IP address of the client for the network interface that it is using to transmit media. IP addresses are partially masked to preserve the personal identity of users.

Public IP

This is the public IP address of the client as seen by the cloud. For enterprises this is the address of the firewall providing the NAT. IP addresses are partially masked to preserve the personal identity of users.

MAC addresses

The MAC address of the client endpoint.

Geolocation

Geo lookup of Public IP address. This address is not accurate, if connected over PNC. If the user is using the Webex App and connecting to the enterprise through a VPN, the location is not accurate.

ISP

Internet Service Provider who provides network connectivity to the Webex Calling Cloud.

Network

The type of network connection that the client used to exchange media. Possible values are:

  • Wi-Fi

  • Ethernet

  • Cellular

  • Unknown

Audio Codec

(Send or Receive) The media encoding and decoding format in use for the media that are transmitted by a client.

Video Codec

(Send or Receive) The media encoding and decoding format in use for the media that are transmitted by a client. Applies only for a video call.

Call ID

The internal identifier that is used to identify the call leg.

Some metrics are masked in the article screenshots to preserve the identity of the user.

Limitations

Media quality metrics aren’t available from the following devices.

  • Analog phones

  • Third-party devices

  • IPv6 endpoints