Catalyst 3900 Troubleshooting
                                                                     Printable Pdf
Document ID: 12269

  • Introduction
  • Using the Statistics Menu
  • Using the Display Summary Screen
  • Using the Message Log Information Screen
  • Using the Switch Statistics Screen to Check the Total Throughput
  • Troubleshooting Port Connectivity and Throughput problems
  • Using the Port Status Screen to Check the Configuration
  • Troubleshooting Address Table Problems
  • Troubleshooting Spanning Tree Problems
  • Checking the VLAN Statistics
  • Checking the VTP Statistics
  • Related Information

Introduction

This document describes how to use the Cisco Catalyst 3900 Statistics screens to help troubleshoot various switching scenarios. It presents some of the most common problems faced with the 3900 switch and how the Statistics screens can be used to identify them.You can use the Token Ring acronyms document to explain some of the acronyms used. The document concentrates on the following areas:


  • Display Summary - The most important configuration and operational settings on the 3900 switch
  • Message Log - Important event and error messages the 39000 switch has logged
  • Switch Statistics - The behavior of the 3900 switch as a whole
  • Port Status and Statistics - Looking at the type of traffic on individual ports
  • Address Tables - Looking at the internal switching tables to see from which port an address has been learned
  • Spannning Tree Information - Looking at the various spanning tree modes running at the TrBRF and TrCFRF level

These areas are explained in more detail below.


Using the Statistics Menu

This is the main Statistics menu for the Catalyst 3900. The screens described below are all accessible from here. The information on all of the statistics menu screens is refreshed approximately every five seconds.


Using the Display Summary Screen

The display summary will probably be the first menu you will go to when troubleshooting the Catalyst 3900 switch. It will provide you with a detailed summary of the most important configuration and operational parameters. In most situations this will be the most powerful tool for you to use to troubleshoot the 3900 switch. The information it will give you will include:


  • Switch/Stack information
  • VLAN configuration
  • IP configuration
  • SNMP configuration
  • CDP configuration
  • Module configuration
  • Spanning Tree information
  • Port Status information
  • Message log
  • Hardware diagnostics

Many of the problems on the Catalyst 3900 switch are caused by running a back level release of software. One of the first uses of the display summary should be to check the software level which can be found under the Switch/stack information section. Make sure that you are running the latest software release available.

A very common cause of problems on the 3900 switch is a software misconfiguration. Check the software configuration which can be found under the VLAN configuration section carefully for any errors. Check especially that ring numbers configured between a 3900 switch and a router match (ring numbers are defined in decimal on a router and in hex on the 3900 switch).

Another very common cause of problems on the 3900 switch is an error in the Spanning Tree modes selected at the BRF and CRF levels. Use the Spanning Tree information section to carefully check that you have the correct Spanning Trees running. Click here and refer to the section Catalyst 3900 Spanning-Tree Support for assistance.

Much of the information in the summary display can be viewed individually by using the menus explained in the following sections of this guide.


Using the Message Log Information Screen

After the Display Summary, the Message Log Information screen will probably be the second menu that you will most commonly go to when troubleshooting the Catalyst 3900 switch. It will provide you with detailed information about events that have happened on the Catalyst 3900. Use this log history information to gain an understanding of events that have been logged on the Catalyst 3900.


Using the Switch Statistics Screen to Check the Total Throughput

The Switch Statistics Menu gives us the following information about the Catalyst 3900 switch as a whole: system up time, environmental statistics, stations in use, traffic and error counters. Use this screen to get an idea of the quantity of traffic the 3900 switch is passing (Frames Transmitted/Received and Currently active Stations/Largest Number of Stations counters). Check the Error Frames Received/Frames Lost counters to identify whether the 3900 switch is losing frames or receiving corrupted or non switchable frames from one or more of its ports.

Notice there is the equivalent of a clear counters at the bottom of the screen. Use the Reset option to set a fresh timemark by which to collect statistics.


Troubleshooting Port Connectivity and Throughput Problems

The following options are displayed from this menu:


  • General Statistics - Select this option to view general port statistics.
  • 802.5 Statistics - Select this option to view 802.5 port statistics.
  • 802.5 State Information - Select this option to view 802.5 state information for each port.
  • 802.5 DTR MAC Information - Select this option to view dedicated Token Ring information for each port.

Selecting the General Statistics option will display the menu below:



The General Statistics screen for an individual port is the closest to a Token Ring sniffer that the Catalyst 3900 switch can become. It gives you detailed information on the amount and type of packets that are being received and transmitted on this port. By monitoring this screen (refreshed every five seconds) you will be able to build an idea of the type and amount of traffic passing through the selected port.

The most common problems you will use this screen to troubleshoot for are packets that are not being forwarded or that are being dropped. Check carefully the NSR, SRF, STE and ARE counters and compare the packet types they represent to the rules of the Catalyst 3900 Spanning-Tree Support found here. A common problem is an incorrectly configured Spanning Tree mode which prevents some of these packet types from being forwarded.

Check the total amount of frames being received and transmitted to see how busy this port is. Check the counters on the right hand side to see if there is congestion or a problem on the physical Token Ring connecting to this port.

Another problem you might see from the STE annd ARE counters on this screen is an explorer storm. If you think that you have a problem with an explorer storm then the following two options are available on the Catalyst 3900 switch:


  • Reduce the hop count limit for ARE and STE explorers to the lowest value needed within your network. Change the ARE and SRE values from the VLAN Parameter Configuration for TrCRF menu below:

  • Reduce the maximum explorer rate permitted on an individual port to the lowest value needed within your network. Change the maximum explorer rate value from the Token Ring Port Configuration menu below:

From the 802.5 DTR MAC Information menu option the following menu is presented:



The following options are displayed on this menu:


  • TXI Information - Select this option to view dedicated Token Ring TXI information.
  • Station-CPort Information - Select this option to view dedicated Token Ring station-Cport information.

Selecting the TXI Information option will display the menu below:



Use this screen to gain information about the physical functioning of the Tokenring. You would typically use this screen if you suspected a problem with the Tokenring itself, such as beaconing.

Selecting the 802.5 Statistics and State Information options will display the menus below:




The information on the above two screens is the Catalyst 3900 switches equivalent of the show controllers tokenring xx command on a router. You should use it in a similar context. Troubleshooting for physical problems with the Tokenring segment the port is connected to.

Remember that a large percentage of physical Tokenring problems are with the cable used. This would be a good place to start looking if you suspected a problem in this area.

The Catalyst 3900 Switch Software Release 4.1(1) and later performs error detection and isolation by monitoring the Report Soft Error MAC frames generated by stations on each port. During the interval you define, the Catalyst 3900 monitors the stations on the port. If the threshold is exceeded, the switch can be configured to generate a trap indicating the port number and station on which the threshold was exceeded. If necessary, you can issue a Remove Ring Station MAC frame to remove the station from the ring. You can configure soft error thresholds and sampling intervals for a port using the SNMP Configuration menu below:



Choose the Soft Error Monitoring option to set the desired threshold and interval for the port:



Using the Port Status Screen to Check the Configuration

Use the above screen to verify the CRF and BRF configuration in the Catalyst 3900 switch. You would use this screen if you wanted to check for, or suspected a misconfiguration of the CRF's and BRF's on the 3900 switch.

A lot of problems on the 3900 switch are caused by the Operational and Forwarding mode settings of the ports. A typical symptom of these problems are stations that fail to insert into the Tokenring. If a station is directly attached, then you need to configure the port for Half Duplex port mode. If the station is attached via a MAU, then you need to configure the port for Half Duplex station mode.

Verify that you have the correct Operational and Forwarding modes configured.

Notice the values here are the same values that you would have configured in the port configuration screen.


Troubleshooting Address Table Problems

The following options are displayed on the above panel:

By far the most common problem found with the Catalyst 3900 switch internal tables is that of an address that has been learned on the wrong port. Bear this in mind at all times when using the screens described below to troubleshoot an internal switching problem. Also remember that in the vast majority of cases the only way to recover from an address that has been learned on the wrong port is to reset the Port Address Table. The difference between the Port Address tables and the Route Descriptor tables is as follows:
  • Master Address Table - View all of the destination MAC address entries in the master address table.
  • Master Route Descriptor Table - View all of the SRS route descriptor entries in the master route descriptor table.
  • VLAN Address Table - View the destination MAC address entries for each VLAN.
  • VLAN Route Descriptor Table - View the SRS route descriptor entries for each VLAN.

By far the most common problem found with the Catalyst 3900 switch internal tables is that of an address that has been learned on the wrong port. Bear this in mind at all times when using the screens described below to troubleshoot an internal switching problem. Also remember that in the vast majority of cases the only way to recover from an address that has been learned on the wrong port is to reset the Port Address Table.

The difference between the Port Address tables and the Route Descriptor tables is as follows:


  • Port address tables contain destination MAC address entries used for SRB or SRT switching of packets between seperate VLANs.
  • Route descriptor tables contain tuplets (ring number/bridge number/ring number) entries for SRS switching of packets within the same VLAN.

If you need to reset the Port Address Table you can do so by selecting the 'Reset' option from the Catalyst 3900 main menu. You will then be taken to the menu below which gives an option to reset the Port Address Table:



Another problem that can be found on the Catalyst 3900 switch is that of not being able to pass traffic between different ports. The twenty ports in a 3900 switch are divided into five logical groups containg four physical ports. Each of the five groups (1-4, 4-8, 9-12, 13-16 and17-20) are controlled by an individual ASIC chip. Therefore, you have five ASIC chips that talk to each other across an AXIS backbone in a 3900 switch

Problems have been found where ports can pass traffic between the other ports that they share the same ASIC with. However, they cannot pass traffic to ports that are on a separate ASIC. Use the screenshots of the various tables shown below to assist you in troubleshooting a problem such as this. The problem has been resolved in the latest software level for the Catalyst 3900 switch. However, if you cannot immediately cause an outage of the 3900 switch to perform the upgrade, then the following three workarounds are available:


  • Change the forwarding mode of the affected ports from Auto or Cutthrough to Store and Forward.
  • If you have redundant ports then move the Token Ring cable that is failing to pass traffic to the redundant port and then configure that port to be in the same TrCRF.
  • Use the advanced configuration 'qtpreset R' command to reset the ASIC chips as described below:
  1. From the Catalyst 3900 main menu enter CNTL-T.

  2. Enter the password for the 3900 switch.

  3. From the vin> command line prompt enter the 'qtprst R' command as in the screenshot below:


  4. Finally from the vin> command line prompt enter the conset command to return to the Catalyst 3900 main menu.

Selecting the Master Address Table option will display the menu below:



The above screen contains all destination MAC address entries and the ports those destination MAC addresses have been learned from. When a packet enters the switch that will be either SRB or SRT switched, it will use the address table entries above to decide on which port(s) to be forwarded. Use the above screen to gain information about all destination MAC addresses that have been learned. This screen will give you the address, type, and port from which the entry was learned. Use this screen if you suspect an internal address table problem causing incorrect SRB or SRT switching of packets.

Note that a problem such as this would probably require a reset of the port address table.

Selecting the Master Route Descriptor Table option will display the menu below:



The above screen contains all tuplets (ring number/bridge number/ring number) that make up the route descriptor entries, together with the port number that the route descriptor entries point to. When a packet enters on a port that will be SRS switched (for example: a packet containing a RIF field with a matching ring number/bridge number/ring number tuplet for one of the route descriptor entries), the 3900 switch will use SRS switching to move the packet to the port number within the same CRF that the route descriptor points to. SRS switching is only used used to move packets between ports that are within the same CRF. Use the above screen to gain information about all route descriptor entries that have been learned. Use this screen if you suspect an internal route descriptor problem causing incorrect SRS switching of packets within the same CRF.

Note that a problem such as this would probably require a reset of the port address table.

Selecting the VLAN Address Table option will display the menu below:



The above screen will provide you with the same information as the master address table, but on a per VLAN basis. Use the above screen to gain information about destination MAC addresses that have been learned for this VLAN. This screen will give you the address, type, and port from which the entry was learned. Use this screen if you suspect an internal address table problem causing incorrect SRB or SRT switching of packets.

Note that a problem such as this would probably require a reset of the port address table.

Selecting the VLAN Route Descriptor Table option will display the menu below:



The above screen will provide you with the same information as the master route descriptor table, but on a per VLAN basis. Use the above screen to gain information about route descriptor entries that have been learned for this VLAN. This screen will give you the route, type and port in which the route was learned. Use this screen if you suspect an internal route descriptor problem causing incorrect SRS switching of packets within the same CRF.

Note that a problem such as this would probably require a reset of the port address table.


Troubleshooting Spanning Tree Problems

The above screen will provide you with the current Spanning Tree Information that you have configured on the BRF. Use this screen to troubleshoot a problem with spanning tree at the BRF level.

Use the rules found here (refer to the section on Catalyst 3900 Spanning-Tree Support) for assistance.to assist you in assuring that you have the correct spanning tree modes running at the BRF and CRF levels.

Note: Remember that there is a different instance of spanning tree running in every VLAN.



The above screen will provide you with the Current Spanning Tree Information that you have configured on the CRF. Use this screen to troubleshoot a problem with spanning tree at the CRF level. Check the Spanning Tree mode you are using in the Protocol field above and make sure that it is correct, using the rules found here (refer to the section on Catalyst 3900 Spanning-Tree Support) to assist you.

Note: Remember that there is a different instance of spanning tree running in every VLAN.


Checking the VLAN Statistics

The above screen will provide you with the VLAN Statistics for the BRF's that you specified on the switch. This screen will provide you with the following information:


  • Currently Active Stations - Number of addresses currently in the master address table that are recognized by ports belonging to this domain
  • Largest Number of Stations - Largest number of addresses in the master address table since the last reset or power cycle that are recognized by ports belonging to this domain

The above screen will provide you with the VLAN Statistic for the TrCRF that you specified on the switch. This screen will provide you with the following information:


  • Parent VLAN - Name of the parent TrBRF
  • Currently Active Stations - Number of addresses currently in the master address table that are recognized by ports belonging to this domain
  • Largest Number of Stations - Largest number of addresses in the master address table since the last reset or power cycle that are recognized by ports belonging to this domain
  • Ports - List of ports that belong to this TrCRF
Checking the VTP Statistics

The above screen will provide you with the VTP Statistics for the switch. Refer to the VTP section of the Managing the 3900 Configuration Guide for more information on how to use this screen.



Related Information
  • Technology Support
  • Product Support
  • Technical Assistance Center

All contents are Copyright © 1992-2005 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.

SPECIALS!
Dual Cisco 2501/2513 Router CCNA Kit
$199.99
Dual 1720 32/8 CCNA Kit
$329.99
Google