Configure EtherChannel Between Catalyst 4500/4000, 5500/5000, and 6500/6000 Switches That Run CatOS System Software
                                                                     Printable Pdf

Interactive: This document offers customized analysis of your Cisco device.


Contents
  Introduction
  Prerequisites
    Requirements
    Components Used
    Conventions
    Background Information
  Network Diagram
  Manually Configure EtherChannel
    Step-by-Step
  Verify the EtherChannel Configuration
  Using PAgP to Configure EtherChannel (Recommended)
  Silent/Non-Silent Mode
  Trunking and EtherChannel
  Troubleshoot EtherChannel
    Mismatched Parameters
    Waiting Too Long Before You Configure the Other Side
    Correcting errdisable State
    Showing What Happens When a Link Breaks and Is Restored
  Commands Used in This Document
    Commands to Set the Configuration
    Commands to Verify the Configuration
    Commands to Troubleshoot the Configuration
    Commands to Help Create the Troubleshooting Scenarios
    Command Summary
  Appendix A: Ethernet Crossover Cables
  NetPro Discussion Forums - Featured Conversations
  Related Information

Introduction

EtherChannel allows multiple physical Fast Ethernet or Gigabit Ethernet links to be combined into one logical channel. A logical channel allows load sharing of traffic among the links in the channel as well as redundancy in the event that one or more links in the channel should fail. EtherChannel can be used to interconnect LAN switches, routers, servers, and clients with unshielded twisted pair (UTP) wiring or single-mode and multimode fiber.


Prerequisites

Requirements

There are no specific requirements for this document.


Components Used

The information in this document is based on these software and hardware versions:


  • A console cable1 suitable for the Supervisor Engine in the switch
  • Two Catalyst 5505 switches in a lab environment with cleared configurations2
  • A Fast Ethernet module capable of EtherChannel in each Catalyst 5505
  • Four RJ-45 Ethernet crossover cables3 to connect the EtherChannel
  1. For more information, refer to the Identifying a Rollover Cable, Straight-through, and Crossover Cable section of the document Connecting a Terminal to the Console Port on Catalyst Switches.

  2. The clear config all command was entered on the switch to ensure that it has a default configuration.

  3. For a pinout of an Ethernet crossover cable, see Appendix A.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.


Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.


Background Information

EtherChannel is an easy way to aggregate bandwidth between critical networking devices. On the Catalyst 5500/5000, a channel can be created from two ports, making it a 200 Mbps link (400 Mbps, full-duplex), or four ports, making it a 400 Mbps link (800 Mbps, full-duplex). Some cards and platforms also support Gigabit EtherChannel and have the ability to use from two to eight ports in an EtherChannel. The concept is the same no matter the speeds or number of links that are involved. Normally, the Spanning Tree Protocol (STP) considers these redundant links between two devices to be loops and causes the redundant links to be in blocking mode. This effectively makes the links inactive (providing only backup capabilities if the main link fails). When using a Catalyst OS (CatOS) software version 3.1(1) or later, STP treats the channel as one large link, so all the ports in the channel can be active at the same time.

This document takes you through the steps to configure EtherChannel between two Catalyst 5500/5000 switches and shows you the results of the commands as they are issued. Catalyst 4500/4000 and 6500/6000 switches that run CatOS can be used in the scenarios presented in this document to obtain the same results. For the Catalyst 2900XL and Catalyst 1900/2820, the command syntax differs, but the EtherChannel concepts are the same. For EtherChannel guidelines and configuration information for the Catalyst 6500/6000 series switches that run Cisco IOS® System Software, refer to the document:


  • Sample Configuration: EtherChannel Between Catalyst Switches Running CatOS and Native IOS

For an overview and comparison of the Catalyst 6500 Cat0S and Cisco IOS Software platforms, refer to the document:


  • Comparison of the Cisco Catalyst and Cisco IOS Operating Systems for the Cisco Catalyst 6500 Series Switch

EtherChannel may be configured manually by executing the appropriate commands, or it may be configured automatically by having the switch negotiate the channel with the other side using Port Aggregation Protocol (PAgP). It is best to use the PAgP desirable mode to configure EtherChannel whenever possible because manually configuring EtherChannel sometimes creates complications. This document provides examples of configuring EtherChannel manually and examples of configuring EtherChannel using PAgP. Also included is how to troubleshoot EtherChannel and how to use trunking with EtherChannel. In this document, the terms EtherChannel, Fast EtherChannel, Gigabit EtherChannel, or channel all refer to EtherChannel.


Network Diagram

The network setup shown in this section illustrates the test environment.

After the configuration of the switches was cleared with the clear config all command, the prompt was changed with the set system name command. An IP address and mask were assigned to the switch for management purposes using the set interface sc0 172.16.84.6 255.255.255.0 command for Switch A and the set interface sc0 172.16.84.17 255.255.255.0 command for Switch B. A default gateway was assigned to both switches with the set ip route default 172.16.84.1 command.

The switch configurations were cleared to start from the default conditions. The switches were given names for identification from the prompt on the command line. To ping between the switches for testing, the IP addresses were assigned. The default gateway was not used.



Many of the commands display more output than is needed for this discussion. Extraneous output is suppressed in this document.


Manually Configure EtherChannel

Step-by-Step

Complete these steps to manually configure EtherChannel:


  1.Issue the show version and show module commands.

The show version command displays the software version the switch is running. The show module command lists which modules are installed in the switch.


  Switch-A> show version
  WS-C5505 Software, Version McpSW: 4.5(1) NmpSW: 4.5(1)
  !--- This is the software version that runs on the switch.
  Copyright (c) 1995-1999 by Cisco Systems
  NMP S/W compiled on Mar 29 1999, 16:09:01
  MCP S/W compiled on Mar 29 1999, 16:06:50
  System Bootstrap Version: 3.1.2
  Hardware Version: 1.0 Model: WS-C5505 Serial #: 066507453
  Mod   Port   Model   Serial #   Versions
  ---   ----   -----   -------   ----------
  1   0   WS-X5530   006841805   Hw : 1.3
                                            Fw : 3.1.2
                                            Fw1: 3.1(2)
                                            Sw : 4.5(1)
  2   24  WS-X5225R  012785227  Hw : 3.2
                                            Fw : 4.3(1)
                                            SW : 4.5(1)
  DRAM FLASH NVRAM
  Module Total Used Free Total Used Free Total Used Free
  ------ ----- ---- ---- ----- ---- ---- ----- ---- -----
  1 32640K 13650K 18990K 8192K 4118K 4074K 512K 108K 404K
  Uptime is 0 day, 3 hours, 32 minutes
  Switch-A> show module
  Mod Module-Name Ports Module-Type Model Serial-Num Status
  --- ----------- ----- ----------- ----- --------- -------
  1 0 Supervisor III WS-X5530 006841805 ok
  !--- These are the modules installed on the switch.
  2 24 10/100BaseTX Ethernet WS-X5225R 012785227 OK
  Mod   MAC-Address(es)                                 Hw     Fw     Sw
  ---   -------------------------------------- ------ ---- -------
  1   00-90-92-b0-84-00 to 00-90-92-b0-87-ff 1.3  3.1.2  4.5(1)
  2   00-50-0f-b2-e2-60 to 00-50-0f-b2-e2-77 3.2  4.3(1)  4.5(1)
  Mod Sub-Type Sub-Model Sub-Serial Sub-Hw
  --- -------- --------- ---------- ------
  1 NFFC WS-F5521 0008728786 1.0
  Switch-B> show version
  WS-C5505 Software, Version McpSW: 4.5(1) NmpSW: 4.5(1)
  !--- This is the software version that runs on the switch.
  Copyright (c) 1995-1999 by Cisco Systems
  NMP S/W compiled on Mar 29 1999, 16:09:01
  MCP S/W compiled on Mar 29 1999, 16:06:50
  System Bootstrap Version: 5.1(2)
  Hardware Version: 1.0 Model: WS-C5505 Serial #: 066509957
  Mod   Port   Model   Serial #   Versions
  ---   ----   -----   -------   ----------
  1   0   WS-X5530   008592453   Hw : 2.3
                                            Fw : 5.1(2)
                                            Fw1: 4.4(1)
                                            SW : 4.5(1)
  2   24   WS-X5234   015388641   Hw : 1.0
                                            Fw : 4.5(2)
                                            SW : 4.5(1)
  DRAM FLASH NVRAM
  Module Total Used Free Total Used Free Total Used Free
  ------ ----- ---- ---- ----- ---- ---- ----- ---- -----
  1  32640K 13548K 19092K 8192K 7300K 892K 512K 119K 393K
  Uptime is 0 day, 3 hours, 36 minutes
  Switch-B> show module
  Mod Module-Name Ports Module-Type Model Serial-Num Status
  --- ----------- ----- ----------- ----- --------- -------
  1 0 Supervisor III WS-X5530 008592453 OK
  !--- These are the modules installed on the switch.
  2 24 10/100BaseTX Ethernet WS-X5234 015388641 OK
  Mod   MAC-Address(es)                                 Hw     Fw     Sw
  ---   -------------------------------------- ------ ---- -------
  1   00-10-0d-b2-8c-00 to 00-10-0d-b2-8f-ff 2.3  5.1(2)  4.5(1)
  2   00-d0-bc-03-58-98 to 00-d0-bc-03-58-af 1.0  4.5(2)  4.5(1)
  Mod Subtype Sub-Model Sub-Serial Sub-Hw
  --- -------- --------- ---------- ------
  1 EARL 1+ WS-F5520 0011591025 1.1
  2. Verify that EtherChannel is supported on the ports.

Note: The show port capabilities command is available in CatOS software versions 4.x and later. If you have a software version earlier than 4.x, you must skip this step. Not every Fast Ethernet module supports EtherChannel. Some of the original EtherChannel modules have "Fast EtherChannel" printed on the bottom left corner of the module (as you face it in the switch), which tells you that the feature is supported. But this convention was abandoned on later modules. The modules in this test do not have "Fast EtherChannel" printed on them, but they do support the feature.


  Switch-A> show port capabilities 2/1
  Model WS-X5225R
  Port 2/1
  Type 10/100BaseTX
  Speed auto,10,100
  Duplex half,full
  Trunk encap type 802.1Q,ISL
  Trunk mode on,off,desirable,auto,nonegotiate
  Channel 2/1-2,2/1-4
  !--- This indicates that EtherChannel can be configured on port 2/1
  !--- with two or four contiguous ports.
  Broadcast suppression percentage(0-100)
  Flow control receive-(off,on),send-(off,on)
  Security yes
  Membership static,dynamic
  Fast start yes
  Rewrite yes
  Switch-B> show port capabilities 2/1
  Model WS-X5234
  Port 2/1
  Type 10/100BaseTX
  Speed auto,10,100
  Duplex half,full
  Trunk encap type 802.1Q,ISL
  Trunk mode on,off,desirable,auto,nonegotiate
  !--- This indicates that EtherChannel can be configured on port 2/1
  !--- with two or four contiguous ports.
  Channel 2/1-2,2/1-4
  Broadcast suppression percentage(0-100)
  Flow control receive-(off,on),send-(off,on)
  Security yes
  Membership static,dynamic
  Fast start yes
  Rewrite no

A port that does not support EtherChannel looks like this:


  Switch> show port capabilities 2/1
  Model WS-X5213A
  Port 2/1
  Type 10/100BaseTX
  Speed 10,100,auto
  Duplex half,full
  Trunk encap type ISL
  Trunk mode on,off,desirable,auto,nonegotiate
  Channel no
  !--- This indicates that EtherChannel is not supported on this port
  !--- or module.
  Broadcast suppression pps(0-150000)
  Flow control no
  Security yes
  Membership static,dynamic
  Fast start yes
  3. Verify that the ports are connected and operational.

Before connecting the cables, the port status is:


  Switch-A> show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1      notconnect 1 normal auto auto 10/100BaseTX
  2/2      notconnect 1 normal auto auto 10/100BaseTX
  2/3      notconnect 1 normal auto auto 10/100BaseTX
  2/4      notconnect 1 normal auto auto 10/100BaseTX

After connecting the cables between the two switches, the status is:


  1999 Dec 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1
  1999 DEC 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/2
  1999 DEC 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/3
  1999 DEC 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/4
  Switch-A> show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1      connected 1 normal a-full a-100 10/100BaseTX
  2/2      connected 1 normal a-full a-100 10/100BaseTX
  2/3      connected 1 normal a-full a-100 10/100BaseTX
  2/4      connected 1 normal a-full a-100 10/100BaseTX
  Switch-B> show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1      connected 1 normal a-full a-100 10/100BaseTX
  2/2      connected 1 normal a-full a-100 10/100BaseTX
  2/3      connected 1 normal a-full a-100 10/100BaseTX
  2/4      connected 1 normal a-full a-100 10/100BaseTX

Since the switch configurations were cleared before starting this test, the ports are in their default conditions. They are all in VLAN 1, and their speed and duplex are set to auto. After connecting the cables, they negotiate to a speed of 100 Mbps and full-duplex. The status is connected. You are now able to ping the other switch.


  Switch-A> ping 172.16.84.17
  172.16.84.17 is alive

In your network, you may want to to set the speeds manually to 100 Mbps and full-duplex instead of relying on autonegotiation because you probably want your ports to always run at the fastest speed. For a discussion of autonegotiation, refer to the document:


  • Configuring and Troubleshooting Ethernet 10/100/1000Mb Half/Full Duplex Auto-Negotiation.
  4. Verify that the ports to be grouped have the same settings.

This is an important point that is covered in more detail in the Troubleshoot EtherChannel section. If the command to set up EtherChannel does not work, it is usually because the ports involved in the channel have configurations that differ from each other. This includes the ports on the other side of the link as well as the local ports. In this case, since the switch configurations were cleared before this test, the ports are in their default conditions. They are all in VLAN 1, their speed and duplex are set to auto, and all spanning tree parameters for each port are set to be the same. You saw from the output above that, after connecting the cables, the ports negotiate to a speed of 100 Mbps and full-duplex. Since STP runs for each VLAN, it is easier to simply configure the channel and respond to error messages than to attempt to check every STP field for consistency for each port and VLAN in the channel.


  5. Identify valid port groups.

On the Catalyst 5500/5000, only certain ports can be put together into a channel. These restrictive dependencies do not apply to all platforms. The ports in a channel on a Catalyst 5500/5000 must be contiguous. The show port capabilities command, issued for port 2/1, shows the possible combinations:


  Switch-A> show port capabilities 2/1
  Model WS-X5225R
  Port 2/1
  ...
  Channel 2/1-2,2/1-4

Notice that this port can be a part of a group of two (2/1-2) or part of a group of four (2/1-4). An Ethernet Bundling Controller (EBC) on the module causes these configuration limitations. Here, show port capabilities is issued for another port:


  Switch-A> show port capabilities 2/3
  Model WS-X5225R
  Port 2/3
  ...
  Channel 2/3-4,2/1-4

This port can be grouped into a group of two ports (2/3-4) or into a group of four ports (2/1-4).

Note: Depending on the hardware, there may be additional restrictions. On certain modules (WS-X5201 and WS-X5203), you cannot form an EtherChannel with the last two ports in a port group unless the first two ports in the group already form an EtherChannel. A port group is a group of ports that are allowed to form an EtherChannel. (In the example above, 2/1-4 is a port group.)

For example, if you are creating separate EtherChannels with only two ports in a channel, you cannot assign ports 2/3-4 to a channel until you have first configured ports 2/1-2 to a channel. This is true only for the modules that have this restriction. Similarly, before you configure ports 2/6-7, you must configure ports 2/5-6. This restriction does not occur on the modules used for this document (WS-X5225R and WSX5234).

Since you are configuring a group of four ports (2/1-4), this is within the approved grouping. You are not able to assign a group of four to ports 2/3-6. This is a group of contiguous ports, but they do not start on the approved boundary, as shown by the show port capabilities command. (Valid groups are ports 1-4, 5-8, 9-12, 13-16, 17-20, and 21-24.)


  6. Create the channel.

To create the channel manually, use the set port channel mod/port on command for each switch. It is best to turn the ports off on one side of the channel using the set port disable command before turning EtherChannel on manually. This avoids possible problems with STP during the configuration process.

STP can shut down some ports (with a port status of errdisable) if one side is configured as a channel before the other side can be configured as a channel. Because of this possibility, it is much easier to create EtherChannels using PAgP, which is covered in the Using PAgP to Configure EtherChannel (Recommended) section of this document. To avoid this situation when configuring EtherChannel manually, you disable the ports on Switch A, configure the channel on Switch A, configure the channel on Switch B, then reenable the ports on Switch A.


  a. Verify that channelling is off.
    Switch-A> (enable) show port channel
    No ports channelling
    Switch-B> (enable) show port channel
    No ports channelling
  b. Disable the ports on Switch A until both switches have been configured for EtherChannel.
    Switch-A> (enable) set port disable 2/1-4
    Ports 2/1-4 disabled.
    [output from Switch A upon disabling ports]
    1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/1 left bridg1
    1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
    1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
    1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4

Now, STP does not generate errors and shut down the ports.


  c. Turn the channel mode to on for Switch A.
  Switch-A> (enable) set port channel 2/1-4 on
  Port(s) 2/1-4 channel mode set to on.

Note: In this case, ports 2/1 to 2/4 are configured for Etherchannel with a single command. If you configure the etherchannel for every port independently without using the port range, remember to mention the same admin-group for all the ports that need to be part of same Etherchannel. If the admin-group is not specified, then every port will belong to different Etherchannel groups and the desired Etherchannel bundle will never be formed.


  d. Check the status of the channel.
  Switch-A> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1 disabled  on      channel
  2/2 disabled  on      channel
  2/3 disabled  on      channel
  2/4 disabled  on      channel
  --- ------ ------- -------- ------------ -------

Notice that the channel mode has been set to on, but the status of the ports is disabled (because you disabled them earlier). The channel is not operational at this point, but it becomes operational when the ports are enabled.

Because Switch A ports were (temporarily) disabled, Switch B ports no longer have a connection.

This message is displayed on the Switch B console when Switch A ports are disabled:


    Switch-B> (enable)
    2000 Jan 13 22:30:03 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1
    2000 Jan 13 22:30:04 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
    2000 Jan 13 22:30:04 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
    2000 Jan 13 22:30:04 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  e. Turn on the channel for Switch B.
  Switch-B> (enable) set port channel 2/1-4 on
  Port(s) 2/1-4 channel mode set to on.
  f. Verify that the channel mode is on for Switch B.
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1 notconnect  on      channel
  2/2 notconnect  on      channel
  2/3 notconnect  on      channel
  2/4 notconnect  on      channel
  --- ------ ------- -------- ------------ -------

Notice that the channel mode for Switch B is on, but the status of the ports is notconnect. This is the case because Switch A ports are still disabled.


  g. Enable the ports on Switch A.
    Switch-A> (enable) set port enable 2/1-4
    Ports 2/1-4 enabled.
    1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
    1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
    1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
    1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
Verify the EtherChannel Configuration

To verify that the channel is set up properly, issue the show port channel command.


  Switch-A> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1 connected on      channel WS-C5505 066509957(SW 2/1
  2/2 connected on      channel WS-C5505 066509957(SW 2/2
  2/3 connected on      channel WS-C5505 066509957(SW 2/3
  2/4 connected on      channel WS-C5505 066509957(SW 2/4
  --- ------ ------- -------- ------------ -------
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1 connected on      channel WS-C5505 066507453(SW 2/1
  2/2 connected on      channel WS-C5505 066507453(SW 2/2
  2/3 connected on      channel WS-C5505 066507453(SW 2/3
  2/4 connected on      channel WS-C5505 066507453(SW 2/4
  --- ------ ------- -------- ------------ -------

If you have the output of a show port channel command from your Cisco device, you can use the Output Interpreter Tool ( registered customers only) , which allows you to view an analysis of show command output.

STP is shown to treat the ports as one logical port in the show spantree command. In this output, when the port is listed as 2/1-4, it means that STP is treating ports 2/1, 2/2, 2/3 and 2/4 as one port.


    Switch-A> (enable) show spantree
    VLAN 1
    Spanning tree enabled
    Spanning tree type ieee
    Designated Root 00-10-0d-b2-8c-00
    Designated Root Priority 32768
    Designated Root Cost 8
    Designated Root Port 2/1-4
    Root Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec
    Bridge ID MAC ADDR 00-90-92-b0-84-00
    Bridge ID Priority 32768
    Bridge Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec
    Port Vlan Port-State Cost Priority Fast-Start Group-Method
    --- ---- ---------- ----- -------- ---------- ------------
    2/1-4 1 forwarding 8 32 disabled channel

If you have the output of a show spantree command from your Cisco device, you can use the Output Interpreter Tool ( registered customers only) , which allows you to view an analysis of show command output.

EtherChannel can be implemented with different ways of distributing the traffic across the ports in a channel.

The EtherChannel specification does not dictate how the traffic should be distributed across the links in a channel. The Catalyst 5500/5000 uses the last bit or the last two bits (depending on how many links are in the channel) of the source and destination MAC addresses in the frame to determine which port in the channel to use. You should see a similar amount of traffic on each of the ports in the channel, assuming that traffic is generated by a normal distribution of MAC addresses on one side of the channel or the other. To verify that traffic is going over all the ports in the channel, you can use the show mac command. If your ports were active before configuring EtherChannel, you can reset the traffic counters to 0 with the clear counters command. The traffic values then represent how EtherChannel has distributed the traffic.

In this test environment, a real-world distribution is not achieved because there are no workstations, servers, or routers generating traffic. The only devices generating traffic are the switches themselves. Pings were issued from Switch A to Switch B; the unicast traffic is using the first port in the channel. (See the output below.) The receive information (Rcv-Unicast) in this case shows how Switch B distributed the traffic across the channel to Switch A. Also in the output, the transmit information (Xmit-Unicast) shows how Switch A distributed the traffic across the channel to Switch B. You also see that a small amount of switch-generated multicast traffic (Dynamic Inter-Switch Link Protocol [ISL], Cisco Discovery Protocol [CDP]) goes out all four ports. The broadcast packets are Address Resolution Protocol (ARP) queries (for the default gateway which does not exist in this lab). If you had workstations sending packets through the switch to a destination on the other side of the channel, you would expect to see traffic going over each of the four links in the channel. You can monitor the packet distribution in your network using the show mac command.


    Switch-A> (enable) clear counters
    This command will reset all MAC and port counters reported in CLI and SNMP.
    Do you want to continue (y/n) [n]? y
    MAC and Port counters cleared.
    Switch-A> (enable) show mac
    Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast
    --- ----------- -------------- -------------
    2/1     9     320     183
    2/2     0     51     0
    2/3     0     47     0
    2/4     0     47     0
    (...)
    Port Xmit-Unicast Xmit-Multicast Xmit-Broadcast
    ---- ------------- ------------- --------------
    2/1     8     47     184
    2/2     0     47     0
    2/3     0     47     0
    2/4     0     47     0
    (...)
    Port Rcv-Octet Xmit-Octet
    --- ---------- ----------
    2/1     35176     17443
    2/2     5304     4851
    2/3     5048     4851
    2/4     5048     4851
    (...)
    Last-Time-Cleared
    --------------------------
    Wed DEC 15 1999, 01:05:33

If you have the output of a show mac command from your Cisco device, you can use the Output Interpreter Tool ( registered customers only) , which allows you to view an analysis of show command output.


Using PAgP to Configure EtherChannel (Recommended)

PAgP facilitates the automatic creation of EtherChannel links by exchanging packets between channel-capable ports. The protocol learns the capabilities of port groups dynamically and informs the neighboring ports.

After PAgP identifies correctly paired channel-capable links, it groups the ports into a channel. The channel is then added to the spanning tree as a single bridge port. A given outbound broadcast or multicast packet is transmitted out one port in the channel only, not out every port in the channel. In addition, outbound broadcast and multicast packets transmitted on one port in a channel are blocked from returning on any other port of the channel.

There are four user-configurable channel modes: on, off, auto, and desirable. PAgP packets are exchanged only between ports in auto and desirable mode. Ports configured in on or off mode do not exchange PAgP packets.

For switches to which you want to form an EtherChannel, it is best to have both switches set to desirable mode.

This gives the most robust behavior if one side or the other encounters error situations or is reset. The default mode of the channel is auto.

Both the auto and desirable modes allow ports to negotiate with connected ports to determine if they can form a channel. The determination is based on criteria such as port speed, trunking state, and native VLAN.

Ports can form an EtherChannel when they are in different channel modes as long as the modes are compatible.

This list provides examples:


  • A port in desirable mode can successfully form an EtherChannel with another port that is in desirable or auto mode.
  • A port in auto mode can form an EtherChannel with another port in desirable mode.
  • A port in auto mode cannot form an EtherChannel with another port that is also in auto mode, since neither port initiates negotiation.
  • A port in on mode can form a channel only with a port in on mode because ports in on mode do not exchange PAgP packets.
  • A port in off mode cannot form a channel with any port.

When using EtherChannel, if this message (or a similar syslog message) is displayed, it indicates a mismatch of EtherChannel modes on the connected ports:


  SPANTREE-2: Channel misconfig - x/x-x will be disabled

Correct the configuration and reenable the ports by issuing the set port enable command. Valid EtherChannel configurations include:



  1 If both the local and neighbor ports are in auto mode, an EtherChannel bundle does not form.

The next table provides a summary of all the possible channelling mode scenarios. Some of these combinations may cause STP to put the ports on the channelling side in errdisable state. (In other words, some of the combinations shut down the ports on the channelling side.)



The channel from the previous example (Step 6b in the section Manually Configure EtherChannel) is turned off using this command on Switch A and Switch B:


  Switch-A> (enable) set port channel 2/1-4 auto
  Port(s) 2/1-4 channel mode set to auto.

The default channel mode for a port that is able to channel is auto. To verify this, issue this command:


  Switch-A> (enable) show port channel 2/1
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1  connected  auto   not channel

The show port channel port command also shows that the ports currently are not channelling. This is another way to verify the channel state:


  Switch-A> (enable) show port channel
  No ports channelling
  Switch-B> (enable) show port channel
  No ports channelling

It is simple to make the channel work with PAgP. At this point, both switches are set to auto mode, which means that they channel if a connected port sends a PAgP request to channel. Setting Switch A to desirable causes Switch A to send PAgP packets to the other switch, asking it to channel.


  Switch-A> (enable) set port channel 2/1-4 desirable
  Port(s) 2/1-4 channel mode set to desirable.
  1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/1 left bridg1
  1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  1999 DEC 15 22:03:19 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  1999 DEC 15 22:03:19 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  1999 DEC 15 22:03:20 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  1999 DEC 15 22:03:23 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
  1999 DEC 15 22:03:23 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
  1999 DEC 15 22:03:23 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
  1999 DEC 15 22:03:24 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4

To view the channel, issue this command:


  Switch-A> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected   desirable channel   WS-C5505 066509957(SW 2/1
  2/2   connected   desirable channel   WS-C5505 066509957(SW 2/2
  2/3   connected   desirable channel   WS-C5505 066509957(SW 2/3
  2/4   connected   desirable channel   WS-C5505 066509957(SW 2/4
  --- ------ ------- -------- ------------ -------
  Switch-A> (enable)

Since Switch B is in auto mode, it responds to the PAgP packets and creates a channel with Switch A.


  Switch-B> (enable)
  2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/1 left bridg1
  2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  2000 Jan 14 20:26:45 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  2000 Jan 14 20:26:45 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  2000 Jan 14 20:26:45 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  2000 Jan 14 20:26:47 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
  2000 Jan 14 20:26:47 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
  2000 Jan 14 20:26:47 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
  2000 Jan 14 20:26:48 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected   auto channel   WS-C5505 066507453(SW 2/1
  2/2   connected   auto channel   WS-C5505 066507453(SW 2/2
  2/3   connected   auto channel   WS-C5505 066507453(SW 2/3
  2/4   connected   auto channel   WS-C5505 066507453(SW 2/4
  --- ------ ------- -------- ------------ -------
  Switch-B> (enable)

Note: It is best to set both sides of the channel to desirable so that both sides try to initiate the channel if one side drops out. Setting the EtherChannel ports on Switch B to desirable mode, even though the channel is currently active and in auto mode, poses no problem. The command is:


  Switch-B> (enable) set port channel 2/1-4 desirable
  Port(s) 2/1-4 channel mode set to desirable.

Note: In this case, all ports 2/1 to 2/4 are configured for Etherchannel with a single command. If you configure the etherchannel for every port independently without using the port range, remember to mention the same admin-group for all the ports that need to be part of same Etherchannel. If the admin-group is not specified, then every port will belong to different Etherchannel groups and the desired Etherchannel bundle will never be formed.


  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected   desirable channel   WS-C5505 066507453(SW 2/1
  2/2   connected   desirable channel   WS-C5505 066507453(SW 2/2
  2/3   connected   desirable channel   WS-C5505 066507453(SW 2/3
  2/4   connected   desirable channel   WS-C5505 066507453(SW 2/4
  --- ------ ------- -------- ------------ -------
  Switch-B> (enable)

If Switch A drops out for some reason, or if new hardware replaces Switch A, Switch B tries to reestablish the channel. If the new equipment cannot channel, Switch B treats its ports 2/1-4 as normal nonchannelling ports.

This is one of the benefits of using the desirable mode. If the channel was configured using the PAgP on mode and one side of the connection has an error of some kind or a reset, it could cause an errdisable state (shutdown) on the other side. With PAgP set in desirable mode on each side, the channel stabilizes and renegotiates the EtherChannel connection.


Silent/Non-Silent Mode

When dealing with fiber connections, it is possible that, even if a receive (Rx) transceiver dies, the transmit (Tx) transceiver on the other end is still up. During a similar scenario, packets can get black holed.

It is important for the switch that is transmitting to remove this port from the EtherChannel bundle. To do so on the Catalyst 5500/5000, you set PAgP in non-silent mode. This means that if the Rx does not receive traffic, the port is not put into the channel. However, this is not enough because this detection happens only when the channel is formed.

To prevent the black holing of traffic when the channel is already formed, this occurs: PAgP detects that the Rx port is not receiving any traffic, so it resets the Tx transceiver of the port that is not receiving. It is reset for 1.6 seconds so that the switch on the other end also resets the port. The faulty port does not join the channel anymore because no traffic is received on that port.

On the Catalyst 5500/5000, it is recommended (default) that non-silent mode is set on fiber strands and silent mode is set on copper strands. The reason for this is that, on fiber connection on the Catalyst 5500/5000, the negotiation is usually not available, so there is no way to detect the problem at a physical layer.


Default PAgP Settings on the Catalyst 4500/4000 and 5500/5000

By default, PAgP is auto for a plug-and-play implementation. It is best to disable PAgP manually from the ports where there is no need to have it.

By default, the silent mode is on. (Non-silent is acceptable as well.) However, since a port can be connected to a device that does not send traffic (for example, a sniffer), it is more general to have silent enabled.


Recommendations
  • Use the non-silent keyword when you are connecting to a device that transmits bridge protocol data units (BPDUs) or other traffic. Use this keyword with the auto or desirable mode. PAgP non-silent adds an extra level of link state detection by listening for BPDUs or other traffic to determine if the link is functioning properly. This adds a form of unidirectional link detection capability not available when you use the default silent PAgP mode.
  • Use the silent keyword when you are connecting to a silent partner (a device that is not generating BPDUs or other traffic). An example of a silent partner is a traffic generator that is not transmitting packets. Use this keyword with the auto or desirable mode. If you do not specify silent or non-silent, silent is assumed.
  • The silent mode does not disable the PAgP capability of detecting unidirectional links. However, if you are configuring a channel, non-silent prevents a unidirectional port from even joining the link.
  • PAgP configuration (the set port channel {desirable | auto} command) is safer than non-PAgP configuration (the set port channel on command) because it provides protection for unidirectional links. It also avoids misconfigurations that can arise from having ports channeling on one side of the link and not on the other side.
  • Refer to Understanding and Configuring the Unidirectional Link Detection Protocol Feature for more information on UniDirectional Link Detection (UDLD).
Trunking and EtherChannel

EtherChannel is independent of trunking. You can turn trunking on or you can leave trunking off. Also, you can turn trunking on for all the ports before creating the channel, or you can turn it on after creating the channel (as in this example). As far as EtherChannel is concerned, it does not matter; trunking and EtherChannel are completely separate features. What does matter is that all the ports involved are in the same mode: either they are all trunking before you configure the channel, or they are all not trunking before you configure the channel.

All the ports must be in the same trunking state before creating the channel.

Once a channel is formed, whatever is changed on one port is also changed for the other ports in the channel.

The modules used in this test bed can do ISL or IEEE 802.1Q trunking. By default, the modules are set to auto trunking and negotiate mode. This means that they trunk if the other side asks them to trunk, and they negotiate whether to use the ISL or 802.1Q method for trunking. If they are not asked to trunk, they work as normal nontrunking ports.


  Switch-A> (enable) show trunk 2
  Port Mode Encapsulation Status Native vlan
  -------- ----------- -------- ------ ------
  2/1   auto   negotiate   not-trunking 1
  2/2   auto   negotiate   not-trunking 1
  2/3   auto   negotiate   not-trunking 1
  2/4   auto   negotiate   not-trunking 1

There are a number of different ways to turn on trunking. For this example, Switch A is set to desirable. Switch A is already set to negotiate. The combination of desirable/negotiate causes Switch A to ask Switch B to trunk and to negotiate the type of trunking to perform (ISL or 802.1Q). Since Switch B defaults to autonegotiate, Switch B responds to the Switch A request. These are the results:


  Switch-A> (enable) set trunk 2/1 desirable
  Port(s) 2/1-4 trunk mode set to desirable.
  Switch-A> (enable)
  1999 DEC 18 20:46:25 %DTP-5-TRUNKPORTON:Port 2/1 has become isl trunk
  1999 DEC 18 20:46:25 %DTP-5-TRUNKPORTON:Port 2/2 has become isl trunk
  1999 DEC 18 20:46:25 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4
  1999 DEC 18 20:46:25 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-4
  1999 DEC 18 20:46:25 %DTP-5-TRUNKPORTON:Port 2/3 has become isl trunk
  1999 DEC 18 20:46:26 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/1-4
  1999 DEC 18 20:46:26 %DTP-5-TRUNKPORTON:Port 2/4 has become isl trunk
  1999 DEC 18 20:46:26 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/1-4
  1999 DEC 18 20:46:28 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
  1999 DEC 18 20:46:29 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
  1999 DEC 18 20:46:29 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
  1999 DEC 18 20:46:29 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
  Switch-A> (enable) show trunk 2
  Port Mode Encapsulation Status Native vlan
  --- ----- ------------- ------ -----------
  2/1 desirable n-isl trunking   1
  2/2 desirable n-isl trunking   1
  2/3 desirable n-isl trunking   1
  2/4 desirable n-isl trunking   1

The trunk mode was set to desirable. The result was that trunking mode was negotiated with the neighbor switch, and they decided on ISL (n-isl). The current status now is trunking. This output shows what happened on Switch B because of the command issued on Switch A:


  Switch-B> (enable)
  2000 Jan 17 19:09:52 %DTP-5-TRUNKPORTON:Port 2/1 has become isl trunk
  2000 Jan 17 19:09:52 %DTP-5-TRUNKPORTON:Port 2/2 has become isl trunk
  2000 Jan 17 19:09:52 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4
  2000 Jan 17 19:09:52 %DTP-5-TRUNKPORTON:Port 2/3 has become isl trunk
  2000 Jan 17 19:09:52 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-4
  2000 Jan 17 19:09:53 %DTP-5-TRUNKPORTON:Port 2/4 has become isl trunk
  2000 Jan 17 19:09:53 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/1-4
  2000 Jan 17 19:09:53 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/1-4
  2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
  2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
  2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
  2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
  Switch-B> (enable) show trunk 2
  Port Mode Encapsulation Status Native vlan
  ---- ---- ------------- ------ -----------
  2/1   auto n-isl trunking   1
  2/2   auto n-isl trunking   1
  2/3   auto n-isl trunking   1
  2/4   auto n-isl trunking   1

Notice that all four ports (2/1-4) became trunking, even though you only specifically changed one port (2/1) to desirable. This is an example of how changing one port in the channel affects all the ports.


Troubleshoot EtherChannel

The challenges for EtherChannel can be divided into two main areas: troubleshooting during the configuration phase and troubleshooting during the execution phase. Configuration errors usually occur because of mismatched parameters on the ports involved (for example, different speeds, different duplex, or different STP port values) However, you can also generate errors during the configuration by setting the channel on one side to on and waiting too long before configuring the channel on the other side. This causes STP loops which generate an error and shut down the port.

When an error is encountered while configuring EtherChannel, be sure to check the status of the ports after correcting the EtherChannel error situation. If the port status is errdisable, it means that the ports have been shut down by the software. They do not come on again until you issue the set port enable command.

Note: If the port status becomes errdisable, you must specifically enable the ports using the set port enable command for the ports to become active. Currently, you can correct all the EtherChannel issues, but the ports do not come up or form a channel until they are enabled again. Later versions of the operating system may periodically check to determine if errdisable ports should be enabled.

These tests are covered in this section. For the tests, trunking and EtherChannel are turned off:


  • Mismatched Parameters
  • Waiting Too Long Before You Configure the Other Side
  • Correcting errdisable State
  • Showing What Happens When a Link Breaks and Is Restored
Mismatched Parameters

Here is an example of mismatched parameters. Port 2/4 is set in VLAN 2 while the other ports are still in VLAN 1. To create a new VLAN, you must assign a VLAN Trunk Protocol (VTP) domain for the switch and then create the VLAN.


  Switch-A> (enable) show port channel
  No ports channelling
  Switch-A> (enable) show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1      connected 1 normal a-full a-100 10/100BaseTX
  2/2      connected 1 normal a-full a-100 10/100BaseTX
  2/3      connected 1 normal a-full a-100 10/100BaseTX
  2/4      connected 1 normal a-full a-100 10/100BaseTX
  Switch-A> (enable) set vlan 2
  Cannot add/modify VLANs on a VTP server without a domain name.
  Switch-A> (enable) set vtp domain testDomain
  VTP domain testDomain modified
  Switch-A> (enable) set vlan 2 name vlan2
  Vlan 2 configuration successful
  Switch-A> (enable) set vlan 2 2/4
  VLAN 2 modified.
  VLAN 1 modified.
  VLAN Mod/Ports
  ---- -----------------------
  2   2/4
  Switch-A> (enable)
  1999 DEC 19 00:19:34 %PAGP-5-PORTFROMSTP:Port 2/4 left bridg4
  Switch-A> (enable) show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1 connected 1 normal a-full a-100 10/100BaseTX
  2/2 connected 1 normal a-full a-100 10/100BaseTX
  2/3 connected 1 normal a-full a-100 10/100BaseTX
  2/4 connected 2 normal a-full a-100 10/100BaseTX
  Switch-A> (enable) set port channel 2/1-4 desirable
  Port(s) 2/1-4 channel mode set to desirable.
  Switch-A> (enable)
  1999 DEC 19 00:20:19 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1
  1999 DEC 19 00:20:19 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  1999 DEC 19 00:20:19 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  1999 DEC 19 00:20:20 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  1999 DEC 19 00:20:20 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  1999 DEC 19 00:20:22 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  1999 DEC 19 00:20:22 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  1999 DEC 19 00:20:24 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-2
  1999 DEC 19 00:20:25 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-2
  1999 DEC 19 00:20:25 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/3
  1999 DEC 19 00:20:25 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/4
  Switch-A> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected desirable channel WS-C5505 066509957(SW 2/1
  2/2   connected desirable channel WS-C5505 066509957(SW 2/2
  --- ------ ------- -------- ------------ -------
  Switch-A> (enable)

Notice that the channel formed only between ports 2/1-2. Ports 2/3-4 were left out because port 2/4 is in a different VLAN. There was no error message; PAgP just did what it could to make the channel work. Watch the results when you create the channel to be sure that it did what you wanted it to do.

Now, set the channel manually to on with port 2/4 in a different VLAN and see what happens. First, set the channel mode back to auto. This tears down the existing channel. Then, manually set the channel to on.


  Switch-A> (enable) set port channel 2/1-4 auto
  Port(s) 2/1-4 channel mode set to auto.
  Switch-A> (enable)
  1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-2
  1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-2
  1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  1999 DEC 19 00:26:18 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1
  1999 DEC 19 00:26:19 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/2
  1999 DEC 19 00:26:19 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/3
  1999 DEC 19 00:26:19 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/4
  Switch-A> (enable) show port channel
  No ports channelling
  Switch-A> (enable) set port channel 2/1-4 on Mismatch in vlan number.
  Failed to set port(s) 2/1-4 channel mode to on.
  Switch-A> (enable) show port channel
  No ports channelling

On Switch B, when you turn the channel on, it indicates that the ports are channelling fine. You know, however, that Switch A is not configured correctly.


  Switch-B> (enable) show port channel
  No ports channelling
  Switch-B> (enable) show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1      connected 1 normal a-full a-100 10/100BaseTX
  2/2      connected 1 normal a-full a-100 10/100BaseTX
  2/3      connected 1 normal a-full a-100 10/100BaseTX
  2/4      connected 1 normal a-full a-100 10/100BaseTX
  Switch-B> (enable) set port channel 2/1-4 on
  Port(s) 2/1-4 channel mode set to on.
  Switch-B> (enable)
  2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1
  2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
  2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
  2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
  2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
  2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
  2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
  2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected on channel WS-C5505 066507453(SW 2/1
  2/2   connected on channel WS-C5505 066507453(SW 2/2
  2/3   connected on channel WS-C5505 066507453(SW 2/3
  2/4   connected on channel WS-C5505 066507453(SW 2/4
  --- ------ ------- -------- ------------ -------

You must check both sides of the channel when manually configuring the channel to ensure that both sides, not just one side, are up. The above output shows that Switch B is set for a channel, but Switch A is not channelling because it has one port that is in the wrong VLAN.


Waiting Too Long Before You Configure the Other Side

In this situation, Switch B has EtherChannel turned on, but Switch A does not because it has a VLAN configuration error. (Ports 2/1-3 are in VLAN 1, and port 2/4 is in VLAN 2.) This is what happens when one side of an EtherChannel is set to on while the other side is still in auto mode: after a few minutes, Switch B shuts down its ports because of a spanning loop detection. This is because Switch B ports 2/1-4 all act as one large port while Switch A ports 2/1-4 are all completely independent ports. A broadcast sent from Switch B to Switch A on port 2/1 is sent back to Switch B on ports 2/2, 2/3, and 2/4 because Switch A treats these ports as independent ports. Switch B interprets this as a spanning tree loop. Notice that the ports on Switch B are now disabled and have a status of errdisable:


  Switch-B> (enable)
  2000 Jan 17 22:55:48 %SPANTREE-2-CHNMISCFG: STP loop - channel 2/1-4 is disabled in vlan 1.
  2000 Jan 17 22:55:49 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4
  2000 Jan 17 22:56:01 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-4
  2000 Jan 17 22:56:13 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/1-4
  2000 Jan 17 22:56:36 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/1-4
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1 errdisable on channel
  2/2 errdisable on channel
  2/3 errdisable on channel
  2/4 errdisable on channel
  --- ------ ------- -------- ------------ -------
  Switch-B> (enable) show port
  Port  Name  Status  Vlan  Level  Duplex  Speed  Type
  ----  ----  ------  ----  -----  ------  -----  -----
  2/1      errdisable 1 normal auto auto 10/100BaseTX
  2/2      errdisable 1 normal auto auto 10/100BaseTX
  2/3      errdisable 1 normal auto auto 10/100BaseTX
  2/4      errdisable 1 normal auto auto 10/100BaseTX
Correcting errdisable State

Sometimes, when you try to configure EtherChannel but the ports are not configured in the same way, the ports on one side of the channel shut down. The link lights are yellow on the port. You can tell this from the console by issuing the show port command. The ports are listed as errdisable. To recover from this, fix the mismatched parameters on the ports involved, and then reenable the ports.

Note: This reenabling of the ports is a separate step that must be done if the ports are to become functional again.

In this example, you know that Switch A had a VLAN mismatch. On Switch A, put port 2/4 back into VLAN 1.

Then, turn on the channel for ports 2/1-4. Switch A does not show that it is connected until you reenable Switch B ports. After you have fixed Switch A and put it in channelling mode, go back to Switch B and reenable the ports.


  Switch-A> (enable) set vlan 1 2/4
  VLAN 1 modified.
  VLAN 2 modified.
  VLAN Mod/Ports
  ---- ----------
  1   2/1-24
  Switch-A> (enable) set port channel 2/1-4 on
  Port(s) 2/1-4 channel mode set to on.
  Switch-A> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   notconnect   on   channel
  2/2   notconnect   on   channel
  2/3   notconnect   on   channel
  2/4   notconnect   on   channel
  --- ------ ------- -------- ------------ -------
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   errdisable   on   channel
  2/2   errdisable   on   channel
  2/3   errdisable   on   channel
  2/4   errdisable   on   channel
  --- ------ ------- -------- ------------ -------
  Switch-B> (enable) set port enable 2/1-4
  Ports 2/1-4 enabled.
  Switch-B> (enable)
  2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/1 joined bridg4
  2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
    2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
  2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
  Switch-B> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected   on   channel
  2/2   connected   on   channel
  2/3   connected   on   channel
  2/4   connected   on   channel
  --- ------ ------- -------- ------------ -------
Showing What Happens When a Link Breaks and Is Restored

When a port in the channel goes down, any packets that are normally sent on that port are shifted over to the next port in the channel. You can verify that this occurs by issuing the show mac command. In this test bed, Switch A sends ping packets to Switch B to determine which link the traffic is using. You clear the counters, issue the show mac command, send three pings, and then issue the show mac command again to determine on which channel the ping responses were received.


  Switch-A> (enable) clear counters
  This command will reset all MAC and port counters reported in CLI and SNMP.
  Do you want to continue (y/n) [n]? y
  MAC and Port counters cleared.
  Switch-A> (enable) show port channel
  Port Status Channel Channel Neighbor Neighbor
                  mode     status     device     port
  --- ------ ------- -------- ------------ -------
  2/1   connected on channel   WS-C5505 066509957(SW 2/1
  2/2   connected on channel   WS-C5505 066509957(SW 2/2
  2/3   connected on channel   WS-C5505 066509957(SW 2/3
  2/4   connected on channel   WS-C5505 066509957(SW 2/4
  --- ------ ------- -------- ------------ -------
  Switch-A> (enable) show Mac
  Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast
  ---- ----------- ------------ -------------
  2/1     0     18     0
  2/2     0     2     0
  2/3     0     2     0
  2/4     0     2     0
  Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive
  Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive
  Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive
  Switch-A> (enable) show Mac
  Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast
  ---- ----------- ------------ -------------
  2/1     3     24     0
  2/2     0     2     0
  2/3     0     2     0
  2/4     0     2     0

At this point, ping responses are received on port 3/1. When the Switch B console sends a response to Switch A, the EtherChannel uses port 2/1. Now, shut down port 2/1 on Switch B. From Switch A, issue another ping and determine on which channel the response comes back.

Note: Switch A sends on the same port to which Switch B is connected. Only the received packets from Switch B are shown because the transmit packets appear later in the show mac command output.


  1999 DEC 19 01:30:23 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4
  Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive
  Switch-A> (enable) show mac
  Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast
  ---- ----------- ------------ -------------
  2/1     3     37     0
  2/2     1     27     0
  2/3     0     7     0
  2/4     0     7     0

Now that port 2/1 is disabled, EtherChannel automatically uses the next port in the channel, 2/2. Reenable port 2/1, and wait for it to join the bridge group. Then, issue two more pings.


  1999 DEC 19 01:31:33 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
  Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive
  Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive
  Switch-A> (enable) show mac
  Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast
  ---- ----------- ------------ -------------
  2/1     5     50     0
  2/2     1     49     0
  2/3     0     12     0
  2/4     0     12     0

Note: These pings are sent from port 2/1. When the link comes back up, EtherChannel again adds it to the bundle and uses it. All this is done transparently to the user.


Commands Used in This Document

Commands to Set the Configuration
  • set port channel on—Turns on the EtherChannel feature.
  • set port channel auto—Resets the ports to their default mode of auto.
  • set port channel desirable—Sends PAgP packets to the other side requesting that a channel be created.
  • set port enable—Enables the ports after set port disable is issued or after an errdisable state.
  • set port disable—Disables a port during other configuration settings.
  • set trunk desirable—Turns on trunking by causing this port to send to the other switch a request that this be a trunk link. Also, if the port is set to negotiate (the default setting), it requests to negotiate the type of trunking to use on the link (ISL or 802.1Q).
Commands to Verify the Configuration
  • show version—Displays the version of software that the switch is running.
  • show module—Displays the modules that are installed in the switch.
  • show port capabilities—Determines if the ports you want to use have EtherChannel capabilities.
  • show port—Determines the status of the port (notconnect or connected) as well as the speed and duplex settings.
  • ping—Tests connectivity to the other switch.
  • show port channel—Shows the current status of the EtherChannel bundle.
  • show port channel mod/port —Provides a more detailed view of the channel status of a single port.
  • show spantree—Verifies that STP viewed the channel as one link.
  • show trunk—Shows the trunking status of ports.
Commands to Troubleshoot the Configuration
  • show port channel—Shows the current status of the EtherChannel bundle.
  • show port—Determines the status of the port (notconnect or connected) as well as the speed and duplex settings.
  • clear counters—Resets the switch packet counters to zero. The counters are visible with the show mac command.
  • show mac—Shows packets received and sent by the switch.
  • ping—Tests connectivity to the other switch and generates traffic that appears in the show mac command output.
Commands to Help Create the Troubleshooting Scenarios
  • set vtp domain testDomain—Gives the switch a VTP domain, which is required to add VLANs on the switch.
  • set vlan 2 name vlan2—Creates VLAN 2 with a name of "vlan2".
  • set vlan 2 2/4—Moves port 2/4 into VLAN 2.
  • set port channel 2/1-4 desirable—Sends PAgP packets to the other side requesting the creation of a channel.
  • set port channel 2/1-4 auto—Resets the ports to their default mode of auto.
  • set port channel 2/1-4 on—Sets the channel mode of these ports to on. No PAgP packets are sent to the other side. This side simply assumes that the other side has formed a channel as well.
  • set vlan 1 2/4—Moves port 2/4 into VLAN 1.
Command Summary

Because this document uses CatOS software version 4.5, the command syntax is taken from the Switch Software Documentation, Release 4.5.



Appendix A: Ethernet Crossover Cables

These cables are available from most computer stores. Also, it is possible to make your own. These two images show the pinouts required for a switch-to-switch crossover cable:



NetPro Discussion Forums - Featured Conversations

Networking Professionals Connection is a forum for networking professionals to share questions, suggestions, and information about networking solutions, products, and technologies. The featured links are some of the most recent conversations available in this technology.




Related Information
  • Multilayer LAN Switches Documentation
  • Configuring Fast EtherChannel and Gigabit EtherChannel
  • Understanding EtherChannel Load Balancing and Redundancy on Catalyst Switches
  • LAN Product Support Pages
  • LAN Switching Support Page
  • Technical Support - Cisco Systems

All contents are Copyright © 1992-2005 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.

SPECIALS!
Dual Cisco 2501/2513 Router CCNA Kit
$199.99
Dual 1720 32/8 CCNA Kit
$329.99
Google