Cisco Cloud Campus LAN Design Guide (CVD)

 

This document provides a pre-validated design and deployment guide for a Cisco Campus LAN with Catalyst^® Switches and Access Points running in either Cloud Managed or Cloud Monitored mode alongside the various design guidelines, topologies, technologies, configurations, and other considerations relevant to the design of any highly available, full-service campus switching fabric. It is also intended to serve as a guide to direct readers to general design and best practices for Cloud-based Cisco Campus LAN.

Overview

The LAN is the networking infrastructure that provides access to network communication services and resources for end users and devices spread over a single floor or building. You create a campus network by interconnecting a group of LANs that are spread over a local geographic area. Campus network design concepts include small networks that use a single LAN switch, up to very large networks with thousands of connections.

The campus wired LAN enables communications between devices in a building or group of buildings, as well as interconnection to the WAN and Internet edge at the network core.

Specifically, this design provides a network foundation and services that enable:

●      Tiered LAN connectivity

●      Wired network access for employees

●      IP Multicast for efficient data distribution

●      Wireless and Wired infrastructure ready for multimedia services

Cisco's Campus LAN architecture offers customers a wide range of options. The Catalyst portfolio with Digital Network Architecture (a.k.a. Cisco Catalyst Center, previously known as Cisco DNA Center) provides a roadmap to digitization and a path to realizing immediate benefits of network automation, assurance and security with an on-prem operating model. The Catalyst portfolio with Meraki Dashboard enables customers to accelerate business evolution through easy-to-use cloud networking technologies that deliver secure customer experiences and simple deployment of network products with a cloud-first operating model.

The proposed architecture enables you to build secure, scalable, and robust enterprise networks. Since the design involves deploying Catalyst platforms in either Cloud Managed or Cloud Monitored modes, special attention should be given to proper planning and design to ensure interoperability and performance.

Introduction

Designing a LAN for the campus use case is not a one-design-fits-all proposition. The scale of campus LAN can be as simple as a single switch and wireless AP at a small remote site or a large, distributed, multi-building complex with high-density wired port and wireless requirements. The deployment may require very high availability for the services offered by the network, with a low tolerance for risk, or there may be tolerance for fix-on-failure approach with extended service outages for a limited number of users considered acceptable. Platform choices for these deployments are often driven by needs for network capacity, the device and network capabilities offered, and the need to meet any compliance requirements that are important to the organization.

This document provides a pre-validated design and deployment guide for a Cisco Campus LAN with Catalyst Switches and Access Points running in either Cloud Managed or Cloud Monitored mode alongside the various design guidelines, topologies, technologies, configurations, and other considerations relevant to the design of any highly available, full-service campus switching fabric. It is also intended to serve as a guide to direct readers to general design and best practices for Cloud-based Cisco Campus LAN.

Cloud management and monitoring for Cisco Catalyst

Cloud monitoring

Selected Cisco Catalyst devices (9200, 9300, and 9500) are capable of connecting to the Meraki Dashboard for monitoring purposes. This offers dashboard monitoring and insights for Catalyst devices including visibility into some configuration items. However, please note that this does not offer full management in Meraki Dashboard. (i.e. No configuration changes in Meraki Dashboard). Please see the following snapshot of C9500 switches/stacks in the Meraki Dashboard:

For more information about Cloud Monitoring, please refer to this article.

Campus LAN architecture with Cloud management

Please refer to the following proposed architecture diagram as a reference for this CVD:

To achieve a robust, reliable, high speed and Future Proof Campus LAN, the following components are part of this architecture:

Logical architecture

This document will provide three options to design this campus architecture from a logical standpoint, which are outlined below (each with its own characteristics):

Layer 2 Access with Native VLAN 1

This option assumes that your Spanning Tree Protocol (STP) domain is extended all the way to your core layer. It offers great flexibility in terms of network segments as you can have your VLANs spanning over the different stacks/closets. However, the STP configuration and tuning is crucial since the Catalyst platforms can run different STP protocols than the Meraki MS390 switches.

Pros:

●      Flexibility in your VLAN design

●      Facilitates Wireless Roaming across the whole campus

●      Easier to deploy and consistent configuration across the entire Campus LAN

Cons:

●      Non-deterministic route failover

●      Slow convergence

●      Different STP protocol support on Cloud Monitored and Cloud Managed Catalyst Switches

●      The possibility of VLAN hopping

Layer 2 access without Native VLAN 1

This option is similar to the above except that VLAN 1 does not exist and the default Native VLAN 1 is replaced with another non-trivial VLAN assignment which can be considered a more preferable option for customers as its separate from the Management VLAN

Pros:

●      Flexibility in your VLAN design

●      Facilitates Wireless Roaming across the whole campus

●      Easier to deploy and consistent configuration across the entire Campus LAN

●      Minimize the risk of VLAN hopping

Cons:

●      Non-deterministic route failover

●      Slow convergence

●      Different STP protocol support on Cloud Monitored and Cloud Managed Catalyst Switches

Layer 3 access

This option assumes that your Open Shortest Path First (OSPF) domain is extended all the way to your core layer and thus there is no need to rely on STP between your Access and Core for convergence. It offers fast convergence since it relies on Equal-cost multi-path routing (ECMP) rather than STP layer 2 paths. However, it doesn't offer great flexibility in your VLAN design as each VLAN cannot span between multiple stacks/closets.

Pros:

●      Deterministic route failover

●      Fast convergence

●      Relies on either stacking or gateway redundancy at upper layers

Cons:

●      VLANs cannot span multiple stacks/closets

●      Your backbone area size can be unmanageable

●      Layer 3 roaming is not possible without a concentrator

This CVD offers the design and configuration guidelines for ALL options above.

Campus LAN planning, design, and configuration

Planning

The following section provides information on planning your solution and ensuring that you have a successful deployment. This will include gathering the design requirements and planning for your Cloud-based Cisco Campus LAN architecture based on your own requirements.

Prior to proceeding to plan for your deployment, please refer to the Campus LAN Design Best Practices Guide which can be used to guide you through the planning phase of designing your Campus LAN.

Meraki cloud administration and management

If you don't have an account on the Meraki Dashboard, create one following these steps:

1.     Generate an API Key for your account following these steps.

2.     Claim your order(s) or serial number(s) into your Meraki Dashboard account.

3.     Add your devices to existing networks or create new networks as required.

4.     Configure firmware upgrades for your network(s) with latest Stable or RC releases for each device type (Please check the firmware changelog for platform-specific details).

5.     Configure your network(s) with the correct time zone from Network-wide > Configure > General (This is key for reporting and firmware upgrades).

6.     Configure your network(s) with the desired upgrade date and time.

7.     Configure the MR upgrade behavior as desired.

8.     Ensure that your Campus LAN has access to the internet for management purposes.

9.     Ensure that Meraki Cloud is accessible and that all required ports are opened where applicable (information can be found in Dashboard).

10.  Ensure that there is sufficient bandwidth for firmware upgrades as they tend to be large in size.

11.  Ensure that only current administrators are added with the correct permissions on the Meraki dashboard (unless SAML is configured for Single Sign-on).

12.  If using Single sign-on integration with Meraki dashboard, please ensure that login to dashboard is scoped such that administrators have the correct level of access where applicable (e.g. Per network, Per switch port, etc.). For more information about dashboard access roles, please refer to the following article.

13.  In case of SAML SSO, it is still required to have one valid administrator account with full rights configured on the Meraki dashboard. However, it is recommended to have at least two accounts to avoid being locked out from dashboard.

14.  Where applicable, ensure that the designated Management VLAN has access to Dynamic Host Configuration Protocol (DHCP) (at least during initial bootup before assigning a static IP address) and also to the internet.

Radius integration (e.g. Cisco ISE)

1.     If using an external Radius server (e.g. Cisco ISE), then ensure that the network segment where ISE is hosted can access the Management VLAN configured on your network devices (or the Alternate Management Interface on MR and/or MS if configured and where applicable).

2.     Ensure that all required ports are opened where applicable (e.g. 1812, 1813, etc.).

Active directory integration

1.     If using an external identity source (e.g. Active Directory), then ensure that the network segment where the AD is hosted can access the Management VLAN configured on your network devices (or the Alternate Management Interface on MR and/or MS if configured with Radius integration).

2.     Ensure that all required ports are opened where applicable (e.g. 3268, 389, etc.).

Catalyst onboarding for cloud monitoring (C9200/9300/9500)

For ease of management, Customers can onboard Cisco C9200/9300/9500 switches/stacks for Cloud Monitoring such that they can be available in the Meraki Dashboard in Monitor only mode. This process enables dashboard monitoring on these switches/stacks and selected configuration parameters will be visible in the Meraki Dashboard. Please refer to the following article for the supported Catalyst 9000 series.

Pre-requisites

Please ensure the following prior to onboarding a switch/stack for Cloud Monitoring:

●      It is a supported model (Please refer to this article)

●      Running IOS-XE 17.3 – 17.10.1

●      It must have an SVI or routed interface that has access to the Internet on port TCP 443

●      It must have a valid DNS server

●      It must have a valid DNA software subscription

●      It must have Telnet for connectivity pre-check (Please refer to this article)

●      A valid Dashboard account and API Key

●      A computer with both access to internet on port 443 and access to the switch(es)

Onboarding catalyst devices for cloud monitoring

The onboarding process for the C9500 core switches is out of scope for the purposes of this CVD. Please refer to the following article for a step by step guide on onboarding Catalyst for Cloud Monitoring.

Switch Status on Meraki dashboard

Once the device has been onboarded for Meraki dashboard monitoring, it should come online on dashboard after several minutes and also the network topology will show all switches in Monitor Only mode.

Design and configuration guidelines

Option 1: STP Based convergence with Native VLAN 1

Overview

This design option allows for flexibility in terms of VLAN and IP addressing across the Campus LAN such that the same VLAN can span across multiple access switches/stacks thanks to Spanning Tree that will ensure that you have a loop-free topology. However, this method of convergence is considered non- deterministic since the path of execution isn't fully determined (unlike Layer 3 routing protocols for example). As a result, convergence can be slow and STP must be tuned to provide best results.

This design is based on consistent STP protocols running in this campus deployment, as such Multiple Spanning Tree Protocol (MST, aka 802.1s) will be configured since it is supported on both the Meraki and Catalyst platforms.

You should consider this option if you need a consistent VLAN assignment across all switching closets. Here are some things to consider about this design option:

Pros:

●      Flexibility in your VLAN design

●      Facilitates Wireless Roaming across the whole campus

●      Easier to deploy and consistent configuration across the entire Campus LAN

Cons:

●      Non-deterministic route failover

●      Slow convergence

●      Different STP protocol support on Cloud Managed and Cloud Monitored Catalyst Switches

Since MST will be used as a loop prevention mechanism, all SVIs will be created on the collapsed core layer.

Logical architecture

The following diagram shows the logical architecture highlighting STP convergence within a campus LAN design leveraging Cloud Managed and Cloud Monitored Catalyst platforms:

Physical architecture

The following diagram shows the physical architecture and port list for this design:

Assumptions

The following assumptions have been considered:

●      It is assumed that Wireless roaming is required everywhere in the Campus

●      It is assumed that VLANs are spanning across multiple zones/closets

●      Corporate SSID (Broadcast in all zones/areas) users are assigned VLAN 10 on all APs. CoA VLAN is VLAN 30 (via Cisco ISE)

●      BYOD SSID (Broadcast in all zones/areas) users are assigned VLAN 20 on all APs. CoA VLAN is VLAN 30 (via Cisco ISE)

●      Guest SSID (Broadcast in all zones/areas) users are assigned VLAN 30 on all APs

●      IoT SSID (Broadcast in all zones/areas) users are assigned VLAN 40 on all APs

●      Access Switches will be running in Layer 2 mode (No SVIs or DHCP)

●      MS390 Access Switches physically stacked together

●      C9300-M (or compatible) Access Switches physically stacked together

●      C9500 Core Switches with Stackwise-virtual stacking using SVLs

●      Access Switch uplinks are in trunk mode with native VLAN = VLAN 1 (Management VLAN^*)

●      STP root is at Distribution/Collapsed-core

●      Distribution/Collapsed-core uplinks are in Trunk mode with Native VLAN = VLAN 1 (Management VLAN)

●      All VLAN SVIs are hosted on the core layer

●      Network devices will be assigned fixed IPs from the management VLAN DHCP pool. Default Gateway is 10.0.1.1

Network segments

Please check the following table for more information about the network segments (e.g. VLANs, SVIs, etc.) for this design:

Device list

Access policies

Port list

Wireless SSID list

Group policies

Configuration and implementation guidelines

1.     Login to your dashboard account (or create an account if you don't have one)

2.     Navigate to Organization > Configure > Inventory

3.     For Co-term license model, click on Claim. And for PDL, please click on Add

4.     Enter the order and/or serial number(s) to claim the devices into your account. For PDL, click Next then please choose to add them to Inventory (Do not add them to a network)

5.     Create a Dashboard Network: Navigate to Organization > Configure > Create network to create a network for your Campus LAN (Or use an existing network if you already have one). If you are creating a new network, please choose "Combined" as this will facilitate a single topology diagram for your Campus LAN. Choose a name (e.g. Campus) and then click Create network

6.     Dashboard Network Settings: Navigate to Network-wide > Configure > General and choose the settings for your network (e.g. Time zone, Traffic Analytics, firmware upgrade day/time, etc.)

7.     Schedule Firmware Upgrade: Navigate to Organization > Monitor > Firmware upgrades to select the firmware settings for your devices such that devices upgrade once they connect to dashboard. Select the device type then click on Schedule upgrade.

8.     Add Devices to a Dashboard Network: Navigate to Organization > Configure > Inventory:

●      For Co-term licensing model, select the MS390 and C9300 switches and the Primary WAN Edge then click on Add then choose the Network Campus

●      For PDL licensing model, select the MS390 and C9300 switches and the Primary WAN Edge then click on Change network assignment and then choose the Network Campus

●      Please DO NOT add the Secondary WAN Edge device at this stage

9.     Rename MX Security Appliance: Navigate to Security and SD-WAN > Monitor > Appliance status then click on the edit button to rename the MX to Primary WAN Edge then click on Save.

10.  MX Connectivity: Plug in your WAN uplink(s) on the Primary WAN Edge MX then power it on and wait for it to come online on dashboard. This might take a few minutes as the MX will download its firmware and configuration. Navigate to Security and SD-WAN > Monitor > Appliance status and verify that the MX has come online and that its firmware and configuration is up to date.

11.  Rename Access Switches: Navigate to Switching > Monitor > Switches then click on each MS390 and C9300 switch and then click on the edit button on top of the page to rename it per the above table then click on Save such that all your switches have their designated names.

12.  Rename MR APs: Navigate to Wireless > Monitor > Access points then click on each AP and then click on the edit button on top of the page to rename it per the above table then click on Save such that all your APs have their designated names.

13.  MR AP Tags: Navigate to Wireless > Monitor >Access points then click on each AP and then click on the edit button next to TAGS to add Tags to your AP per the above table then click on Save such that all your APs have their designated tags.

14.  MX Addressing and VLANs: Navigate to Security and SD-WAN > Configure > Addressing and VLANs, and in the Deployment Settings menu select Routed mode. Further down the page on the Routing menu, click on VLANs then click on Add VLAN to add your management VLAN then click on Create. Then for the per-port VLAN settings, select your downlink ports (19 and 20) and click on Edit and configure them as access with VLAN 1 and click on Update. Finally, click on Save at the bottom of the page.

15.  Campus LAN Static Routes: Create Static Routes for your Campus network by navigating further down the page to Static routes then click on Add Static Route. Start by adding your Corporate LAN subnet then click on Update and then add static routes to all other subnets (e.g. BYOD, Guest and IoT). Finally, click on Save at the bottom of the page. (The Next hop IP that you have used here will be used to create a fixed assignment for the Core Stack later in DHCP settings).

16.  Optional - If you are accessing any resources over Meraki SD-WAN, please navigate to Security and SD-WAN > Configure > Site-to-site VPN and enable VPN based on your topology and traffic flow requirements. (In this case we will configure this Campus as Spoke with Split Tunneling)

●      Choose Type: Spoke then click on Add a hub and select your hub site where you need access to resources via VPN. You can also add multiple hubs for resiliency. To choose Split Tunneling, please leave the box next to the Hub unticked as shown below.

●      Under VPN Settings, choose which subnet to be Enabled in VPN (e.g. Management VLAN will be required for Radius authentication purposes as the MR/MS390/C9300 devices will reach out to Cisco ISE using their management IP). Any Subnet that needs to access resources via VPN must be Enabled otherwise keep it as Disabled.

●      Finally, click on Save at the bottom of the page

●      On the Hub site, please make sure to advertise the subnets that are required to be reachable via VPN. Navigate to Security and SD-WAN > Configure > Site-to-site VPN then add a local network then click Save at the bottom of the page (Please make sure that you are configuring this on the Hub's dashboard network)

17.  Optional - Verify that your VPN has come up by selecting your Campus LAN dashboard network from the Top-Left Network drop down list and then navigate to Security and SD-WAN > Monitor >VPN status then check the status of your VPN peers. Next, navigate to Security and SD-WAN > Monitor > Route table and check the status of your remote subnets that are reachable via VPN. You can also verify connectivity by pinging a remote subnet(e.g. 172.31.16.32 which is Cisco ISE) by navigating to Security and SD-WAN > Monitor > Appliance status then click on Tools and ping the specified IP address (Please note that the MX will choose the highest IP participating in VPN by default as the source).

Please note that in order to ping a remote subnet, you must either have BGP enabled or have static routes at the far-end pointing back to the Campus LAN local subnets.

In this example, the VPC in AWS has been configured with a Route Entry to route 10.0.1.0/24 via the vMX deployed in AWS that has a VPN tunnel back to the Campus LAN site.

If the remote VPN peer (e.g. AWS) is configured in Routed mode, the static route is not required since traffic will always be NAT'd to a local reachable IP address.

18.  SD-WAN and Traffic Shaping Configuration: To configure Traffic Shaping settings for your Campus LAN site. Navigate to Security and SD-WAN > Configure > SD-WAN and Traffic Shaping to configure your preferred settings. For the purpose of this CVD, the default traffic shaping rules will be used to mark traffic with a DSCP tag without policing egress traffic (except for traffic marked with DSCP 46) or applying any traffic limits. (Please adjust these settings based on your requirements such as traffic limits or priority queue values. For more information about traffic shaping settings on the MX devices, please refer to the following article).

19.  Optional - Configure Threat Protection (Requires Advanced License or above) for your Campus LAN site. Navigate to Security and SD-WAN > Configure > Threat Protection and choose the settings that meet your site requirements. Please see the following configuration example:

20.  Click on Save at the bottom of the page.

21.  Optional - Configure Content Filtering Settings (Requires Advanced License or above) for your Campus LAN site. Navigate to Security and SD-WAN > Configure > Content filtering and choose the settings that meet your site requirements. Please see the following configuration example:

22.  Click on Save at the bottom of the page.

23.  Core Switch Uplinks: On the Catalyst 9500 core switches, Connect their uplinks to the Primary WAN Edge MX and power them both on.

24.  Core Switch Network Access: Connect to first C9500 switch via console and configure it with the following commands:

25.  Core Switch Network Access: Connect to the second C9500 switch via console and configure it with the following commands:

26.  SVL Configuration: Now that both C9500 switches have access to the network, proceed to configure the Stackwise Virtual Links per the port list provided above (In this case with using two ports as part of the SVL providing a total stacking bandwidth of 80 Gbps).

27.  Connect Stacking Cables: Whilst the C9500 switches are reloading, connect the stacking cables on both switches.

28.  Verify Stackwise Configuration: Please wait for about 10 minutes for the switches to come back up and initialize the stack. Then, connect to the 9500-01 (Stack Master) via console to verify that the stack is operational. The stackwise-virtual link should be U (Up) and R (Ready).

29.  Optional - Attach and configure stackwise-virtual dual-active-detection: DAD is a feature used to avoid a dual- active situation within a stack of switches. It will rely on a direct attachment link between the two switches to send hello packets and determine if the active switch is responding or not. Please note that DAD cannot be applied to any SVL links and has to be a dedicated interface. For the purpose of this CVD, interface HundredGigE1/0/27 and HundredGigE2/0/27 will be used for enabling DAD between the two C9500 switches.

30.  Configure Multiple Spanning Tree Protocol (802.1s). Connect to the 9500-01 (Stack Master) via console and use the following commands:

31.  Verify Spanning Tree Configuration (Please note that interface Twe2/0/1 will be in STP blocking state due to the fact that both uplinks are connected to the same MX edge device at this stage).

32.  Configure STP Root Guard and UDLD on the Core Stack Downlinks:

33.  Optional - STP Hygiene: It is recommended to configure STP Root Guard on all C9500 Core Stack downlinks to avoid any new introduced downstream switches from claiming root bridge status.

34.  Optional - STP Hygiene: It is recommended to configure STP Loop Guard on all C9500 Core Stack un-used stacking links.

35.  Configure SVIs for your Campus LAN on the Core Stack:

36.  Verify your DHCP pool configuration:

37.  Verify your SVI configuration:

38.  Configure Layer 2 Switchports, SGTs and CST (Cisco TrustSec) on your Core Stack interfaces. (Please note that enforcement has been disabled on downlink ports allowing it to happen downstream):

39.  Spare WAN Edge Connectivity: Follow these steps to create warm-spare with two MX appliances: (Please note that this might result in a brief interruption of packet forwarding on the MX Appliance):

●      Navigate to Security and SD-WAN > Monitor > Appliance status and click on Configure warm spare

●      Now click on Enabled then choose the Spare MX from the drop-down menu and then choose the Uplink IP option that suits your requirements (Please note that choosing Virtual IPs requires an additional IP address on the upstream network and a single broadcast domain between the two MXs) then click on Update

●      Now click on Spare to access the Appliance status page of your Spare MX and click on the Edit button to rename the spare unit (e.g. Secondary WAN Edge)

 

●      Then configure the following on your C9500 Core Stack:

●      Then connect the Spare MX downlinks to your C9500 Core Stack (e.g. Spare MX port 19 to Twe1/0/2 and port 20 to Twe2/0/2)

●      Then connect the Spare MX with its uplinks (This must match the uplink configuration on your Primary WAN Edge)

●      Power on the Spare MX and wait for it to come online on dashboard

 

 

●      You can also verify that your C9500 Core Stack interfaces to the Spare MX are up, and that the redundant uplinks are in STP BLK mode

40.  Access Policy configuration: When you're logged in dashboard, Navigate to Switching > Configure > Access policies to configure Access Policies as required for your Campus LAN. Please see the following example for two Access Policies; 802.1x and MAB.

41.  Adaptive Policy Configuration: Configure Adaptive Policy for your Campus LAN. When you're logged in dashboard, Navigate to Organization > Configure > Adaptive Policy then click on the Groups tab on the top.

There should be two groups (Unknown, Infrastructure) that are already available. Click on Add group to add each group required for your Campus LAN. You need to fill in the Name, the SGT value, and a description then click on Review changes then click on Submit. Please see the following examples:

42.  Adaptive Policy Configuration: Configure Adaptive Policy for your Campus LAN. When you're logged in dashboard, Navigate to Organization > Configure > Adaptive Policy then click on the Policies tab on the top. The source groups are on the left side, and the destination groups are on the right side. Select a source group from the left side then select all destination groups on the right side that should be allowed then click on Allow and click on Save at the bottom of the page. Next, select a source group from the left side then select all destination groups on the right side that should be denied (i.e. Blocked) then click on Deny and click on Save at the bottom of the page. After creating the policy for that specific source group, the allowed destination groups will be displayed with a green tab and the denied destination groups will be displayed with a red tab. Repeat this step for all policies required for all Groups (Allow and Deny).

43.  Access Switch Ports Configuration: Configure Uplink Ports on your Access Switches. When you're logged in dashboard, Navigate to Switching > Monitor > Switch Ports, then select your uplink ports and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard)

44.  Optional - For ease of management, it is recommended that you rename the ports connecting to your Core switches with the actual switch name / Connecting port as shown below.

45.  Access Switch Ports Configuration: Configure Wired Client Ports (802.1x) on your Access Switches. Navigate to or Refresh Switching > Monitor > Switch Ports, then select your Wired Client ports (5-8) and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard)

46.  Access Switch Ports Configuration: Configure Wired Client Ports (MAB) on your Access Switches. Navigate to or Refresh Switching > Monitor > Switch Ports, then select your Wired Client ports (9-12) and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard)

47.  Access Switch Ports Configuration: Configure MR Ports on your Access Switches. Navigate to or Refresh Switching > Monitor > Switch Ports, then select your ports connecting to MR Access Points (13-16) and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard)

48.  Optional - Access Switch Ports Configuration: Configure unused ports on your Access Switches such that they are disabled and mapped to an unrouted VLAN (e.g. VLAN 999). Navigate to Switching > Configure > Switch Ports and filter for any unused ports (e.g. 17-24) and configure them as shown below.

49.  Rename Wireless SSIDs: To configure your SSIDs per the above table, first navigate to Wireless > Configure SSIDs then rename the SSIDs per your requirements (Refer to the above table for guidance).

●      SSID#1 (First column, aka vap:0, enabled by default): Click on rename and change it to Acme Corp

●      SSID#2 (Second column, aka vap:1): Click on rename and change it to Acme BYOD, then click on the top drop-down menu to enable it

●      SSID#3 (Third column, aka vap:2): Click on rename and change it to Guest, then click on the top drop-down menu to enable it

●      SSID#4 (Fourth column, aka vap:3): Click on rename and change it to Acme IoT, then click on the top drop- down menu to enable it

●      Click Save at the bottom of the page

50.  Configure Access Control for Acme Corp: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Acme Corp.

●      Click Save at the bottom of the page

●      Please Note: Adaptive Policy Group feature is not currently available in the New Version of the Access. You will need to click on View old version

View old version

which is available at the top right corner of the page to be able to access this and configure the Adaptive Policy Group (10: Corp). Then, please click Save at the bottom of the page

51.  Configure Access Control for Acme BYOD: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Acme BYOD.

 

 

 

 

 

 

●      Click on

View old Version

which is available on the top right corner of the page, then choose the Adaptive Policy Group 20: BYOD and then click on Save at the bottom of the page.

52.  Configure Access Control for Guest: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Guest.

●      Click on

View old Version

at the top right corner of the page then choose the Adaptive Policy Group 30: Guest then click on Save at the bottom of the page

53.  Configure Access Control for Acme IoT: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Acme IoT.

●      Click on

View old version

at the top right corner of the page then choose the Adaptive Policy Group 40: IoT then click on Save at the

●      bottom of the page

54.  Enabling Stacking on your MS390 and C9300 Switches in Meraki Dashboard; please follow these steps:

A.   Connect a single uplink to each switch (e.g. Port 1 on MS390-01 to Port TwentyFiveGigE1/0/23 on C9500)

B.   Make sure all stacking cables are unplugged from all switches

C.   Power up all switches

D.   Verify that your C9500 Stack downlinks are up and not shutdown

E.   Wait for them to come online on dashboard. Navigate to Switching > Monitor > Switches and check the status of your Access Switches

F.   After they come online and download their configuration and firmware (Up to date) you can proceed to the next step. You can see their Configuration status and Firmware version from Switching > Monitor > Switches

G.   Enable stacking in dashboard by Navigating to Switching > Monitor > Switch stacks then click on add one

H.   Then give your stack a name and select its members and click on Create

 

I.    Now click on Add a stack to create all other stacks in your Campus LAN access layer by repeating the above steps

 

 

J.   Power off all access switches

K.   Disconnect all uplink cables from all switches

L.   Nominate your master switch for each stack (e.g. MS390-01 for stack1 and C9300-01 for stack2)

M.  On the master switches, plug the uplink again

N.   Plug stacking cables on all switches in each stack to form a ring topology and make sure that the Cisco logo is upright

O.  Power on your master switches first, then power other stack members

P.   Wait for the stack to come online on dashboard. To check the status of your stack, Navigate to Switching > Monitor > Switch stacks and then click on each stack to verify that all members are online and that stacking cables show as connected

 

Q.  Plug uplinks on all other non-master members and verify that the uplink is online in dashboard by navigating to Switching > Monitor > Switch stacks and then click on each stack to verify that all uplinks are showing as connected however they should be in STP discarding mode

 

R.   Configure the same Static IP for all members in each stack by navigating to Switching > Monitor > Switches then click on the master switch (e.g. MS390-01 for Stack1) and under LAN IP menu copy the IP address then click on the edit button to specify the Static IP address information (You can use the same IP address that was assigned using DHCP) then click Save. The same Static IP address information should now be copied for all members of the same stack. You can verify this by navigating to Switch > Monitor > Switches (Tip: Click on the configure button on the right-hand side of the table to add Local IP information display).

 

 

 

 

S.   Finally, configure etherchannels on both your Access Switch Stacks and your Core Switch Stacks so that all uplinks can be operational (STP forwarding mode) at the same time. Follow these steps:

◦    First, disconnect the downlinks to non-master switches from your C9500 Core Stack (e.g. Port TwentyFiveGigE2/0/23 and TwentyFiveGigE2/0/24)

◦    Navigate to Switching > Monitor > Switch ports and search for uplink then select all uplinks in the same stack (in case you have tagged your ports otherwise search for them manually and select them all) then click on Aggregate. Please note that all port members of the same Ether Channel must have the same configuration otherwise Dashboard will not allow you to click the aggregate button.

 

 

 

 

◦    Please repeat above steps for all stacks in your network

◦    Please note that the above step will cause all members within the stack to go offline in Dashboard

●      On your C9500 Core Stack, please configure etherchannel Settings for your downlinks such that each Stack downlinks should be in a separate Port-channel and that the mode is active:

 

●      Plug all uplinks to non-master switches

●      Now all your switches should come back online on Dashboard

 

●      And now all your uplinks from each stack should be in STP Forwarding mode, which you can verify on Dashboard by navigating to Switching > Monitor > Switch stacks and checking the uplink port status. Also, you can check that on your C9500 Core Stack:

 

 

 

55.  Configure Multiple Spanning Tree Protocol (802.1s) in Dashboard for MS390 and C9300 switches: Navigate to Switching > Configure > Switch settings and select your stack and choose the appropriate STP priority per stack (61440 for all Access Switch Stacks) then click Save at the bottom of the page.

●      Verify that the Access Stacks are seeing the C9500 Core Stack as the root by navigating to Switching > Monitor > Switches then click on any switch and under the RSTP root menu check the root bridge information

56.  Configure Dynamic ARP Inspection (DAI) on your C9500 Core Switches: All Downlinks to Access Switches and Uplinks to MX Edge must be configured as Trusted and all other interfaces as Untrusted. (Please note that the order of commands is important to avoid loss of connectivity)

57.  Configure Dynamic Arp Inspection (DAI) on your Access Switch Stacks: Navigate to Switching > Monitor > DHCP Servers and ARP and scroll down to Dynamic ARP Inspection and enable it. Then click Save at the bottom of the page.

58.  Setting up your Access Points: Connect your APs to the respective ports on the Access Switches (e.g. Ports 13-16) and wait for them to come online on dashboard and download their firmware and configuration files. To check the status of your APs navigate to Wireless > Monitor > Access points and check the status, configuration and firmware of your APs.

59.  Re-addressing your Network Devices: In this step, you will adjust your IP addressing configuration to align with your network design. This step could have been done earlier in the process however it will be easier to adjust after all your network devices have come online since the MX (The DHCP server for Management VLAN 1) has kept a record of the actual MAC addresses of all DHCP clients. Follow these steps to re-assign the desired IP addresses: (Please note that this will cause disruption to your network connectivity)

A.   Navigate to Organization > Monitor > Overview then click on Devices tab to check the current IP addressing for your network devices

B.   Navigate to Security and SD-WAN > Monitor > Appliance status then click on the Tools tab and click on Run next to ARP Table

C.   Take a note of the MAC addresses of your network devices

D.   Navigate to Security and SD-WAN > Configure > DHCP then under Fixed IP assignments click on Add a fixed IP assignment and add entries for your network devices using the MAC addresses you have from Step #3 above then click on Save at the bottom of the page

E.   Navigate to Switching > Configure > Switch ports then filter for MR (in case you have previously tagged your ports or select ports manually if you haven't) then select those ports and click on Edit, then set Port status to Disabled then click on Save.

 

F.   After a few minutes (For configuration to be up to date) Navigate to Switching > Configure > Switch ports then filter for MR (in case you have previously tagged your ports or select ports manually if you haven't) then select those ports and click on Edit, then set Port status to Enabled then click on Save.

 

G.   Navigate to Switching > Monitor > Switches then click on each master switch to change its IP address to the one desired using Static IP configuration (remember that all members of the same stack need to have the same static IP address)

H.   On your C9500 Core Stack, bounce your VLAN 1 interface. Then verify that the interface VLAN 1 came up with the correct IP address (e.g. 10.0.0.2 per this design)

 

I.    Navigate to Organization > Monitor > Overview then click on Devices tab to check the current IP addressing for your network devices

60.  Configure QoS in your Campus LAN: Quality of Service configuration needs to be consistent across the whole Campus LAN. Please refer to the above table as an example. To configure QoS, please follow these steps: (For the purpose of this CVD, Default traffic shaping rules will be used to mark traffic with DSCP values without setting any traffic limits. Please adjust traffic shaping rules based on your own requirements)

A.    Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the Acme Corp SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules. Click Save at the bottom of the page when you are done. Click Save at the bottom of the page when you are done.

B.     Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the Acme BYOD SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules.

C.    Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the Guest SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules. Click Save at the bottom of the page when you are done.

D.   Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the IoT SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules. Click Save at the bottom of the page when you are done.

E.   Navigate to Switching > Configure > Switch settings and under the Quality of Service menu configure the VLAN to DSCP mappings. Please click on Edit DSCP to CoS map to change settings per your requirements. (For more information on MS QoS settings and operation, please refer to the following article) Click Save at the bottom of the page when you are done. (Please note that the ports used in the below example are based on Cisco Webex traffic flow)

 

F.   Please ensure that your C9500 Core Stack is configured to trust incoming QoS. Here's a reference of the configuration needed to be applied:

G.   Navigate to Security and SD-WAN > Configure > SD-WAN and Traffic shaping and make sure your Uplink configuration matches your WAN speed. Then, under Uplink selection choose the settings that match your requirements (e.g. Load balancing). Under Traffic shaping rules, select Enable default traffic shaping rules then click on Add a new shaping rule to create the rules needed for your network (for more information about Traffic shaping rules on MX appliances, please refer to the following article). Please see the following example:

 

 

 

For more information about any of the above configurations, please refer to Meraki Documentation for further guidance on configuring Etherchannels, stacking, switch ports, SSId configuration and more. Here is a useful MR – Wireless section and a MS – Switching section.

Testing and Verification

Firmware

The following table indicates the firmware versions used in this Campus LAN:

Device Connectivity

MX WAN Edge

Upstream Connectivity

Internet/Cloud Connectivity

Downstream Connectivity

C9500 Core Stack

Upstream Connectivity

 

Internet Connectivity

 

Downstream Connectivity (Please note that the MS390 and C9300-M platforms will prioritize packet forwarding over ICMP echo replies so it's expected behavior that you might get some drops)

In case of connectivity issues, please check the following:

MS390 Access Stack

Upstream Connectivity

Internet/Cloud Connectivity

Downstream Connectivity

C9300 Access Stack

Upstream Connectivity

Internet/Cloud Connectivity

Downstream Connectivity

MR Access Points

Client Connectivity

802.1x Authentication

802.1x authentication has been tested on both Corp and BYOD SSIDs. Dashboard will be checked to verify the correct IP address assignment and username. Packet captures will also be checked to verify the correct SGT assignment. In the final section, ISE logs will show the authentication status and authorization policy applied.

Authentication Details

Wireless roaming

Wireless roaming has been tested between two zones and APs homed to different switch stacks whilst being on a Webex meeting with Audio/Video and Content share. Device and Client details in the following table:

 

First association

Second Association (The video overlay is the stream from a Webex meeting while the client was roaming)

Traffic Flow (Packet #27)

Webex meeting statistics (Snapshot taken after roaming)

Dashboard logs

STP Convergence

STP convergence will be tested using several methods as outlined below. Please see the following table for steady-state of the Campus LAN before testing:

Introducing loops (Access to Core)

A loop was introduced by adding a link between C9300-01 /NM Port 2 and C9500 Core Stack / Port TwentyFiveGigE1/0/22 (Please note that for the purposes of this test, the interface has been unshut and configured as a Trunk port with Native VLAN 1 with STP guards on that interface).

Introducing Loops (Access Layer, with STP Guard: Loop Guard)

For the purposes of this test and in addition to the previous loop connections, the following ports were connected: MS390-01 / Port 11 < - > C9300-01 / Port 11

Introducing Loops (Access Layer, without STP Guard)

For the purposes of this test and in addition to the previous loop connections, the following ports were connected:

MS390-02 / Port 12 < - > C9300-02 / Port 12

 

Introducing Loops (Core Layer)

For the purpose of this test and in addition to the previous loop connections, the following ports were connected:

Port Twe1/0/10 to port Twe2/0/10 on the C9500 Core switches.

 

Introducing Rogue Bridge in VLAN 1

For the purpose of this test and in addition to the previous loop connections, the Bridge priority on C9300 Stack will be reduced to 4096 (likely root) and increasing the Bridge priority on C9500 to 8192.

●      Downlinks on C9500 are configured with STP Root Guard

●      Access Layer Links (Stack to Stack) are configured with STP Loop Guard + UDLD

 

High Availability and Failover

Here's the steady-state physical architecture for reference:

MX WAN Edge Failover

C9500 Core Stack Loss of Uplink

For the purpose of this test, ports TwentyFiveGigE1/0/1 and TwentyFiveGigE1/0/2 will be disconnected.

 

C9300 Stack Loss of Uplink

For the purpose of this test, NM Port 1 on C9300-01 (Master switch) will be disconnected.

MS390 Stack Loss of Uplink

For the purpose of this test, port 1 on MS390-01 (Master switch) will be disconnected.

QoS

For the purpose of this test, packet capture will be taken between two clients running a Webex session. Packet capture will be taken on the Edge (i.e. MR wireless and wired interfaces) then on the Access (i.e. the MS390 or C9300 uplink port) then on the MX WAN Downlink and finally on the MX WAN Uplink. The table below shows the testing components and the expected QoS behavior:

Access Point Wireless Port pcaps

Client #1

Client #2

Access Point Wired Port pcaps

Client #1

Client #2

Access Switch Uplink pcaps

Client #1

Client #2

MX appliance Downlink pcaps

Client #1

Client #2

MX Appliance Uplink pcaps

Option 2: STP-Based Convergence without Native VLAN 1

Overview

This option is similar to the above except that the default VLAN 1 does not exist and the Native VLAN is replaced with another non-trivial VLAN assignment which can be considered a more preferable option for customers as it's separate from the Management VLAN. Also, a Transit VLAN has been introduced between the C9500 Core Stack and the MX WAN Edge to facilitate the separation between Management traffic (VLAN 100) and Client Traffic (Transit VLAN 192)

This design is based on consistent STP protocols running in this campus deployment, as such Multiple Spanning Tree Protocol (MST, aka 802.1s) will be configured since it is supported on both the Meraki and Catalyst platforms.

You should consider this option if you need to steer away from having VLAN 1 in your Campus LAN. Here's some things to consider about this design option:

Pros:

●      Flexibility in your VLAN design

●      Facilitates Wireless Roaming across the whole campus

●      Easier to deploy and consistent configuration across the entire Campus LAN

●      Minimize the risk of VLAN hopping

●      Considered more secure due to separation between Management traffic and Client traffic

Cons:

●      Non-deterministic route failover

●      Slow convergence

●      Different STP protocols on Cloud Managed and Cloud Monitored Catalyst Switches

Logical Architecture

The following diagram shows the logical architecture highlighting STP convergence within a campus LAN design leveraging Cloud Managed and Cloud Monitored Catalyst platforms:

Physical Architecture

The following diagram shows the physical architecture and port list for this design:

Assumptions

The following assumptions have been considered:

●      VLAN 1 should not be configured on any switchport in this Campus LAN

●      It is assumed that Wireless roaming is required everywhere in the Campus

●      It is assumed that VLANs are spanning across multiple zones

●      Corporate SSID (Broadcast in all zones) users are assigned VLAN 10 on all APs. CoA VLAN is VLAN 30 (Via Cisco ISE)

●      BYOD SSID (Broadcast in all zones) users are assigned VLAN 20 on all APs. CoA VLAN is VLAN 30 (Via Cisco ISE)

●      Guest SSID (Broadcast in all zones) users are assigned VLAN 30 on all APs

●      IoT SSID (Broadcast in all zones) users are assigned VLAN 40 on all APs

●      Access Switches will be running in Layer 2 mode (No SVIs or DHCP)

●      MS390-M Access Switches physically stacked together

●      C9300-M Access Switches physically stacked together

●      C9500 Core Switches with Stackwise-virtual stacking using SVLs

●      Access Switch uplinks are in trunk mode with native VLAN = VLAN 1 (Management VLAN^*)

●      STP root is at Distribution/Collapsed-core

●      Distribution/Collapsed-core uplinks are in Trunk mode with Native VLAN = VLAN 1 (Management VLAN)

●      All VLAN SVIs are hosted on the core layer

●      Network devices will be assigned fixed IPs from the management VLAN DHCP pool. Default Gateway is 10.0.100.1

Network Segments

Please check the following table for more information about the network segments (e.g. VLANs, SVIs, etc.) for this design:

Quality of Service

Device list

Access policies

Port list

Wireless SSID list

Configuration and implementation guidelines

The following section will take you through the steps to amend your design by removing VLAN 1 and creating the desired new Native VLAN (e.g. VLAN 100) across your Campus LAN. The steps below should not be followed in isolation as first you have to complete the configuration of your Campus LAN based on the above previous section. The below steps are meant to replace VLAN 1 in your Campus LAN with a new one.

1.     Login to your dashboard account

2.     MX Addressing and VLANs; Navigate to Security and SD-WAN > Configure > Addressing and VLANs, then click on VLANs then click on Add VLAN to add your new infrastructure and Transit VLANs then click on Create. Please do not delete the existing VLAN 1 yet. Then, click on Save at the bottom of the page.

●      As seen above, VLAN 1 needs to be kept at this stage to avoid losing connectivity to all downstream devices.

3.     MX Addressing and VLANs: Navigate to Security and SD-WAN > Configure > DHCP, then under VLAN 100 AND 192 click on Fixed IP assignments and add entries for your network devices. (Tip: You can copy the MAC addresses from VLAN 1 and make sure to add the correct IP assignment to them). Then, click on Save at the bottom of the page.

4.     Create VLAN 100 and 192 on your C9500 Core Stack

5.     Navigate to Switching > Configure > Switch ports and filter for MR (if you have tagged the ports accordingly, otherwise select your downlink ports manually), then change the Native VLAN on these switchports from Native VLAN 1 to Native VLAN 100. Also, please add VLAN 100 to the list of Allowed VLANs and remove VLAN 1 from the allowed list of VLANs. Then, click on Save at the bottom of the page.

●      Please note that this will cause disruption to client traffic

6.     Navigate to Switching > Monitor > Switches and click on the first master switch then change the IP address settings from Static to DHCP and please leave the VLAN field blank. (DO NOT add VLAN 100 at this stage). Then, click on Save at the bottom of the window. Please repeat this for all master switches in your network.

●      As seen from the above screen shot, the VLAN value has been kept empty at this stage

7.     On your C9500 Core Stack, add an MST instance in VLAN 100 and VLAN 192

8.     Navigate to Switching > Monitor > Switch ports and filter for uplink (if you have tagged the ports accordingly, otherwise select your uplink ports manually), then change the Native VLAN on these switchports from Native VLAN 1 to Native VLAN 100. Also, please add VLAN 100 to the list of Allowed VLANs and remove VLAN 1 from the allowed list of VLANs. Then, click on Save at the bottom of the page.

●      Please note that this will cause the Access Stacks to go offline on the Meraki dashboard

9.     On your C9500 Core Stack, change the Native VLAN on your downlink Port-channels to VLAN 100

10.  Shutdown all uplinks from C9500 Core Stack to Port 19 and 20 on your Secondary WAN Edge appliance to avoid having a dual-active situation.

11.  MX Addressing and VLANs: Navigate to Security and SD-WAN > Configure > Addressing and VLANs, then under Per-port settings, change the Native VLAN on your downlinks to VLAN 100 and allow both VLAN 100 and 192.

12.  On your C9500 Core Stack, change the Native VLAN on your uplink to VLAN 100 and allow VLANs 100 and 192 (Please note that you will need to connect to your C9500 Core Stack via console access since VLAN 1 does not exist anymore on the upstream device which is the MX WAN Edge in this case):

13.  On your C9500 Core Stack, create a default route for your SVI interfaces:

14.  Adjust your Static Routes on the MX to point to the transit VLAN instead of VLAN 1. Navigate to Security and SD-WAN > Configure > Addressing and VLANs and under Static routes click on a static route to change the next-hop. Please repeat that for all your static routes. Then, click on Save at the bottom of the page:

15.  Wait for your Access Switches to come back online and acquire an IP address in the new Native VLAN 100. Then, proceed to the next step.

16.  Now your switches should have acquired an IP address per the fixed IP assignment configuration. Navigate to Switching > Monitor > Switches then click on the first master switch and then change the IP address settings to static. Then, click on Save at the bottom of the window. Repeat this for all master switches in your network.

●      Please repeat the above step for all stacks in your network

17.  Navigate to your Primary WAN Edge device and ping 10.0.100.2 to make sure that it is reachable via VLAN 100. Then proceed to the next step.

18.  Unshut the uplinks on your C9500 Core Stack to the Secondary WAN Edge appliance:

19.  Verify that all your devices have come back online and acquired an IP address in the new Management VLAN. Navigate to Organization > Monitor > Overview then click on the devices tab:

20.  Navigate to Switching > Configure > Switch settings then change the Management VLAN configuration to VLAN 100. Then, click on Save at the bottom of the page.

21.  Delete VLAN 1 from your MX appliance. Navigate to Security and SD-WAN > Configure > Addressing and VLANs and select the old Management VLAN 1 and then click on Delete. Then, click on Save at the bottom of the page.

22.  Where applicable - Please remember to adjust any routing between your Campus LAN and remote servers (e.g. Cisco ISE for 802.1x auth) as in this case devices will use the new Management VLAN 100 as the source of Radius requests. To verify that you have connectivity to your remote servers, Navigate to Wireless > Monitor > Access points then click on any AP and from the Tools section ping your remote server. Repeat this process from one of your switches.

●      With the current scope of the design, Cisco ISE resides in AWS and is reachable via AutoVPN which terminates on the vMX in AWS as well. As such, it was required to add a route on the VPC to 10.0.100.0/24 pointing to the vMX

●      Also, please ensure that the new Management VLAN has been enabled with AutoVPN by navigating to Security and SD-WAN > Configure > Site-to-site VPN and ensure that VLAN 100 is enabled.

23.  Where applicable - Please remember to adjust your Radius server configuration (e.g. Cisco ISE) as the Network devices now are grouped in a new Management VLAN 100. Please see the below example for Cisco ISE:

Option 3: Layer 3 Access

Overview

This option assumes that your OSPF domain is extended all the way to your core layer and thus there is no need to rely on STP between your Access and Core for convergence (as long as there are separate broadcast domains between Access and Core). It offers fast convergence since it relies on ECMP rather than STP layer 2 paths. However, it doesn't offer great flexibility in your VLAN design as each VLAN cannot span between multiple stacks/closets.

Pros:

●      Deterministic route failover

●      Fast convergence

●      Relies on either stacking or gateway redundancy at upper layers

●      Complete end to end separation between Management traffic and Client traffic

Cons:

●      VLANs cannot span multiple stacks/closets

●      Your backbone area size can be unmanageable

●      Forces Layer 3 roaming across the Campus LAN

●      Additional VLANs needed to route traffic between Campus LAN layers (aka Transit VLAN)

Logical Architecture

The following diagram shows the logical architecture for Layer 3 convergence within a campus LAN design leveraging Cloud Managed and Cloud Monitored Catalyst platform components:

Physical Architecture

The following diagram shows the physical architecture and port list for this design:

Assumptions

The following assumptions have been considered:

●      It is assumed that Wireless roaming is required only within a specific Campus Zone

●      It is assumed that VLANs are NOT spanning across multiple zones

●      There will be NO use of VLAN 1 across the Campus LAN

●      Corporate SSID (Broadcast in all zones) users are assigned VLAN 11/12 based on the AP zone.

●      BYOD SSID (Broadcast in all zones) users are assigned VLAN 21/22 based on the AP zone.

●      Guest SSID (Broadcast in Zone1) users are assigned VLAN 30 on all APs in that zone

●      IoT SSID (Broadcast in zone2) users are assigned VLAN 40 on all APs in that Zone

●      Access Switches will be running Layer 3 (SVIs and DHCP)

●      MS390 Access Switches physically stacked together

●      C9300-M Access Switches physically stacked together

●      C9500 Core Switches with Stackwise-virtual stacking using SVLs

●      Access Switch uplinks are in trunk mode with native VLAN = VLAN 1 (Management VLAN)

●      STP root is at Distribution/Collapsed-core

●      Network devices will be assigned fixed IPs from the management VLAN DHCP pool. Default Gateway will vary based on the Zone and stack.

Network Segments

Please check the following table for more information about the network segments (e.g. VLANs, SVIs, etc.) for this design:

Quality of Service

Device List

Access policies

Port List

Wireless SSID List

Configuration and Implementation Guidelines

It is assumed that by this stage, Catalyst devices have been added to dashboard for either Monitoring (e.g. C9500) and/or Management (e.g. C9300). For more information, please refer to the above section.

Before proceeding, please make sure that you have the appropriate licenses claimed into your dashboard account.

1.     Login to your dashboard account (or create an account if you don't have one)

2.     Navigate to Organization > Configure > Inventory

3.     For Co-term license model, click on Claim. And for PDL, please click on Add

4.     Enter the order and/or serial number(s) to claim the devices into your account. For PDL, click Next then please choose to add them to Inventory (Do not add them to a network)

5.     Create a Dashboard Network: Navigate to Organization > Configure > Create network to create a network for your Campus LAN (Or use an existing network if you already have one). If you are creating a new network, please choose "Combined" as this will facilitate a single topology diagram for your Campus LAN. Choose a name (e.g. Campus) and then click Create network

6.     Dashboard Network Settings: Navigate to Network-wide > Configure > General and choose the settings for your network (e.g. Time zone, Traffic Analytics, firmware upgrade day/time, etc.)

7.     Schedule Firmware Upgrade: Navigate to Organization > Configure > Firmware upgrades to select the firmware for your devices such that devices upgrade once they connect to dashboard. Select the device type then click on Schedule upgrade.

8.     Add Devices to a Dashboard Network: Navigate to Organization > Configure > Inventory.

●      For Co-term licensing model, select the MS390 and C9300 switches and the Primary WAN Edge then click on Add then choose the Network Campus

●      For PDL licensing model, select the MS390 and C9300 switches and the Primary WAN Edge then click on Change network assignment and then choose the Network Campus

●      Please DO NOT add the Secondary WAN Edge device at this stage

9.     Rename MX Security Appliance: Navigate to Security and SD-WAN > Monitor > Appliance status then click on the edit button to rename the MX to Primary WAN Edge then click on Save.

10.  MX Connectivity: Plug in your WAN uplink(s) on the Primary WAN Edge MX then power it on and wait for it to come online on dashboard. This might take a few minutes as the MX will download its firmware and configuration. Navigate to Security and SD-WAN > Configure > Appliance status and verify that the MX has come online and that its firmware and configuration is up to date.

11.  Rename Access Switches: Navigate to Switching > Monitor > Switches then click on each MS390 and C9300 switch and then click on the edit button on top of the page to rename it per the above table then click on Save such that all your switches have their designated names.

12.  Rename MR APs: Navigate to Wireless > Monitor > Access points then click on each AP and then click on the edit button on top of the page to rename it per the above table then click on Save such that all your APs have their designated names.

13.  MR AP Tags: Navigate to Wireless > Monitor > Access points then click on each AP and then click on the edit button next to TAGS to add Tags to your AP per the above table then click on Save such that all your APs have their designated tags.

14.  MX Addressing and VLANs: Navigate to Security and SD-WAN > Configure > Addressing and VLANs, and in the Deployment Settings menu select Routed mode. Further down the page on the Routing menu, click on VLANs then click on Add VLAN to add your Management and Transit VLANs then click on Create. Then for the per-port VLAN settings, select your downlink ports (19 and 20) and click on Edit and configure them as Trunk with VLAN 3 (Allowed VLANs 3, 100, 200, 1923) and click on Update. Finally, click on Save at the bottom of the page.

●      Please repeat the above steps to create VLANs 100 and 200

15.  Campus LAN Static Routes: Create Static Routes for your Campus network by navigating further down the page to Static routes then click on Add Static Route. Start by adding your Corporate LAN subnet then click on Update and then add static routes to all other subnets (e.g. BYOD, Guest and IoT). Finally, click on Save at the bottom of the page. (The Next hop IP that you have used here will be used to create a fixed assignment for the Core Stack later in DHCP settings).

16.  Optional - If you are accessing any resources over Meraki SD-WAN, please navigate to Security and SD-WAN > Configure > Site-to-site VPN and enable VPN based on your topology and traffic flow requirements. (In this case, we will configure this Campus as Spoke with Split Tunneling)

●      Choose Type: Spoke then click on Add a hub and select your hub site where you need access to resources via VPN. You can also add multiple hubs for resiliency. To choose Split Tunneling, please leave the box next to the Hub unticked as shown below.

●      Under VPN Settings, choose which subnet to be Enabled in VPN (e.g. Management VLAN will be required for Radius authentication purposes as the MR/MS390/C9300 devices will reach out to Cisco ISE using their management IP). Any Subnet that needs to access resources via VPN must be Enabled otherwise keep it as Disabled.

●      Finally, click on Save at the bottom of the page on the Hub site, please make sure to advertise the subnets that are required to be reachable via VPN. Navigate to Security and SD-WAN > Configure > Site-to-site VPN then add a local network then click Save at the bottom of the page (Please make sure that you are configuring this on the Hub's dashboard network).

17.  Optional - Verify that your VPN has come up by selecting your Campus LAN dashboard network from the Top-Left Network drop-down list and then navigate to Security and SD-WAN > Monitor > VPN status then check the status of your VPN peers. Next, navigate to Security and SD-WAN > Monitor > Route table and check the status of your remote subnets that are reachable via VPN. You can also verify connectivity by pinging a remote subnet (e.g. 172.31.16.32 which is Cisco ISE) by navigating to Security and SD-WAN > Monitor > Appliance status then click on Tools and ping the specified IP address (Please note that the MX will choose the highest VLANs interface IP participating in VPN by default as the source).

Please note that in order to ping a remote subnet, you must either have BGP enabled or have static routes at the far-end pointing back to the Campus LAN local subnets. (In other words, the source of your traffic which for ping by default is the highest VLAN participating in AutoVPN if not otherwise specified).

In this example, the VPC in AWS has been configured with a Route Entry to route 10.0.100.0/24 and 10.0.200.0/24 via the vMX deployed in AWS that has a VPN tunnel back to the Campus LAN site.

If the remote VPN peer (e.g. AWS) is configured in Routed mode, the static route is not required since traffic will always be NAT'd to a local reachable IP address. Please also don't forget to create Network Device groups on Cisco ISE for your network devices to be able to send authentication messages to Cisco ISE. See the below example:

18.  SD-WAN and Traffic Shaping Configuration: To configure Traffic Shaping settings for your Campus LAN site. Navigate to Security and SD-WAN > Configure > SD-WAN and Traffic Shaping to configure your preferred settings. For the purpose of this CVD, the default traffic shaping rules will be used to mark traffic with a DSCP tag without policing egress traffic (except for traffic marked with DSCP 46) or applying any traffic limits. (Please adjust these settings based on your requirements such as traffic limits or priority queue values. For more information about traffic shaping settings on the MX devices, please refer to the following article).

19.  Optional - Configure Threat Protection (Requires Advanced License or above) for your Campus LAN site. Navigate to Security and SD-WAN > Configure > Threat Protection and choose the settings that meet your site requirements. Please see the following configuration example:

20.  Click on Save at the bottom of the page.

21.  Optional - Configure Content Filtering Settings (Requires Advanced License or above) for your Campus LAN site. Navigate to Security and SD-WAN > Configure > Content filtering and choose the settings that meet your site requirements. Please see the following configuration example:

22.  Click on Save at the bottom of the page.

23.  Core Switch Uplinks: On the Catalyst 9500 core switches, Connect their uplinks to the Primary WAN Edge MX and power them both on.

24.  Core Switch Network Access: Connect to the first C9500 switch via console and configure it with the following commands:

25.  Core Switch Network Access: Connect to the second C9500 switch via console and configure it with the following commands:

26.  SVL Configuration: Now that both C9500 switches have access to the network, proceed to configure the Stackwise Virtual Links per the port list provided above (In this case using two ports for the SVL providing a total stacking bandwidth of 80 Gbps).

27.  Connect Stacking Cables: Whilst the C9500 switches are reloading, connect the stacking cables on both switches.

28.  Verify Stackwise Configuration: Please wait for about 10 minutes for the switches to come back up and initialize the stack. Then, connect to the 9500-01 (Stack Master) via console to verify that the stack is operational. The stackwise-virtual link should be U (Up) and R (Ready).

29.  Optional - Attach and configure stackwise-virtual dual-active-detection: DAD is a feature used to avoid a dual- active situation within a stack of switches. It will rely on a direct attachment link between the two switches to send hello packets and determine if the active switch is responding or not. Please note that DAD cannot be applied to any SVL links and has to be a dedicated interface. For the purpose of this CVD, interface HundredGigE1/0/27 and HundredGigE2/0/27 will be used for enabling DAD between the two C9500 switches.

30.  Configure Multiple Spanning Tree Protocol (802.1s). Connect to the 9500-01 (Stack Master) via console and use the following commands:

31.  Verify Spanning Tree Configuration (Please note that interface Twe2/0/1 will be in STP blocking state due to the fact that both uplinks are connected to the same MX edge device at this stage).

32.  Configure STP Root Guard and UDLD on the Core Stack Downlinks:

33.  Optional - STP Hygiene: It is recommended to configure STP Root Guard on all C9500 Core Stack downlinks to avoid any new introduced downstream switches from claiming root bridge status.

34.  Optional - STP Hygiene: It is recommended to configure STP Loop Guard on all C9500 Core Stack un-used stacking links.

35.  Configure SVIs for your Campus LAN on the Core Stack:

36.  Verify your DHCP pool configuration:

37.  Verify your SVI configuration:

38.  Configure Layer 2 Switchports, SGTs, and CST (Cisco TrustSec) on your Core Stack interfaces. (Please note that enforcement has been disabled on downlink ports allowing it to happen downstream)

39.  Spare WAN Edge Connectivity: Follow these steps to create warm-spare with two MX appliances: (Please note that this might result in a brief interruption of packet forwarding on the MX Appliance)

●      Navigate to Security and SD-WAN > Monitor > Appliance status and click on Configure warm spare

●      Now click on Enabled then choose the Spare MX from the drop-down menu and then choose the Uplink IP option that suits your requirements (Please note that choosing Virtual IPs requires an additional IP address on the upstream network and a single broadcast domain between the two MXs) then click on Update

●      Now click on Spare to access the Appliance status page of your Spare MX and click on the Edit button to rename the spare unit (e.g. Secondary WAN Edge)

 

●      Then configure the following on your C9500 Core Stack:

 

●      Then connect the Spare MX downlinks to your C9500 Core Stack (e.g. Spare MX port 19 to Twe1/0/2 and port 20 to Twe2/0/2)

●      Then connect the Spare MX with its uplinks (This must match the uplink configuration on your Primary WAN Edge)

●      Power on the Spare MX and wait for it to come online on dashboard

 

 

●      You can also verify that your C9500 Core Stack interfaces to the Spare MX are up, and that the redundant uplinks are in STP BLK mode

 

40.  Access Policy configuration: When you're logged in dashboard, Navigate to Switching > Configure > Access policies to configure Access Policies as required for your Campus LAN. Please see the following example for two Access Policies; 802.1x and MAB.

41.  Adaptive Policy Configuration: Configure Adaptive Policy for your Campus LAN. When you're logged in dashboard, Navigate to Organization > Configure > Adaptive Policy then click on the Groups tab on the top. There should be two groups (Unknown, Infrastructure) that are already available. Click on Add group to add each group required for your Campus LAN. You need to fill in the Name, the SGT value, and a description then click on Review changes then click on Submit. Please see the following examples.

42.  Adaptive Policy Configuration: Configure Adaptive Policy for your Campus LAN. When you're logged in dashboard, Navigate to Organization > Configure > Adaptive Policy then click on the Policies tab on the top. The source groups are on the left side, and the destination groups are on the right side. Select a source group from the left side then select all destination groups on the right side that should be allowed then click on Allow and click on Save at the bottom of the page. Next, select a source group from the left side then select all destination groups on the right side that should be denied (i.e. Blocked) then click on Deny and click on Save at the bottom of the page. After creating the policy for that specific source group, the allowed destination groups will be displayed with a green tab and the denied destination groups will be displayed with a red tab. Repeat this step for all policies required for all Groups (Allow and Deny).

43.  Access Switch Ports Preparation: MS390 switches support a maximum of 1000 configured VLANs and given that the default configuration has all switchports in Trunk mode with Native VLAN 1 and allowed VLANs 1-1000 (consuming the 1000 limit already), Dashboard will not allow for the configuration of this design to be saved (i.e. configuring VLAN 1921/1922 as this will breach the 1000 VLANs limit). As such, ports will need to be configured with a different range or VLAN set other than the default settings before applying the configuration needed for this design. It is therefore recommended to configure ALL ports in your network as access in a parking VLAN such as 999. To do that, Navigate to Switching > Monitor > Switch ports then select all ports (Please be mindful of the page overflow and make sure to browse the different pages and apply configuration to ALL ports) and then make sure to deselect stacking ports (as you cannot change configuration on dedicated stacking ports) then click on the Edit button and configure all ports as shown below:

●      IMPORTANT - The above step is essential before proceeding to the next steps. If you proceed to the next step and receive an error on Dashboard then it means that some switchports are still configured with the default configuration. Please revisit the Switching > Monitor > Switch ports page and ensure that no ports have a Trunk with allowed VLANs 1-1000

44.  Access Switch Ports Configuration: Configure Uplink Ports on your Access Switches. When you're logged in dashboard, Navigate to Switching > Monitor > Switch ports, then select your uplink ports and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard):

45.  Optional - For ease of management, it is recommended that you rename the ports connecting to your Core switches with the actual switch name / Connecting port as shown below.

46.  Access Switch Ports Configuration: Configure Wired Client Ports (802.1x) on your Access Switches. Navigate to or Refresh Switching > Monitor > Switch Ports, then select your Wired Client ports (5-8) and configure them as◦ shown below. (Tip: You can filter for ports by using search terms in dashboard)

47.  Access Switch Ports Configuration: Configure Wired Client Ports (MAB) on your Access Switches. Navigate to or Refresh Switching > Monitor > Switch Ports, then select your Wired Client ports (9-12) and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard)

48.  Access Switch Ports Configuration: Configure MR Ports on your Access Switches. Navigate to or Refresh Switching > Configure > Switch Ports, then select your ports connecting to MR Access Points (13-16) and configure them as shown below. (Tip: You can filter for ports by using search terms in dashboard)

49.  Optional - Access Switch Ports Configuration: Configure unused ports on your Access Switches such that they are disabled and mapped to a parking VLAN such as 999. Navigate to Switching > Monitor > Switch Ports and filter for any unused ports (e.g. 17-24) and configure them as shown below.

50.  Rename Wireless SSIDs: To configure your SSIDs per the above table, first navigate to Wireless > Configure SSIDs then rename the SSIDs per your requirements (Refer to the above table for guidance).

●      SSID#1 (First column, aka vap:0, enabled by default): Click on rename and change it to Acme Corp

●      SSID#2 (Second column, aka vap:1): Click on rename and change it to Acme BYOD, then click on the top drop-down menu to enable it

●      SSID#3 (Third column, aka vap:2): Click on rename and change it to Guest, then click on the top drop-down menu to enable it

●      SSID#4 (Fourth column, aka vap:3): Click on rename and change it to Acme IoT, then click on the top drop- down menu to enable it

●      Click Save at the bottom of the page

51.  Configure Access Control for Acme Corp: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Acme Corp.

●      Click Save at the bottom of the page

●      Please Note: Adaptive Policy Group feature is not currently available in the New Version of the Access. You will need to click on View old version

View old Version

which is available at the top right corner of the page to be able to access this and configure the Adaptive Policy Group (10: Corp). Then, please click Save at the bottom of the page.

52.  Configure Access Control for Acme BYOD: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Acme BYOD.

●      Click on

View old Version

which is available on the top right corner of the page, then choose the Adaptive Policy Group 20: BYOD and then click on Save at the bottom of the page.

53.  Configure Access Control for Guest: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Guest.

●      Click Save at the bottom of the page

●      Click on the top right corner of the page on "View Old Version" then choose the Adaptive Policy Group 30:Guest then click on Save at the bottom of the page

●      Navigate to Wireless > Configure > SSID availability and configure broadcast via Tag = Zone 1

54.  Configure Access Control for Acme IoT: Navigate to Wireless > Configure > Access control then from the top drop-down menu choose Acme IoT. (Please note that in this example Acme IoT SSID has been configured with iPSK without Radius).

●      Navigate to Network-wide > Configure > Group policies, then create a group policy for IoT devices and click Save at the bottom of the page

●      Then, Navigate to Wireless > Configure > Access control and choose Acme IoT from the top drop-menu and configure settings as shown below, First choose iPSK without Radius from the Security menu:

●      Then, click on Add an identity PSK:

 

 

 

●      Click on Save at the bottom of the page

●      Click on

View old Version

at the top right corner of the page then choose the Adaptive Policy Group 40: IoT then click on Save at the bottom of the page.

●      Navigate to Wireless > Configure > SSID availability and configure broadcast via Tag = Zone 2

55.  Enabling Stacking on your MS390 and C9300 Switches in Meraki Dashboard: Please follow these steps.

A.   Connect a single uplink to each switch (e.g. Port 1 on MS390-01 to Port TwentyFiveGigE1/0/23 on C9500)

B.   Make sure all stacking cables are unplugged from all switches

C.   Power up all switches

D.   Verify that your C9500 Stack downlinks are up and not shutdown

 

E.   Wait for them to come online on dashboard. Navigate to Switching > Configure > Switches and check the status of your Access Switches

F.   After they come online and download their configuration and firmware (Up to date) you can proceed to the next step. You can see their Configuration status and Firmware version from Switching > Configure > Switches

G.   Enable stacking in dashboard by Navigating to Switching > Monitor > Switch stacks then click on add one

H.   Then give your stack a name and select it's members and click on Create

 

I.    Now click on Add a stack to create all other stacks in your Campus LAN access layer by repeating the above steps

 

 

J.   Power off all access switches

K.   Disconnect all uplink cables from all switches

L.   Nominate your master switch for each stack (e.g. MS390-01 for stack1 and C9300-01 for stack2)

M.  On the master switches, plug the uplink again

N.   Plug stacking cables on all switches in each stack to form a ring topology and make sure that the Cisco logo is upright

O.  Power on your master switches first, then power other stack members

P.   Wait for the stack to come online on dashboard. To check the status of your stack, Navigate to Switching > Monitor > Switch stacks and then click on each stack to verify that all members are online and that stacking cables show as connected

 

Q.  Plug uplinks on all other non-master members and verify that the uplink is online in dashboard by navigating to Switching > Monitor > Switch stacks and then click on each stack to verify that all uplinks are showing as connected however they should be in STP discarding mode.

 

R.   Configure the same Static IP for all members in each stack by navigating to Switching > Monitor > Switches then click on the master switch (e.g. MS390-01 for Stack1) and under LAN IP menu copy the IP address then click on the edit button to specify the Static IP address information (You can use the same IP address that was assigned using DHCP) then click Save. The same Static IP address information should now be copied for all members of the same stack. You can verify this by navigating to Switching > Monitor > Switches (Tip: Click on the configure button on the right-hand side of the table to add Local IP information display).

 

●      And on your Stack2-9300 Master Switch:

 

S.   Finally, configure etherchannels on both your Access Switch Stacks and your Core Switch Stacks so that all uplinks can be operational (STP forwarding mode) at the same time. Follow these steps:

◦    First, disconnect the downlinks to non-master switches from your C9500 Core Stack (e.g. Port TwentyFiveGigE2/0/23 and TwentyFiveGigE2/0/24)

◦    Navigate to Switching > Monitor > Switch ports and search for uplink then select all uplinks in the same stack (in case you have tagged your ports otherwise search for them manually and select them all) then click on Aggregate. Please note that all port members of the same Ether Channel must have the same configuration otherwise Dashboard will not allow you to click the aggregate button.

 

 

 

◦    Please repeat above steps for all stacks in your network

◦    Please note that the above step will cause all members within the stack to go offline in Dashboard

●      On your C9500 Core Stack, please configure etherchannel Settings for your downlinks such that each Stack downlinks should be in a separate Port-channel and that the mode is active:

●      Plug all uplinks to non-master switches

●      Now all your switches should come back online on Dashboard

●      And now all your uplinks from each stack should be in STP Forwarding mode, which you can verify on Dashboard by navigating to Switching > Monitor > Switch stacks and checking the uplink port status. Also, you can check that on your C9500 Core Stack.

 

 

56.  Configure Multiple Spanning Tree Protocol (802.1s) in Dashboard for MS390 and C9300 switches: Navigate to Switch > Configure > Switch settings and select your stack and choose the appropriate STP priority per stack (61440 for all Access Switch Stacks) then click Save at the bottom of the page.

●      Please note that changing the STP priority will cause a brief outage as the STP topology will be recalculated.

●      Verify that the Access Stacks are seeing the C9500 Core Stack as the root by navigating to Switching > Monitor > Switches then click on any switch and under the RSTP root menu check the root bridge information

57.  Configure Dynamic ARP Inspection (DAI) on your C9500 Core Switches: All Downlinks to Access Switches and Uplinks to MX Edge must be configured as Trusted and all other interfaces as Untrusted. (Please note that the order of commands is important to avoid loss of connectivity)

58.  Configure Dynamic Arp Inspection (DAI) on your Access Switch Stacks: Navigate to Switch > Monitor > DHCP Servers and ARP and scroll down to Dynamic ARP Inspection and enable it, then click Save at the bottom of the page.

59.  Setting up your Access Points: Connect your APs to the respective ports on the Access Switches (e.g. Ports 13-16) and wait for them to come online on dashboard and download their firmware and configuration files. To check the status of your APs navigate to Wireless > Monitor > Access points and check the status, configuration and firmware of your APs.

60.  Re-addressing your Network Devices: In this step, you will adjust your IP addressing configuration - if required - to align with your network design. This step could have been done earlier in the process however it will be easier to adjust after all your network devices have come online since the MX (The DHCP server for Management VLAN 1) has kept a record of the actual MAC addresses of all DHCP clients. Follow these steps to re-assign the desired IP addresses. (Please note that this will cause disruption to your network connectivity)

A.   Navigate to Organization > Monitor > Overview then click on Devices tab to check the current IP addressing for your network devices

B.   Navigate to Security and SD-WAN > Monitor > Appliance status then click on the Tools tab and click on Run next to ARP Table

C.   Take a note of the MAC addresses of your network devices

D.   Navigate to Security and SD-WAN > Configure > DHCP then under Fixed IP assignments click on Add a fixed IP assignment and add entries under each DHCP Pool as shown below for your network devices using the MAC addresses you have from Step #3 above then click on Save at the bottom of the page.

 

 

 

E.   Navigate to Switching > Monitor > Switch ports then filter for MR (in case you have previously tagged your ports or select ports manually if you haven't) then select those ports and click on Edit, then set Port status to Disabled then click on Save.

 

F.   After a few minutes (For configuration to be up to date) navigate to Switching > Monitor > Switch ports, then filter for MR (in case you have previously tagged your ports or select ports manually if you haven't) then select those ports and click on Edit, then set Port status to Enabled then click on Save.

 

G.   Navigate to Switching > Monitor > Switches, then click on each master switch to change its IP address to the one desired using Static IP configuration (remember that all members of the same stack need to have the same static IP address)

H.   On your C9500 Core Stack, bounce your VLAN 3,100,200 interfaces. Then verify that the interfaces VLAN 3/ 100/200 came up with the correct IP address (e.g. 10.0.3.2 per this design)

I.    Navigate to Organization > Monitor > Overview then click on Devices tab to check the current IP addressing for your network devices:

61.  Configure QoS in your Campus LAN: Quality of Service configuration needs to be consistent across the whole Campus LAN. Please refer to the above table as an example. (For the purpose of this CVD, Default traffic shaping rules will be used to mark traffic with DSCP values without setting any traffic limits. Please adjust traffic shaping rules based on your own requirements). To configure QoS, please follow these steps.

A.   Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the Acme Corp SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules. Click Save at the bottom of the page when you are done. Click Save at the bottom of the page when you are done.

B.   Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the Acme BYOD SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules.

C.   Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the Guest SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules. Click Save at the bottom of the page when you are done.

D.   Navigate to Wireless > Configure > Firewall and Traffic Shaping and choose the IoT SSID from the above drop-down menu. Under Traffic Shaping rules, choose the per-client and per-SSID limits desired and select Shape traffic on this SSID then select Enable default traffic shaping rules. Click Save at the bottom of the page when you are done.

E.   Navigate to Switching > Configure > Switch settings and under the Quality of Service menu configure the VLAN to DSCP mappings. Please click on Edit DSCP to CoS map to change settings per your requirements. Click Save at the bottom of the page when you are done. (Please note that the ports used in the below example are based on Cisco Webex traffic flow)

 

F.   Please ensure that your C9500 Core Stack is configured to trust incoming QoS. Here's a reference of the configuration needed to be applied:

 

G.   Navigate to Security and SD-WAN > Configure > SD-WAN and Traffic shaping and make sure your Uplink configuration matches your WAN speed. Then, under Uplink selection choose the settings that match your requirements (e.g. Load balancing). Under Traffic shaping rules, select Enable default traffic shaping rules then click on Add a new shaping rule to create the rules needed for your network. (for more information about Traffic shaping rules on MX appliances, please refer to the following article). Please see the following example:

 

 

 

62.  Enable OSPF Routing: Navigate to Switching > Configure > OSPF routing and then click on Enabled to enable OSPF. Add the details required and create an OSPF area for your Campus Network. Then, click Save at the bottom of the page.

63.  Enable OSPF Routing on your Core Stack: Please use the following commands to add an OSPF instance and create OSPF neighbors.

64.  Create SVI Interfaces on your Access Switch Stacks: Navigate to Switching > Configure > Routing and DHCP and click on CREATE INTERFACE and start adding your interfaces but first start with the Transit VLANs. Once you have created an interface click on Save and add another at the bottom of the page to add more interfaces.

●      Please note that the Static Routes shown above are automatically created per stack and they reflect the default gateway settings that you have configured with the first SVI interface created which is in this case the Transit VLAN interface for each Stack

65.  Verify that your Core Stack is receiving OSPF routes from its neighbors:

66.  And that concludes the configuration requirements for this design option. Please remember to always click Save at the bottom of the page once you have finished configuring each item on the Meraki Dashboard.

Testing and Verification

Firmware

The following table indicates the firmware versions used in this Campus LAN:

Device Connectivity

MX WAN Edge

Upstream Connectivity

Internet/Cloud Connectivity

Downstream Connectivity

C9500 Core Stack

Upstream Connectivity

Internet Connectivity

Downstream Connectivity (Please note that the MS390 and C9300-M platforms will prioritize packet forwarding over ICMP echo replies so it's expected behavior that you might get some drops when you ping the management interface)

In case of connectivity issues, please check the following:

MS390 Access Stack

Upstream Connectivity

Internet/Cloud Connectivity

Downstream Connectivity

C9300 Access Stack

Upstream Connectivity

Internet/Cloud Connectivity

Downstream Connectivity

MR Access Points

Downstream Connectivity

Client Connectivity

802.1x Authentication

802.1x authentication has been tested on both Corp and BYOD SSIDs. Dashboard will be checked to verify the correct IP address assignment and username. Packet captures will also be checked to verify the correct SGT assignment. In the final section, ISE logs will show the authentication status and authorization policy applied.

 

 

 

VLAN Assignment

This section will validate that VLANs are assigned correctly based on the VLAN tag. The following client was used to test the connectivity in the designated VLAN:

 

STP Convergence

STP convergence will be tested using several methods as outlined below. Please see the following table for steady-state of the Campus LAN before testing:

Introducing loops (Access to Core)

A loop was introduced by adding a link between C9300-01 /NM Port 2 and C9500 Core Stack / Port TwentyFiveGigE1/0/22 (Please note that for the purposes of this test, the interface has been unshut and configured as a Trunk port with Native VLAN 1 with STP guards on that interface)

Interface Twe1/0/22 is in STP FWD state (As expected since this is the Root bridge)

Interface 26 is in STP BLK state (As expected since the Ether-channel is in FWD state)

Introducing Loops (Access Layer, with STP Guard: Loop Guard)

For the purposes of this test and in addition to the previous loop connections, the following ports were connected: MS390-01 / Port 11 < - > C9300-01 / Port 11

Please note that the port configuration for both ports was changed to assign a common VLAN (in this case VLAN 99). Please see the following configuration that has been applied to both ports:

 

Introducing Loops (Access Layer, without STP Guard)

For the purposes of this test and in addition to the previous loop connections, the following ports were connected: MS390-02 / Port 12 < - > C9300-02 / Port 12.

Please note that the port configuration for both ports was changed to assign a common VLAN (in this case VLAN 99). Please see the following configuration that has been applied to both ports:

Introducing Loops (Core Layer)

For the purpose of this test and in addition to the previous loop connections, the following ports were connected:

Port Twe1/0/10 to port Twe2/0/10 on the C9500 Core switches.

Introducing Rogue Bridge in VLAN 200

For the purpose of this test and in addition to the previous loop connections, the Bridge priority on C9300 Stack will be reduced to 4096 (likely root) and increasing the Bridge priority on C9500 to 8192.

●      Downlinks on C9500 are configured with STP Root Guard

●      Access Layer Links (Stack to Stack) are configured with STP Loop Guard + UDLD

C9500 Core Stack is still the Root Bridge (i.e. The root Bridge placement has been enforced).

Downlinks to C9300 and MS390 stacks are in STP Root Inconsistent State which caused all access switches to go offline on Dashboard.

To recover access switches, you will need to change the STP priority on the C9500 Core stack to 0 which ensures that your core stack becomes the root of the CIST. Alternatively, you can configure STP root Guard on the MS390 ports facing the C9300 and thus the MS390s will come back online.

The reason why all access switches went online on dashboard is that the C9300 was the root for the access layer (priority 4096) and thus the MS390s were passing traffic to Dashboard via the C9300s. Configuring STP Root Guard on the ports facing C9300 recovered the MS390s and client connectivity.

On the other hand, changing the STP priority on the C9500 core stack pulled back the Root to the core layer and recovered all switches on the access layer.

 

Reverting all configurations back to its original state:

1.     Disconnect and shutdown interface TwentyFiveGigE1/0/22

2.     Disconnect port 11 on MS390-01 and C9300-01 and remove Loop Guard and UDLD

3.     Disconnect port 12 on MS390-02 and C9300-02

4.     Disconnect and revert port TwentyFiveGigE1/0/10 and TwentyFiveGigE20/10 back to access with VLAN 1 and shutdown

5.     Change MST priority on C9300 stack to 61440

6.     Change MST priority on C9500 Core Stack to 4096

High Availability and Failover

Here's the steady-state physical architecture for reference:

MX WAN Edge Failover

 

C9500 Core Stack Loss of Uplink

For the purpose of this test, ports TwentyFiveGigE1/0/1 and TwentyFiveGigE1/0/2 will be disconnected.

 

C9300 Stack Loss of Uplink

For the purpose of this test, NM Port 1 on C9300-01 (Master switch) will be disconnected.

MS390 Stack Loss of Uplink

For the purpose of this test, port 1 on MS390-01 (Master switch) will be disconnected.

QoS

For the purpose of this test, packet capture will be taken between two clients running a Webex session. Packet capture will be taken on the Edge (i.e. MR wireless and wired interfaces) then on the Access (i.e. the MS390 or C9300 uplink port) then on the MX WAN Downlink and finally on the MX WAN Uplink. The table below shows the testing components and the expected QoS behavior:

Access Point Wired Port pcaps

Client #1

Client #2

Access Point Wired Port pcaps

Client #1

Client #2

Access Switch Uplink pcaps

Client #1

Client #2

MX appliance Downlink pcaps

Client #1

Client #2

Layer 3 Roaming with concentrator

The previous design which extends the Layer 3 domain to the Access Layer offered several benefits but one of the drawbacks was that VLANs cannot span between different stacks and therefore roaming is restricted within a single zone/closet. As such, to enable Layer 3 roaming in this Campus network the SSID needs to be tunneled to a Meraki MX operating as a concentrator. Please see the below diagram for the logical architecture of this design option:

The design will not change any of the elements previously configured except that the Acme Corp SSID will be configured in Layer 3 Roaming with Concentrator mode which requires having a Meraki MX Appliance configured as a concentrator. Subsequently, VLANs 11 and 12 will not be required anymore and the SVI for the new Corp VLAN will move to the WAN Edge MX. The WAN Edge MX in this case needs to provide DHCP services to roaming clients.

Special considerations for this design option:

●      APs will create a Layer 2 AutoVPN tunnel to the MX Concentrator using their management IP address

●      Radius requests from the Acme Corp SSID will have the NAS ID referring to the AP's management IP address where the client is attached however the device IP in the request will refer to the uplink IP address of the MX concentrator (e.g. 10.0.3.4 in this case)

●      The Radius server (in our case Cisco ISE) will require an IP route to the MX concentrator's uplink IP address (e.g. 10.0.3.4)

●      The Radius server will also need to be configured with the concentrator as a network device since the Radius requests will have its IP address as the device IP address (Otherwise testing 802.1x auth failed)

●      If the Radius server is reachable from the Campus via VPN tunnel (e.g. AutoVPN) then the Concentrator's uplink IP address/network will need to be advertised via the VPN as well

The following steps will outline the configuration changes to enable Layer 3 Roaming in this Campus LAN:

1.     Please ensure that you have an additional MX appliance in your dashboard and the appropriate license(s) claimed

2.     Add the appliance(s) to a new network (e.g. Roaming)

3.     Navigate to your Roaming network

4.     Navigate to Security and SD-WAN > Configure > Addressing and VLANs

5.     Select Passthrough or VPN Concentrator and click Save at the bottom of the page

6.     Navigate to your Campus Network

7.     Navigate to Security and SD-WAN > Addressing and VLANs and create a new VLAN for the Roaming SSID (e.g. VLAN 10)

8.     Navigate further down the page to the Per-port VLAN settings and configure the port connecting the MX Concentrator (e.g. Port 3 in this design) with a Native VLAN (e.g. VLAN 3) and allow both the native VLAN and the Roaming SSI VLAN that you have just created in the above step

9.     Click Save at the bottom of the page

10.  Plug your MX Concentrator and connect it to the designated port (Port #3) on the WAN Edge MX. Please note that the MX concentrator needs to be connected ONLY via a single uplink (No other uplinks or LAN ports)

11.  Once the MX Concentrator comes online on dashboard you can proceed to the next step (Waiting for the concentrator to come online will allow you to test the tunnel connectivity from the APs to the Concentrator)

12.  Navigate to Wireless > Configure > Access control and from the top drop-down menu select the Acme Corp SSID

13.  Navigate further down the page and under the Client IP assignment menu, select the Layer 3 with Concentrator option then choose VLAN 10 as the terminating VLAN for this SSID. Click Save at the bottom of the page.

14.  To test the Tunnel connectivity, click on Test Connectivity

●      The test above will check the IP connectivity between the APs with the Acme Corp SSID (AP's uplink IP address) and the MX concentrator (MX's uplink IP address) and return back how many APs passed the test (valid IP route) and how many failed (due to IP routing issues)

15.  Navigate to Security and SD-WAN > Configure > Site-to-site VPN and enable the upstream network of the MX Concentrator in AutoVPN (e.g. VLAN 3 in our case)

●      As explained earlier, this step is essential for the Cisco ISE server to accept Access-Requests from the MX concentrator

16.  After you have configured the appropriate routing on the Radius server side to allow it to communicate with VLAN 3, you can proceed with testing IP connectivity between the MX concentrator and the Radius Server

●      Please note that you won't be able to ping unless the Upstream network of the MX Concentrator has been enabled in AutoVPN and that the Radius Server has an IP route back to the Campus LAN. Please check the following example for this implementation of Cisco ISE in AWS where a route has been added on the VPC where the ISE server resides

17.  After you have added the MX concentrator on your Radius server as a network device, you can test using a client attached to the Acme Corp SSID

Testing and Verification:

The following client was used for testing and verification:

Device Connectivity

 

Radius Authentication

Layer 3 Wireless Roaming