Table of Contents
Introduction
In this post, we’re exploring the world of application hosting on Catalyst routers, specifically focusing on deploying ThousandEyes on a Catalyst 8000v router in a Cisco Modeling Labs (CML) environment.
Wait a minute… ThousandEyes on a Catalyst 8000v?! Yes, you read that right! For all the Cisco employees and Business Unit folks reading this: I know it’s not officially supported, but as a showcase, it works just fine. Let’s face it, not everyone has access to physical Catalyst 8k routers lying around for testing purposes, so CML is a great alternative.
I decided to write this article after some recent discussions with one of my customers, who is exploring the possibility of deploying the ThousandEyes Agent on their routers. This deployment serves as a practical example of how Catalyst routers can be leveraged beyond traditional routing, giving edge compute capabilities to monitor network performance right where it matters most.
Let’s dive in! 🚀
I wish my lab would look like this 😜
Why Thousandeyes on a network device?
Here are five reasons why deploying a ThousandEyes agent on a router, whether in a branch office or a data center, makes a lot of sense, especially when considering the impact of network changes and the need for pre-and post-checks, which was a topic that has been heavily discussed with my client:
-
Real-Time Network Visibility at Critical Points:
- Deploying ThousandEyes directly on a router offers real-time insight into the network performance at key locations, such as branch offices or data centers. This allows IT teams to identify issues at the network edge or in core locations quickly, and resolve them before users feel the impact.
- Pre-check: Before implementing changes, you can use the ThousandEyes agent to monitor existing performance and ensure that everything is functioning optimally.
- Post-check: After changes (e.g., firmware updates or routing adjustments), the agent helps verify that performance remains stable.
-
End-to-End Path Monitoring:
- Hosting the agent on a router enables end-to-end monitoring of the network path. This is crucial during network changes or upgrades, as the agent can detect whether new configurations have affected critical paths between branches or data centers.
- Pre-check: Baseline the end-to-end performance before applying changes.
- Post-check: Ensure that the modifications have not introduced latency, jitter, or packet loss.
-
Proactive Problem Detection:
- With the ThousandEyes agent directly on a router, you can detect issues before they escalate. This is particularly important during maintenance windows when changes are being made. The agent allows you to monitor for early signs of degradation in the network.
- Pre-check: Detect potential bottlenecks or problem areas before making changes.
- Post-check: Validate that no new issues have emerged after updates or configuration changes.
-
Service Assurance and Application Performance:
- The ThousandEyes agent on routers can monitor not just network metrics but also application performance, especially when critical business apps traverse that router. This ensures that changes do not negatively impact application performance or cloud services accessed through the network.
- Pre-check: Ensure application performance meets SLA before updates.
- Post-check: Validate that applications continue to meet performance requirements post-change.
-
Impact Analysis of Network Changes:
- Routers are often the first and last hops for traffic entering and exiting your network. Placing a ThousandEyes agent here helps you assess how changes to routing policies, firewall rules, or other network adjustments affect overall traffic flow and performance.
- Pre-check: Analyze how the current configuration impacts traffic.
- Post-check: Compare the pre- and post-change performance to assess the impact of the changes on traffic flow and routing efficiency.
In summary, deploying a ThousandEyes agent on routers gives you end-to-end network visibility, proactive monitoring, and post-change validation, helping you maintain a stable and reliable network environment while making configuration changes with confidence. So can safely wear the Thousandeyes T-shirt with a slogan that says: “Go ahead blame the network!” 😅
Keep in mind that Catalyst 8k is not the only platform where the Thousandeyes client can be deployed. The client is also supported on certain Nexus 9k, ASR, ISR 4k and Catalyst 9k devices. Guess what, not on all models as there are some requirements that I will cover later during the post.
CML Lab setup
In this Cisco Modeling Labs (CML) topology, a multi-location network is being simulated across three key locations: London, New York, and Frankfurt. Each location is configured with its own Edge and Core routers, connected to form a cohesive network.
Here’s a breakdown of the configuration:
1. Autonomous Systems (AS Numbers):
- AS 656801: Assigned to the London network.
- AS 656802: Assigned to the New York network.
- AS 656803: Assigned to the Frankfurt network.
2. Core and Edge Routers:
- Each location (London, New York, Frankfurt) has two core components:
- Core Routers: Represented as London-Core, NewYork-Core, and Frankfurt-Core, they manage the internal network and route traffic outside the AS.
- Edge Routers: Represented as London-Edge, NewYork-Edge, and Frankfurt-Edge, these routers handle traffic coming into the network. These devices are hosting the ThousandEyes application.
3. Clients:
- There are three clients in the topology:
- London-Client (connected to London-Edge),
- NewYork-Client (connected to NewYork-Edge),
- Frankfurt-Client (connected to Frankfurt-Edge).
- These clients represent end devices within the respective locations, likely used for testing network connectivity, and performance.
4. Unmanaged Switch:
- The unmanaged switch connects the network to an external connection labeled ext-conn-0, which bridges the network to VMs NIC which can forward traffic to the internet. This is needed in to get the ThousandEyes agent connected.
My use case: ThousandEyes Deployment
The topology shows the Core Routers from the three locations connected, forming inter-AS connections that enable communication between London, New York, and Frankfurt. This simulates a geographically distributed network, where traffic can flow between these major hubs through BGP.
This network setup could be used for deploying and testing ThousandEyes agents at each edge location (London, New York, Frankfurt). These agents will monitor network performance, latency, packet loss, and other metrics between the various sites, as well as traffic to services like AWS and WebEx.
This simulation allows you to test how the network performs under different conditions, assess routing between locations, and monitor network behavior using tools like ThousandEyes for pre and post-change validation.
Here is an example of the BGP config, which is very basic:
Core Router:
router bgp 65003
bgp log-neighbor-changes
network 10.20.0.0 mask 255.255.0.0
redistribute connected
neighbor 10.20.0.1 remote-as 65001
neighbor 10.21.0.2 remote-as 65003
neighbor 10.30.0.2 remote-as 65002
Edge Router:
router bgp 65003
bgp log-neighbor-changes
redistribute connected
neighbor 10.21.0.1 remote-as 65003
Lets deploy the enterprise agent on Catalyst 8000V
In my previous article, I explained what an enterprise agent is. If you don’t know, make sure to check that out.
Deploying an enterprise agent on the device itself is very easy due to the good documentation and the interactive formula on the ThousandEyes site once you add an enterprise agent as shown in the screenshot below
Once you fill out the fields, it will generate you a config. Here is an example:
configure terminal
iox
exit
!
app-hosting install appid thousandeyes package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-4.4.4.cisco.tar
configure terminal
app-hosting appid thousandeyes
app-vnic gateway1 virtualportgroup 0 guest-interface 0
guest-ipaddress 10.13.0.2 netmask 255.255.255.252
exit
app-default-gateway 10.13.0.1 guest-interface 0
name-server0 198.18.128.1
app-resource docker
prepend-pkg-opts
run-opts 1 "-e TEAGENT_ACCOUNT_TOKEN=abcdefghij123456789"
run-opts 2 "--hostname frankfurt-edge"
end
!
app-hosting activate appid thousandeyes
app-hosting start appid thousandeyes
Let’s break down the configuration step by step:
Enable IOx on the Catalyst Router:
- First, ensure that IOx is enabled on the router. IOx allows you to run applications in a container or virtual machine environment directly on the router.
- Enable the service using the following commands:
iox
Installing the ThousandEyes Agent
app-hosting install appid thousandeyes package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-4.4.4.cisco.tar
- This command downloads and installs the ThousandEyes Enterprise Agent from the provided URL on the Catalyst router using the
appididentifierthousandeyes. Theappidis the unique identifier for the application on the IOx platform.
Configure the application
app-hosting appid thousandeyes
- Enter the global configuration mode on the router and start configuring the application with the
appidthousandeyes.
Assign a Virtual Interface
app-vnic gateway1 virtualportgroup 0 guest-interface 0
- The Virtual Network Interface Card (vNIC) for the application is assigned using
gateway1. This vNIC connects to Virtual Port Group (VPG) 0, which defines the network interface the application uses for communication. - The guest interface is assigned to the
guest-interface 0.
Assign IP Address
guest-ipaddress 10.13.0.2 netmask 255.255.255.252
- Here, the ThousandEyes Agent is given the static IP address
10.13.0.2with the subnet mask255.255.255.252. This IP will be used by the application inside the virtual environment on the router.
Default Gateway Configuration
app-default-gateway 10.13.0.1 guest-interface 0
- This command sets the default gateway for the application to
10.13.0.1. Theguest-interface 0indicates that the default gateway is configured on the virtual interface we previously set up.
Configure DNS Server
name-server0 198.18.128.1
- The name-server or DNS server is set to
198.18.128.1to resolve domain names.
Set Resource Type for Application
app-resource docker
- Specifies that the application will run as a Docker container within the IOx framework. ThousandEyes leverages Docker as its runtime environment.
Prepend Package Options
prepend-pkg-opts
- This indicates that additional package options will be added before launching the application.
Run-Time Options
run-opts 1 "-e TEAGENT_ACCOUNT_TOKEN=abcdefghijklmn123456789"
run-opts 2 "--hostname frankfurt-edge"
-
Run-opts 1 sets an environment variable
TEAGENT_ACCOUNT_TOKENused by the ThousandEyes agent to authenticate with your ThousandEyes account. -
Run-opts 2 defines the hostname for the agent as
frankfurt-edge, helping you identify where the agent is deployed, especially useful when managing multiple agents across various locations. - More options like proxy config etc. are available here
Activate and Start the Application
app-hosting activate appid thousandeyesapp-hosting start appid thousandeyes
- Activate: This prepares the application for execution.
- Start: This launches the ThousandEyes application and begins monitoring.
In my case, this configuration was insufficient because my external network is not aware of the internal application network. That’s why I need to perform NAT on the application interface:
interface GigabitEthernet2
description to port1.unmanaged-switch-0
ip address 198.18.1.181 255.255.255.0
ip nat outside
negotiation auto
!
interface VirtualPortGroup0
ip address 10.13.0.1 255.255.255.0
ip nat inside
!
ip nat inside source list TS_NAT_ACL interface GigabitEthernet2 overload
!
ip access-list standard TS_NAT_ACL
10 permit 10.13.0.0 0.0.0.255
!
Network options for application hosting
When deploying applications on Catalyst routers, you have two interface connection options for communication:
1. Management GigabitEthernet0 Interface Connection (Not for the 8000v):
- Layer 2-3 packet support: This interface supports both Layer 2 and Layer 3 traffic.
- VRF Awareness: Applications using this interface are not aware of any VRF configuration. All traffic on this interface uses the global routing table.
- IP Addressing: The application can use an IP address in the same subnet as the GigabitEthernet0 interface. This simplifies configuration, as there’s no need to manage separate IP pools or subnets.
2. Virtual Port Group (VPG) Interface Connection:
- Layer 3 Routed Mode: The VPG operates in Layer 3 routed mode, which allows the hosted application to have routing awareness.
- NAT Support: VPG supports Network Address Translation (NAT), allowing applications to translate internal addresses to external ones for communication beyond the router.
- IP Unnumbered: You can configure VPG interfaces with IP unnumbered, meaning they borrow the IP address from another interface, simplifying the IP configuration when multiple devices or applications need to share an IP space.
Validate the ThousandEyes agent
If you encounter any issues and the agent is not connecting, the following commands might help you pinpoint the problem:
london-edge#show app-hosting list
App id State
---------------------------------------------------------
thousandeyes RUNNING
london-edge#show app-hosting detail appid thousandeyes
App id : thousandeyes
Owner : iox
State : RUNNING
Application
Type : docker
Name : ThousandEyes Enterprise Agent
Version : 4.4.4
Description :
Author : ThousandEyes <support>
Path : bootflash:thousandeyes-enterprise-agent-4.4.4.cisco.tar
URL Path :
Activated profile name : custom
Resource reservation
Memory : 500 MB
Disk : 1 MB
CPU : 1850 units
CPU-percent : 53 %
VCPU : 1
Platform resource profiles
Profile Name CPU(unit) CPU(percent) Memory(MB) Disk(MB)
---------------------------------------------------------------------------
Attached devices
Type Name Alias
---------------------------------------------
serial/shell iox_console_shell serial0
serial/aux iox_console_aux serial1
serial/syslog iox_syslog serial2
serial/trace iox_trace serial3
Network interfaces
---------------------------------------
eth0:
MAC address : 52:54:dd:de:ae:95
IPv4 address : 10.13.0.2
IPv6 address : ::
Network name : VPG0
Multicast : No
Mirroring : No
Docker
------
Run-time information
Command :
Entry-point : /sbin/my_pre_init
Run options in use : -e TEAGENT_ACCOUNT_TOKEN=abcdefghij123456789
Package run options : -e TEAGENT_ACCOUNT_TOKEN=TOKEN_NOT_SET --hostname=$(SYSTEM_NAME) -e TEAGENT_PROXY_TYPE=DIRECT -e TEAGENT_PROXY_LOCATION= -e TEAGENT_PROXY_USER= -e TEAGENT_PROXY_AUTH_TYPE= -e TEAGENT_PROXY_PASS= -e TEAGENT_PROXY_BYPASS_LIST= -e TEAGENT_KDC_USER= -e TEAGENT_KDC_PASS= -e TEAGENT_KDC_REALM= -e TEAGENT_KDC_HOST= -e TEAGENT_KDC_PORT=88 -e TEAGENT_KERBEROS_WHITELIST= -e TEAGENT_KERBEROS_RDNS=1 -e PROXY_APT= -e APT_PROXY_USER= -e APT_PROXY_PASS= -e APT_PROXY_LOCATION= -e TEAGENT_AUTO_UPDATES=1 -e TEAGENT_SERVICES_ALLOW_IPV6=0
Application health information
Status : 0
Last probe error :
Last probe output : </support>
london-edge#show app-hosting resource
CPU:
Quota: 7(Percentage)
Available: 3(Percentage)
VCPU:
Count: 0
Memory:
Quota: 1024(MB)
Available: 524(MB)
Storage device: bootflash
Quota: 4934(MB)
Available: 2696(MB)
Storage device: IOx persist-disk
Quota: 4934(MB)
Available: 2561(MB)
london-edge#show app-hosting utilization appid thousandeyes
Application: thousandeyes
CPU Utilization:
CPU Allocation: 1850 units
CPU Used: 0.03 %
CPU Cores: 0
Memory Utilization:
Memory Allocation: 500 MB
Memory Used: 142824 KB
Disk Utilization:
Disk Allocation: 1 MB
Disk Used: 0.00 MB
If you need to access the docker container’s bash in order to execute troubleshooting commands or check the logs:
london-edge#app-hosting connect appid thousandeyes session /bin/bash
root@london-edge:/# less /var/log/agent/te-agent.log
If the configuration of an app has changed, make sure to deactivate and activate the app again!
Create and run ThousandEyes tests
Now it’s time to put the ThousandEyes agents to work by setting up a test that measures the network performance between the Frankfurt-Edge and NewYork-Edge agents.
For this, we will create an Agent to Server test in the ThousandEyes platform. This test will allow us to evaluate various network performance metrics such as latency, packet loss, and jitter, giving insights into how well the Frankfurt-Edge agent can communicate with the NewYork-Edge.
But wait! If we run the test with a perfect link, it would be way too smooth and… well, boring 😄. To make things more interesting, let’s simulate some latency and packet loss on the link between Frankfurt-Edge and NewYork-Edge.
Luckily, in Cisco Modeling Labs (CML), we have the capability to add link conditions to simulate real-world network imperfections. This helps us observe how the network reacts to adverse conditions like congestion or poor-quality links.
Now that we’ve added the latency and packet loss conditions to the link between Frankfurt-Edge and NewYork-Edge, the ThousandEyes Agent to Server test is running, and we can observe packet loss on the link.
Summary
In this blog post, we explored application hosting on Catalyst routers, specifically focusing on deploying ThousandEyes on a Catalyst 8000v in a Cisco Modeling Labs (CML) environment. We walked through the deployment process, from enabling IOx services to installing and configuring the ThousandEyes agent. After setting up the test environment, we created an Agent to Server test between Frankfurt-Edge and NewYork-Edge agents to monitor network performance.
To make things more interesting, we added latency and packet loss in CML using link conditions. After running the test, we observed network degradation and saw how the ThousandEyes agent detected packet loss and increased latency. Once we removed the simulated link conditions, the network performance returned to normal, proving that our setup was functioning correctly.
The post highlighted how ThousandEyes on Catalyst routers provides real-time network monitoring, offering insights into network performance and resilience under different conditions.
To wrap up, it’s important to keep in mind that deploying ThousandEyes on Catalyst routers requires specific hardware and software configurations. While we demonstrated the process using a Catalyst 8000v in a lab environment, only certain Catalyst models officially support application hosting, such as the Catalyst 8300 and Catalyst 8500 series.
Additionally, when planning a deployment, ensure your router has sufficient RAM and CPU resources. For example, running ThousandEyes typically requires at least 4GB of RAM for smooth operation. You’ll also need to check if your router supports the IOx platform and ensure you’re using a supported IOS-XE software version.
Lastly, don’t forget about the software licenses. For most deployments involving application hosting, you may need the appropriate DNA Advantage or Premier licenses to enable application hosting and advanced features like ThousandEyes integration.
Taking all these requirements into account will help ensure your setup is fully supported and optimized for the best performance.
