Redirect ACL With C9300 Switches

In this post I will share with you one caveat and its fix with redirect ACL with C9300 switches. In the last few months I was working on a project for a medium size customer. The main requirements were to implement Firepower IPS, dot1x, pxGrid, AnyConnect client provisioning and posture assessment for both VPN and local clients. The customer has a few sites spread across the globe, and all of them are connected through VPLS. There are different network devices that we were working on and in one of the sites we had a stack of Cisco C9300 switches. The customer has ISE deployed for identity management.

Another part of the project was to move some sites into monitor mode for a couple of days and then to closed mode. After we applied all the necessary configuration on ISE and the Cisco switches, we started moving site A. That site was running on 2960-X switches, so far so good, we could move the site to monitor mode and then to closed mode successfully.

After that we started moving site B to monitor mode. At that time one of the customer engineers started doing some tests to verify if the Unknown dACL applied by ISE to the clients’ authentication sessions for posture assessment was working as expected.

Surprisingly, the dACL was not working as expected although we could see it applied to the dot1x authentication sessions along with the redirect url. The dACL should have denied the traffic to any IP address except a few, mainly the remediation servers. However, that was not the case and the dACL was allowing all traffic to pass through as if the deny rules were totally ignored.

We tried to check if the same behavior is also in site A, however, that was not the case, and in site A the dACL was working as expected denying what was configured to be denied and allowing what was configured to be allowed.

At that point we started doing some troubleshooting and we found out that if we remove the redirect ACL from ISE authZ profile, the dACL works. In other words the combination of the redirect ACL and the dACL in the authZ profile was causing the issue.

I then raised a Cisco TAC, and we started collecting some data capture for further investigation by the TAC engineer. TAC came back to me then suggesting to upgrade the current release that was running on the C9300 to a specific version stating that the issue might have been caused by a software bug on the release.

We followed TAC suggestion, but nothing changed. Then TAC suggested to downgrade yet to another release. We did so, and nothing changed. This kept going back and forth for quite some time, and we ended up trying about 7 different releases. The result was exactly the same, the Unknown dACL seemed to ignore totally any deny statement we configured.

In the meantime, I also asked a couple friends of mine who work in Cisco TAC, and I was told that they have never experienced a similar issue with the C9300 switches and that it should work. In the meantime, Cisco TAC engineer was working on this case which came back to me after some time asking to rewrite the redirect ACL we have configured because they believed the issue was on the content of the ACL.

I asked the customer engineers to do some changes on the redirect ACL based on what TAC suggested, and the big surprise was that when the engineers removed the the deny ip any any statement that was added by them the issue was gone.

Typically I don’t suggest using the deny ip any any on the redirect ACL because it is not required and it won’t add any benefit. Also, as the redirect ACL is not a security ACL it does not really make sense to add a deny ip any any rule. However, in that case the customer engineers changed the redirect ACL content by adding this deny statement at the end, which I had seen and again, it was working just fine in site A on the 2960-X switches.

Discussing this further with TAC engineer, I was told that the order of operation of the redirect ACL on the 2960-X (IOS) is totally different than the IOS XE code, and that includes the C3650, C3850 and the C9000 switches. Essentially these switches apply the dACL only on the traffic that was not matched by any rule on the redirect ACL, and because the redirect ACL we had was denying (not to redirect) the traffic to ISE nodes, permitting (to redirect) web traffic, and denying (not to redirect) anything else, that was basically matching all the traffic, and accordingly the dACL could not apply any of its rules.

So the takeaway here is not to configure any explicit deny ip any any in the redirect ACL with C9300 switches or any of the other switches series I mentioned above. This should be also a general rule with redirect ACL regardless of the switches platforms, because no need to configure a deny ip any any statement at all, and also please note that there is no implicit deny consideration with redirect ACL neither.

An example of a healthy redirect ACL for AnyConnect posture assessment would be similar to this:

 

 

You can tune up that ACL based on your environment requirements, but mainly you would need to deny (not to redirect) the traffic to ISE, DNS and DHCP traffic, and to permit (to redirect) the web traffic to enroll.cisco.com which is used in AnyConnect probes for redirection.

Also, there is no need to configure multiple permit rules for web traffic as long as you configure one IP address that will be used for redirection. With AnyConnect, it would be enough to configure enroll.cisco.com IP address and allow port 80/tcp.

With this it wraps up this post about redirect ACL with C9300 switches. I hope this has been informative and thank you for reading.