This post covers a potential issue that might cause a Palo Alto VPN tunnel to be up but with no traffic flowing between the encryption domains. Here is the scenario I came across with a site to site VPN tunnel between a Palo Alto and a Cisco ASA behind a NAT device. Basically, the VPN tunnel was configured with no NAT-T enabled where I could see both Phase 1 and 2 being successfully established between the two firewalls.
However, there was no traffic passing through between the local and the remote encryptions domains. When I tried to initiate the traffic from the Palo Alto side, I could see the encaps increasing on the IPSec tunnel, but zero decaps. And on the ASA side I could not see anything landing into the IPsec tunnel or even hitting the ASA outside interface.
However, when I tried to initiate the traffic from the ASA side, everything was working as expected. Digging deeper into this and asking some help on https://www.pangurus.com/forum (which I highly recommend to anyone wants to put on the table any Palo Alto topic to be discussed and received prompted answers from expert people), I started enabling some debugs on the Palo Alto and ASA. Interestingly I could not see anything worth mentioning on the debugs or logs output.
On the other side, when I tried to enable NAT-T on the Palo Alto, Phase 1 tunnel could not be established, and I got a similar snippet as the following from the debugs (the IP addresses have been replaced):
[INFO]: { 2: 4}: IPsec-SA request for 172.16.1.22 queued since no phase1 found
[PNTF]: { 2: }: ====> PHASE-1 NEGOTIATION STARTED AS INITIATOR, MAIN MODE <====
====> Initiated SA: 172.16.1.254[500]-172.16.1.22[500] cookie:e4339610a73de31f:0000000000000000 <====
[INFO]: { 2: }: received Vendor ID: RFC 3947
[INFO]: { 2: }: received Vendor ID: FRAGMENTATION
[INFO]: { 2: }: Selected NAT-T version: RFC 3947
[INFO]: { 2: }: Hashing 172.16.1.22[500] with algo #2
[INFO]: { 2: }: Hashing 172.16.1.254[500] with algo #2
[INFO]: { 2: }: Adding remote and local NAT-D payloads.
[INFO]: { 2: }: received Vendor ID: CISCO-UNITY
[INFO]: { 2: }: received Vendor ID: draft-ietf-ipsra-isakmp-xauth-06.txt
[INFO]: { 2: }: Hashing 172.16.1.254[500] with algo #2
[INFO]: { 2: }: NAT-D payload #0 verified
[INFO]: { 2: }: Hashing 172.16.1.22[500] with algo #2
[INFO]: { 2: }: NAT-D payload #1 doesn't match
[INFO]: { 2: }: NAT detected: PEER
[INFO]: { 2: }: KA list add: 172.16.1.254[4500]->172.16.1.22[4500], (in_use=1),total=1
[INFO]: the packet is retransmitted from 172.16.1.22[500] to 172.16.1.254[500].
[INFO]: the packet is retransmitted from 172.16.1.22[500] to 172.16.1.254[500].
[INFO]: the packet is retransmitted from 172.16.1.22[500] to 172.16.1.254[500].
[PWRN]: { 2: }: remote address mismatched. db=172.16.1.22[4500], act=172.16.1.22[500]
[PERR]: { 2: }: ignore information because ISAKMP-SA has not been established yet.
[PNTF]: { 2: }: ====> PHASE-1 NEGOTIATION FAILED AS INITIATOR, MAIN MODE <====
====> Failed SA: 172.16.1.254[4500]-172.16.1.22[4500] cookie:e4339610a73de31f:d3688b4a77bcf18f <==== Due to timeout.
[INFO]: { 2: }: ====> PHASE-1 SA DELETED <====
====> Deleted SA: 172.16.1.254[4500]-172.16.1.22[4500] cookie:e4339610a73de31f:d3688b4a77bcf18f <====
Looking at the above snippet there was a couple of interesting things. NAT-T seemed to be failing between the two firewalls, and it looked like the remote ASA was not using port 4500/udp as it should have. Doing some more investigation, it ended up that the ASA ISP router had the NAT-T port 4500/udp closed. After opening up that port the data started flowing from both ends successfully.
More specifically the issue was that, without NAT-T enabled, the Palo Alto was sending the ESP packets across the VPN tunnel as expected, and because the ESP packets encrypts the L4 headers, the remote ASA's ISP router could not route them to the ASA, hence it was discarding them.
However, with NAT-T enabled, without port 4500/udp opened on the ASA's ISP router, the traffic was sent encapsulated into a UDP packet using port 4500/udp as the source and destination port, but because the ASA's ISP router had that port closed it was dropping that traffic.
Now why it worked without enabling NAT-T and only when the traffic was initiated from the ASA side?!, I think that was because in that case the ASA's ISP router would have applied NAT'ing for the traffic sourced from the ASA destined to the Palo Alto. This means the ISP router was creating state entries for the traffic leaving it going to the Palo Alto. Therefore, when the traffic was received back from the Palo Alto, the ISP router could associate it to those state entries created for the ASA.
This wraps up this little post about Palo Alto VPN tunnel up with no traffic.
Thank you for reading!
how to open 4500/UDP port on cisco ASA?
I am a new on this. Please help