WR21 Strange behavior[SOLVED]

Digi TransPort WR (Series)
, , ,
1

Hello:

I have a strange problem with WR21s. I was using Fw 8003 but upgraded to 8101 but the behavior did not change.

First problem:
When I load in an edited config.da0, after the router reboots it does NOT come up in port isolate mode even though the router where I copied the config.da0 from was in port isolate mode. All I changed what some IP addressing and router naming entries.
So, I have to manually put it into port isolate mode.
WHY?

Ok, I get it into port isolate mode. Now, I have eth0 statically assigned and connected to some local instrumentation and eth1 is connected to an ADSL router. When in port isolate mode eth1 WILL NOT pickup a dhcp address. I have tried this against this BEETEL 777VR1 ADSL router, against a windows server and against a cisco router, it REFUSES to pick up a DHCP address.
Why?

The WR21 Router is setup as follows:
Eth1 primary ADSL(conected via Ethernet, not in bridge mode), default route metric 1,
PPP1 Secondary 1 GPRS, default route metric 10,
PPP3 Secondary 2 GPRS, default route metric 10,

Now, when the system first reboots after loading a config.da0 with the associated pwds.da0 and fw.txt, it comes up and refuses to negotiate an IPsec tunnel with the cisco over GPRS. The Eth1 doesn’t come up.
I manually change the Eth1 to port isolate mode and reboot with Eth1 as a static IP.

I cannot ping the ADSL router from the Digi. If I unplug the digi and plug in a laptop, not problem with the ADSL router the laptop gets an address and works fine, this particular digi won’t ping with or without a static IP. The last digi WOULD ping with a static IP but would not pick up a DHCP address, same as this one.
The digi is unable to negotiate and IPSec tunnel against A Cisco over GPRS.
19:42:07, 29 Aug 2020,(4) New Phase 1 IKE Session 103.205.244.106,Initiator
19:42:07, 29 Aug 2020,IKE Request Received From Eroute 0
19:42:07, 29 Aug 2020,(3) IKE SA Removed. Peer: ,Negotiation Failure
19:42:07, 29 Aug 2020,(3) IKE Negotiation Failed. Peer: ,Retries Exceeded
19:41:57, 29 Aug 2020,IKE Request Received From Eroute 0
19:41:47, 29 Aug 2020,IKE Request Received From Eroute 0
19:41:37, 29 Aug 2020,(3) New Phase 1 IKE Session 103.205.244.106,Initiator
19:41:37, 29 Aug 2020,IKE Request Received From Eroute 0
19:41:37, 29 Aug 2020,(2) IKE SA Removed. Peer: ,Negotiation Failure
19:41:37, 29 Aug 2020,(2) IKE Negotiation Failed. Peer: ,Retries Exceeded
19:41:27, 29 Aug 2020,IKE Request Received From Eroute 0
19:41:17, 29 Aug 2020,IKE Request Received From Eroute 0

Change Eth1 to dhcp, don’t touch anything else on the router.
Now the IPSec successfully negotiates.
19:44:41, 29 Aug 2020,(10) IKE SA Removed. Peer: 103.205.244.106,Successful Negotiation
19:44:40, 29 Aug 2020,Eroute 0 VPN up peer: 103.205.244.106
19:44:40, 29 Aug 2020,New IPSec SA created by 103.205.244.106
19:44:40, 29 Aug 2020,(10) IKE Notification: Responder Lifetime,RX
19:44:40, 29 Aug 2020,(10) IKE Notification: Responder Lifetime,RX
19:44:39, 29 Aug 2020,(11) IKE Notification: Responder Lifetime,RX
19:44:39, 29 Aug 2020,(10) New Phase 2 IKE Session 103.205.244.106,Initiator
19:44:39, 29 Aug 2020,(9) IKE Keys Negotiated. Peer:
19:44:39, 29 Aug 2020,(9) New Phase 1 IKE Session 103.205.244.106,Initiator
19:44:39, 29 Aug 2020,IKE Request Received From Eroute 0
19:44:39, 29 Aug 2020,(8) IKE SA Removed. Peer: ,Negotiation Failure
19:44:39, 29 Aug 2020,(8) IKE Negotiation Failed. Peer: ,Retries Exceeded

But I cannot ping the Cisco router and it cannot ping the digi.
But the digi reports ISAKMP SAs and IPSec SAs as does the cisco, just can’t ping.
Routing table:
Destination Gateway Metric Protocol Idx Interface Status
100.67.46.172/30 100.67.46.173 1 Local - PPP 1 UP
192.168.51.24/29 192.168.51.25 1 Local - ETH 0 UP
192.168.0.0/16 192.168.190.13 2 Static 0 TUN 0 DOWN

Default Routes
Destination Gateway Metric Protocol Idx Interface Status
0.0.0.0/0 0.0.0.0 2 Static 2 ETH 1 UP
0.0.0.0/0 100.67.46.173 11 Static 0 PPP 1 UP
0.0.0.0/0 0.0.0.0 - Static 1 PPP 3 OOS

I go back into Eth1, change it to “Use the following settings”
and enter in a static IP, mask and GW…
Pings start.
Now its all working and all I did was change the addressing of eth1.
Routing table:
Destination Gateway Metric Protocol Idx Interface Status
100.67.46.172/30 100.67.46.173 1 Local - PPP 1 UP
192.168.1.0/24 192.168.1.201 - Local - ETH 1 OOS
192.168.51.24/29 192.168.51.25 1 Local - ETH 0 UP
192.168.190.12/30 192.168.190.13 1 Local - TUN 0 UP
192.168.0.0/16 192.168.190.13 2 Static 0 TUN 0 UP

Default Routes
Destination Gateway Metric Protocol Idx Interface Status
0.0.0.0/0 100.67.46.173 11 Static 0 PPP 1 UP
0.0.0.0/0 0.0.0.0 - Static 1 PPP 3 OOS
0.0.0.0/0 192.168.1.1 - Static 2 ETH 1 OOS

So great its up…
But if I reboot, I need to do this whole ritual again. This is completely repeatable.

The router will wake up after the reboot with eth1 with a static IP, it does not come up, it cannot ping the adsl router, it shows red all the time, and so the router comes up on GPRS and cannot negotiate and IPSec tunnel.
I go into eth1, change it to dhcp, and the IPSec successfully negotiates but I cannot ping.
Go back into eth1, assign a static IP with mask and GW, and pings start.

What the hell is going on here?

This appears to be related to port isolate mode. We have another 70 routers to deploy and I can’t be fighting with everyone like this.

So, is there a way to get each Ethernet port its own IP address without being in port isolate mode? I seem to remember some king of group mode or something.

If there isn’t, I need help to figure this out.
I can send over the config.da0 and the fw.txt files if that will help.

Cheers,
John

2

Update:

Had a bad connection on Eth1 to the ADSL. It works on static IP only, will not pick up an DHCP address.

I took the switch out of port isolate mode and the behavior is exactly the same AND even in HUB mode, I can route between interfaces? I’m fine with that but it seems odd.

Here is the problem:
If I boot the router with all interfaces up and plugged in, it comes up using the ADSL on Eth1, PPP1 is up, and PPP3 is OOS.

There are two failure modes for Eth1:

  1. The Eth1 interface stays up but the ADSL goes down,
  2. The Eth1 goes down.

Now, if the unit is up and running and I unplug Eth1 OR disconnect the phone line form the ADSL router, it switches to GPRS, all good.
I put the ADSL back, it switched back…all good.

If I power up with either the phone line disconnected or Eth1 disconnect, when the router comes up, it says that PPP1 is up and trying to connect to the Cisco, looks like this:
16:49:20, 31 Aug 2020,IKE Request Received From Eroute 0
16:49:10, 31 Aug 2020,IKE Request Received From Eroute 0
16:49:00, 31 Aug 2020,(4) New Phase 1 IKE Session 103.205.244.106,Initiator
16:49:00, 31 Aug 2020,IKE Request Received From Eroute 0
16:49:00, 31 Aug 2020,(3) IKE SA Removed. Peer: ,Negotiation Failure
16:49:00, 31 Aug 2020,(3) IKE Negotiation Failed. Peer: ,Retries Exceeded
16:48:50, 31 Aug 2020,IKE Request Received From Eroute 0
16:48:40, 31 Aug 2020,IKE Request Received From Eroute 0
16:48:30, 31 Aug 2020,(3) New Phase 1 IKE Session 103.205.244.106,Initiator
16:48:30, 31 Aug 2020,IKE Request Received From Eroute 0
16:48:30, 31 Aug 2020,(2) IKE SA Removed. Peer: ,Negotiation Failure
16:48:30, 31 Aug 2020,(2) IKE Negotiation Failed. Peer: ,Retries Exceeded
16:48:20, 31 Aug 2020,IKE Request Received From Eroute 0
16:48:10, 31 Aug 2020,IKE Request Received From Eroute 0

But here’s the thing, the Cisco on debug ISAKMP shows NOTHING. There are no IKE initiation packets coming form the WR21 even though the WR21 event log looks like the above.

If I go into the WR21, to Network-Interfaces-Ethernet-Eth1 and click “Get an IP address automatically using DHCP” the cisco debug screen fills up and the IPSec tunnels come up…
I then put the interface back to static IP. And its all good. If I reconnect the telephone line or reconnect Eth1 the ADSL will come back up and the router will switch back to ADSL.

So, if Eth1 is down for any reason at power up, the IPSec tunnels will not come up.

Does anyone know a work around for this?
I have the “activate interface after X seconds after power up” set to 20, a set that back to 0, no change.
I will experiment with this but…
Why does chnaging Eth1 from static to dynamic IP all of a sudden get the IPSec to start working? That is like a bug deep-deep in the WR21. That is weird behavior.

Need a work around for the PPP1 issue on power up if Eth1 is OOS for either ping issues or interface down issues.

Cheers,
John

3

Hello:

I have downloaded AN41 and meticulously followed this and found the following:

  1. If I use a different IP in the FW for the ping than in the interface setup, the router does NOT detect Eth1 has gone down.

But, I have the same issues:

  1. I cannot get a DHCP IP on Eth1,
  2. I put Eth1 as static,
  3. If I either unplug Eth1 or unplug the phone line from the ADSL router, when ppp1 comes up it cannot negotiate an IPSec tunnel with the Cisco. In fact, the Cisco debug shows NO IPSec initiations coming in but the Digi sure believes its sending some. Where these packets are being sent I dunno.
    To fix this, all I need to is change Eth1 to DHCP and the next IPSec initiation attempt succeeds.

This bug is a SERIOUS problem for me, I need these connections to work as a backup. This inability for Eth1 to pickup a DHCP is a huge issue. I’m not sure if the router would behave properly if Eth1 was always on DHCP or if the toggling does something.

I get upload the FW.txt and config.da0 for someone to look at. I have 72 sites to deploy and we cannot go forward until this is fixed.

Cheers,
John

4

I moved the WAN interface to Eth0 and Eth0 can get a DHCP address in HUb more and port isolate mode so there is a trick for new players. So, my instrumentation is now on the the statically assigned Eth1 and the ADSL router is on Eth0.

I had to modify some things out of AN41. I had to allow ppp1 to power up immediately at power by selecting in service all the time,
I took out the inhibit ppp1 and ppp3 from the default route setting for Eth0. The FW script was also wrong, I changed the IP to 8.8.8.8.
I got it working in the following scenarios:

  1. At boot Eth0 plugged in, DSL line plugged in:
    A. IPSec comes up about 40 seconds after reboot,
    B. Can disconnect Eth0, PPP1 takes the IPSec tunnel about 20 seconds later,
    C. Plug Eth0 back in, switched back about 20 seconds later with no dropouts.
    D. Can disconnect the phone line, ppp1 takes the link about 40 seconds.
    E. Put the phone line back, takes about a minute to switch back, no drop outs on the switch back to Eth0.
  2. At boot Eth0 is unplugged, IPSec comes up on ppp1 after about 160 seconds. The delay is from the GPRS network, it takes some time sot get the IP.
  3. At boot, Eth0 is plugged in but the phone line is unplugged form the ADSL router. FAIL. This does not work. This is the WR21 event log:
    15:54:56, 01 Sep 2020,IKE Request Received From Eroute 0
    15:54:46, 01 Sep 2020,IKE Request Received From Eroute 0
    15:54:36, 01 Sep 2020,(3) New Phase 1 IKE Session 103.205.244.106,Initiator
    15:54:36, 01 Sep 2020,IKE Request Received From Eroute 0
    15:54:36, 01 Sep 2020,(2) IKE SA Removed. Peer: ,Negotiation Failure
    15:54:36, 01 Sep 2020,(2) IKE Negotiation Failed. Peer: ,Retries Exceeded
    15:54:26, 01 Sep 2020,IKE Request Received From Eroute 0
    15:54:16, 01 Sep 2020,IKE Request Received From Eroute 0
    15:54:12, 01 Sep 2020,Default Route 2 Out Of Service,Activation
    15:54:11, 01 Sep 2020,IP Act_Rq to PPP 3-0: s_ip[2.2.2.4] d_ip[1.1.1.10]
    15:54:06, 01 Sep 2020,(2) New Phase 1 IKE Session 103.205.244.106,Initiator
    15:54:06, 01 Sep 2020,IKE Request Received From Eroute 0
    15:54:06, 01 Sep 2020,(1) IKE SA Removed. Peer: ,Negotiation Failure
    15:54:06, 01 Sep 2020,(1) IKE Negotiation Failed. Peer: ,Retries Exceeded
    15:54:05, 01 Sep 2020,PPP 1 up
    15:54:05, 01 Sep 2020,PPP 1 Start
    15:54:05, 01 Sep 2020,Modem connected on asy 2
    15:54:05, 01 Sep 2020,GPRS: Context activation successful
    15:54:04, 01 Sep 2020,GPRS: Activating context 1
    15:54:04, 01 Sep 2020,GPRS: Activating context 3
    15:53:59, 01 Sep 2020,Default Route 0 Out Of Service,Firewall
    15:53:59, 01 Sep 2020,ETH 0 Out Of Service,Firewall
    15:53:56, 01 Sep 2020,IKE Request Received From Eroute 0
    15:53:53, 01 Sep 2020,GPRS Connection Status:Normal, unspecified
    15:53:53, 01 Sep 2020,Network technology changed to LTE
    15:53:53, 01 Sep 2020,GPRS Registration On
    15:53:53, 01 Sep 2020,GPRS Attachment On
    15:53:50, 01 Sep 2020,WEB Login OK by SOIroot lvl 0
    15:53:46, 01 Sep 2020,IKE Request Received From Eroute 0

You can see that the router is attempting to bring up and IPSec tunnel over Eth0 before it is declared OOS. Because of the delay of getting a GPRS IP, it cannot immediately switch to GPRS, when it starts it appears to be failing…but it actually isn’t sending anything to the Cisco. The Cisco debug shows nothing coming from the WR21 which means the WR21 is sending those requests out another interface but thinks they are going out ppp1.
I can ping the Cisco public IP form the WR21 so ppp1 is up, its jsut that those IPsec initiations aren’t going out ppp1.
I can fix this like before, got to Eth0, switch it to static IP, hit Appply, switch it back to DHCP, hit apply and presto, on the next IPSec request the tunnel comes up.

The only problem here is the most realistic scenario is the phone line down but the ADSL router up at reboot and that is the only situation that doesn’t work.
Question:
Is there a way to slow the activation of the router up after reboot to allow ppp1 to come up BEFORE Eth0 is declared OOS?
Is the a way to reset eth0 automatically to get the ppp1 ipsec to come up, something that would emulate what I am doing manually?

Almost there.

Cheers,
John

5

Ok, the short answer to this is…
DHCPc ONLY works on Eth0, it will not work on Eth1 whether in Port isolate mode (it actually only makes sense in port isolate mode) or in hub mode.

We’re using the “disable interface” function in the default route section for the PPP interfaces to use as backup as per one of the app notes, works fine.

Cheers,
john