unstable network

I am experiencing serious stability issues with my network of xbee pros, it seems more or less random how many end devices are able to join the coordinator.

I have a setup with 1 coordinator and 8 end devices using API firmware.
the coordinator and end devices share one unique PAN id.
end devices are set to cyclic sleep mode (sleep cycle 6 seconds).
SD,SC set to default
NJ=0xFF, so its a open network
NT set to default

While testing all devices are in my studio, so there are no range issues involved.
For the actual application they will be out in a forest, so it is a disaster if I don´t resolve this…

I have gone through the documentation in 4.1.1-4.1.6 quite thoroughly, and this is the behaviour I expected:
First I activate coordinator, it finds a channel and starts up with the given pan ID.
I then turn on the end devices, they search for a valid panID (they are all set to look for the same one), and when they find the network they ask to join and supposedly get accepted.
The network info does not get lost unless there is a network reset, so if the coordinator is turned off and on again, and the end devices are turned off/on, the network should be reestablished.

What actually happens is very unpredictable.
I start the coordinator, then the end devices.
A random number of children (1-8) joins the network (My devices have a led connected to the on/sleep pin so I can see when they behave as expected).
If I do a NC (number of children connected) I usually get 8, but if I do a ND it corresponds to the actual number of devices joined.

I do a NR0 to forcereset the network.
that gives different results, but one pattern is this:
If 5 nodes were active before the reset, 3 nodes becomes active after the reset, which usually are the ones which weren´t active before.
The strange thing is that the 5 first ones behave as if they are still connected in the network (the led blinks as if in sleepmode). A NC would return 8, ND 3.
If I then turn off the end devices and then on again, quite often all 8 end devices show up, but not always.

If you are still with me, to sum up:
NC doesnt seem to reflect the number of active children, ND is more reliable.
End devices don´t seem to behave as expected (try to rejoin a network) when they lose contact with the coordinator, either through a NR0 command or if the coordinator is turned off.
It seems random the number of end devices which are allowed to join when they are switched on.

A final note, since my setup is static, I was thinking to use a closed network (NJ<0xFF), but this didn´t work any better.

best,
hc gilje

Hello HC Gilje,

I am sorry to hear you are having trouble. I have a few questions to ask you.

What firmware versions are you running (VR)? What is the fiwmare version of the coordinator? What is the firmware version of the end devices?

Are you installing the XBee end device modules into custom hardware?

If so, have you wired up pin 20 of the XBee module to a push button? When D0 is set to “1” and DIO0 is attached to a momentary push button it becomes the “deploment & commissioning push button” and it can make debugging network join problems in the field much, much easier.

Do you have the association LED wired to anything? This can also make debugging joining problems much, much simpler.

Are your end devices sleeping?

If so, you may be surprised to learn that ND may not always be a reliable means of discovering all of your nodes on your network. This is especially true if your nodes are sleeping for a time greater than the value of SP on their associated parent (in this case, the coordinator).

Furthermore, in sleeping networks the AT command mode firmware version of the ND command will timeout after a time and only show you the responses it has heard from. If sleeping nodes wake and respond after the ND command has terminated, you will never see their responses although the nodes may in fact be joined to the network.

If you are running the API mode firmware, you will in fact see the delayed responses (and the join announcment messages) as API frames. In general, the API mode firmware is much better for being able to see network diagnostic information.

Do your end devices monitor their association status (AI)? If so, do they think they are associated? Who do they report as being their parent, channel, and PAN ID?

Jordan

the answer from tech support has been to disable JV (off by deafault) and the undocumented JN (join notification) parameters.
This has not solved my problem, so I still have to do it the way I figured out:
it seems that if more than 5 nodes join at the same time, the network is jammed somehow. So I start with 5 end devices and then add the 3 others laters, and then 3 out of 5 times I have a network with 8 nodes.

This is problematic since if the coordinator goes off the network (which it needs to do sometimes if I am upgrading the software controlling it) I first need to turn off all the end devices, then turn on five, and then the three remaining

hi,
thanks for your quick answer.
I am running coordinator firmware 1141, end device firmware 1341, so I am in API mode.

the xbees are mounted on custom circuit boards, running together with a atmega168 chip. Unfortunately I don´t have the comissioning button or associate led hooked up, which was a obvious design mistake. I do have the on/sleep indicator though, which at least seems to show different behaviour when the end devices are active in the network or not. If the sleep led remains on, or have a different blinking rate than the normal sleep mode, I assume something is not working.

my end devices are sleeping, SP set to 1770 on both end devices and coordinator.
On my previous setups with xbee 2.5 (not the pros) I had 4 end devices and 1 coordinator and didnt experience any of these problems.
My coordinator is hooked up to a terminal program, so I see the ND results as they come in, and I can also watch on the end devices when they wake up when receiving the ND command.
and since the devices not responding to the ND command doesnt respond to other broadcast messages either I would say they are for all practical reasons not part of the network?
I don´t know if the pro modules (which I only received last week) is the problem or if having the maximum number of end devices could cause problems?

I have tried increasing SD to 4 on both end devices and coordinator, but it didnt make a difference.

I appreciate any suggestions.

best,
hc gilje

Thank you for the information. Interesting problem you are having here.

From a software/joining perspective there should be nothing different about the behavior of the PRO modules vs. the original modules you have.

What is interesting to me is that your coordinator is reporting that it has 8 children associated. In the 1141 firmware, 8 children is the maximum number so once that number is hit the coordinator will not allow any more nodes to use it as a parent. If there are not any routers with free slots within range of the sleeping child the node will not be able to join the network. Is it possible that something else is associating itself to the coordinator?

One strategy may be to try and change only the nodes under test to associate to a different PAN or use a security key and see if that helps.

Jordan

the thing is, even if I issue a NR0 command to clear the network, not all the end devices are able to join (or as I wrote, it is completely unpredictable how many will join).
I have no other xbees active except the ones I am testing, so what else could be associating itself to the coordinator?

Tomorrow I will connect some of the xbees with commissioning button and associate led, and maybe also one to a serial port to see what is happening on the end devices.
Anything in particular I should try out?

Could there be something wrong in the setup on the coordinator that makes it not update the NC status?

Is there any chance the xbees can interfere with each other when they are placed next to each other (I would assume not)?

Is there some other firmware I could try?

I have hooked up 5 of the end devices with associate led and commisioning push-buttons, and the 3 others are still on the custom boards.

steps:
turn on coordinator
turn on the end devices
7 end devices start up as normal, 1 doesnt connect.
NC gives me 8.
I press the button on the one not connecting, it blinks once, which would indicate error code 0x21, scan found no pans.
I reset the network on the coordinator, NR0
NC=0
None of the end devices seem to respond to this, the association led stays the same as before the reset.

I press the button on one of the end devices for it to join a network, but without success:
the association led blinks rapidly for the whole 30 seconds it is awake, which would indicate it has joined a network?

NC=0
turn all end devices off, then on
NC=8, ND=8

turn all end devices off, then on
6 end devices start up as normal, 2 doesnt.
NC=8, ND=6
The two that doesnt join, blinks differently than the others:
I assume it is following the sleep cycle, because every 5th second both the associate led and on/sleep led stays on for a bit less than 2 seconds, then off 5 seconds.

when I press the button on these two devices I get 7 blinks, which is 0x27, node joining attempt failed (typically due to incompatible security settings)

This makes some sense, since the coordinator is indicating it has 8 children already.

I reset network, NR0
NC=2,ND=2
and now the end devices which didn´t join before has joined while the others have not joined (but their associate led indicate they are still part of a network)
I press the button on two of the devices, they blink continously for about 30 seconds, before going into sleep mode, seemingly part of a network, but not.
NC=2

Hopefully this gives you some additional information to help me solve the problem?
Should I also report this to tech support?

update:
upgraded coordinator to 1147, end devices to 1347

start coordinator (before powering up the end devices).
NC=8
NR0
NC=8
?
start end devices
5 join, 3 don´t
NC=0,ND=5
NR0
NC=0,ND=8

This last pattern I seem to get consistently:
with the end devices on, I send NR0 to the coordinator
all the devices appear.
If I turn off and on the end devices though, not all of them appear, so it seems I have to do a network reset everytime?
The strange thing now is that NC stays at 0 no matter what I do.

Message was edited by: hcgilje

2nd update:
this seems to work consistently, which means I have to do a network reset for the end devices to join:
First turn on end devices, then the coordinator, then issue a NR0 command on the coordinator, and voila I have 8 nodes connected (but NC still gives 0).
It would be great to be able to avoid the reset.

Message was edited by: hcgilje