Get data from multiple nodes

Hi,
I’m trying to collect data from multiple nodes by running the following code i made:

import sys
import os
import zigbee
import binascii
import time
from socket import *
import struct

sd = socket(AF_ZIGBEE, SOCK_DGRAM, ZBS_PROT_TRANSPORT)
socket.ZBS_PROT_TRANSPORT)
sd.bind((‘’, 0xe8, 0, 0))
nodes = zigbee.getnodelist()

for node in nodes:
while 1:
payload, src_addr = sd.recvfrom(255)
#print payload
output = binascii.b2a_hex(payload)
print “Time:%s Reading:%s From:%s” % (time.strftime(“%X”,time.localtime()),output, src_addr)


The only problem is that i can only get data from 1 node.
I currently have 2 nodes, node1 is connected to a pc’s serial port, node2 is connected to another device via serial port. Even when i connect node2 to a serial port of another pc, i still cant receive its data. It seems like it can only read from 1 node. Can someone help me understand what i’m doing wrong please or what else i need to do???

P.S. the extra import modules are there because i want to eventually do some converting of the data i receive once i’m able to read data from both nodes.

Thanks!

Hi meme,

To help answer why you may be receiving data from one node and not the other may be because of the node’s settings:

Node2’s DH and DL values must match the SH and SL values of the gateway.

If you are sending to a different endpoint (Not 0xe8), the socket defined in your code will not receive the data. The DE parameter on node2 defines this.

Make sure that node2 matches your gateway’s encryption enable setting (EE).

One way of diagnosing this problem would be to enable a trace while sending from the node to the gateway. To do this, you can use the command ‘set trace mask=mesh:* state=on’ to view all XBee traffic.

On a side note, you should become familiar with the select() function from the select python module for socket behavior.

Plus this code does nothing useful:
__nodes = zigbee.getnodelist()
__for node in nodes:
____while 1:
______…

Literally, it runs forever with node = the first node returned by getnodelist(), but you’re not using the value ‘node’.

If you bind on the socket/end-point 0xE8, then any number of nodes can send you data (if as Max says the DH/DL are correct) and you’d use the src_addr to understand which Xbee sent the data.

So remove the getnodelist and for statements

Thank You for the prompt replies!!

Oh! Then what do i need to do in order to read from all my nodes? Does that mean i dont need a for loop etc?

I basically want to send everything to the gateway (connectportX2) so my understanding was to set DH,DL to zero??

That DH and DL setting will work on ZNet2.5/ZB radios. Otherwise use the SH and SL of the gateway.

The AF_XBEE socket descriptor will receive all traffic that is received from the XBee radio that has the destination endpoint of 0xe8. Whether it is one node or ten, a single socket will receive it all.

If you want to receive from more then one endpoint, you can create multiple sockets, and bind them to other endpoints. In this case, using a select statement inside the while loop would allow receiving from either socket descriptor in a non-blocking way.

Have you run the trace?

You cannot ‘fetch’ serial data this way from nodes - the mesh is a peer-to-peer system. If you feed serial data into a remote XBee, it will automatically (like it or not) forward that data to the DL+DH ‘peer’. You have no control over order, so if 2 remotes each send 120 bytes (making 2 RF packets) at the same time, you cannot be sure in which order the 4 packets created arrive.

So you’ll need to understand this, using the ‘src_addr’ to collect and reassemble any higher level protocol packets.

But literally - if the remote XBee act as client/master, then your gateway code just sits and waits for any message sent its way. In my systems with 50 tanks guages which send levels hourly, literally the central code just idles and waits for the data to show up like magic. The Xbee take care of avoiding collision and limited retry behavior.

Thanks. Ok i will try and use multiple sockets. Do you know where i can find some examples on using this? What is the trace please?

Thanks lynnl.
From your post, im a little confused. however, your 50 tanks gauges example is very similar, in fact exactly what i’m trying to do. Except they’re not tank gauges, their gas sensors that typically send a reading every 5 seconds to the central host, which is my gateway. All i want to do is collect this data on my pc connected to my gateway.

so what i understand from you post is that i need to make sure all the nodes have DH+DL = 0 in order to send everything to my gateway? + using src_addr to reassemble my packets?

Yes, all Xbee’s act as clients … can you show me the code for your tank gauges system please? Im sorry, im hopeless with programming.

> But literally - if the remote XBee act as
> client/master, then your gateway code just sits and
> waits for any message sent its way. In my systems
> with 50 tanks guages which send levels hourly,
> literally the central code just idles and waits for
> the data to show up like magic. The Xbee take care of
> avoiding collision and limited retry behavior.

How did you implemented “idle and wait” and “data show up” actions? I managed to loop all nodes and read their IS with zero delay and then wait for a few seconds.

Simon

On ZNet and ZB, a DH/DL pair of 0 causes packets to be sent to a central aggregater (which happens to be the gateway). In DigiMesh it causes a discard, which is why we tend to encourage people to load the DH/DL of the gateway to be more universal.

Max’s suggestion to use 5 sockets for 5 nodes is ideal when you DO NOT understand the protocol, because that eliminates the mix-up of packets and/or need to defragment them.

In my case, the guages send a serial protocol & all XBee/guage-originated packets are less than 60 bytes so a single socket can handle all 50 nodes. They also work in API mode, so I’m 99.9% sure to always get a full message in 1 packet (occasionally they get lost or garbled - so write good code!).

If you just send raw serial data to an XBee with the AT-mode firmware, you probably want to set the RO/packetization timeout to be high enough to reduce risk of packets being sent in 2 or 3 pieces. I’d change the default of 3 to something larger - 10 perhaps or even 50. A 50 character delay (50msec at 9600 baud) shouldn’t hurt your sensor readings any.

Also be smart - if the packet should be 20 bytes, and you only see 12, be careful as the ‘next’ packet from that node might be the 8 missing bytes. A higher RO setting should eliminate this, but just be mindful and design code to understand this risk.

I can’t speak for the tank gauge code, but I can speak to a general principle involved…

If the device connected to the remote radio already is capable of automatic, asynchronous transmission (i.e. not polled), then its XBee radio will simply forward that serial data as it is received to the device established by DH/DL. In such an environment, the gateway should be the DH/DL address for all of the “child” radios, and the Python script simply blocks in “select” on the socket, until the next frame arrives.

In other words, “idle and wait” is implemented by “select”, and “data show up” is a function of the remote endpoint. In cases where these remote devices are not “awake all of the time”, such a solution is almost mandatory, otherwise the gateway must be “lucky” to catch the device awake when it polls.

Message was edited by: 52637p

Look at this Wiki page.
http://www.digi.com/wiki/developer/index.php/UDP_to_XBee_Network

Look at the section around:
rl, wl, el = select.select(sock_list, sock_list, [])

rl will be a list of sockets with data which can be read, which for you might just be your ZB socket. In this demo script it also includes UDP sockets (and can include the CPX4’s serial port). That’s the nice thing about select() - without it you might need 1 thread for every “comm path” since a single thread can only watch 1 socket in your idle-wait loop.

I normally add a fourth option parameter of 5.0 or 15.0 to select, which causes the select to return after 5 or 15 seconds with rl/wl/el all set to empty. This allows your script to do something else once in a while. Don’t worry, you won’t miss packets while you’re not blocked in select - they are buffered.

Hi there,

I’ve altered my code as below:

import sys
import os
import zigbee
import binascii
import time
from socket import *
import struct

sd = socket(AF_ZIGBEE, SOCK_DGRAM, ZBS_PROT_TRANSPORT)
socket.ZBS_PROT_TRANSPORT)
sd.bind((‘’, 0xe8, 0, 0))
node_list = zigbee.getnodelist()

print “%12s %24s %24s” % (“Time”, “From”, “Reading”)
print “%24s %24s %24s” % (“-” * 24, “-” * 24, “-” * 24)

for node in node_list:

while 1:
	print node.label
	payload, src_addr = sd.recvfrom(255)
	output = binascii.b2a_hex(payload)
	a = output[4:12]
	c = binascii.unhexlify(a.replace(' ', ''))
	h = struct.unpack('" etc.
  1. what does: <232,49413,17> stand for?
  2. how can i get my program to print out the labels instead of the addresses? I’ve labelled my nodes node1 and node2 respectively.

thanks

The ‘node.label’ is frequently the NI parameter in the XBee - however, the NI is NOT a normal part of node-routing over a mesh with multiple routers.

So you will find that a zigbee.getnodelist() does NOT always return a useful NI/Node label. For example, if the node discovery is satisfied by an external router on behalf of a end-device, the NI may be missing. You should see the same behavior on the gateway “net view/display Xbee” screen - the names of sleeping nodes come and go and then return over time.

The easiest solution is to just print the last 5 or 6 chars of the MAC, so you’d think of a node as ‘3C:A3’ instead of a name.

Or explicitly fetch the NI yourself using the getddo function, then create your own lookup dictionary of MAC keyed to return a NI/name. This way you know that the actual NI has been received (or that it was not received).

Sorry for not answering all, the numbers like <232,49413,17> refer to the endpoints, cluster id and so on. It is Zigbee stuff - much like the source/destination ports on TCP/IP … but different.

See the XBee manual 90000976 for the Zigbee XBee module (or any ZIgbee docs) to learn more … but if you are like me and just doing serial encap on mesh then these values work as-is. The 0xE8 or 232 ‘end-point’ is the raw serial encap ‘socket or channel’ (call it what you will)

Hi Lynnl,
Thank you so much for your help! Really appreciate it!
I’ve just got a few more questions:

  1. How do i print the last 5 or 6 chars of the mac address? I’ve tried using “short_addr” to do this, but it only prints [‘00:00’] (which i assume is the coordinator) all the time. Why does this happen?

  2. When i use the ‘node.label’ to print the NI, it only prints out ‘coordinator’, eventho i’ve changed the NI for both nodes to be unique.

  3. How do i create my own lookup dictionary of MAC keyed to return a NI/name?

Most of these are standard Python data manipulations. Resources like the Python tutorial (http://www.python.org/doc/2.4.3/tut/) introduce some of these concepts.

For instance, the last 5 characters of a string “msg” can be expressed as:

msg[-5:]

Regarding the “NI” string… I think that the recent comment that the remote node be directly queried using the ddo_get_param function is a good approach. It only needs to be done the first time a mac arrives that isn’t already in the dictionary. The empty dictionary can be created in a variety of ways, including:

ni_string_dict = {}

The lookup based on a mac string is just:

ni_string = ni_string_dict[mac]

Assigning an NI string is just the inverse:

ni_string_dict[mac] = ni_string

… etc.

Since the XBee packets look like UDP data, the normal method is to use 2 tasks/threads. One uses the select call to sleep/block on input. It can use a queue to send the received data to your other thread which is running your logic in a normal sleep loop - or wait on various queue inputs.

Unfortunately, async-event driven programs are more complex than old-fashioned linear step-by-step programs because there is no A-follows-B logic you can trust.

Hi There, it’s an old post, but i found the problem with multiple nodes. I’m using X8 (not X2). I Could receive all the data from multiple nodes. I wrote all the received data into my flash memory inise X8.
The data come like these:
[address1] [data1]
[address2] [data2]

But the problem is, sometimes when two nodes (or more) sending data in same time or nearly same, Coordinator will receive all the data as same data like these
[address1] [data1] [address2] [data2]

I’m using sd.recvfrom(255). When I’m trying to cut the buffer size into 7bytes, the program will goes out or break (i’m using “try”).
How to receive in data one by one format? How to hold any data in a buffer?

Thank you so much for the help, need it badly niw

You can use select, for non blocking IO operations. Buffer size here does not reflect to only payload, it also calculate for the address bytes, start bytes, check sum and other overheads. So, What you can do in this case is reduce it by 22 approx. to recover 7 bytes of actual data/ payload.