I’m using a Digi XBee3 Cellular LTE-M/NB-IoT for data collection and sending data to a web application. I’m using micropython, sockets and ussl.wrap_socket.
I believe I’ve found a bug in wrap_socket; the problem is that it truncates data sent to the web application. It appears to be sending only the first packet because the truncated size is consistently 1500 bytes. If I use regular sockets (ie. no encryption), all the data is sent.
Things I’ve tried:
- the write() method (should send all the data)
- the send() method with my own loop to ensure all the data is sent
- setblocking(True)
- checking the return values; wrap_socket.write() reports that it sends all the bytes even though it hasn’t
This is a bit difficult to demonstrate because you need a web server to receive the data and the ability to configure it for both HTTP and HTTPS to see the success and failure cases. I can provide minimal code samples for everything, but didn’t want to paste them in here unformated…
Firmware version is 11410, and python version info is:
MicroPython v1.9.4-927-g7565eb7 on 2018-11-08; XBC LTE-M/NB-IoT Global with EFR32MG
2 Likes
Hmm… no answers. I’ll try posting some code then (hopefully bbcode format works…)
host = "example.com"
page = "/postlen.php"
head = "POST {:s} HTTP/1.1
" \
"Host: {:s}
" \
"Content-Type: application/x-www-form-urlencoded
" \
"Content-Length:{:d}
"
data = "data=" + ("abcdefghijklmnopqrstuvwxyz" * 100)
headFmt = head.format(page, host, len(data))
import usocket
import ussl
sock = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM, usocket.IPPROTO_SEC)
wrap = ussl.wrap_socket(sock)
wrap.connect((host, 443))
sent = wrap.write(headFmt + data)
if sent < len(headFmt) + len(data):
raise RuntimeError("incomplete")
wrap.close()
The code above fails to send all the data. Note that the RuntimeError(“incomplete”) should be triggered but is not, so wrap.write() is reporting that it sends all the data.
If you remove SSL, it works fine:
# (everything above here is unchanged...)
import usocket
sock = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM)
sock.connect((host, 80))
sent = sock.write(headFmt + data)
if sent < len(headFmt) + len(data):
raise RuntimeError("incomplete")
sock.close()
postlen.php is a minimal PHP script to receive the data and write the number of bytes to the error log. Note this only includes the bytes of “data=…” since PHP can’t get the raw HTTP header…
I’m possibly having a similar problem in which I don’t believe the write call is writing the data it says it is. See my question over here: https://forums.digi.com/70190/mutual-tls-authentication-with-azure.
Although, looking at your code I believe the write call requires a byte array, not a string. Try encoding it as a byte array before calling write like I am in my script and see what happens.
Interesting, it looks like you made it further than me with authentication. I lost a day or more trying various certificate options for any sort of authentication, but always hit EIO errors. Eventually gave up hope.
As for your suggestion, I believe my actual code was using bytearrays. The above examples were just the simplest demonstration I could come up with. But I’ll check just to be sure. Thanks…
EDIT: I just checked and no, sending a bytearray made no difference.
Please try with the 11411 firmware and let us know if it still fails to work correctly.
Hate to tell you, but that didn’t fix it. I get the exact same number of bytes transmitted using the new firmware.
Here’s my version info after the update:
FW ver 11411 Prod Bootloader: 181 Build: Apr 9 2019 15:13:14
MicroPython v1.10-1179-g94f4f29 on 2019-04-09; XBC LTE-M/NB-IoT Global with EFR32MG
Thanks for the detailed report. I’ve opened a bug in our issue tracker to investigate the problem.
Ok, thanks for looking into it.
This has been confirmed as a bug with the LTE-M/NB-IoT product, but I don’t know when a fix might be released.
A temporary workaround is to wait for a response from the server before closing the socket.
A temporary workaround is to wait for a response from the server before closing the socket.
Interesting. I don’t have the xbee available now, but the next time I have a chance I’ll try to confirm that. Thx.