Some Python people on the 'net say that in-place alterations of built-in types such as lists - may or may NOT be atomic. Depends on the interpreter implementation.
So if list y is used by thread 1 and thread 2, and y.append is used, or del y[n] is used, must one put a MUTEX Lock on the list to assure that the list alteration is done without a thread switch?
What about this one:
thread 1 does a for x in y where y is a list.
thread 2 alters list y in the midst of thread 1’s for loop. Is this an example of a need for a MUTEX lock?
(apologies: I’m a newbie to Python, not newbie to RTOSes in embedded).
Yes, you always need to use at least a Lock or RLock object.
Do not even try to ‘guess’ what operations are atomic. plus look at the ‘try: except: finally:’ to make sure a thrown exception frees up the lock when your routine ends/aborts.
Another solution is to pass objects via a Queue object, so only 1 thread owns the object at a time. So thread #1 works on the object, sends it to thread 2 which has been waitign for it. Thread #2 does some work & sends it back.
Thank you. The confusion is that most experts posting on python.org and other places say that in-place object alterations such as list.append are atomic. A few say it “may not be”, depending on the specific interpreter.
Therefore, for now, I use Lock() for altering lists in my (first) threaded program.
Now, I think I know the answer to this:
Thread 1 does: for x in y, where y is a list. In that for loop are some socket transmissions that take time, may block.
Thread 2 may (use locks) and alter the list that thread 1 is iterating. The list contains references to class objects. Thread 2 can remove the 0th list item or append to the list. What happens to the integrity of the iterations that thread 1 is doing if thread 2 alters the same list while thread 1 is suspended due to I/O within the loop?
This part of my program wouldn’t benefit from a queue, because thread 1 is iterating the list (or queue) and reporting what objects are in the list (or queue) at this moment. This report is done at regular time intervals.
Taking a copy of the list to iterate on (a snapshot if you will), wouldn’t work either, it seems, since the each item in the list (or queue) is a reference to a class object that may get deleted at any time by thread 2.
I know how to manage these things in when using C/C++, but I’m struggling with Python’s interpreter and the vagueness of what happens inside the libraries and preemption that’s in the thread switching.
A Windows/Linux PC is ‘big resource’, so threads are really threads. Running 2 Python threads gives you 2 completely unrelated contexts which follow OS rules.
But the Digi boxes are ‘small resource’, so for example there is only a single Python instance to limit RAM size. So 2 threads are a mix, but not so clear cut. they share some things, don’t share other things. Therefore the rules of context switching may not behave like on a PC.
