How is a referenced library module included in the python ZIP file ?

Python
1

I have been diving into both the IDigi Python and Dia samples for about a week now using the ESP IDE. When I first started it seemed like all the required libraries and modules were automatically pulled and placed into the /src/libs directory which then was included in the corresponding zip file that is transferred to the physical digi device along with the python script.

Now I am experiencing some problems when building and modifying new projects. Either some ESP build configuration has changed or I dont really understand the build mechanism … probably the latter :slight_smile:

Questions:

  1. if a library module is referenced within a python script (digi_data for example), will the ESP automatically place it into the src/lib folder and include it in the zip? Or does one have to find it in the python/digi tree and manually copy it over?

  2. assuming the library module IS in the /src/lib folder and is so referenced within the python script, will it automatically get included in the zip file?

  3. do such modules have to be in the zip file, or can they be manually placed in the python folder on the physical digi device?

I had assumed that all of this was handled magically by the ESP and python build, but am now finding that it is not always. Perhaps this is a product of me moving from the predefined samples and demos. How should it work?

  • Gary
2

Hello Gary,

I’ll try to explain you the build process of the Digi ESP for both, Python and Dia projects.

Digi Python Application projects.

The src folder of the Python project contains the main module of the project, which is automatically generated. This folder can contain more Python modules used by your application, or Python packages, which are standard folders with a dummy init.py file. Inside that or those Python packages you can add more Python modules to be used by your application. In addition, your Python application can make use standard Python modules (such as time, thread, etc.) without needing to copy them anywhere.

The build process of Python Application projects will look for all the required modules of your application inside your src folder and the standard Python ones. The process will zip them (if needed) inside a file that will be uploaded to the remote device when the application is launched. You don’t need to look and copy anything, the build process will take care of that.

The remote device contains a zip file named python.zip. That file includes several standard Python modules to be used by any Python or Dia application. Because of that, the build process won’t add a module to the zip file if it is already contained inside the python.zip one.

iDigi Dia projects.

The build process of these kind of projects is pretty similar to the Python application ones. Apart from the described in the previous topic, the build process of Dia projects will also look for required Python modules inside the lib and src source folders of the iDigi Dia, which is located at \Digi\DigiPython\Dia\Dia_1.3.8. It will also look for required modules in the custom_devices, custom_presentations, custom_loggers and custom_services source folders of the Dia project.

All the required modules will be zipped in the dia.zip file when building the project.

So, basically, yes, the Digi ESP magically finds and copies inside a zip file all the Python modules required by your Python or Dia project to run in a remote Digi device.

Answers:

  1. The Digi ESP won’t copy the required module anywhere, it will include the module inside the generated zip file.

  2. A module located in the /src/lib folder will be included in the zip file only it is required by the application to run (it is imported anywhere in the code)

  3. They are placed in a zip folder because the Digi ConnectPort devices don’t have filesystem, zip files are cached at startup and they maintain the folders structure for the imports in a virtual manner.

Hope it helps.

Regards.

3

Ok, what kind of project are you developing?

If you are creating a Digi Python Application project, then the build process won’t look for Python modules in the Dia source path. The idigi_data module is located in the Dia source path, so it will be only auto discovered by iDigi Dia projects. If you want to use it in your Python Application project you will need to copy that module in your project source folder, or inside a Python package located in your source folder. Then you will be able to use it.

Regards.

4

For some reason, this stopped working a few days ago. I had created a new project that required the idigi_data() command and it was not included in the zip and was not available at run-time on the digi device.

I had run previous projects with it without issue.

When I cheated and used the zip file from one of those projects on the target, it ran.

I will have to attempt to recreate.

5

Yes, this still happened to me this morning. A module imported

I cut and pasted the last three imports to a python script below from the idigi_stats demo along with a snuppet of its code. A run-time error was given when the idigi_data file could not be found. Once I manually moved a copy into the projects/src/lib it evidently did place it into the zip file and the script ran on the digi target.

python script

import sys, os
import time
from libs.localain import CURRENTLOOP, TENV, LocalAIN
import rci
import cStringIO
from libs import idigi_data

  try:
    success, error, errmsg= idigi_data.send_idigi_data (data, filename, \
                                                    collection,
                                                        secure=False)
    # validate upload
    if success:
        print "Successful upload.

"
else:
print "Failed upload.
"
except:
print "Failed upload.
"

Digi Run-time

#> python dpdsrv.py
Launching Python application ‘local_ain_voltage3.py’ …

Traceback (most recent call last):
File “WEB/python/dpdsrv.py”, line 17, in ?
execfile(os.path.join(‘WEB/python’, ‘local_ain_voltage3.py’))
File “WEB/python/local_ain_voltage3.py”, line 48, in ?
import idigi_data
ImportError: No module named idigi_data
#>

6

Python. I am combining derivatives of the Digi “idigi_stats” and “local_ain” samples.

I had assumed that they both had created their own libs. Based on your post I now know to manually copy the libraries of the parent projects over by hand.

Makes sense now that I know what caused it.

Thanks