Memory usage of Dia - paring it down

I’m trying to get my Dia app to run reliably on a X2. At bootup there is ~2800KB of memory. After Dia starts there is around 700KB and after my device driver starts going it drops to around ~330KB.

If I run a basic presentation it drops to below 200KB and then I start to have memory errors and then it freezes up.

I’ve started to comment out things I’m not using, such as presentations, channels etc… and now when my Driver starts I have ~700KB, but once it sends to the server using urllib it drops to ~380 – an improvement, but I’d like more room.

Anyone else tried to run a minimal version of Dia?

Turning off HTTP, SNMP, NTP etc… seem to have negligible effect on lowering memory usage.

Any other suggestions on how to lower startup memory usage?

Thanks!

One option: dig around in the make.py & tools.

In theory there is a compiler option to strip doc-strings out of the PYC. Results vary, but I’ve seen up to 45% drop in ZIP file size by stripping doc-strings from other non-Dia projects, and I know Dia has been gaining new doc-strings rapidly.

I have NOT looked into how to enable this in Dia since I am working on the big X4H with 32MB RAM (so starts with 18MB free).

If, when using make.py, you use the ‘-OO’ option to the Python interpreter, the resulting .pyo files in the zip will not have docstrings. This can indeed make a meaningful difference in the installed footprint. How much still depends on what is being imported at runtime.

Thanks for the suggestions. I rain the python2.4 -OO make.py and it dropped the zip by 78KB. When running in Dia the memory footprint is down to 444KB instead of 380KB. That is a nice gain from a simple change.

looking through the files in the dia.zip, there are some that I don’t use. For example:

email/Generator.pyc
email/quopriMIME.pyc
email/Iterators.pyc
email/Utils.pyc
plus a bunch more in the email dir

encodings/ascii.pyc

does anyone know why the email stuff is included? If I delete them from the zip, my driver seems to run fine. Would be nice to know where they are imported to comment them out.

Also, there is support for XML:
xml/parsers/expat.pyc

I don’t use xml, so I think I can take that as well. I am thinking of using iDigi – does it require these files?

Thanks again for the -OO tip.

Many of the files (incl. those you listed) that get added to the dia.zip, are there not because they are directly called out, but because they are the result of the module dependency calculations.

The way in which the modules are identified for inclusion in that archive is pretty simple, but because of that does identify things that are not guaranteed to be needed by the system. It looks at the import calls that it finds while scanning the Python code and adds those files. We then prune the resulting list to remove files we know should not be included. That does leave a number of modules that may never be imported but could be due to un-executed paths in the system.

Until the module is imported, it consumes space in the ZIP file in flash, but won’t be pulled into memory. So, you won’t be taking on added RAM foot print due to these modules.

RE: the email modules. If you have the flash, I would leave them in. However, if your testing shows the system to be functional it’s likely to be safe to remove them.

I would leave the encoding. Encodings are a special type of module to the Python interpreter, and this one is explicitly added so that certain portions of the system will work without needed to be kludged around.

If you’re not doing anything with XML, you should be able to get rid of expat.pyc. However, once again, it’s only a ZIP file size burden until used. The iDigi code does do XML, but it generates it through templated forms and doesn’t use the parser, so shouldn’t call out expat. The RCI presentation is a common way for XML use to enter the system.

It removed the email files and so far so good. As you mention, the memory footprint didn’t really change, but the file size did. Since I’m using the flash to store log files when there is no connectivity, the more flash I can save the better.

It would be nice to be able to remove other files off the flash (like the web interface html files etc…) Anyone know how to do this?

Also, if you turn off the web server, what is the command line way to turn it back on?
I couldn’t see that in the docs, so did it via iDigi.

I’d much rather have an X2 version which ran ssh and got rid of the web interface.

The ‘set service’ command allows you to turn on and off network services on the X2, including HTTP and HTTPS.

RE: (removing the web interface html files etc…)

The normal Digi web interface is dynamically generated from the EOS or software image. There is no way to remove it, and even if you did it remove it, you would free up space in a special area NOT directly linked to your Python file area.

Thanks, but when I check “set service ?” I get:
#> set service

syntax: set service [options…]

options:

range=(1 to 11)
state=(on|off)
ipport=(network port)
keepalive=(on|off)
nodelay=(on|off)
delayed_ack=(0 - 1000)
reduced_buffer=(on|off)

as options. I don’t see HTTP/HTTPS.

Right, with ‘set service ?’, you’re listing the command usage. If you just do a ‘set service’ or ‘show service’ the command will display the table that you modify with that usage information. On my system HTTP is index 5 and HTTPS is index 4, so I can do a

set service ra=4,5 state=on

to turn those services on.