Since version 2.6, Python has done basic "constant folding" - the replacement of expressions that involve only constant values with their resulting value, rather than recalculating them every time the code is run. This is an even more valuable feature in a resource-limited environment like SNAPpy, but unfortunately Python doesn't go quite as far as one might hope:
* The only things recognized as constants for this purpose are literal values. Global variables that are never reassigned are considered as constants by SNAPpy (the 64 global variable limit would be unbearably restrictive otherwise), but Python doesn't apply constant folding to them. (Python really has no choice here, because it can never prove that a global variable cannot be reassigned. SNAPpy lacks the dynamic mechanisms that would make indirect reassignment possible.)
* Only operators applied to constants get folded - there's no possibility of folding a function call with constant inputs, even if it has no side-effects. (Again, Python has no choice - it can never be sure that the function apparently being called will be the one actually found at runtime).
These limitations were severely bloating the code on a current project of mine, so I wrote a little post-processor that rewrites the bytecode of your functions to apply various optimizations. To use it:
1. Download the attached files: foldy.py (my code) and byteplay.py (a dependency, not written by me), and place them in your snappyImages folder.
2. At the top of your main script, and the top of each imported file that needs to use any of the function decorators described here, place the line:
from foldy import *
3. At the very bottom of your main script, add the line:
fold_constants(globals(), False)
(change False to True to get verbose descriptions of the optimizations being performed in the event log)
With just those changes, you get the following benefits:
* Python's constant folding is extended to handle globals that are never reassigned, and the five built-in functions for which folding makes sense (int, len, ord, str, c_h_r).
* Functions with names starting with an underscore, that aren't actually called in your script, are deleted. There's no point in including them, since RPCs (or otherwise calling a function by name) cannot access them.
You can request further optimization by placing one of the following decorators above your function definitions:
@constfunc
Makes calls to this function, in which all parameters are themselves constant, eligible for constant folding. You are asserting that the function has no side-effects other than its return value, and that it uses only functions that will work in both the Python and SNAPpy environments (since it will actually be called at script compile time). Furthermore, the function itself will be deleted if there are no remaining runtime calls to it after optimization (which includes the possibility of there being no calls to it in the first place).
@constfunc_keep
Makes the function eligible for constant folding as described above, but unconditionally keeps it in the final script so that you can RPC it.
@nokeep
Makes the function eligible for deletion if there are no calls to it. This would be useful for library functions that aren't of interest to all users of the library. Note that functions with names starting with an underscore are automatically @nokeep.
@consolidate
This can be applied to a function with a single string parameter, that returns nothing. Consecutive calls to this function, with the strings all being (possibly folded) constants, get combined into a single call with the concatenation of those strings (as long as that wouldn't put it over the 255 byte limit on SNAPpy string constants). This met a specific need I had, and is likely not useful to you.
A bit of background: this was inspired by a project involving a library for talking to a LCD display, that had dozens of little functions that take the parameters for a specific display function and build the actual binary command string to be sent via SPI. Here's one of the more complicated ones:
(As usual, c_h_r substituted for The Function Which Shall Not Be Named)
These functions are quite commonly called with constant parameters, so all of that bit-thrashing simply ends up producing a 4-character constant string. Writing that constant directly in the script would be vastly more efficient, but of course it would make the script incomprehensible and uneditable. With foldy.py, I get the best of both worlds - applying @constfunc to these functions, and @consolidate to the function that actually sends the commands to the LCD, I can have all of the parameters written out in the source code, yet the script image will contain nothing more than the precompiled command. On a test script using this library, that nearly filled a RF100 module, I saved about 6% script space from the constant folding alone, 31% including the removal of unneeded functions.
Please note that this is deep magic, relies on undocumented internal details of Portal, and has had about one day of testing at this point. If you use it, consider the possibility that your script might not be doing exactly what you wrote anymore.
* The only things recognized as constants for this purpose are literal values. Global variables that are never reassigned are considered as constants by SNAPpy (the 64 global variable limit would be unbearably restrictive otherwise), but Python doesn't apply constant folding to them. (Python really has no choice here, because it can never prove that a global variable cannot be reassigned. SNAPpy lacks the dynamic mechanisms that would make indirect reassignment possible.)
* Only operators applied to constants get folded - there's no possibility of folding a function call with constant inputs, even if it has no side-effects. (Again, Python has no choice - it can never be sure that the function apparently being called will be the one actually found at runtime).
These limitations were severely bloating the code on a current project of mine, so I wrote a little post-processor that rewrites the bytecode of your functions to apply various optimizations. To use it:
1. Download the attached files: foldy.py (my code) and byteplay.py (a dependency, not written by me), and place them in your snappyImages folder.
2. At the top of your main script, and the top of each imported file that needs to use any of the function decorators described here, place the line:
from foldy import *
3. At the very bottom of your main script, add the line:
fold_constants(globals(), False)
(change False to True to get verbose descriptions of the optimizations being performed in the event log)
With just those changes, you get the following benefits:
* Python's constant folding is extended to handle globals that are never reassigned, and the five built-in functions for which folding makes sense (int, len, ord, str, c_h_r).
* Functions with names starting with an underscore, that aren't actually called in your script, are deleted. There's no point in including them, since RPCs (or otherwise calling a function by name) cannot access them.
You can request further optimization by placing one of the following decorators above your function definitions:
@constfunc
Makes calls to this function, in which all parameters are themselves constant, eligible for constant folding. You are asserting that the function has no side-effects other than its return value, and that it uses only functions that will work in both the Python and SNAPpy environments (since it will actually be called at script compile time). Furthermore, the function itself will be deleted if there are no remaining runtime calls to it after optimization (which includes the possibility of there being no calls to it in the first place).
@constfunc_keep
Makes the function eligible for constant folding as described above, but unconditionally keeps it in the final script so that you can RPC it.
@nokeep
Makes the function eligible for deletion if there are no calls to it. This would be useful for library functions that aren't of interest to all users of the library. Note that functions with names starting with an underscore are automatically @nokeep.
@consolidate
This can be applied to a function with a single string parameter, that returns nothing. Consecutive calls to this function, with the strings all being (possibly folded) constants, get combined into a single call with the concatenation of those strings (as long as that wouldn't put it over the 255 byte limit on SNAPpy string constants). This met a specific need I had, and is likely not useful to you.
A bit of background: this was inspired by a project involving a library for talking to a LCD display, that had dozens of little functions that take the parameters for a specific display function and build the actual binary command string to be sent via SPI. Here's one of the more complicated ones:
Code:
def Vertex2ii(x, y, handle, cell):
return c_h_r(cell | ((handle << 7) & 0xFF)) + c_h_r(handle >> 1 | ((y << 4) & 0xFF)) + \
c_h_r(y >> 4 | ((x << 5) & 0xFF)) + c_h_r(x >> 3 | 0x80)
These functions are quite commonly called with constant parameters, so all of that bit-thrashing simply ends up producing a 4-character constant string. Writing that constant directly in the script would be vastly more efficient, but of course it would make the script incomprehensible and uneditable. With foldy.py, I get the best of both worlds - applying @constfunc to these functions, and @consolidate to the function that actually sends the commands to the LCD, I can have all of the parameters written out in the source code, yet the script image will contain nothing more than the precompiled command. On a test script using this library, that nearly filled a RF100 module, I saved about 6% script space from the constant folding alone, 31% including the removal of unneeded functions.
Please note that this is deep magic, relies on undocumented internal details of Portal, and has had about one day of testing at this point. If you use it, consider the possibility that your script might not be doing exactly what you wrote anymore.
Comment