nabla.py - limited generator support for SNAPpy

[jasonharper]

SNAPpy's single-threaded nature, and the limited amount of buffer space, mean that your functions often cannot do everything they need to do within the timespan of a single call - they have to do a little bit of work, and set up global variables so that they know to do the next bit of work sometime in the future, an approach known as a 'state machine'. However, writing code in this style gets increasingly annoying as the complexity increases. Case in point - a current project of mine requires displaying a bunch of messages sequentially on a 1-line text LCD. The bulk of the messages are generated by two nested loops; there's another loop that might be empty, and a couple of individual messages that might be conditionally shown. Each call to the 1-second timer hook needs to generate exactly one of these messages, since I cannot block execution for anywhere near a full second. After repeatedly screwing up every attempt to write this function, I finally decided to look for a better approach. After all, this is an entirely solved problem in regular Python - something like this could trivially be written as a generator function, calling yield whenever it needs to pause for a while. So, what I've done here is a bytecode hack that transforms a normal Python generator function into something compatible with the SNAPpy runtime environment. In short, it does this by splitting the function into chunks at the yield points, then replacing the original function with a little driver function that calls one or more of the chunks as needed (using a global variable to keep track of which chunk is next).

This necessarily works a bit different than generators in full Python, since there is no possibility of user-defined iterators in SNAPpy. Instead, the generator function itself acts as an iterator: you just call it repeatedly, each call runs until the next yield and then returns the specified value (None by default). If execution falls off the end of the function, or you do an explicit return, it will yield None, and execution will restart from the top on the next call.

Here's a simple demonstration - it will print numbers 1 thru 9 repeatedly, one number per second, without any blocking.

@setHook(HOOK_1S)
def tick(ms):
    i = 0
    while True:
        i += 1
        if i == 10:
            break
        print i
        yield
             
import nabla
nabla.transform(globals())

Those last two lines have to go in your main script, after any generator functions (which are simply any functions containing a yield).

Further limitations and details:

1. This is the biggie - for loops do NOT work in generator functions, and I see no way at this time to fix it. The specific problem is an "incompatible type" error when storing the loop's iterator object in a global variable, so that it can be accessed from another function chunk. These objects are normally never stored in a variable at all (they exist only in the evaluation stack), so I guess Synapse took some shortcut in their implementation based on that assumption. Just use while loops for now - after all, that's the only kind of loop we had prior to the 2.6 firmware.
2. Generator functions cannot use parameters - it's unclear what they'd even mean on the subsequent calls to the function, as opposed to the initial call. However, since it would be very handy to use a generator function as a timer or other hook function (which automatically get passed a parameter by the system), I do allow parameters to be declared - you just can't access them.
3. yield is only supported as a statement, not as an expression that can receive values sent in from outside the function. (Expressions will in general involve other values on the evaluation stack; there's no practical way to transfer these values between the chunks of the original function.)
4. Since the generator function itself acts as an iterator, you cannot have multiple instances of a single generator active at once. In particular, a generator absolutely cannot call itself recursively. (Multiple instances of a generator would imply multiple sets of variables that it could access. Nothing with such capability currently exists in SNAPpy.)
5. Any local variable that is used in more than one of the chunks automatically gets promoted to a global variable, as that's the only way to keep them around between calls. This has some consequences:
   • This could push your script over the global variable limit.
   • This could keep string buffers in use a lot longer than you expect. Consider del (or assignment with None) of any local variable containing a dynamic string, if it's not going to be reassigned soon.
   • If you reuse the same variable name in multiple chunks, but each use fits entirely within a chunk, the promotion to global is unnecessary (but would be enormously difficult to detect this situation). Consider using distinct variable names for each distinct use.
   
   
6. The transformation generates a bunch of global variables with names based on the generator function's name. There is currently no check for accidental use of these names elsewhere in your script, which would likely be disastrous - you just need to be careful to avoid them.
   • funcname_ is the state variable containing the index of the next chunk to call. It will be 0 if the generator has never been called, or has reached the end and will restart on the next call; setting it to 0 (from outside the generator only!) will force it to restart from the top on the next call. (No other manipulation is valid.)
   • funcname_0 is the tuple of chunk functions.
   • funcname_1, _2, etc. are the variables holding for loop iterators, the number being the nesting level. (Not relevant, since these loops do not work.)
   • funcname_varname are local variables that have been promoted to global.
   • _funcname_number are the individual chunk functions. These are never called by name (the funcname_0 tuple is used instead), but for some reason SNAPpy functions won't work unless there is a global name referring to them. The leading underscore makes the name private, you'd certainly never want to RPC one of the chunks directly!
   
   

To install, copy the two attached files into your snappyImages folder. If you've already downloaded my foldy.py script, you will already have byteplay.py - the file hasn't changed.

[berwyn]

Cool! We could have used this any number of times to replace state machines.