Enable ++x and --x expressions in Python
By default, Python supports neither pre-increments (like ++x) nor post-increments (like x++), commonly used in other languages. However, the first ones are syntactically correct since Python parses them as two subsequent +x operations, where + is the unary plus operator (same with --x and the unary minus). They both have no effect, since in practice -(-x) == +(+x) == x.
I'd like to share the plusplus module that turns the ++x-like expressions into x += 1 at the bytecode level, using pure Python only.
Unlike x += 1, ++x is still an expression, so the increments work fine inside other expressions, if/while conditions, lambda functions, and list/dict comprehensions:
array[++index] = new_value
if --connection.num_users == 0:
connection.close()
button.add_click_callback(lambda: ++counter)
index = 0
indexed_cells = {++index: cell for row in table for cell in row}
Note: I don't claim that allowing increments is good for real projects (it may confuse new developers and give opportunities to write less readable code), though some situations when they simplify the code do exist. I've made this module for fun, as a demonstration of Python flexibility and bytecode manipulation techniques.
The module works by replacing the bytecode patterns corresponding to the ++x and --x expressions with the bytecode for actual incrementing. For example, this is what happens for the y = ++x line:
It's not always that simple: incrementing object attributes and collection items requires much trickier bytecode manipulation (see the "How it works" section in the docs for details).
To use the module, you can just run pip install plusplus and add two lines of code enabling the increments. You may do this for just one function or for the whole package you're working on (see the "How to use it?" section).
Updates:
- The same approach could be used to implement the assignment expressions for the Python versions that don't support them. For example, we could replace the
x <-- valueexpressions (two unary minuses + one comparison) with actual assignments (settingxtovalue). - See also cpmoptimize - my older project about Python bytecode manipulation. It optimizes loops calculating linear recurrences, reducing their time complexity from O(n) to O(log n). The source code is available on GitHub as well.
83 comments · 96% upvoted · other discussions
I hate both the idea of using the unary operator and the ability to manipulate the behavior of the bytecode.
This is the most interesting post I've seen on this sub in a while.
13steinj 36 points · Sep 2021
For obvious reasons this just shouldn't be used, as there's plans to bring some basic optimizations to Python.
[deleted] 16 points · Sep 2021
Do you have a link for this?
13steinj 47 points · Sep 2021
https://github.com/faster-cpython/ideas
By Guido and Mark.
End goal IIRC is 5x speedup.
[deleted] 23 points · Sep 2021
Thanks.
That's really overdue. Just by reading the bytecode we can spot so many obvious optimizations, good thing they finally decided to do it.
hx-zero [OP] 5 points · Sep 2021
Over the years, I saw many bytecode optimization projects doing things like constant folding, constant binding, etc. (e.g. astoptimizer, foldy.py, and others that I review in the end of this post). While the bytecode optimization could have provided a significant speedup, unfortunately, most of these projects are dead now.
I hope this will improve with the officially supported implementation.
SittingWave 2 points · Sep 2021
I remember there was a discussion somewhere about being able to alter the parser itself, basically to allow modules to extend the language.
to7m 1 points · Sep 2021
That would be cool. Hope they do it by making things like `while` and `[` objects which can be subclassed.
mindofmateo 1 points · Sep 2021
That would be nuckin' futs