Every now and then I double check that our view type annotations are going to be compatible with future Python versions.
Good news! They will be.
In fact, it looks like runtime support for type introspection is more or less guaranteed for the near future (including Python 4). There were also a bunch of neat things which I thought others might be interested in:
In PEP 557 a new decorator has been added called
dataclass which uses the type annotation to dynamically generate a class definition with common “dunder” methods like
__init__. This was actually referenced as a reason in some other typing-related PEPs to preserve the ability to introspect types at runtime. If you think about it, our transformers are very similar in concept to this new decorator, in that they add boilerplate code dynamically to transform data between formats/objects.
@dataclass classes themselves would be a nicer way to handle our lists/dicts of
<Thing>Record which we have so many of inside the SDK (right now they are all
In PEP 568 context variables are an async analogue to
threading.local. This could be a good way to implement
ctx configuration (used behind the scenes) and warrants a deeper look.
A more dramatic change in PEP 563 makes
__annotations__ a string if you import
from __future__ import annotations. This will become default behavior in Python 4. This might require some changes in our codebase, but it is ultimately entirely compatible with what we are doing. The worst that would happen is we would use
typing.parse_type_hints to eval the strings in the
__annotations__ dictionary into classes once more (according to the PEP, we’re supposed to be doing that right now, oops). Incidentally, our transformation API technically does exact matching, so we would really only be impacted if we wanted to do better things with the
Speaking of which! In PEP 560,
typing is overhauled to use
__mro_entries__ which means we will be able to pickle these annotations across processes (this is what is currently preventing us from doing nice things with it). That means in the future, instead of creating transformers to something like
_PlotQualView so that you can accept disparate semantic types, you could say something like
Union[FormatA, FormatB] and just check at runtime which one you’ve received (and write 0 new transformers).
All in all, it may be worth jumping straight from Python 3.5 to Python 3.7 (and pinning that as minimum) as some of these changes would significantly improve how the framework operates.
Right now 3.7 isn’t around on conda , but it’ll be there someday soon I’m sure .