packing.md

Python's Star Operator

A feature that packs a punch (pun intended)

Introduction

You have probably seen the * operator in something like this:

def f(*args, **kwargs):
    pass

I'll start somewhere else completely, and we'll build our understanding up to the function arguments.

The * operator is a.k.a.

• packing/unpacking
• spreading
• destructuring
• variadic arguments (for Python function arguments specifically)
• the ... operator in JavaScript (I'll also give examples in JavaScript)

Different terms, different operators, sometimes implicit... but really one concept. I'll try to showcase it my own words, hopefully connecting some dots and giving you ideas on where to use it.

Let's start simple

You have also probably seen two variables declared on a single line, like this:

Python	JavaScript
first, second = 1, 2	[first, second] = [1, 2]
It is particularly cool to switch the values of two variables, without the need of a third (temporary) variable:
first, second = second, first	[first, second] = [second, first]
That hints at the fact there is some magic going on. Another clue is that if the number of items doesn't match on both sides of the = operator, you get an error like "too many values to unpack (expected 2)". That's it: packing and unpacking. In Python, the comma operator implicitly instantiates a tuple. This is what happened here on the right side of the = operator. On the left side, we are unpacking the tuple into two variables. The JavaScript syntax that I used to get a similar result is a lot more explicit, with the array brackets. So, we can generalize that assigning an iterable to a tuple of variables will implicitly unpack the values into the respective variables. I say "assigning" in general, like for the = operator, but also for what happens for example to k, v in for k, v in ..., or for function arguments when you call a function.
first, second = my_iterable	[first, second] = my_array
Another way to unpack an iterable, is explicitly using the * operator (or ... in JavaScript):
my_tuple = first, *items	my_array = [first, ...items]
This is equivalent to enumerating all the items, but without having to know how many items there are.
# pseudocode my_tuple = first, items[0], items[1], ..., items[n]	// pseudocode my_array = [first, items[0], items[1], ..., items[n]]
And this syntax is symmetric! Earlier, I mentioned that the number of items on both sides has to match, otherwise there are "too many values to unpack", but there are ways around this. You can re-pack the rest of the items in a new tuple.
first, *rest = my_iterable	[first, ...rest] = my_array
This is equivalent to enumerating all the "cells" where we want the items to go.
# pseudocode first, rest[0], rest[1], ..., rest[n] = my_iterable	// pseudocode [first, rest[0], rest[1], ..., rest[n]] = my_array
In Python, you can also ignore the rest with the variable _ (by convention). JavaScript doesn't complain if there are "too many values to unpack", so it's not even necessary to save the rest in a variable.
first, *_ = my_iterable	[first] = my_array
Obviously, you don't need that fancy syntax to access the first element of an iterable, but it comes in handy in more complex situations like follows. I find JavaScript remarkably straightforward here.
first, _, third, *_ = my_iterable	[first, , third] = my_array
You get an error if you write "multiple starred expressions in assignment". For example, in *_, a, *_ = my_iterable, Python wouldn't know which item to assign to a. But, as long you have a single *, Python accepts it anywhere; at the beginning, at the end, or in the middle.
first, *_, second_last, last = my_iterable	😢
Unfortunately, in JavaScript, the "Rest element must be last element". It can't be anywhere else. To work around that and unpack the end of the array, you can slice it first, like in Python.
second_last, last = my_iterable[-2:]	[second_last, last] = my_array.slice(-2)
Back to function arguments! Positional arguments work just like in a tuple.
def f(first, *args): pass	function f(first, ...args) { }
This can be viewed as packing all the remaining arguments in one args tuple/array.
# pseudocode def f(first, args[0], args[1], ..., args[n]): pass	// pseudocode function f(first, args[0], args[1], ..., args[n]) { }
This is still symmetric, so you can also unpack arguments, just like instantiating a tuple.
f(first, *args)	f(first, ...args)
Double up! In Python, the ** operator is very similar to the * operator, but for dictionaries instead of tuples. In fact, JavaScript uses the same ... operator for both objects and arrays. This can be used to unpack a dictionary/object into a new one.
my_dict = { "foo": 42, "bar": 5, **another_dict }	my_object = { foo: 42, bar: 5, ...another_object }
Only JavaScript also supports unpacking objects into multiple variables (on the left side of the assignment). That's awesome!
😭	{ foo, bar, ...rest } = my_object // Equivalent to foo = my_object.foo bar = my_object.bar // pseudocode rest = { key0: my_object.key0, key1: my_object.key1, ..., keyN: my_object.keyN }
Python has a special syntax for functions' keyword arguments (a.k.a. named arguments).
f(foo=42, bar=5)	🤔
You can think of this as passing a single dictionary of arguments to the function, for example dict(foo=42, bar=5). In the function, you can then unpack this dictionary, grab foo, and re-pack the rest of the keyword arguments in **kwargs. That's exactly what I did here in Javascript to get the same result.
def f(foo, **kwargs): print(foo, kwargs) f(foo=42, bar=5) # 42 {'bar': 5}	function f({ foo, ...kwargs }) { console.log(foo, kwargs); } f({ foo: 42, bar: 5 }) // 42 {'bar': 5}
And of course positional arguments and keyword arguments can be combined.
def f(first, *args, kwarg1, kwarg2, **kwargs): print(first, args, kwarg1, kwarg2, kwargs) f(1, 2, 3, kwarg1=42, kwarg2=5, kwarg3=0) # 1 (2, 3) 42 5 {'kwarg3': 0}	function f({ kwarg1, kwarg2, ...kwargs }, first, ...args) { console.log(first, args, kwarg1, kwarg2, kwargs); } f({ kwarg1: 42, kwarg2: 5, kwarg3: 0 }, 1, 2, 3) // 1 [2, 3] 42 5 {'kwarg3': 0}

In Python, you pass positional arguments first, then the keyword arguments.

So, all arguments defined after the *args have to be passed as keyword arguments (those are called keyword-only arguments). That can be leveraged, for example, to avoid ambiguity between two arguments. Bear with me, I'll give an example in a moment.

If you don't need the *args, you could use

• *_ like before to pack additional positional arguments into oblivion and ignore them; or
• * alone to forbid additional positional arguments. This is a special syntax specific to function arguments. You can think of it as packing additional positional arguments into... well, there is no variable to collect them, so it raises an exception.

For example, the following assert_privileges() function has two arguments of the same type, actual and expected, but the > operator used inside the function is not commutative, so it is important that the caller doesn't flip the arguments. Therefore, I used the * to force callers to specify expected as a keyword argument.

def assert_privileges(actual: Privileges, *, expected: Privileges):
    if PRIVILEGES_DESCENDING_ORDER.index(actual) > PRIVILEGES_DESCENDING_ORDER.index(expected):
        raise HTTPException

# So that works:
assert_privileges(actual=privileges, expected=expected_privileges)
# Or at least that:
assert_privileges(privileges, expected=expected_privileges)

# But that doesn't work:
assert_privileges(privileges, expected_privileges)
# TypeError: assert_privileges() takes 1 positional argument but 2 were given
assert_privileges(expected_privileges, privileges)
# TypeError: assert_privileges() takes 1 positional argument but 2 were given
assert_privileges(expected_privileges)
# TypeError: assert_privileges() missing 1 required keyword-only argument: 'expected'

Conclusion

The * operator is quite versatile, although quite simple once you grasp it. I hope that helped.

Next time you're extracting items from iterables, aim for the stars!