Another Approach to Building Exercise Problems

Here recently I've been working with those in my developer book club group to get some of the others to be presenters, and with that we've gone through and covered chapters 2 and 3 of Effective Python with others from the group acting as presenters, and the two presenters took a different approach to coming up with the exercise problems than I did, and it worked pretty well, so I figured that I'd go over their approach a little bit here.

To start off, here's a small sampling of some of the exercise problems and their answers for the Effective Python chapter 2 presentation, as presented by Nathan Malloch:

from typing import List
from collections import defaultdict

print('create a list with 10 elements')
l = [4, 65, 34, 56, 78, 23, 54, 76, 653, 66]
print(l)

print('print the last 3')
print(l[-3:])

print('print all but the last 5')
print(l[:-5])

print('print the first 4')
print(l[:4])

print('make a referenced copy of the list and print them')
g = l
print(g)
print(l)

print('remove the last two indexes from the first list while also updating the second')
l[:] = l[:-2]
print(g)
print(l)

print('make a deep copy of the list, print both')
b = l[:]
print(b)
print(l)

print('remove 2 through 4 index of first list, but don\'t effect the second list')
l[2:5] = []
print(l)
print(b)

So here we see an approach where instead of giving a single, high level problem to solve, we are given problems that are much closer to the code and are very clearly tied to the principles taught in chapter 2.

I find that this approach and the high level approach can both work well in helping to emphasize and teach different things. This low level approach allows you to focus in on a specific language feature and test it in many different ways so as to get an in depth understanding of how that specific feature works, whereas in the high level approach, you aren't told specifically what tools or features to use, and choosing which features to use is part of the problem solving process.

This makes me think of an analogy in math. In your typical math textbook you will run into two types of problems. The first type are your standard problems, which will typically look something like this:

    6
   x3 
    

And then there's the story problem, which looks a bit more like this:

• A box of donuts has 6 donuts, and you have 3 boxes. How many donuts do you have?

Now with the first type of problem, solving is pretty straightforward. You're expected to practice your multiplication, like so:

    6
   x3 
   18

The second type of problem also involves setting up the problem, and the best setup would turn out the same:

    6
   x3 
   18

But that being said, it's quite valid for someone to choose to solve the problem in this way, even if it isn't quite ideal:

    6
    6
   +6 
   18

So with that, a story problem gives the opportunity to exercise your ability to choose the right tool for the job. On the flip side of the coin, though, standard practice problems give the ability to explore important concepts that don't easily fit into an nice little story, such as:

(2+i)(3-4i)=

These same concepts apply to high level and low level exercise problems for programming, where high level also exercises tool selection, and low level allows experimentation with principles that would be hard to fit into a concise story problem.

There is also another benefit to low level exercise problems for the presenter, which is that they are easier to come up with. There's no need to come up with a story, and no need to try and fit unrelated features into the same problem.

One word of caution when using this style, though. Because of how small and simple the problems are, it can be tempting to prepare more material than can be covered in a single meetup. Just as with the story problems, it's important that you don't try to teach everything from the chapter, but instead pick out a few of the more important principles to practice.

And it's important to expect that as you're presenting the problem, that people in the group will want to test and experiment with each of the problems along the way, seeing what happens with one tweak or another. It is important that there is time enough for this experimentation, because that's what's going to help it to stick in people's minds.

So with that, feel free to mix it up between high level and low level exercise problems. They both make for great ways to help solidify the ideas being taught.

Caveat for Optimizing PostgreSQL Migrations

In last week's post I wrote about minimizing ACCESS EXCLUSIVE lock time on a table when writing a migration, and gave the example of taking a migration that looked like this:

alter table people
add column if not exists guid varchar(50) default uuid_generate_v4() not null;

create index if not exists people_guid_index
    on people using btree(guid);

And changing it to this:

alter table people
add column if not exists guid varchar(50);

alter table people
alter column guid set default uuid_generate_v4();

update people set guid = uuid_generate_v4()
where guid is null;

alter table people
add constraint temp_null_check check ( guid is not null ) not valid;

alter table people
validate constraint temp_null_check;

alter table people
alter column guid set not null;

alter table people
drop constraint temp_null_check;

create index concurrently if not exists people_guid_index
    on people using btree(guid);

Because on our example data set the first version caused the people table to have 1 m 6 s 455 ms of ACCESS EXCLUSIVE lock time and 25 s 751 ms of SHARE lock time during the migration, which meant that for over a minute, no select, update, insert, or delete queries could be run against the people table, and then for nearly another 30 seconds following that select queries could run, but update, insert, and delete queries still could not. Alternatively, the second version resulted in 35 ms of ACCESS EXCLUSIVE lock time, 41 s 450 ms of SHARE UPDATE EXCLUSIVE lock time, and 1 m 46 s 636 ms of ROW EXCLUSIVE lock time, which meant that your select, update, insert, and delete queries would only be blocked from running for a total of 35 milliseconds.

Now one important caveat that I'd like to mention here is that this only matters if it's important to still be able to run queries against the table during the migration. This is typically important if you have a zero-downtime deployment pipeline in place. But I have worked at places where the deployment pipeline would intentionally take the servers down before running the migrations, and then bring them back up afterwards. Now, is this an ideal pipeline? No, not really. But many times in the real world we need to work in less than ideal situations. While it would be nice to fix the pipeline in such a situation, it is likely to take a lot of time and effort that may or may not be able to spare at the current time, and it is unrealistic to expect all other work to halt until the pipeline is addressed.

With that being said, which of the two versions of the migration given above would be better in the situation with the pipeline described? The first version. Why? Because at this point we don't care about the lock times. What we care about is how long the server will be down, and the first version's total run time is 1 m 32 s 206 ms, whereas the second version's total run time is 2 m 28 s 121 ms. As such, the second version would cause the downtime to be nearly a minute longer.

It is always important when optimizing to know which metrics are important to optimize to.

Minimize Access Exclusive Lock Time When Running a Migration in PostgreSQL

I recently ran into an issue where a data migration written for a PostgreSQL caused some issues because it locked up a table for a rather long period of time, preventing any other queries from being run against that table until the migration was finished. Because the table in question was one that was heavily relied on for core parts of the application, it caused a short but extensive outage.

Now, with the proper foresight, this situation can be avoided, but it requires that a developer be mindful of the type of lock any given query will cause, and for how long. As an example, let's create a similar situation. Let's say that we have the following table called people:

create table people (
  id serial primary key,
  first_name text,
  last_name text
);

And we want to create a guid column, because in the future we want to start moving away from using a sequence for our ids and instead move to using guids. To do this, it would seem to be straightforward to write this migration by building a simple alter table query, like this:

alter table people
add column if not exists guid varchar(50) default uuid_generate_v4() not null;

It seems simple enough. What could go wrong? A lot, as it turns out. And the more data you have in your people table, and the more that data is used, the worse the problems with this query become. To better understand this problem, we need to dive in to the Postgres documentation a bit. In particular we need to understand Table-Level Lock Modes. We can find documentation on this at https://www.postgresql.org/docs/12/explicit-locking.html.

According to the documentation, there are eight different table-level locks, moving from least restrictive to most restrictive in this order: ACCESS SHARE, ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE. An alter table query can have one of three different locks: SHARE UPDATE EXCLUSIVE, SHARE ROW EXCLUSIVE, or ACCESS EXCLUSIVE. Now can potentially have one of multiple different locks, it's always best to assume it's the most restrictive lock unless you've explicitly looked at the documentation and know that it is otherwise, and that rule would hold up in the case of the query above, because it does acquire an ACCESS EXCLUSIVE lock.

Now the important thing to understand about locks is what other lock types they conflict with. This in turn tells you what kind of queries will be locked out from accessing the table while the given query is running. In the case of ACCESS EXCLUSIVE, it conflicts with all other locks. Now this in and of itself isn't bad. There are a lot of important things that you wouldn't be able to reliably do for your database without the use of this kind of lock. Where it becomes bad is when a query that needs an ACCESS EXCLUSIVE lock is also a long running query, with the biggest issue caused by this being that it will block all of your standard CRUD operations, which includes all select, insert, update, or delete queries that need access to the table that has been locked.

So how does this apply to the example above? Well, let's find out. In order to test this, we'll want to have a large amount of data in our table. So after creating our table by running this:

create table people (
  id serial primary key,
  first_name text,
  last_name text
);

We can put a little bit of data into the table with the following query:

insert into people (first_name, last_name) values
  ('John', 'Doe'),
  ('Jane', 'Doe'),
  ('Bob', 'Smith'),
  ('Jill', 'Hill'),
  ('Jack', 'Hill');

And then we can turn that into a lot of data by running this query a number of times:

insert into people (first_name, last_name)
  select first_name, last_name
  from people;

Which essentially takes all the data in the table, copies it, and reinserts into the table, effectively doubling the size of the table each time it is run. By running it 20 times, your table will have over 5 million users, which should be enough to give us a clear idea of what's going on. At this point if I run the alter table query from above, it'll take 1 m 6 s 455 ms to complete (Note that your times will likely vary). So that is over a minute where the table is locked with an ACCESS EXCLUSIVE lock, meaning that no other queries can be run against it. This is a problem. Ideally any query that needs an ACCESS EXCLUSIVE lock should be running in the millisecond range, or at worst upwards of a couple of seconds. That's definitely not the case here. So let's figure out how to fix it. Let's start by reverting our table to its pre migration state by running the following query:

alter table people
drop column guid;

Now, the part of the query that is taking so long to run is the part that is trying to generate a new guid for all of the 5 million plus rows that already exist in our database. There isn't really a reason why we need to generate all of those guids as part of the alter table query, and that part of the migration could ultimately be moved out into an update query, which according to the documentation only needs a ROW EXCLUSIVE lock, which is much more permissive and will allow other select, insert, update, and delete queries to run while it is runnning.

So let's start out with modifying our alter table query to remove the default value for the time being. Note that by removing the default value we are also required to remove the not null constraint.

alter table people
add column if not exists guid varchar(50);

This pared down query runs in 14 ms and will give you a resulting table that looks something like this:

id	first_name	last_name	guid
1	'John'	'Doe'	NULL
2	'Jane'	'Doe'	NULL
3	'Bob'	'Smith'	NULL
4	'Jill'	'Hill'	NULL
5	'Jack'	'Hill'	NULL
...	...	...	...

At this point, we can add on our default value.

alter table people
alter column guid set default uuid_generate_v4();

This query runs in 4 ms, which is good, because this query also requires an ACCESS EXCLUSIVE lock. At this point our total ACCESS EXCLUSIVE lock time is 18 ms. The reason why this query is so quick is because it doesn't set the guid value for the 5 million plus existing rows, but instead ensures that newly inserted rows will have a guid. So for instance if at this point I ran the query:

insert into people (first_name, last_name) values
  ('Susie', 'Queue');

I'd end up with a table that looks something like this:

id	first_name	last_name	guid
1	'John'	'Doe'	NULL
2	'Jane'	'Doe'	NULL
3	'Bob'	'Smith'	NULL
4	'Jill'	'Hill'	NULL
5	'Jack'	'Hill'	NULL
...	...	...	...
5242881	'Susie'	'Queue'	'60f029c8-3818-43ba-bb0a-9a483a4c5529'

Now we can run an update query to add guids to all existing rows:

update people set guid = uuid_generate_v4()
where guid is null;

Note the conditional 'where guid is null'. This prevents us from overriding the guid values of any newly inserted rows, such as the Susie Queue row in the example above. Running this query takes 1 m 46 s 636 ms. Note that this takes longer than our original alter table query, but that's ok, because this is 1 m 46 s 636 ms of Row EXCLUSIVE lock time, which is a much more permissive lock that will still allow for select, insert, update, and delete queries to run. So at this point we have 18 ms of ACCESS EXCLUSIVE lock time, and 1 m 46 s 636 ms of ROW EXCLUSIVE lock time, with a table that looks like this:

id	first_name	last_name	guid
1	'John'	'Doe'	'cf8cd62a-5849-491a-a0b6-fd96dddb9562'
2	'Jane'	'Doe'	'ea9722ae-f4af-4d52-bea9-fd38691d37a7'
3	'Bob'	'Smith'	'a75d722c-217d-4a02-bb94-f4246fcf1a5c'
4	'Jill'	'Hill'	'ab09e9ca-75b9-4bf1-a3cf-b34cc8fe6965'
5	'Jack'	'Hill'	'14c31156-c57e-4250-93d1-110ba78d2b5d'
...	...	...	...
5242881	'Susie'	'Queue'	'60f029c8-3818-43ba-bb0a-9a483a4c5529'

But we're still not finished yet. We still need to add on our not null constraint. We can do this with the following query:

alter table people
alter column guid set not null;

This query took 6 s 290 ms to run, and requires an ACCESS EXCLUSIVE lock. This would bring our total ACCESS EXCLUSIVE lock time up to 6 s 308 ms. In many situations this is probably acceptable, and you could stop here. But with that being said, 6 seconds is still a long time to lock down all access to a table, especially if that table is critical to your application. So can we improve this and get the time down further? We can, but it requires diving deeper into the alter table documentation, which can be found at https://www.postgresql.org/docs/12/sql-altertable.html.

In the documentation for set not null, it says: "SET NOT NULL may only be applied to a column providing none of the records in the table contain a NULL value for the column. Ordinarily this is checked during the ALTER TABLE by scanning the entire table". This gives us the explanation of why the query takes so long. After it locks the table, it has to go through every single row and verify that there are no nulls before it can apply the constraint. Why does it need to do this? Simple, it's because while at this point you and I know that there are no nulls in any of the rows, Postgres has no way of knowing this. This is because at this current point it would still be completely valid to run a query such as:

insert into people (first_name, last_name, guid) values
  ('Joe', 'Johnson', null);

Or like:

update people set guid = null
where id = 1

While you and I know that no such query has been run, Postgres has no way of knowing that for certain, hence the need to check every single row to verify that there are no nulls. But in the very next line of the documentation it gives us a way in which this very expensive scan can be skipped. It says: "however, if a valid CHECK constraint is found which proves no NULL can exist, then the table scan is skipped." So at this point we'll want to take a closer look at the documentation for the check constraint.

As we look over the documentation for the check constraint, we know that it will also require an ACCESS EXCLUSIVE lock, because there is no explicit documentation stating otherwise, and at the beginning of the alter table documentation it states that "[a]n ACCESS EXCLUSIVE lock is held unless explicitly noted." So with that in mind, let's take a closer look at what the documentation has to say about how the check constraint works. It states: "Normally, this form will cause a scan of the table to verify that all existing rows in the table satisfy the new constraint. But if the NOT VALID option is used, this potentially-lengthy scan is skipped." Furthermore, it states that: "The constraint will still be enforced against subsequent inserts or updates (that is [...] they'll fail unless the new row matches the specified check condition). But the database will not assume that the constraint holds for all rows in the table, until it is validated by using the VALIDATE CONSTRAINT option."

From this we can assume that adding a check constraint will result in the same lengthy ACCESS EXCLUSIVE lock time as the not null constraint from before, unless we use the NOT VALID option. By doing so, the constraint will only be enforced for new changes to the data, and not any of the current data. This constraint can then be validated for the rest of the existing data later via the use of the VALIDATE CONSTRAINT option. So with this in mind, let's go take a closer look at the VALIDATE CONSTRAINT documentation.

The VALIDATE CONSTRAINT section points us to the notes section, and the notes section tells us the following: "[A] VALIDATE CONSTRAINT command can be issued to verify that existing rows satisfy the constraint. The validation step does not need to lock out concurrent updates, since it knows that other transactions will be enforcing the constraint for rows that they insert or update; only pre-existing rows need to be checked. Hence, validation acquires only a SHARE UPDATE EXCLUSIVE lock on the table being altered."

Going back over to the documentation on locks, we can that the SHARE UPDATE EXCLUSIVE lock only conflicts with locks at the SHARE UPDATE EXCLUSIVE level or above. This means that it won't prevent our select, update, insert, or delete queries. As such, we should be able to add a check constraint with the NOT VALID option, then validate that constraint, followed by adding the not null constraint, and then finally dropping the check constraint. Let's do that.

alter table people
add constraint temp_null_check check ( guid is not null ) not valid;

This query took 9 ms to run, and required an ACCESS EXCLUSIVE lock.

alter table people
validate constraint temp_null_check;

This query took 6 s 748 ms to run, and required a SHARE UPDATE EXCLUSIVE lock.

alter table people
alter column guid set not null;

This query took 4 ms to run, and required an ACCESS EXCLUSIVE lock.

alter table people
drop constraint temp_null_check;

And this final query took 4 ms to run, and required an ACCESS EXCLUSIVE lock.

At this point our entire migration from start to finish requires 35 ms of ACCESS EXCLUSIVE lock time, 6 s 748 ms of SHARE UPDATE EXCLUSIVE lock time, and 1 m 46 s 636 ms of ROW EXCLUSIVE lock time, resulting in a total migration time of 1 m 53 s 419 ms, of which time, only 35 ms caused CRUD queries to be blocked.

As an additional note, as part of a migration such as this, it's very likely that you'd also want to create an index on the newly created guid column, with the original migration looking something like this:

alter table people
add column if not exists guid varchar(50) default uuid_generate_v4() not null;

create index if not exists people_guid_index
    on people using btree(guid);

This create index call is also problematic, in that it takes 25 s 751 ms to run on our example problem, and it requires a SHARE lock. A SHARE lock conflicts with a ROW EXCLUSIVE lock, so while this would allow select queries to be made, it would block insert, update, and delete queries. Luckily, this is easily fixed. Simply add the keyword concurrently, like so:

create index concurrently if not exists people_guid_index
    on people using btree(guid);

When the keyword concurrently is added, it only requires a SHARE UPDATE EXCLUSIVE lock, which will in turn allow for insert, update, and delete queries to run. Note that adding concurrently will make the query take longer to run, with this taking 34 s 702 ms to run on our example, but again, this is preferred, because it doesn't block anything. So with that, that would bring our final migration to look something like this:

alter table people
add column if not exists guid varchar(50);

alter table people
alter column guid set default uuid_generate_v4();

update people set guid = uuid_generate_v4()
where guid is null;

alter table people
add constraint temp_null_check check ( guid is not null ) not valid;

alter table people
validate constraint temp_null_check;

alter table people
alter column guid set not null;

alter table people
drop constraint temp_null_check;

create index concurrently if not exists people_guid_index
    on people using btree(guid);

With it requiring 35 ms of ACCESS EXCLUSIVE lock time, 41 s 450 ms of SHARE UPDATE EXCLUSIVE lock time, and 1 m 46 s 636 ms of ROW EXCLUSIVE lock time, with a total time of 2 m 28 s 121 ms, with only 35 ms of that blocking CRUD operations. And that is how you fix a migration to minimize lock times and keep everything running smoothly. The important lesson here is to always be aware of your lock times when writing migrations around heavily used and sensitive tables.

My Thought Process For Creating an Exercise Problem

In order to help others in the process of coming up with practice problems with the intention of having people exercise the principles that they've read in a chapter, I've put together an outline of the thought process that I used when coming up with the practice problem for Chapter 1 of Effective Python.

After having read the complete chapter, I looked back through the sub topics of the chapter and made the following mental thoughts and notes:

• Item 1: Know Which Version of Python You're Using
• This section covers python versions and making sure that you're on version 3.
• Nothing to practice here. This section can be skipped.
• Item 2: Follow the PEP 8 Style Guide
• There's a lot of little things in this section. I won't be able to cover everything here.
• Though the whitespace and naming will undoubtedly be covered no matter what problem I come up with.
• It might be good to try to cover inline negation or empty container checks.
• Item 3: Know the Differences Between bytes and str
• While the information here is useful to know about under the right circumstances, it is not something that I expect my group of developers to be running into often, and as such I am fine with skipping this point in the practice problem.
• Item 4: Prefer Interpolated F-Strings Over C-style Format Strings and str.format
• It would be very worthwhile to cover f-strings, potentially something that allows for a complex f-string
• With f-string being recommended as the best practice, there is no need to cover any of the older string formatting variants.
• Item 5: Write Helper Functions Instead of Complex Expressions
• Covering this point is a maybe, if I can get it to fit into the problem.
• That being said, this is a principle that should be universal to programming in general, and not unique to python, so I don't believe that it is something that the developers will be unfamiliar with.
• Item 6: Prefer Multiple Assignment Unpacking Over Indexing
• This could potentially be good to cover. While this feature isn't unique to Python, the number of languages that have this feature are limited, and as such it could be good to practice.
• Item 7: Prefer enumerate Over range
• This is a maybe. It might fit well with f-strings.
• It could also tie in nicely Item 6 from above.
• Item 8: Use zip to Process Iterators in Parallel
• This is another maybe, but I could likely pass on it. Of the developers that I'm presenting to, a large number of them are very familiar with rxJava, and have used its zip function, which works in a very similar way.
• Item 9: Avoid else Blocks After for and while Loops
• This covers a Python specific feature that I've not seen in other languages that the author explicitly recommends avoiding. Easy enough to do. We can skip it for the practice problem.
• Item 10: Prevent Repetition with Assignment Expressions
• This covers the walrus operator, which would be worthwhile to cover.
• It might be difficult to come up with a good case, though.

So with these points in mind, I started thinking through what might make for a good exercise. My thought process went somewhat as follows:

In order to use the f-string I'll probably want to take some data and format it to print prettily on the screen, and most likely I'll want this data to be a list of some sort to allow for different variations of the string to be formatted. So I imagine that I'll want something similar to the grocery list example from the book. Allowing for an empty list case and having a requirement to handle it differently would also pull in another one of the points that I'm trying to cover.

With these thoughts in mind, I looked into how to perhaps pull in some of my other points, and by looking over enumerate I figured that it would be simple enough to pull it in by simply making the list an ordered list of rankings. I would also get the added bonus of pulling in multiple assignment in that way. So with that emerged the idea to have a score board display.

With that, I started considering how I might be able to pull an assignment expression into the problem, and I had to think on that one for a while. Ultimately I figured that what would lead to its likely use would be to have a complex object for each item in my list, where certain fields may or may not be there.

This then led to the idea of displaying a competition score board where you would be given a list of users where a user would have a user name and a score, and may or may not have a team name.

After coming up with the problem, I went through and coded up a basic solution to the problem, and as I coded up the solution, it made me aware of holes in my problem definition that needed to be specified, such as a max length for the user name and the team name.

In the end, I had a simple problem definition put together, which can be found here, and an example solution put together that I could reference as needed as I took my group of developers through the practice problem, which I've shared below. Note that the code that I came up with here and the code that came out of the mob programming exercise with the group (found here) is slightly different. Also note that the code from the mob programming in addition to completing the exercise also to the exercise a step further. This is the result of the experimentation phase that I recommend following the mob programming practice problem, which can be read about in the Presenter section in this post.

With that, here's the example solution that I had come up with while putting the problem together:

def format_score_board(users: List[dict]) -> str:
    if not users:
        return "Awaiting Final Ranking"
    result = ""
    for i, user in enumerate(users):
        if team_name := user.get('team_name', ''):
            team_name = f" ({team_name})"
        line = f"{i+1} {(user['user_name'] + team_name):<43} {user['score']:8.2f}"
        result += "\n" + line
    return result

Answer: Super Epic Competition Score Board

Answer to Effective Python Chapter 1 Exercise

This is the answer to the exercise defined here.

To begin with, let's throw together some basic scaffolding to help us test and build out our solution. We can take the input examples from the exercise and put them into python as follows:

example_1 = []

example_2 = [
    {
        "user_name": "hamster dance",
        "score": 1337
    },
    {
        "user_name": "success kid",
        "team_name": "nerf herders",
        "score": 1200.333333
    },
    {
        "user_name": "rickroll",
        "team_name": "the fellowship",
        "score": 1051.4
    },
    {
        "user_name": "ArrowToTheKnee",
        "team_name": "nerf herders",
        "score": 999.99
    },
    {
        "user_name": "grumpy_cat",
        "team_name": "the fellowship",
        "score": 999.98
    },
    {
        "user_name": "anonymous",
        "score": 561.12
    },
    {
        "user_name": "lol cat",
        "team_name": "allz te lolz",
        "score": 98.0432
    },
    {
        "user_name": "awkward seal",
        "score": -20
    }
]

And we can set up a basic way to test it by doing this:

print(format_score_board(example_1),
      format_score_board(example_2),
      sep="\n\n")

After that, we just need to define our format_score_board function. We can solve the exercise by defining the following code:

def format_score_board(users: List[dict]) -> str:
    if not users:
        return "Awaiting Final Ranking"
    result = ""
    for rank, user in enumerate(users, 1):
        if team_name := user.get('team_name', ''):
            team_name = f" ({team_name})"
        line = f"{rank} {(user['user_name'] + team_name):<43} {user['score']:8.2f}"
        result += "\n" + line
    return result

Now for a little bonus extra credit. What if we wanted to make this just a little bit nicer? What if we wanted to add some leader dots to the blank space between the left and right sides, and perhaps add commas to the scores to make them more readable? How could we change this to do that? With a couple of simple changes, we can get this code:

def format_score_board(users: List[dict]) -> str:
    if not users:
        return "Awaiting Final Ranking"
    result = ""
    for rank, user in enumerate(users, 1):
        user_name = user['user_name']
        if team_name := user.get('team_name', ''):
            team_name = f" ({team_name})"
        score = user['score']
        line = f"{rank} {(user_name + team_name):{'.'}<43}.{score:{'.'}>9,.2f}"
        result += "\n" + line
    return result

Exercise: Super Epic Competition Score Board

Effective Python Chapter 1 Exercise

We have been tasked with creating a score board for the Super Epic Competition. All the work for scoring the competition is already done, and we will be given a list of users that are ordered by the place they took in the competition. We need to take that list and format and print out a score board according to the following spec:

The score board will print out all participants ordered by their place in the competition. There are a max of 8 participants per competition. Each line will have a single participant. On the left side we start with their rank, followed by their user name. If the user is part of a team (which can be determined by whether or not the user object has a team name field), then the team name should follow in parentheses. Both user name and team name have a max limit of 20 characters. On the right their score will be displayed. In this competition the score is a decimal number that can have any number of decimal places. On the score board this number will be formatted to two decimal places. Also note that is is possible for the list of users to be returned as empty, in which case the message "Awaiting Final Ranking" should be displayed. Here are a few examples of input and expected output:

Example 1

Input

[]

Expected Output

Awaiting Final Ranking

Example 2

Input

[
  {
    "user_name": "hamster dance",
    "score": 1337
  },
  {
    "user_name": "success kid",
    "team_name": "nerf herders",
    "score": 1200.333333
  },
  {
    "user_name": "rickroll",
    "team_name": "the fellowship",
    "score": 1051.4
  },
  {
    "user_name": "ArrowToTheKnee",
    "team_name": "nerf herders",
    "score": 999.99
  },
  {
    "user_name": "grumpy_cat",
    "team_name": "the fellowship",
    "score": 999.98
  },
  {
    "user_name": "anonymous",
    "score": 561.12
  },
  {
    "user_name": "lol cat",
    "team_name": "allz te lolz",
    "score": 98.0432
  },
  {
    "user_name": "awkward seal",
    "score": -20
  }
]

Expected Output

1 hamster dance                                   1337.00
2 success kid (nerf herders)                      1200.33
3 rickroll (the fellowship)                       1051.40
4 ArrowToTheKnee (nerf herders)                    999.99
5 grumpy_cat (the fellowship)                      999.98
6 anonymous                                        561.12
7 lol cat (allz te lolz)                            98.04
8 awkward seal                                     -20.00

The answer to this exercise can be found here.

The Developer Book Club System

Over the last year I've experimented with what it takes to make a successful book club experience for software developers, and while I imagine that there's still plenty to learn and improve upon, I've found that the following system and roles work fairly well when tackling a technical book.

The System

When reading a technical book as a group, a good general rule of thumb is to cover a chapter a week. These kinds of books tend to be fairly dense, and trying to cover any more than that can overwhelm members of the group. But note that this can vary on a case by case basis, where some chapters in a book can be very short and could be combined, and other chapters can be overly long, and need to be further broken down.

Additionally reading speeds will very from group to group and from topic to topic. So during the first few weeks of a new book, be sure to gauge the how well the group is keeping up with the pace, and either slow down or speed up accordingly. With this being the case, it will typically take 2 to 3 months to go through a book, varying from book to book.

Meeting regularly to discuss the book is important and valuable, and meeting weekly for about an hour is a good cadence. During this time it's useful have people discuss any points they found interesting from the book, share example code, have a presenter present a practice problem, which the group can then solve, mob programming style, and generally give opportunities for the group to experiment with what they read about. In addition, a facilitator may need a few minutes to address points with keeping the book club running smoothly, such as meeting times, reading schedules, next book, etc. We'll further discuss the facilitator and present roles in the following sections.

The Facilitator

The facilitator is there to keep things running smoothly. This includes a number of responsibilities, such as:

• Determining what book to read. This involves first determining a topic, which can come from polling the group about interests or from determining business needs, and then from there researching what books are available on the topic. Note that while some technical books age well, many do not simply because of the rapid pace of change in the industry.
• Setting a reading schedule. Again, a good rule of thumb is a chapter a week, but be sure to look through the book and take into consideration the people in your group when determining this. And this doesn't need to be set in stone. If after a week or two, you find that the current pace isn't working for people, change it. Also note that it can sometimes be worthwhile to schedule in some break time, such as around holidays or in between books.
• Arranging a time and place to meet. One hour once a week should suffice. Ideally in a setting that would encourage round table discussion instead of a lecture or presentation. As such, a room with a large table and many chairs surrounding it works better than a lecture hall with all seats facing a front stage. Additionally, it is useful to have a large TV or screen in the room and a method to allow anyone in the group to easily cast their laptop screen to it, such as using a Chromecast or similar device.
• Sending reminders about the meetup. People get busy and even with the best of intentions will forget about the meeting. Sending out a simple reminder the day of or the day before drastically improves turnout.
• Arranging presenters for each meetup. When first getting things up and running, it may make sense for the facilitator to also be the sole presenter, but over time it is worthwhile for the facilitator to encourage members of the group to volunteer taking turns being the presenter because it puts less of a load on the facilitator, gives the group the opportunity to more fully learn the material (those who learn the material best are those who teach it), and ensures the long term success of the meetup because it doesn't heavily depend on any one person. Note that it's still worthwhile for the facilitator to take his or her turn as the presenter, as well. You should make yourself available to the presenters to help them come up with ideas for their presentations, as needed. Additionally, as the facilitator it is worthwhile to have at least a few thoughts put together of what could be presented at each meetup so that you could step in and present if the scheduled presenter is unable to make it.
• Coordinate the hand off with the next facilitator. When first getting things up and running, it may make sense to have a single person be the facilitator over the course of many books, but over time it is worthwhile to encourage other members of the group to take a turn at being the facilitator. This ensures stability of the group, since it doesn't heavily rely on any one single person. The ideal time to switch facilitators is as you're finishing one book and preparing to start another.
• Get feedback from the group. Regularly check in with the group and see how things are going. Get feedback and make changes and adjustments based off of their suggestions. Ultimately every group is different, and it is always best to tailor the experience accordingly.

The Presenter

The presenter's job is to help the group get the most out of what they're learning by coming up with a practice problem for the group to tackle.

• Come up with a practice problem. As you're doing the reading for the upcoming meeting, take note of the points being taught and which of those points would make for good candidates to cover in a practice problem. Think of a problem that could cover the use of multiple points from the reading. Note that you don't need to exhaustively cover all points brought up in the reading. Just a handful will do. If the points that you would like to cover don't fit together nicely into a problem, then trying making two smaller problems. It's better to have two smaller practice problems than to try to shoehorn multiple unrelated elements into the same problem.
• Solve the practice problem once yourself before the meetup. This allows you to give some guidance on good ways to approach the problem as needed.
• Lead the discussion at the meetup. My suggested format is to start out with letting the group share points they found interesting from the reading, and/or share example code, then move in to tackling the practice problem via mob programming, followed by allowing the group to make suggestions of other things to test or experiment with in the code, and end with a few minutes for the facilitator to take care of any necessary points that need to be addressed.
• Present the practice problem in the meetup. I've found that taking a mob programming approach works well, where the presenter is at the keyboard and lets the group tell him or her what to type. In general, let the group guide the code, but feel free to give suggestions here and there.
• Invite further experimentation on the code. After the group has finished solving the practice problem, it is a perfect time to invite the group to give any suggestions of anything that they'd like to test out on the code that you now have up on screen. This give the group an opportunity to further experiment and play with the code and discover things that they didn't know.
• Publish a write up for the group to reference later. At minimum take the practice problem definition and the code generated from the mob programming session and publish the two in an article that the group can go back to and reference as need be. That being said, by all means feel free to take it further than that. Add explanations about how the code works or why to write the code the way that it is. Expand further on the problem. Show different ways that it could be solved. If you feel like it would be useful to the group, feel free to add it. And then publish it. While you could put it on an internal company wiki, I'd recommend against this unless it happens to contain private company information. Instead I'd recommend that the article be published as either a github gist, a medium.com article, a dev.to article, a blog post on your own personal blog, or any other public facing option. Why? Because it can prove useful in your future as a way to show a potential employer that you have knowledge of a topic.