Domain Driven Design Chapter 1 Summary

Part I: Putting the Domain Model to Work

• An eighteenth-century Chinese map of the world shows China as the central and largest location, and is a reflection of their society at that time, which had intentionally turned inward.
• "Maps are models, and every model represents some aspect of reality or an idea that is of interest."
• "A model is a simplification. It is an interpretation of reality that abstracts the aspects relevant to solving the problem at hand and ignores extraneous detail."
• Software relates to some activity or interest of the user, which is the domain of the software.
• Software domains usually have little to do with computers, though there are exceptions:
• The domain of a source-code control system is software development itself.
• A model is a rigorously organized and selective abstraction of domain knowledge
• While a model can be represented by the following, it is not one of the following:
• A diagram
• Carefully written code
• An English sentence
• "Domain modeling is not a matter of making as "realistic" a model as possible. [...] Nor is it just the construction of a software mechanism that gives the necessary results. It is more like moviemaking, loosely representing reality to a particular purpose. Even a documentary film does not show unedited real life. Just as a moviemaker selects aspects of experience and presents them in an idiosyncratic way to tell a story or make a point, a domain modeler chooses a particular model for its utility."
• The Utility of a Model in Domain-Driven Design
• Three basic choices determine the choice of a model
• The model and the heart of the design shape each other
• Intimate link between model and implementation makes the model relevant
• It also helps during maintenance and continuing development, because the code can be interpreted based on understanding the model
• The model is the backbone of a language used by all team members
• Binding of model and implementation means developers can talk about the program in this language
• The can communicate with domain experts without translation
• The model is distilled knowledge
• The model is the team's agreed-upon way of structuring domain knowledge and distinguishing the elements of most interest.
• The Heart of Software
• "The heart of software is its ability to solve domain-related problems for its user. All other features, vital though they may be, support this basic purpose."
• "Yet these are not the priorities on most software projects. [...] Instead, the technical talent goes to work on elaborate frameworks, trying to solve domain problems with technology. Learning about and modeling the domain is left to others. Complexity in the heart of software has to be tackled head-on. To do otherwise is to risk irrelevance."

Chapter 1: Crunching Knowledge

• "A few years ago, I set out to design a specialized software tool for printed-circuit board (PCB) design. One catch: I didn't know anything about electronic hardware. I had access to some PCB designers, of course, but they typically got my head spinning in three minutes. How was I going to understand enough to write this software? I certainly wasn't going to become an electrical engineer before the delivery deadline!"
• Tried having the PCB designers tell exactly what the software should do. Bad idea.
• Their software ideas usually involved reading in an ASCII file, sorting it, writing it back out with some annotation, and producing a report.
• Through various example dialogs between the developer and the PCB experts, the author shows the process of building up a domain model and gaining a clearer understanding in coordination with the experts.
• It starts very basic, showing the relationship between nets (a wire conductor that can connect any number of components) and chips.
• Ideas of how signal work gets added in.
• Chips are replaced with component instances, and the ideas of pin and topology are added.
• "To focus our exploration, we limited ourselves, for a while, to studying one particular feature. A "probe simulation" would trace the propagation of a signal to detect likely sites of certain kinds of problems in the design."
• "So, now, where does the Topology come into it? [...] That's not used for the probe simulation. [...] Then I'm going to drop it out for now, OK? We can bring it back when we get to those features."
• "And so it went (with much more stumbling than is shown here). Brainstorming and refining; questioning and explaining. The model developed along with my understanding of the domain and their understanding of how the model would play into the solution."
• "After a couple more part-time days of this, I felt I understood enough to attempt some code. I wrote a very simple prototype, driven by an automated test framework. I avoided all infrastructure. There was no persistence, and no user interface (UI). This allowed me to concentrate on the behavior. I was able to demonstrate a simple probe simulation in just a few more days. Although it used dummy data and wrote raw text to the console, it was nonetheless doing the actual computation of path lengths using Java objects. Those Java objects reflected a model shared by the domain experts and myself."
• "The correctness of this prototype made clearer to the domain experts what the model meant and how it related to the functioning software. From that point, our model discussions became more interactive, as they could see how I incorporated my newly acquired knowledge into the model and then into the software. And they had concrete feedback from the prototype to evaluate their own thoughts."
• Ingredients of Effective Modeling
• Binding the model and the implementation.
• That crude prototype forged the essential link, and it was maintained through all subsequent iterations.
• Cultivating a language based on the model.
• At first, the engineers had to explain elementary PCB issues to me, and I had to explain what a class diagram meant. But as the project proceeded, any of us could take terms straight out of the model, organize them into sentences consistent with the structure of the model, and be unambiguously understood without translation.
• Developing a knowledge-rich model.
• The objects had behavior and enforced rules. The model wasn't just a data schema; it was integral to solving a complex problem. It captured knowledge of various kinds.
• Distilling the model.
• Important concepts were added to the model as it became more complete, but equally important, concepts were dropped when they didn't prove useful or central. When an unneeded concept was tied to one that was needed, a new model was found that distinguished the essential concept so that the other could be dropped.
• Brainstorming and experimenting.
• The language, combined with sketches and a brainstorming attitude, turned our discussions into laboratories of the model, in which hundreds of experimental variations could be exercised, tried, and judged. As the team went through scenarios, the spoken expressions themselves provided a quick viability test of a proposed model, as the ear could quickly detect either the clarity and ease or the awkwardness of expression.
• Knowledge Crunching
• Similar to how financial analysts crunch numbers, effective domain modelers are knowledge crunchers.
• "They take a torrent of information and probe for the relevant trickle. They try one organizing idea after another, searching for the simple view that makes sense of the mass."
• "Good programmers will naturally start to abstract and develop a model that can do more work. But when this happens only in a technical setting, without collaboration with domain experts, the concepts are naive. That shallowness of knowledge produces software that does a basic job but lacks a deep connection to the domain expert's way of thinking."
• "The constant refinement of the domain model forces the developers to learn the important principles of the business they are assisting [...] The domain experts often refine their own understanding by being forced to distill what they know to essentials, and they come to understand the conceptual rigor that software projects require."
• "These models are never perfect; they evolve. They must be practical and useful in making sense of the domain. They must be rigorous enough to make the application simple to implement and understand."
• Continuous Learning
• "When we set out to write software, we never know enough."
• "At this point, stop and ask yourself a question. Did you learn something about the PCB design process? Although this example has been a superficial treatment of that domain, there should be some learning when a domain model is discussed. I learned an enormous amount. I did not learn how to be a PCB engineer. That was not the goal. I learned to talk to PCB experts, understand the major concepts relevant to the application, and sanity-check what we were building."
• Knowledge-Rich Design
• "The kind of knowledge captured in a model such as the PCB example goes beyond "find the nouns." Business activities and rules are as central to a domain as are the entities involved; any domain will have various categories of concepts. Knowledge crunching yields models that reflect this kind of insight. In parallel with model changes, developers refactor the implementation to express the model, giving the application use of that knowledge."
• "Domain experts are usually not aware of how complex their mental processes are as, in the course of their work, they navigate all these rules, reconcile contradictions, and fill in gaps with common sense. Software can't do this. It is through knowledge crunching in close collaboration with software experts that the rules are clarified, fleshed out, reconciled, or placed out of scope."
• An example is given in the domain of booking cargos onto a voyage of a ship.
• Overbooking is a basic strategy in the industry.
• Code examples are given showing how to better reflect and emphasize this business rule as it relates to the domain model.
• "Now, I am not recommending that such an elaborate design be applied to every detail of the domain. Chapter 15, "Distillation," goes into depth on how to focus on the important and minimize or separate everything else. This example is meant to show that a domain model and corresponding design can be used to secure and share knowledge."
• The more explicit design has these advantages:
• In order to bring the design to this stage, the programmers and everyone else involved will have to come to understand the nature of overbooking as a distinct and important business rule, not just an obscure calculation.
• Programmers can show business experts technical artifacts, even code, that should be intelligible to domain experts (with guidance), thereby closing the feedback loop.
• Deep Models
• "Useful models seldom lie on the surface. As we come to understand the domain and the needs of the application, we usually discard superficial model elements that seemed important in the beginning, or we shift their perspective. Subtle abstractions emerge that would not have occurred to us at the outset but that pierce to the heart of the matter."
• "[In the container shipping system] project, because a shipment begins with the act of booking cargo, we developed a model that allowed us to describe the cargo, its itinerary, and so on. This was all necessary and useful, yet the domain experts felt dissatisfied. There was a way they looked at the business that we were missing."
• "Eventually, after months of knowledge crunching, we realized that the handling of cargo, the physical loading and unloading, the movements from place to place, was largely carried out by subcontractors or by operational people in the company. In the view of our shipping experts, there was a series of transfers of responsibility between parties. A process governed that transfer of legal and practical responsibility, from the shipper to some local carrier, from one carrier to another, and finally to the consignee. Often, the cargo would sit in a warehouse while important steps were being taken. At other times, the cargo would move through complex physical steps that were not relevant to the shipping company's business decisions. Rather than the logistics of the itinerary, what came to the fore were legal documents such as the bill of lading, and processes leading to the release of payments."
• "Knowledge crunching is an exploration, and you can't know where you'll end up."

Effective Python Book Review and Book Club Experience

Well, if you want to get an in depth understanding of best practices and how to write Pythonic code with the latest version of Python (3.8 as of this writing), then Effective Python, Second Edition by Brett Slatkin is a good book for doing just that. In addition to giving current best practices, it also present how a number of things were done in the past, which is quite useful if you need to look through legacy code that might not by following all the current best practices. And let's face it, in the world of programming, that's a pretty common occurrence.

As I took the book club through this book, I took things in a bit of a different direction. I had the preparation of practice problems for each chapter, and the presentation of them, assigned out to different member of the book club. I did this for a couple of reasons. First, I know from experience that the one who teaches is the one who learns the most by far, and it was good to spread this opportunity around. Secondly, to be quite honest, it freed up some of my time to focus on other projects on my plate.

In preparation for this, I wrote an article going over how I figured the developer book club could be systematized, which can be seen here:

The Developer Book Club System

Following this, I was the presenter for the first chapter of the book:

Chapter 1: exercise and answer

In addition to this, while coming up with the exercise, I recorded my thought process for coming up with a practice problem, which can be found in this article:

My Thought Process For Creating an Exercise Problem

After this, it was time for others in the group to take a turn. During this process, there were plenty in the group that had vastly different styles of presenting compared to what I typically do, and it gave me an opportunity to learn some of the benefits of these other presentation styles. I wrote up a number of posts covering my thoughts on some of these presentations:

Another Approach to Building Exercise Problems
Presenting by Using Examples from the Book
When Presenting, It can be Worthwhile to Add a Fun Twist to the Problem

It is also worth noting that people stepped up and made it work. The only other presentations that I did for the book were on chapter 8, where I needed to cover for someone due to extraneous circumstances, and the last chapter, chapter 10, where I simply didn't find someone to fill that spot. My practice problems for these chapters can be found here:

Chapter 8: exercise and answer
Chapter 10: exercise and answer

And with that, we've managed to get through another book, and learned a few more things. The group decided that we needed a breather before getting started in on the next book, so we'll plan to take about a month and a half off, during which time we'll work on getting through the busy season for our company, and after that, we'll look into picking up our next book.

Effective Python Exercise Answer: Virtual Environments and Modules

Let's get to answering the exercise problems from the last post.

Step 1: Virtual Environments

We'll start off with creating our projects folder:

mkdir projects
cd projects/

And after that we can create myproject using venv:

python3 -m venv myproject
cd myproject/

We can then activate our new virtual environment by doing this:

source bin/activate

Once we're in the virtual environment, we can install Flask and PyTest:

python3 -m pip install flask
python3 -m pip install pytest

And at this point if we run:

python3 -m pip list

We'll get an output that looks something like this:

Package            Version
------------------ -------
attrs              19.3.0 
Click              7.0    
Flask              1.1.1  
importlib-metadata 1.5.0  
itsdangerous       1.1.0  
Jinja2             2.11.1 
MarkupSafe         1.1.1  
more-itertools     8.2.0  
packaging          20.3   
pip                18.1   
pkg-resources      0.0.0  
pluggy             0.13.1 
py                 1.8.1  
pyparsing          2.4.6  
pytest             5.3.5  
setuptools         40.8.0 
six                1.14.0 
wcwidth            0.1.8  
Werkzeug           1.0.0  
zipp               3.1.0  

Showing that we have Flask and PyTest (and their dependencies) installed. At this point we can run pip freeze and save the output into a requirements.txt file.

python3 -m pip freeze > requirements.txt

Now, let's see if our requirements.txt file works. We'll start by leaving our current virtual environment and leaving the myproject folder

deactivate
cd ..

At this point we can use venv to create otherproject and activate it's virtual environment:

python3 -m venv otherproject
cd otherproject/
source bin/activate

If we were to use pip list and look at what's currently installed:

python3 -m pip list

We'll then see something like this:

Package       Version
------------- -------
pip           18.1   
pkg-resources 0.0.0  
setuptools    40.8.0 

Which is pretty bare bones. Let's pull over the requirements.txt file from myproject and use it to set up the virtual environment.

cp ../myproject/requirements.txt requirements.txt
python3 -m pip install -r requirements.txt 

At this point if we run:

python3 -m pip list

Then we should get a list that looks like the list from myproject:

Package            Version
------------------ -------
attrs              19.3.0 
Click              7.0    
Flask              1.1.1  
importlib-metadata 1.5.0  
itsdangerous       1.1.0  
Jinja2             2.11.1 
MarkupSafe         1.1.1  
more-itertools     8.2.0  
packaging          20.3   
pip                18.1   
pkg-resources      0.0.0  
pluggy             0.13.1 
py                 1.8.1  
pyparsing          2.4.6  
pytest             5.3.5  
setuptools         40.8.0 
six                1.14.0 
wcwidth            0.1.8  
Werkzeug           1.0.0  
zipp               3.1.0 

Step 2: Modules

Let's start out by moving back over to myproject:

deactivate
cd ../myproject
source bin/activate

At this point we can create two folders, controllers and repositories:

mkdir controllers
mkdir repositories

Create a main.py file that we'll use to fire off our little program:

touch main.py

And inside of both the controllers folder and the repositories folder we'll create an __init__.py and a todo.py file.

cd controllers
touch __init__.py
touch todo.py
cd ../repositories
touch __init__.py
touch todo.py

At this point the modules are set up, and we can add code to the main.py, controllers/todo.py, and repositories/todo.py files like this:

main.py

from controllers import todo

print("In Main")
todo.makeTodo("Test")

controllers/todo.py

from repositories import todo as repo

def makeTodo(name: str):
    print("In controller")
    repo.makeTodo(name)

repositories/todo.py

def makeTodo(name: str):
    print("In repository")
    print(f"Made todo: {name}")

At which point if you run this from inside the myproject folder:

python3 main.py

Then you'll get this output:

In Main
In controller
In repository
Made todo: Test

Which gives a small taste of how modules can be used to avoid naming collisions.

Effective Python Exercise: Virtual Environments and Modules

This week we covered the last chapter of the Effective Python book, chapter 10: Collaboration. There are two specific points from the chapter that I'd like to focus on for the exercises, which are virtual environments and modules. We'll go over the problem in two steps:

Exercise: Virtual Environments and Modules

Step 1: Virtual Environments

Within a folder called projects, use venv to create a folder called myproject with it's own virtual environment. Install Flask and PyTest into the environment. Use pip to create a requirements.txt file that reflects this environment setup. To verify that the requirements.txt file works as expected, back out and use venv to create an otherproject folder with its own virtual environment. Use the requirements.txt file previously created to set up the otherproject folder so that it has the exact same environment as the myproject folder.

Step 2: Modules

Returning back to the original myproject, set it up via modules to have a controllers module and a repositories module. In both the controllers module and the repositories module create a todo.py. Create a main.py, and have it call a function in the todo controller, which in turn calls a function in the todo repository.

The answer to this exercise can be found here.

Answer to Effective Python Exercise: Wizardly Wiles

In the last post we set up a problem to solve with a few bugs and a couple of things to optimize. Our sales golem is not working at optimal efficiency, and so we've set out to fix that. But before we get to that, let's explore some of the problem cases. First up, if we run this code:

customer = HumanCustomer(["4 grams of magic powder"])
print(golem.help_customer(customer))

We will see this output:

('0.00', [4 grams of magic powder])

Which shows us that the order for 4 grams of magic powder was fulfilled, but the price charged to the customer was 0.00. That's not acceptable. We can't be giving things away for free.

This problem is fairly straightforward to fix. in the _process_payment method, we'll want to switch from using float to Decimal, and we'll set the Decimal to round up. Also note that for the best accuracy, it's typically best to instantiate a Decimal using a string. The code below should fix this problem.

    def _process_payment(self, order: List[Union[MagicPowderContainer, Book, None]]) -> str:
        price = Decimal("0")
        for item in order:
            if isinstance(item, MagicPowderContainer):
                price += Decimal(item.measure()) * \
                         Decimal(magic_powder_price_per_gram_sign)
            elif isinstance(item, Book):
                price += Decimal(item.price)
        price = price.quantize(Decimal("0.01"), rounding=ROUND_UP)
        return f"{price:.2f}"

Once this code change is in place, then running the example from above will output the following:

('0.01', [4 grams of magic powder])

The next problem is a little tricky, but take a look at what you get when you run this code:

print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["100000 grams of magic powder"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = OrcCustomer(["Magical Maladies"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
magic_powder_container.fill(100000)
print("Filled")
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["10 grams of magic powder"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = OrcCustomer(["10 grams of magic powder"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["10 grams of magic powder"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")

It outputs the following:

Magic powder: 100000
('100.00', [100000 grams of magic powder])
Magic powder: 0
('0.00', [])
Magic powder: 0
Filled
Magic powder: 100000
('0.01', [10 grams of magic powder])
Magic powder: 99989
('0.01', [10 grams of magic powder])
Magic powder: 99978
('0.01', [10 grams of magic powder])
Magic powder: 99968

So what we're seeing here is the first customer comes in and places an order for all of your magic powder. The order gets fulfilled and the customer is charged 100.00.

After this, the next customer, an orc, comes and tries to place an order. Because the magic powder is gone, the golem is unable to use the translation amulet, and is unable to fulfill the customer's order. The customer is given nothing, and is charged nothing. After this the magic powder container gets refilled (undoubtedly because that last orc customer came and complained to you about the service, at which point you realized that the magic powder had run out, and you refilled it from out of the back room).

After this the golem serves the next customer, a human, who asks for 10 grams of magic powder. After this interaction you check the magic powder container, expecting to find it 10 grams lighter, but instead it is 11 grams lighter. What happened? As it turns out, when the golem determined that it was unable to translate for the previous orc customer, it also failed to turn the translation amulet off. So the translation amulet is left on, and will stay on, until a translation is actually needed (such as with the next customer, which is an orc, and 11 grams are removed as expected) at which point in time the golem will finally turn it off, and things will start working as expected (such as with the customer after that, which is human, and 10 grams are removed as expected).

To fix this, after activating the translation amulet, the code that will end up using the amulet should be in a try block, and the code that deactivates the amulet should be in a finally block, like this:

    def _ask_for_order(self, customer: Customer) -> List[str]:
        try:
            if customer.race != "human":
                translation_amulet.activate()
                try:
                    order = customer.get_order()
                finally:
                    translation_amulet.deactivate()
            else:
                order = customer.get_order()
            return order
        except:
            return []

Now it's worth noting at this point that this is not the only place that uses the translation amulet. It is also used for translating the titles of books. We could also add a try finally block there, but at this point it might be worthwhile to consider creating a function using the contextmanager annotation so that we can use the amulet via a with statement. For instance, if we were to create the following function:

@contextmanager
def use_amulet(amulet: TranslationAmulet):
    amulet.activate()
    try:
        yield
    finally:
        amulet.deactivate()

Then the _ask_for_order method could be changed to this:

    def _ask_for_order(self, customer: Customer) -> List[str]:
        try:
            if customer.race != "human":
                with use_amulet(translation_amulet):
                    order = customer.get_order()
            else:
                order = customer.get_order()
            return order
        except:
            return []

And the _find_book method could be changed to this:

    def _find_book(self, title: str) -> Union[Book, None]:
        for book in books:
            if book.language != "english":
                with use_amulet(translation_amulet):
                    book_title = book.get_title()
            else:
                book_title = book.get_title()
            if title == book_title:
                books.remove(book)
                return book
        return None

At this point if we were to rerun the example from above, we'd get the following output:

Magic powder: 100000
('100.00', [100000 grams of magic powder])
Magic powder: 0
('0.00', [])
Magic powder: 0
Filled
Magic powder: 100000
('0.01', [10 grams of magic powder])
Magic powder: 99990
('0.01', [10 grams of magic powder])
Magic powder: 99979
('0.01', [10 grams of magic powder])
Magic powder: 99969

You can now see that when the third customer is served, the magic powder is reduced by only 10 grams instead of the 11 grams that it was before.

The final issue that we'll take a look at isn't a bug, but rather it's an optimization issue. When the sales golem is looking for a book to fill an order, it always starts with the first book and looks through all the books in order until it finds the book that it is looking for. Considering that the bookshelf is alphabetized, this is slow, and not only is is slow, it will use up extra magic powder translating more books than it needs to. Take for instance this example:

print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Wand Maintenance"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")

This will print out:

Magic powder: 100000
('2.37', [Wand Maintenance])
Magic powder: 99996

Notice that it had to use 4 grams of magic powder to translate the titles of the orcish books "Adventures of Thorg" and the three copies of "Sorcery 101" before it was able to locate the book that it was looking for.

On the other hand, had it been using a binary search, it would have only needed to use 1 gram of magic powder, since it would have started with the book in the middle (one of the copies of "Sorcery 101"), used a gram of magic powder to translate that title, at which point it would realize the the book needed will be to the right, and it will rule out all books to the left, then move to the middle book of the books to the right, and find the copy of "Wand Maintenance" and be done.

So how can we do this in Python? Unfortunately it's not as straightforward as I'd like it to be for this case. Python has the functions bisect and bisect_left, which will perform this kind of search, but these functions do not currently allow for a key parameter to be passed in in similar fashion to how many other Python functions work. (Note that there have been conversations to add this functionality to the bisect functions, and there is a code being written for it. See here for the conversation history, and here for the code.) If it did, then we could simply:

    # This won't work because bisect_left doesn't have a key parameter
    def _find_book(self, title: str) -> Union[Book, None]:
        book_title = None
        
        def key(book: Book):
            nonlocal book_title
            if book.language != "english":
                with use_amulet(translation_amulet):
                    book_title = book.get_title()
            else:
                book_title = book.get_title()
            return book_title
        
        index = bisect_left(books, title, key=key)
        if book_title == title:
            book = books.pop(index)
            return book
        else:
            return None

But since it doesn't, what are some options that we could do to make this work? The two primary recommendations are to either 1) maintain a separate list in parallel to the list that you need to search, which in this case would mean having the sales golem maintain a list of the translated titles of the books separate from the books themselves. Or 2) have the object implement the __lt__ method, which in this case would mean adding the __lt__ method to the Book class.

So long as we're sticking to the strictest sense of the rules for the problem, then we would be unable to modify the Book class, which would mean going with option 1, which would potentially look something like this:

class SalesGolem:
    def __init__(self):
        self._book_titles = []
        for book in books:
            if book.language != "english":
                with use_amulet(translation_amulet):
                    book_title = book.get_title()
            else:
                book_title = book.get_title()
            self._book_titles.append(book_title)
    
    ...
    
    def _find_book(self, title: str) -> Union[Book, None]:
        index = bisect_left(self._book_titles, title)
        book_title = self._book_titles[index]
        if book_title == title:
            book = books.pop(index)
            self._book_titles.pop(index)
            return book
        else:
            return None
    
    ...

If we run this with the previous example, we will get the following output:

Magic powder: 99995
('2.37', [Wand Maintenance])
Magic powder: 99995

You'll notice that this is even more expensive than the previous version, using 5 grams of magic powder because it also needed to translate the title for the orcish book "What's the Difference Between a Wizard and a Warlock?", and it had to do all this before even helping the customer. Now, for helping a single customer, this is clearly worse, but the benefits to it would add up over the course of helping multiple customers because all title translations happened up front, and does not need to be repeated for each individual customer. For instance, if we run this scenario:

print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Wand Maintenance"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["The Vanishing Book on Vanishing"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Sorcery 101"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Magical Maladies"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")

Then you'll get the following output:

Magic powder: 99995
('2.37', [Wand Maintenance])
Magic powder: 99995
('0.00', [None])
Magic powder: 99995
('15.47', [Sorcery 101])
Magic powder: 99995
('9.99', [Magical Maladies])
Magic powder: 99995

Notice that outside of the up front cost, no further cost is incurred, so this can work well, if you expect a lot of customers to be buying books. But this does still have a problem. What happens if you add another book to the shelf? Well then the sales golem's internal list of titles gets out of sync, and without resyncing the list, the golem could end up giving the customer the wrong book. Take a look at this scenario:

print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Wand Maintenance"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["The Vanishing Book on Vanishing"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
print(f"Adding orc book, Arcane Arts")
books.insert(1, OrcishBook("Arcane Arts", "13.33"))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Sorcery 101"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Magical Maladies"])
print(golem.help_customer(customer))

Which outputs:

Magic powder: 99995
('2.37', [Wand Maintenance])
Magic powder: 99995
('0.00', [None])
Magic powder: 99995
Adding orc book, Arcane Arts
Magic powder: 99995
('39.99', [See the Sites of Middle Earth])
Magic powder: 99995
('13.33', [Arcane Arts])
Magic powder: 99995

The last two customers both get the wrong books. This could be fixed by adding a way to resync the list, which could be something like this:

class SalesGolem:
    def __init__(self):
        self._book_titles = []
        self.sync_book_titles()

    def sync_book_titles(self):
        self._book_titles = []
        for book in books:
            if book.language != "english":
                with use_amulet(translation_amulet):
                    book_title = book.get_title()
            else:
                book_title = book.get_title()
            self._book_titles.append(book_title)

    ...

Then the example needs to be changed to allow for the resync:

print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Wand Maintenance"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["The Vanishing Book on Vanishing"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
print(f"Adding orc book, Arcane Arts")
books.insert(1, OrcishBook("Arcane Arts", "13.33"))
golem.sync_book_titles()
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Sorcery 101"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Magical Maladies"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")

This will change the output to:

Magic powder: 99995
('2.37', [Wand Maintenance])
Magic powder: 99995
('0.00', [None])
Magic powder: 99995
Adding orc book, Arcane Arts
Magic powder: 99989
('15.47', [Sorcery 101])
Magic powder: 99989
('9.99', [Magical Maladies])
Magic powder: 99989

This gets us back to getting the customers the right books, but cost of resyncing is high, and if it's something that we have to do regularly, then this solution is less than ideal.

Another way to approach this is instead of putting the new book on the shelf and telling the golem to resync its internal list, we could instead add some functionality to the golem so that we can give the book to the golem, it can determine where it needs to be in it's internal list, and then it can add it to the bookshelf itself. We could this by adding the following method to the SalesGolem class:

    def add_book(self, book: Book):
        if book.language != "english":
            with use_amulet(translation_amulet):
                book_title = book.get_title()
        else:
            book_title = book.get_title()
        index = bisect_left(self._book_titles, book_title)
        self._book_titles.insert(index, book_title)
        books.insert(index, book)

And then changing the scenario to use this method:

print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Wand Maintenance"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["The Vanishing Book on Vanishing"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
print(f"Adding orc book, Arcane Arts")
golem.add_book(OrcishBook("Arcane Arts", "13.33"))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Sorcery 101"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")
customer = HumanCustomer(["Magical Maladies"])
print(golem.help_customer(customer))
print(f"Magic powder: {magic_powder_container.measure()}")

Which then has this output:

Magic powder: 99995
('2.37', [Wand Maintenance])
Magic powder: 99995
('0.00', [None])
Magic powder: 99995
Adding orc book, Arcane Arts
Magic powder: 99994
('15.47', [Sorcery 101])
Magic powder: 99994
('9.99', [Magical Maladies])
Magic powder: 99994

This is much better, it only needs to use 1 more gram of magic powder when the new Orcish book is added, and if an English book is added, it won't need to use any at all.

So is that it? Or could we do better? While the cached titles are nice, it would be better if we could get them lazily instead of eagerly. Is there a way that we could do this? Perhaps instead of having our internal title list being a list of strings that we have to pre populate before we can use it, we could instead fill it with proxy objects that will only get the titles as needed. So if we create this class:

class BookProxy:
    def __init__(self, book: Book):
        self._book = book
        self._title = None

    def get_title(self):
        if not self._title:
            if self._book.language != "english":
                with use_amulet(translation_amulet):
                    self._title = self._book.get_title()
            else:
                self._title = self._book.get_title()
        return self._title

    def __lt__(self, other):
        if isinstance(other, BookProxy):
            return self.get_title() < other.get_title()
        elif isinstance(other, Book):
            return self.get_title() < other.get_title()
        else:
            return self.get_title() < other

Then we can take the SalesGolem class and modify it as follows:

class SalesGolem:
    def __init__(self):
        self._book_titles = [BookProxy(book) for book in books]
    
    ...
    
    def add_book(self, book: Book):
        book_proxy = BookProxy(book)
        index = bisect_left(self._book_titles, book_proxy)
        self._book_titles.insert(index, book_proxy)
        books.insert(index, book)

    def _find_book(self, title: str) -> Union[Book, None]:
        index = bisect_left(self._book_titles, title)
        book_proxy = self._book_titles[index]
        if book_proxy.get_title() == title:
            book = books.pop(index)
            self._book_titles.pop(index)
            return book
        else:
            return None

    ...

Then we're able to run our scenario and get back the following output:

Magic powder: 100000
('2.37', [Wand Maintenance])
Magic powder: 99999
('0.00', [None])
Magic powder: 99998
Adding orc book, Arcane Arts
Magic powder: 99996
('15.47', [Sorcery 101])
Magic powder: 99995
('9.99', [Magical Maladies])
Magic powder: 99995

At which point everything is running as expected, and the total amount of magic powder used is even less. What's more, we've also managed to optimize for the single customer scenario as well. If we run the single customer scenario from above, then we get this output:

Magic powder: 100000
('2.37', [Wand Maintenance])
Magic powder: 99999

So with that, our final SalesGolem class (and supporting functions and classes) looks like this:

class SalesGolem:
    def __init__(self):
        self._book_titles = [BookProxy(book) for book in books]

    def help_customer(self, customer: Customer) -> (str, List[Union[MagicPowderContainer,
                                                                    Book,
                                                                    None]]):
        order_request = self._ask_for_order(customer)
        order = self._fulfill_order(order_request)
        payment = self._process_payment(order)
        return payment, order

    def _ask_for_order(self, customer: Customer) -> List[str]:
        try:
            if customer.race != "human":
                with use_amulet(translation_amulet):
                    order = customer.get_order()
            else:
                order = customer.get_order()
            return order
        except:
            return []

    def _fulfill_order(self, order_request: List[str]) -> List[Union[MagicPowderContainer,
                                                                     Book,
                                                                     None]]:
        order = []
        for item in order_request:
            if item.endswith(" grams of magic powder"):
                ordered_grams = int(item.split(" ")[0])
                try:
                    grams = magic_powder_container.take_magic_powder(ordered_grams)
                except:
                    grams = magic_powder_container.take_remainder()
                order.append(MagicPowderContainer(grams))
            else:
                try:
                    book = self._find_book(item)
                except:
                    book = None
                order.append(book)
        return order

    def add_book(self, book: Book):
        book_proxy = BookProxy(book)
        index = bisect_left(self._book_titles, book_proxy)
        self._book_titles.insert(index, book_proxy)
        books.insert(index, book)

    def _find_book(self, title: str) -> Union[Book, None]:
        index = bisect_left(self._book_titles, title)
        book_proxy = self._book_titles[index]
        if book_proxy.get_title() == title:
            book = books.pop(index)
            self._book_titles.pop(index)
            return book
        else:
            return None

    def _process_payment(self, order: List[Union[MagicPowderContainer, Book, None]]) -> str:
        price = Decimal("0")
        for item in order:
            if isinstance(item, MagicPowderContainer):
                price += Decimal(item.measure()) * \
                         Decimal(magic_powder_price_per_gram_sign)
            elif isinstance(item, Book):
                price += Decimal(item.price)
        price = price.quantize(Decimal("0.01"), rounding=ROUND_UP)
        return f"{price:.2f}"


@contextmanager
def use_amulet(amulet: TranslationAmulet):
    amulet.activate()
    try:
        yield
    finally:
        amulet.deactivate()


class BookProxy:
    def __init__(self, book: Book):
        self._book = book
        self._title = None

    def get_title(self):
        if not self._title:
            if self._book.language != "english":
                with use_amulet(translation_amulet):
                    self._title = self._book.get_title()
            else:
                self._title = self._book.get_title()
        return self._title

    def __lt__(self, other):
        if isinstance(other, BookProxy):
            return self.get_title() < other.get_title()
        elif isinstance(other, Book):
            return self.get_title() < other.get_title()
        else:
            return self.get_title() < other

Now while this exercise is clearly not grounded in the real world, the issues and solutions do very much apply to real world scenarios. Using a with statement to ensure that translation amulet gets turned off can easily be applied to making sure that an open file gets closed. The issue with the finances and the use of the Decimal class to solve them can clearly be applied to finances in the real world. And minimizing the magic powder usage while speeding up the process of finding the right book could be construed to a real world example of needing to find specific resources, where some of them are local and easy to access, while others would require a network call and would be fairly expensive to make.

Effective Python Exercise: Wizardly Wiles

Sticking with the D&D Adventure style problems we've seen before, in this exercise for chapter 8 of Effective Python we are a wizard that owns a magic shop called Wizardly Wiles. Being the savvy saleswizard that you are, you've purchased a sales golem to help run the shop and take orders from customers. This sales golem was marketed as being able to serve customers, fill orders, and tally the total price. It is even able to use a translation amulet to help you with your increasing orders from the local Orcish population Sadly, after getting your magic sales golem, you find that there are several inefficiencies in the spells (code) that make it tick. It sometimes won't deactivate the translation amulet, eating up your precious magic powder. It fails to calculate the correct price for small amounts of magic powder. It's slow at searching for the book that the customer wants. So you decide to void the warranty on your sales golem, and make a few modifications to help it run more efficiently in your shop.

What changes could we make to the spells (code)? Note that you're only allow to change the SalesGolem code permanently, but you can experiment with different books on the bookshelf, different customers, different amounts of magic powder, etc, but the optimizations should be applied to the SalesGolem, and not to other code.

from typing import List, Union


class MagicPowderContainer:
    def __init__(self, grams_of_magic_powder: int):
        self.grams_of_magic_powder = grams_of_magic_powder

    def take_magic_powder(self, grams: int) -> int:
        if self.grams_of_magic_powder > grams:
            self.grams_of_magic_powder -= grams
            return grams
        else:
            raise Exception("Not enough magic powder")

    def take_remainder(self) -> int:
        remainder = self.grams_of_magic_powder
        self.grams_of_magic_powder = 0
        return remainder

    def measure(self) -> int:
        return self.grams_of_magic_powder

    def fill(self, grams: int):
        self.grams_of_magic_powder += grams

    def __repr__(self):
        return f"{self.grams_of_magic_powder} grams of magic powder"


class TranslationAmulet:
    # noinspection PyShadowingNames
    def __init__(self, magic_powder_container: MagicPowderContainer):
        self.is_active = False
        self.magic_powder_container = magic_powder_container

    def activate(self):
        self.is_active = True

    def deactivate(self):
        self.is_active = False

    def use(self):
        self.magic_powder_container.take_magic_powder(1)


class Customer:
    def __init__(self, race: str, order: List[str], translated_order: List[str]):
        self.race = race
        self._order = order
        self._translated_order = translated_order

    def get_order(self) -> List[str]:
        if translation_amulet.is_active:
            translation_amulet.use()
            return self._translated_order
        else:
            return self._order


class HumanCustomer(Customer):
    def __init__(self, order: List[str]):
        super().__init__("human", order, order)


class OrcCustomer(Customer):
    def __init__(self, order: List[str]):
        super().__init__("orc", ["*gibberish*" for _ in order], order)


class Book:
    def __init__(self, language: str, title: str, translated_title: str, price: str):
        self.language = language
        self._title = title
        self._translated_title = translated_title
        self.price = price

    def get_title(self) -> str:
        if translation_amulet.is_active:
            translation_amulet.use()
            return self._translated_title
        else:
            return self._title

    def __repr__(self):
        return self._translated_title


class EnglishBook(Book):
    def __init__(self, title: str, price: str):
        super().__init__("english", title, title, price)


class OrcishBook(Book):
    def __init__(self, title: str, price: str):
        super().__init__("orcish", "*gibberish*", title, price)


# noinspection PyMethodMayBeStatic
class SalesGolem:
    def help_customer(self, customer: Customer) -> (str, List[Union[MagicPowderContainer,
                                                                    Book,
                                                                    None]]):
        order_request = self._ask_for_order(customer)
        order = self._fulfill_order(order_request)
        payment = self._process_payment(order)
        return payment, order

    def _ask_for_order(self, customer: Customer) -> List[str]:
        try:
            if customer.race != "human":
                translation_amulet.activate()
                order = customer.get_order()
                translation_amulet.deactivate()
            else:
                order = customer.get_order()
            return order
        except:
            return []

    def _fulfill_order(self, order_request: List[str]) -> List[Union[MagicPowderContainer,
                                                                     Book,
                                                                     None]]:
        order = []
        for item in order_request:
            if item.endswith(" grams of magic powder"):
                ordered_grams = int(item.split(" ")[0])
                try:
                    grams = magic_powder_container.take_magic_powder(ordered_grams)
                except:
                    grams = magic_powder_container.take_remainder()
                order.append(MagicPowderContainer(grams))
            else:
                try:
                    book = self._find_book(item)
                except:
                    book = None
                order.append(book)
        return order

    def _find_book(self, title: str) -> Union[Book, None]:
        for book in books:
            if book.language != "english":
                translation_amulet.activate()
                book_title = book.get_title()
                translation_amulet.deactivate()
            else:
                book_title = book.get_title()
            if title == book_title:
                books.remove(book)
                return book
        return None

    def _process_payment(self, order: List[Union[MagicPowderContainer, Book, None]]) -> str:
        price = 0.
        for item in order:
            if isinstance(item, MagicPowderContainer):
                price += item.measure() * float(magic_powder_price_per_gram_sign)
            elif isinstance(item, Book):
                price += float(item.price)
        return f"{price:.2f}"


magic_powder_container = MagicPowderContainer(100000)
magic_powder_price_per_gram_sign = "0.001"

books = [
    OrcishBook("Adventures of Thorg", "12.57"),
    EnglishBook("Magical Maladies", "9.99"),
    EnglishBook("See the Sites of Middle Earth", "39.99"),
    EnglishBook("See the Sites of Middle Earth", "39.99"),
    OrcishBook("Sorcery 101", "15.47"),
    OrcishBook("Sorcery 101", "15.47"),
    OrcishBook("Sorcery 101", "15.47"),
    EnglishBook("Tales of Treachery", "5.97"),
    EnglishBook("Wand Maintenance", "2.37"),
    OrcishBook("What's the Difference Between a Wizard and a Warlock?", "42.88"),
    EnglishBook("White Wizard", "25.00")
]
# We cheat and access the translated title here without cost so that we can
# make the assumption that the books are sorted, regardless of the order that
# they are specified in the above array
# noinspection PyProtectedMember
books.sort(key=lambda x: x._translated_title)

translation_amulet = TranslationAmulet(magic_powder_container)
golem = SalesGolem()

You can find the answer here.

When Presenting, It can be Worthwhile to Add a Fun Twist to the Problem

As a presenter, something that you can do to get more energy and interaction out of your group is to take the topic that you intend to present on, and tie it around a fun concept. A good example of this came from a developer book club that I was facilitating when going over the book Effective Python.

Tyler Brocious was the presenter covering Chapter 5: Classes and Interfaces, and he had the brilliant idea of taking the concepts and tying them in to character classes from D&D, with the exercise being to code up character classes in such a way that we could have classes that were a modification of another class, or a combination of other classes, or a class with a special modification for stealth checks. This led to some entertaining conversations that kept people more engaged with the topic as we talked through classes, inheritance, mixins, and how these things work in Python. It's also worth noting that there was no need for us to stick strictly to how D&D actually works, and we varied away from the standard in order to make or tie in important concepts and points. Here's the code that resulted from the exercise:

import random
import math


class MixinStealthCheck:
    def stealth_check(self):
        roll = random.randint(1, 20)
        if not self.dexterity:
            modifier = 0
        else:
            modifier = math.floor(self.dexterity % 10 / 2)
            if self.dexterity < 10:
                modifier = modifier * -1

        print(f'{self.name} rolled {roll} with modifier {modifier}')
        return roll


class Character:
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        self.name = name
        self.strength = strength
        self.dexterity = dexterity
        self.intelligence = intelligence
        self.charisma = charisma


class Ranger(Character, MixinStealthCheck):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity+1, intelligence, charisma)


class Warlock(Character, MixinStealthCheck):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity, intelligence, charisma+1)


class Beastmaster(Ranger):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity, intelligence, charisma+1)


class Witcher(Warlock, Ranger):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity, intelligence, charisma)


heroes = [Beastmaster('Bally', strength=12, dexterity=17, intelligence=12, charisma=8),
          Warlock('Tiresias', strength=7, dexterity=6, intelligence=18, charisma=18),
          Witcher('Barron', strength=10, dexterity=10, intelligence=10, charisma=10)]

print()
for hero in heroes:
    print(hero.__dict__)
    hero.stealth_check()
print(Witcher.mro())

Then as an interesting follow up, the presenter for the following week was Rhys Childs covering Chapter 6: Metaclasses and Attributes, and he decided to play off of the previous week's presentation by Tyler, and utilized the same D&D character classes to explore the property and setter annotations, and take a look at one way in which to do descriptors. We looked at the problem such that we started with the previous week's code, and we need to modify it such that a given stat should have a min value and a max value, first exploring doing so for a single stat using annotations, and then exploring a solution that could be better applied to all stats by using a descriptor. Here's the code that resulted from the exercise:

class Stat:
    def __init__(self):
        self.name = None
        self.internal_name = None
​
    def __set_name__(self, owner, name):
        self.name = name
        self.internal_name = '_' + name
​
    def __get__(self, instance, instance_type):
        if instance is None:
            return self
        return getattr(instance, self.internal_name, '')
​
    def __set__(self, instance, value):
            if 6 <= value <= 20:
                setattr(instance, self.internal_name, value)
            elif value <= 6:
                setattr(instance, self.internal_name, 6)
            else:
                setattr(instance, self.internal_name, 20)
​
​
class Character:
    strength = Stat()
    dexterity = Stat()
    intelligence = Stat()
    charisma = Stat()
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        self.name = name
        self.strength = strength
        self.dexterity = dexterity
        self.intelligence = intelligence
        self.charisma = charisma
​
    # @property
    # def intelligence(self):
    #     return self._intelligence
    #
    # @intelligence.setter
    # def intelligence(self, value):
    #     if 6 <= value <= 20:
    #         self._intelligence = value
    #     elif value <= 6:
    #         self._intelligence = 6
    #     else:
    #         self._intelligence = 20
​
​
class Ranger(Character):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity+1, intelligence, charisma)
class Warlock(Character):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity, intelligence, charisma+1)
class Beastmaster(Ranger):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity, intelligence, charisma+1)
class Witcher(Warlock, Ranger):
    def __init__(self, name, strength, dexterity, intelligence, charisma):
        super().__init__(name, strength, dexterity, intelligence, charisma)
​
heroes = [Beastmaster('Bally', strength=12, dexterity=17, intelligence=12, charisma=8),
          Warlock('Tiresias', strength=7, dexterity=6, intelligence=18, charisma=18),
          Witcher('Barron', strength=10, dexterity=10, intelligence=10, charisma=10)]
​
​
print()
for hero in heroes:
    print(f'Before: {hero.__dict__}')
    hero.intelligence += 3
    print(f'After: {hero.__dict__}')

Overall, it seemed that these presentations worked quite well, in that they were able to take concepts that were perhaps a little complicated from the reading, and brought them across in a way that was both entertaining and relatable, which helped to solidify an understanding of those principles for a number of people in the group.