7 f-strings Powerful Techniques That Will Blow Your Mind

What will you learn?

What is an f-string? Most know that but almost nobody knows the real power of f-strings. In this guide you will learn about it.

An f-string helps you get the string representations of variables.

name = 'Rune'
age = 32
print(f'Hi {name} you are {age} years old')

This will result in the output: ‘Hi Rune you are 32 years old’.

Most know that this is the structure of an f-string.

f'Some text {variable_a} and {variable_b}'

The structure.

  • It starts with f and has quotes afterwards: f’String content’ or f”String content”.
  • Then it will add the string representation of any variable within curly brackets f’String content {varriable_a}’

But there is more power to unleash.

#1 String representation of a class

This is actually a great way to know about Objects in general. If you implement a __str__(self) method it will be the string representation of object. And the best part is that f-string will get use that value as string representation of it.

class Item:
    def __init__(self, a):
        self.a = a
    def __str__(self):
        return str(self.a)
item = Item(12)
print(f'Item: {item}')

This will print ‘Item: 12‘.

#2 Date and time formatting

This is an awesome feature. You can format a date object as you wish.

from datetime import datetime
today = datetime.now()

print(f'Today is {today}')
# 'Today is 2022-04-13 13:13:47.090745'
print(f'Today is {today:%Y-%m-%d}')
# 'Today is 2022-04-13'
print(f'Time is {today:%H:%M%:%S}')
# 'Time is 13:13:47'

#3 Variable names

Another great one is you can actually include the variable names in the output. This is a great feature when you debug or add variables to the log.

x = 10
y = 20
print(f'{x = }, {y = }')
# 'x = 10, y = 20'
print(f'{x=}, {y=}')
# 'x=10, y=20'

#4 Class representation

Now this is not the same as the first one. An Object can have a class representation.

class Price:
    def __init__(self, item, price):
        self.item = item
        self.price = price
        
    def __str__(self):
        return f'{self.item} {self.price}'
    
    def __repr__(self):
        return f'Item {self.item}  costs {self.price} dollars'
p = Price('Car', 10)
print(f'{p}')
# 'Car 10'
print(f'{p!r}')
# 'Item Car  costs 10 dollars'

#5 Formatting specification

Now you can make a lot of formatting of the output.

Here are a few of them.

s = 'Hello, World!'
# Center output
print(f'{s:^40}')
# '             Hello, World!              '
# Left align
print(f'{s:<40}')
# 'Hello, World!                           '
# Right align
print(f'{s:>40}')
# '                           Hello, World!'
n = 9000000
print(f'{n:,}')
# '9,000,000'
print(f'{n:+015}')
# '+00000009000000'

#6 Nested f-strings

You can actually have f-strings within f-strings. This can have a few use-cases like these.

number = 254.3463
print(f"{f'${number:.2f}':>20s}")
# '             $254.35'
v = 3.1415
width = 10
precision = 3
print(f'output {v:{width}.{precision}}')
# 'output       3.14'

#7 Conditional formatting

There might be cases where this is useful.

v = 42.0
print(f'output {v:{"4.3" if v < 100 else "3.2"}}')
# 'output 42.0'
v = 142.0
f'output {v:{"4.3" if v < 100 else "3.2"}}'
# 'output 1.4e+02'

Want to learn more?

If this is something you like and you want to get started with Python, then check my 8 hours FREE video course with full explanations, projects on each levels, and guided solutions.

The course is structured with the following resources to improve your learning experience.

  • 17 video lessons teaching you everything you need to know to get started with Python.
  • 34 Jupyter Notebooks with lesson code and projects.
  • A FREE 70+ pages eBook with all the learnings from the lessons.

How to Use Generators in Python and 3 Use-cases That Simplify Your Code

What will you learn?

What is a Generator in Python and how to use them to work with large datasets in a Pythonic fashion.

What is a Generator?

A Generator is a function that returns a lazy iterator. Said, differently, you can iterate over the iterator, but it is lazy, that is, it will first execute the code when iterated.

A simple example could be as follows.

def my_generator():
    # Do something
    yield 5
    # Do something more
    yield 8
    # Do something else
    yield 12

Then you can iterate over the generator as follows.

for item in my_generator():
    print(item)

This will print 5, 8, and 12.

At first sight, this doesn’t look very useful. But let’s undestand it a bit better what happens.

When we make the first iteration in the for-loop, then it will execute the code in the my_generator function until it reaches the first yield.

Then it stops and returns the value after yield.

In the next iteration, it will continue where it left off and execute until it reaches the next yield.

Then it stops and returns the value after yield.

And so forth until no more yield statements are there.

Now why is that powerful?

Let’s explore some use-cases.

#1 Pre-processing a work item

If you have a pipeline of work items, where there is a pre-processing step. Often you would combine the pre-processing together with the actual processing. But actually, it will make your code more readable and maintainable if you divide it up.

Explore the example.

def pre_process_items():
    for row in open('data.txt'):
        row = row.strip()
        freq = {c: row.count(c) for c in set(row)}
        yield freq
freq = {}
for item in process_items():
    for k, v in item.items():
        freq[k] = freq.get(k, 0) + v

In this case you prepare the work item in pre_process_items().

If you want to learn about the Dict Comprehension read this guide.

This way you divide your code into a piece that prepares data and another one where you process the data. This makes the code easier to understand.

#2 Filtering work items

Often you have a list of work possible work items that need to be processed, but only a few of them actually need to be processed.

A simple example is processing a Log-file, where we are only interested in a specific log-level.

def get_warnings(log_file):
    for row in open(log_file):
        if 'WARNING' in row:
            yield row
for warning in get_warnings('log_file.txt'):
    print(warning)

This example shows how this simplifies how to filter.

If you want to learn more about text processing in Python read this guide.

#3 API calls

A great use-case is if you need to make an API call. This might require setup and filtering the result and possible reformatting.

import pandas_datareader as pdr
from datetime import datetime, timedelta
def get_stocks(tickers):
    d = datetime.now() - timedelta(days=7)
    for ticker in tickers:
        data = pdr.get_data_yahoo(ticker, d)
        close_price = list(data['Close'])
        yield close_price
for prices in get_stocks(['AAPL', 'TWTR']):
    print(prices)

The advantage of this, is, that it will first make the call to the API when you need the data (lazy load). Say, you have a list of 1000s of tickers, if you had to make all the calls before you can start to process, it could be a long waiting time.

With Generators you can utilize the power of lazy-loading.

Want to learn more?

If this is something you like and you want to get started with Python, then check my 8 hours FREE video course with full explanations, projects on each levels, and guided solutions.

The course is structured with the following resources to improve your learning experience.

  • 17 video lessons teaching you everything you need to know to get started with Python.
  • 34 Jupyter Notebooks with lesson code and projects.
  • A FREE 70+ pages eBook with all the learnings from the lessons.

15 String Methods That Changed the Way I Work With Text in Python

What will you learn?

It is (almost) impossible not to work with strings and text in Python at some point. In this guide you will learn 15 ways of working with strings that will help you be more efficient.

Make sure to check the last ones, they are awesome and I use them all the time.

#1 replace()

This is one of my favorite. How often do you need to change a substring to another. This can be as simple as changing a character to another.

input_str = 'this is my slug'
slug = input_str.replace(' ', '_')

This will give ‘this_is_my_slug’ in slug.

#2 split()

You have a string and want to split it into words. This can be done with split()

input_str = 'this is my awesome string'
words = input_str.split()

Then words will contain the list [‘this’, ‘is’, ‘my’, ‘awesome’, ‘string’].

Notice you can set the separator as you wish, say to comma as follows: split(‘,’) then it will separate on comma.

#3 join()

Another favorite. You split something than you want to join it again, right?

input_str = 'this is my awesome string'
my_list = input_str.split()
output_str = '_'.join(my_list)

Then you have output_str to be ‘this_is_my_awesome_string’.

#4 in

In what?

Yes, you want to check if something is a substring of another.

input_str = 'this is my awesome string'
if 'awesome' in input_str:
    print('awesome')
if 'dull' in input_str:
    print('dll')

This will only print awesome.

#5 strip()

Often when you work with strings they will contain spaces in front and at the end as well as a new line.

input_str = '   I love this    '
output_str = input_str.strip()

This will give ‘I love this’ in output_str.

#6 isdigit()

Want to check if a string is a digit value?

input_str = '313'
if input_str.isdigit():
    print(f'{input_str} is digit')
input_str = '313a'
if input_str.isdigit():
    print(f'{input_str} is digit'

This will only be True for the top one.

Read this guide to learn some awesome f-strings (used above).

#7 Concatenate strings

This is such a great feature of Python. Yes, you can simply concatenate strings with a plus sign.

str_a = 'This is Cool'
str_b = ' and Amazing'
output_str = str_a + str_b

This output_str will be ‘This is Cool and Amazing’.

If you notice, this could be done with join as well.

output_str = ''.join((str_a, str_b))

This will give the same result.

#8 Formatted strings

Formatted strings are amazing and makes it easy to output variables. Formatted strings will output string representation of a variable between curly brackets. The formatted string starts with an f.

var_a = 27
var_b = 'Icecream'
print(f'Bought {var_a} items of {var_b}')

This will output ‘Bought 27 items of Icecream’.

#9 lower()

Want a string in lower case.

input_str = 'Hi My Best Friend'
print(input_str.lower())

This will give ‘hi my best friend’.

#10 startswith()

When processing strings, you might want to check if it starts with a specific part.

url = 'https://www.foobar.com/'
if url.startswith('https://'):
    print('HTTPS')

This will obviously print HTTPS for url.

#11 endswith()

Almost similar.

url = 'https://www.foobar.com/'
if url.endswith('/'):
    print('Slash')

It will print Slash. Not from Guns’n’Roses – no not him.

#12 count()

How many occurrences of a substring?

print('foo bar foobar barfoo'.count('foo'))

This will print 3.

#13 splitlines()

This one is amazing when you read a full text with new lines.

text = '''This is my text
and it is long
over multiple lines'''
lines = text.splitlines()

This will have lines [‘This is my text’, ‘and it is long’, ‘over multiple lines’].

#14 Check if string only contains certain characters

I use this all the time. You need to check if a string only contains specific characters. These characters are not standard or very specific to your use case.

Here is how you can solve it.

ef valid_characters(string: str):
    legal_characters = 'abcdefghijklmnopqrstuvwxyz0123456789._-'
    if set(string) <= set(legal_characters):
        return True
    else:
        return False
print(valid_characters('FooBar'))
print(valid_characters('foobar'))
print(valid_characters('foo_bar'))

This will print False, True, True.

#15 Replace and remove last item in itemized string

Am I the only one, which uses this all the time?

It could be comma separated string, and you need to remove the last one of them.

def convert(string: str):
    return ','.join(string.split(',')[:-1])
print(convert('this,is,my,test,remove_me'))

This will print ‘this_is_my_test’.

Want to learn more?

If this is something you like and you want to get started with Python, then check my 8 hours FREE video course with full explanations, projects on each levels, and guided solutions.

The course is structured with the following resources to improve your learning experience.

  • 17 video lessons teaching you everything you need to know to get started with Python.
  • 34 Jupyter Notebooks with lesson code and projects.
  • A FREE 70+ pages eBook with all the learnings from the lessons.

21 Built-in Python Function That Increases Your Productivity

What will you learn?

Built-in functions help programmers do complex things they do all the time with one function. Here are the ones you need to know – and some of them I am sure you didn’t know.

#1 zip()

This is a favorite. If you have multiple lists with connected elements by position, then you can iterate over them as follows.

words = ['as', 'you', 'wish']
counts = [3, 2, 5]
for word, count in zip(words, counts):
    print(word, count)

Which gives.

as 3
you 2
wish 5

Also see how this can be done using List Comprehension.

#2 type()

I don’t really understand how little known this function is. It can help you a lot to understand your code by giving the types of your variables.

words = ['as', 'you', 'wish']
print(type(words))
a = 3.4
print(type(a))

This will give.

list
float

#3 sum()

Often you need to get the sum of an iterable like a list.

my_list = [43, 35, 2, 78, 23, 45, 56]
sum(my_list)

Printing 282 in this case.

#4 set()

If you have a list of elements and you only need to iterate over the unique elements, then set is a great built-in function.

my_list = [3, 2, 6, 3 ,6 ,3 ,5 ,2 ,5, 4, 6, 2, 8, 4, 3]
for item in set(my_list):
    print(item)

This will print the items 2, 3, 4, 5, 6, 8.

But only once each.

#5 list()

If you have an iterable and want it as a list.

my_list = [3, 2, 6, 3 ,6 ,3 ,5 ,2 ,5, 4, 6, 2, 8, 4, 3]
unique_items = list(set(my_list))

Then unique_items will be a list with the elements [2, 3, 4, 5, 6, 8].

#6 sorted()

If you have a list but want a sorted copy of it.

my_list = [3, 2, 6, 3 ,6 ,3 ,5 ,2 ,5, 4, 6, 2, 8, 4, 3]
sorted_list = sorted(my_list)

Which will be [2, 2, 2, 3, 3, 3, 3, 4, 4, 5, 5, 6, 6, 6, 8].

#7 range()

I love this one and use it all the time. It will give you all the numbers in a range. I often use it with for-loops.

for i in range(10):
    print(i)

It will print the numbers 0, 1, …, 9.

You can also generate a list as follows.

my_list = list(range(10))

#8 round()

When you calculate with floats you often get a lot of digits that you actually don’t need. Then round is a great built-in function.

pi = 3.1415
print(round(pi, 2))

This will give you 3.14 only.

#9+10 min() and max()

Gives the minimum and maximum value of a list.

my_list = [3, 2, 6, 3, 6, 3, 5, 2, 5, 4, 6, 2, 8, 4, 3]
print(min(my_list))
print(max(my_list))

Which will print 2 and 8.

#11 map()

If you want to apply a function to all the elements in an iterable like a list.

def my_func(x):
    return 'x'*x
my_list = list(map(my_func, [1, 2, 3, 4]))

Here we also convert to a list with the elements from the map function [‘x’, ‘xx’, ‘xxx’, ‘xxxx’].

#12 isinstance()

Very useful to check the type of a variable.

Here we combine List Comprehension to filter all items of type str (string).

this_list = ['foo', 3, 4.14, 'bar', 'foobar']
my_list = [item for item in this_list if isinstance(item, str)]

Then my_list will contain [‘foo’, ‘bar’, ‘foobar’].

Also check the 7 List Comprehensions to improve your code.

#13 help()

Wow! Did you know that? You can get help.

help(isinstance)

This will give you.

Help on built-in function isinstance in module builtins:
isinstance(obj, class_or_tuple, /)
    Return whether an object is an instance of a class or of a subclass thereof.
    
    A tuple, as in ``isinstance(x, (A, B, ...))``, may be given as the target to
    check against. This is equivalent to ``isinstance(x, A) or isinstance(x, B)
    or ...`` etc.

#14-17 int(), str(), float(), and bool()

These are so useful and easy to use. They basically convert a variable to the given type.

a = '13'
# converts to an integer (here the string a)
b = int(a)
# converts to a float (here the string a)
c = float(a)
# converts to a string (here the float c)
d = str(c)
# converts to a bool - an empty list will be False
e = bool([])
# A list with items will be True
f = bool([2, 3, 4])

#18-19 any() and all()

Given a list of items that can be True of False, if you want to check if any element is True or all elements are True.

a_list = [False, False, True, False, False, False]
if any(a_list):
    print('at least one item is True')
if all(a_list):
    print('all elements are True in the list')

#20 abs()

If you work with numbers and values, then you must often need to take the absolute value. There is a built-in function that does that for you in Python.

Love to Python.

a = -12
b = abs(a) # will be 12
c = 12
d = abs(c)  # will be 12

#21 input()

Sometimes you need to interact with the user of your Python program from the command line. I try to avoid it, but sometimes it actually makes sense. Here Python has done it simple to achieve.

user_input = input('How do you feel?')
print(f'You feel {user_input}')

That is cool.

Check out this guide on f-strings (used above) if you want learn about them.

Final thoughts

The Python built-in functions are actually amazing. I used to program in C, then Java, and some other languages. But the built-in functions in Python just make a lot of simple tasks that are difficult to make in other languages easy to achieve.

Thank you Python.

Want to learn more?

If this is something you like and you want to get started with Python, then check my 8 hours FREE video course with full explanations, projects on each levels, and guided solutions.

The course is structured with the following resources to improve your learning experience.

  • 17 video lessons teaching you everything you need to know to get started with Python.
  • 34 Jupyter Notebooks with lesson code and projects.
  • A FREE 70+ pages eBook with all the learnings from the lessons.

7 Useful List Comprehensions You Didn’t Think Of

What will you learn?

Once you understand list comprehension they actually improve your code readability – still I often advice to comment what the list comprehension does.

First I will show you what List Comprehension is and how the basic case works including with an enclosed if-statement. Then I am going through the 7 use cases you din’t think of and some final thoughts and an alternatives.

What is List Comprehension?

A list comprehension in Python includes three elements.

  1. Expression The member itself, a call to a method, or any other valid expression that returns a value. In the example above, the expression i * i is the square of the member value.
  2. Member The object or value in the list or iterable. In the example above, the member value is i.
  3. Iterable A list, set, sequence, generator, or any other object that can return its elements one at a time. In the example above, the iterable is range(10).

I like to show some examples to explain it better. A List Comprehension is on the following form.

my_list = [do_this(element) for element in this_list]

Instead of this.

my_list = []
for element in this_list:
    my_list.append(element)

A List Comprehension with if-statement.

my_list = [do_this(element) for element in this_list if this_is_true(element)]

Instead of this.

my_list = []
for element in this_list:
    if this_is_true(element):
        my_list.append(element)

Now let’s go through the 7 use cases you didn’t think of.

#1 List Comprehension for Filtering

Say you want to filter all temperatures between 30 and 34 degrees (both excluded here).

temperaturs = [12, 32, 34, 36, 34, 12, 32]
filtered_temps = [t for t in temperaturs if 34 > t > 30]

This will give the items [32, 32] in filtered_temps.

Another example would to find all the strings that are digits.

alphanumeric = ["47", "abcd", "21st", "n0w4y", "test", "55123"]
filtered_aphanumeric = [int(string) for string in alphanumeric if string.isdigit()]

This will give the items [47, 55123]. Notice that we also convert them to integers.

#2 Combining Lists

If you have two lists and you want to combine all combinations from each list.

colors = ["red", "blue", "black"]
models = ["12", "12 mini", "12 Pro"]
combined = [(model, color) for model in models for color in colors]

This will give the following list in combined.

[('12', 'red'),
 ('12', 'blue'),
 ('12', 'black'),
 ('12 mini', 'red'),
 ('12 mini', 'blue'),
 ('12 mini', 'black'),
 ('12 Pro', 'red'),
 ('12 Pro', 'blue'),
 ('12 Pro', 'black')]

A list of tuples of all combinations. If you want to become better at working with strings in Python check this guide.

#3 Finding common elements

Imagine you have two lists and you want to find the elements which are in both lists.

students_a = ["Anna", "Elsa", "Tanja", "Freja", "Frigg"]
students_b = ["Ranja", "Natascha", "Anna", "Tanja"]
common = [student for student in students_a if student in students_b]

This will give you the following items in common.

['Anna', 'Tanja']

#4 Combining Elements with the Same Position

Imagine you have multiple lists with elements that are connected by position.

names = ["John", "Mary", "Lea"]
surnames = ["Smith", "Wonder", "Singer"]
ages = ["22", "19", "25"]
combined = [F"{name} {surname} - {age}" for name, surname, age in zip(names, surnames, ages)]

Then combined will be as follows.

['John Smith - 22', 'Mary Wonder - 19', 'Lea Singer - 25']

Also check out how zip can be used and other built-in functions in Python.

#5 Convert Values

Say you have a list of elements that all need to be converted. Using a function for the conversion can be convenient and also make the transformation of the list easy with List Comprehension.

def convert_to_dol(eur):
    return round(eur * 1.19, 2)

prices = [22.30, 12.00, 0.99, 1.10]
dollar_prices = [convert_to_dol(price) for price in prices]

This will give the following values in dollars_prices.

[26.54, 14.28, 1.18, 1.31]

#6 Frequency Count

I love this one. It is done on a Dict Comprehension, but is useful in many cases.

string = 'this is my string of letters that we will count'
freq = {c: string.count(c) for c in set(string)}

This will give the following dictionary in freq.

{'w': 2, 'n': 2, 'u': 1, 'e': 3, 't': 7, 'r': 2, 'h': 2, 'o': 2, 'm': 1, 'f': 1, 'i': 4, 's': 4, 'y': 1, 'l': 3, ' ': 9, 'a': 1, 'c': 1, 'g': 1}

#7 Generators

Generators are a great tool to master and can be combined with List Comprehension.

def return_next():
    for i in range(10):
        yield i
        
my_list = [i for i in return_next()]

This is a simple example but the power should not be underestimated from it.

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Final thoughts on List Comprehension

List Comprehension is one of the most popular paradigms in Python. That said, you should always keep readability in mind. If you create long and non-intuitive List Comprehensions, maybe you should construct it in another way. Your goal is to create easy to undrestand code – not complex code.

If you want to see a great use case of List Comprehension – then check out how to make a Word Cloud.

Want to learn more?

If this is something you like and you want to get started with Python, then check my 8 hours FREE video course with full explanations, projects on each levels, and guided solutions.

The course is structured with the following resources to improve your learning experience.

  • 17 video lessons teaching you everything you need to know to get started with Python.
  • 34 Jupyter Notebooks with lesson code and projects.
  • A FREE 70+ pages eBook with all the learnings from the lessons.

See the full FREE course page here.

How To Solve Tower of Hanoi with Recursion

What will we cover in this tutorial?

We will first explain and understand Tower of Hanoi programming challenge. It is one of those programming challenges that are highly liked among programmers.

  • Understanding the problem is easy.
  • Solving it seems difficult.
  • Using recursion makes it easy and elegant to solve.

You should be one of those that masters the Tower of Hanoi.

Step 1: Understand the Tower of Hanoi challenge

Tower of Hanoi is a mathematical game, which has three rules. Before we set the rules, let’s see how our universe looks like.

A basic setup of Tower of Hanoi with 3 disks and 3 towers (often called rods)

The disk all have different sizes as pictured above.

The goal is to move all the disks from on tower (rod) to another one with the following 3 rules.

  1. You can only move one disk at the time.
  2. You can only take the top disk and place on top of another tower (rod).
  3. You cannot place a bigger disk on top of a smaller disk.

The first two rules combined means that you can only take one top disk and move it.

Say, in the above we have moved the disk 1 from the first to the second tower (rod).

After that move, we can move disk 2 or disk 1.

The third rule says, that we cannot move disk 2 on top of disk 1.

Those are the 3 rules of the game.

Now do yourself a favor and try to think how you would solve that. How do you get from here.

To here following the 3 rules.

Step 2: Recall recursion and unleash the power of it

Recursion is a method of solving a problem where the solution depends on solutions to smaller instances of the same problem.

https://en.wikipedia.org/wiki/Recursion_(computer_science)

While that is a beautiful and perfect definition – there is still need to exemplify what that means.

A simple example is to sum up the numbers from 1 to n.

It can be a bit difficult to connect the definition of recursions to getting the sum of the integers 1 + 2+ 3 + … + (n – 1) + n.

Let’s first try to do in the iterative way.

def summation(n):
    sum = 0
    for i in range(1, n + 1):
        sum += i
    return sum
print(summation(10))

While there is thing wrong with the above solution, you can turn that into a recursive function by the following.

def summation(n):
    if n == 0:
        return 0
    return n + summation(n - 1)
print(summation(10))

As you see, the problem of summing the numbers from 1 to n, is actually reversed to sum the numbers n, n – 1, n – 2, …, 1. You also notice, that it is true that the summation(n) is equal to n + summation(n – 1).

Wow, that was it. We are breaking the problem down to a problem of smaller size. Just like the definition said.

Also, notice the importance to have a base case in the function. In the above we choose for n == 0 to return 0. This ensures that the recursive calls to not continue forever (or when the Python interpreters stops due to maximum recursion depth).

Try it.

def summation(n):
    return n + summation(n - 1)
print(summation(10))

Well, what did we gain from making the function recursive?

Good question. The above might not be a good example of how recursion helps you. The example of Tower of Hanoi will show you the benefit. It will make your code easy and straight forward.

Step 3: Implement Tower of Hanoi with a recursive function

Now we need to think recursive. Consider the problem again.

How can we break that down to a smaller problem?

Think backwards. Just like the summation from Step 2. What do we need to make happen if we should move disk 3 from first tower (rod) to the last tower (rod)?

Exactly. Then we can move disk 3 to the final destination.

And after that, we should move the smaller problem of the 2 disks on top fo disk 3.

Wow. That is the formula. It is all you need to know.

  1. Move the smaller problem of 2 disks from first tower (rod) to second tower (rod).
  2. Move the big disk from first tower (rod) to last tower (rod).
  3. Move the smaller problem of 2 disks from second tower (rod) to last tower (rod).

Now we need to generalize that.

First understand that there can be any number of disks in an instance of Tower of Hanoi. This means that the problem starts with n disks. If we number the towers (rods) 0, 1, and 2.

Then we have that all n disks start on tower (rod) 0 and should end in tower (rod) 2.

Now we can break down the problem to the following. Given n disks that needs to be moved from start_tower to dest_tower (destination), using aux_tower (auxiliary).

  1. Move subproblem of n – 1 disks from start_tower to aux_tower.
  2. Move disk n to dest_tower.
  3. Move subproblem of n – 1 disk from aux_tower to dest_tower.

See the code below.

class Towers:
    def __init__(self, disks=3):
        self.disks = disks
        self.towers = [[]]*3
        self.towers[0] = [i for i in range(self.disks, 0, -1)]
        self.towers[1] = []
        self.towers[2] = []
    def __str__(self):
        output = ""
        for i in range(self.disks, -1, -1):
            for j in range(3):
                if len(self.towers[j]) > i:
                    output += " " + str(self.towers[j][i])
                else:
                    output += "  "
            output += "\n"
        return output + "-------"
    def move(self, from_tower, dest_tower):
        disk = self.towers[from_tower].pop()
        self.towers[dest_tower].append(disk)

def solve_tower_of_hanoi(towers, n, start_tower, dest_tower, aux_tower):
    # Base case - do nothing
    if n == 0:
        return
    # Move subproblem of n - 1 disks from start_tower to aux_tower.
    solve_tower_of_hanoi(towers, n - 1, start_tower, aux_tower, dest_tower)
    # Move disk n to dest_tower.
    towers.move(start_tower, dest_tower)
    print(towers)
    # Move subproblem of n - 1 disk from aux_tower to dest_tower.
    solve_tower_of_hanoi(towers, n - 1, aux_tower, dest_tower, start_tower)

t = Towers()
print(t)
solve_tower_of_hanoi(t, len(t.towers), 0, 2, 1)

The code includes a simple print function to see the trace of the movings.

Too fast?

Now this often seems a bit too fast. Didn’t we leave out all the subproblems?

That is the beauty of it. We actually didn’t

You tell the machine how to solve the problem using a smaller instance of the problem. This was done with the three things we did. First, move the subproblem away. Second, move the the biggest disk. Finally, move the subproblem on top of biggest disk.

How does that solve it all. See, you solve it for general n, hence, the smaller subproblems solves it by the same formula and we ensure the base case when there are no disks to move.

Deleting Elements of a Python List while Iterating

What will we cover in this tutorial?

  • Understand the challenge with deleting elements while iterating over a Python list.
  • How to delete element from a Python list while iterating over it.

Step 1: What happens when you just delete elements from a Python list while iterating over it?

Let’s first try this simple example to understand the challenge of deleting element in a Python list while iterating over it.

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
for e in a:
  a.remove(e)
print(a)

Now, looking at this piece of code, it would seem to be intended to delete all elements. But that is not happening. See, the output is.

[1, 3, 5, 7, 9]

Seems like every second element is deleted. Right?

Let’s try to understand that. When we enter the the loop we see the following view.

for e (= 0, first element) in a (= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]):
  a.remove(e)

Then the first element is removed on the second line, then the view is.

for e (= 0, first element) in a (= [1, 2, 3, 4, 5, 6, 7, 8, 9]):
  a.remove(e) (a = [1, 2, 3, 4, 5, 6, 7, 8, 9])

Going into the second iteration it looks like this.

for e (= 2, second element) in a (= [1, 2, 3, 4, 5, 6, 7, 8, 9]):
  a.remove(e)

Hence, we see that the iterator takes the second element, which now is the number 2.

This explains why the every second number is deleted from the list.

Step 2: What if we use index instead

Good idea. Let’s see what happens.

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
for i, e in enumerate(a):
  a.pop(i)
print(a)

Which results in the same.

[1, 3, 5, 7, 9]

What if we iterate directly over the index by using the length of the list.

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
for i in range(len(a)):
  a.pop(i)
print(a)

Oh, no.

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    a.pop(i)
IndexError: pop index out of range

I get it. It is because the len(a) is invoked in the first iteration and results to 10. Then when we reach i = 5, we have already pop’ed 5 elements and have only 5 elements left. Hence, out of bound.

Not convinced?

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
for i in range(len(a)):
  print(i, len(a), a)
  a.pop(i)
print(a)

Resulting to.

0 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
1 9 [1, 2, 3, 4, 5, 6, 7, 8, 9]
2 8 [1, 3, 4, 5, 6, 7, 8, 9]
3 7 [1, 3, 5, 6, 7, 8, 9]
4 6 [1, 3, 5, 7, 8, 9]
5 5 [1, 3, 5, 7, 9]
Traceback (most recent call last):
  File "main.py", line 4, in <module>
    a.pop(i)
IndexError: pop index out of range

But what to do?

Step 3: How to delete elements while iterating over a list

The problem we want to solve is not to delete all the element. It is to delete entries based on their values or some conditions, where we need to interpret the values of the elements.

How can we do that?

By using list comprehension or by making a copy. Or is it the same, as list comprehension is creating a new copy, right?

Okay, one step at the time. Just see the following example.

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
a = [i for i in a if i % 2 == 0]
print(a)

Resulting in a copy of the the original list with only the even elements.

[0, 2, 4, 6, 8]

To see it is a copy you can evaluate the following code.

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
b = a
a = [i for i in a if i % 2 == 0]
print(a)
print(b)

Resulting in the following, where you see the variable a get’s a new copy of it and the variable b refers to the original (and unmodified version).

[0, 2, 4, 6, 8]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Hence, the effect of the list comprehension construction above is as the following code shows.

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# a = [i for i in a if i % 2 == 0]
c = []
for i in a:
    if i % 2 == 0:
        c.append(i)
a = c
print(a)

Getting the what you want.

[0, 2, 4, 6, 8]

Next steps

You can make the criteria more advanced by making the criteria by a function call.

def criteria(v):
  # some advanced code that returns True of False
a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
a = [i for i in a if criteria(i)]

And if you want to keep a state of all previous criteria, then you can even use an Object to keep that stored.

class State:
  # ...
  def criteria(self, v):
    # ...
s = State()
a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
a = [i for i in a if s.criteria(i)]

Also, check out this tutorial that makes some observations on performance on list comprehensions.

How to Reverse a Python List in-place and Understand Common Mistakes

Understand the difference

What does in-place mean? When reversing a list it means to not create a new list.

In-place reversing can be done by calling reverse().

a = [i for i in range(19)]
b = a
print("Before reverse")
print(a)
print(b)
a.reverse()
print("After reverse")
print(a)
print(b)

Will result in the following.

Before reverse
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
After reverse
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

While using the slicing to get the list reversed will create a copy.

a = [i for i in range(19)]
b = a
print("Before reverse")
print(a)
print(b)
a = a[::-1] # Reverse using slicing
print("After reverse")
print(a)
print(b)

Resulting in.

Before reverse
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
After reverse
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]

It creates a new copy of the list a and keeps the original, which b points at, untouched.

Correct method: Swapping in a loop

Reversing a list in-place can be achieved with a call to reverse() as was shown above. A common question to get asked during a job-interview is to actually show an implementation of reversing.

As you might guess, you can achieve that by swapping elements.

[0, 1, 2, 3, 4, 5, 6]
# Swap first and last
[6, 1, 2, 3, 4, 5, 0]
# Swap second and second last
[6, 5, 2, 3, 4, 1, 0]
# Swap third and third last
[6, 5, 4, 3, 2, 1, 0]
# Swap the middle with, ah, itself (just kidding, this step is not needed)
[6, 5, 4, 3, 2, 1, 0]

How can you implement that? Let’s try.

a = [i for i in range(19)]
b = a
print(a)
print(b)
for i in range(len(a)//2):
    a[i], a[len(a) - i - 1] = a[len(a) - i - 1], a[i]
print(a)
print(b)

Resulting in the following expected output.

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

Again notice, that I make b point to a, in order to ensure and convince you that we make the reversing in-place and not of a copy. If the print(b) would not print the identical output as print(a), we would have a problem to explain.

Pitfall: List comprehension (incorrect)

But wait a minute? Doesn’t list comprehension mean making a new list based on an existing list?

Correct!

But we can actually circumvent that by using some syntax (or can we? Recommend you read it all).

a = [i for i in range(19)]
b = a
print(a)
print(b)
a[:] = [a[len(a) - i - 1] for i in range(len(a))]
print(a)
print(b)

The slice assignment a[:] enforces Python to do assign, or override, the original values of a. Even if the output shows the following.

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
[18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

The catch is that [a[len(a) – i – 1] for i in range(len(a))] creates a new list, but the slice assignment ensures to override the values of a.

Bummer.

Conclusion

It is easy to make false conclusions like the example above shows. One of the beauties of Python is the extreme power on a high level. The price is that many things are hard to understand unless you dive deep into it.

List Comprehension in Python made Easy with Comparisons

What will we cover in this tutorial?

  • Understand how list comprehension works in Python.
  • Updating and creation of new list is a memory aspect.
  • Test the performance difference between list comprehension and updating a list through a for-loop.

Step 1: Understand what is list comprehension

On wikipedia.org it is defined as follows.

list comprehension is a syntactic construct available in some programming languages for creating a list based on existing lists

Wikipedia.org

Then how does that translate into Python? Or is it, how does Python translate that into code?

If this is the first time you hear about list comprehension, but you have been programming for some time in Python and stumbled upon code pieces like this.

l1 = ['1', '2', '3', '4']
l2 = [int(s) for s in l1]
print(l2)

Which will result in a list of integers in l2.

[1, 2, 3, 4]

The construction for l2 is based on l1. Inspecting it closely, you can see a for-loop inside the creation of the square brackets. You could take the for-loop outside and have the same effect.

l1 = ['1', '2', '3', '4']
l2 = []
for s in l1:
  l2.append(int(s))
print(l2)

Nice.

Step 2: Updating and creation

Sometimes you see code like this.

l1 = [1, 2, 3, 4, 5, 6, 7]
l2 = [i + 1 for i in l1]
print(l2)

And you also notice that the l1 is not used after.

So what is the problem?

Let’s see an alternative way to do it.

l = [1, 2, 3, 4, 5, 6, 7]
for i in range(len(l)):
  l[i] += 1
print(l)

Which will result in the same effect. So what is the difference?

The first one, with list comprehension, creates a new list, while the second one updates the values of the list.

Not convinced? Investigate this piece of code.

def list_comprehension(l):
  return [i + 1 for i in l]
def update_loop(l):
  for i in range(len(l)):
    l[i] += 1
  return l
l1 = [1, 2, 3, 4, 5, 6, 7]
l2 = list_comprehension(l1)
print(l1, l2)
l1 = [1, 2, 3, 4, 5, 6, 7]
l2 = update_loop(l1)
print(l1, l2)

Which results in the following output.

[1, 2, 3, 4, 5, 6, 7] [2, 3, 4, 5, 6, 7, 8]
[2, 3, 4, 5, 6, 7, 8] [2, 3, 4, 5, 6, 7, 8]

As you see, the first one (list comprehension) creates a new list, while the other one updates the values in the existing.

From a memory perspective, this can be an issue with extremely large lists. But what about performance?

Step 3: Performance comparison between the two methods

This is interesting. To compare the run-time (performance) of the two functions we can use the cProfile standart Python library.

import cProfile
import random

def list_comprehension(l):
    return [i + 1 for i in l]

def update_loop(l):
    for i in range(len(l)):
        l[i] += 1
    return l

def test(n, it):
    l = [random.randint(0, n) for i in range(n)]
    for i in range(it):
        list_comprehension(l)
    l = [random.randint(0, n) for i in range(n)]
    for i in range(it):
        update_loop(l)

cProfile.run('test(10000, 100000)')

This results in the following output.

         152917 function calls in 16.837 seconds
   Ordered by: standard name
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   16.837   16.837 <string>:1(<module>)
        1    0.869    0.869   16.837   16.837 TEST.py:15(test)
        1    0.008    0.008    0.040    0.040 TEST.py:16(<listcomp>)
        1    0.003    0.003    0.023    0.023 TEST.py:20(<listcomp>)
    10000    0.013    0.000    4.739    0.000 TEST.py:5(list_comprehension)
    10000    4.726    0.000    4.726    0.000 TEST.py:6(<listcomp>)
    10000   11.164    0.001   11.166    0.001 TEST.py:9(update_loop)
    20000    0.019    0.000    0.041    0.000 random.py:200(randrange)
    20000    0.010    0.000    0.052    0.000 random.py:244(randint)
    20000    0.014    0.000    0.022    0.000 random.py:250(_randbelow_with_getrandbits)
        1    0.000    0.000   16.837   16.837 {built-in method builtins.exec}
    10000    0.002    0.000    0.002    0.000 {built-in method builtins.len}
    20000    0.002    0.000    0.002    0.000 {method 'bit_length' of 'int' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    32911    0.006    0.000    0.006    0.000 {method 'getrandbits' of '_random.Random' objects}

Where we can see that the accumulated time spend in list_comprehension is 4.739 seconds, while the accumulated time spend in update_loop is 11.166 seconds.

Wait? Is it faster to create a new list than update an existing one?

Let’s do some more testing.

Performance of list comprehension vs updating a list

Seems to be no doubt about it.

Let’s just remember that Python is an interpreter and each instruction is highly optimized. Hence, keeping the code as list comprehension, can be highly optimized, while updating the loop is more flexible and takes more lines of interpretation.

Step 4 (Bonus): Use list comprehension with function

One aspect of list comprehension, is that it limits the possibility, while the for-loop construct is more flexible.

But wait, what if you use a function inside the list comprehension construction, then you should be able to regain a lot of that flexibility.

Let’s try to see how that affects the performance.

import cProfile
import random
def add_one(v):
  return v + 1
def list_comprehension(l):
    return [add_one(i) for i in l]

def update_loop(l):
    for i in range(len(l)):
        l[i] += 1
    return l

def test(n, it):
    l = [random.randint(0, n) for i in range(n)]
    for i in range(it):
        list_comprehension(l)
    l = [random.randint(0, n) for i in range(n)]
    for i in range(it):
        update_loop(l)

cProfile.run('test(1000, 10000)')

Giving the following output.

         10050065 function calls in 15.826 seconds
   Ordered by: standard name
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   15.826   15.826 <string>:1(<module>)
    10000    3.960    0.000    3.964    0.000 main.py:11(update_loop)
        1    0.296    0.296   15.826   15.826 main.py:17(test)
        1    0.001    0.001    0.005    0.005 main.py:18(<listcomp>)
        1    0.004    0.004    0.008    0.008 main.py:22(<listcomp>)
 10000000    4.389    0.000    4.389    0.000 main.py:4(add_one)
    10000    0.077    0.000   11.554    0.001 main.py:7(list_comprehension)
    10000    7.088    0.001   11.476    0.001 main.py:8(<listcomp>)
     2000    0.003    0.000    0.006    0.000 random.py:200(randrange)
     2000    0.002    0.000    0.008    0.000 random.py:244(randint)
     2000    0.002    0.000    0.003    0.000 random.py:250(_randbelow_with_getrandbits)
        1    0.000    0.000   15.826   15.826 {built-in method builtins.exec}
    10000    0.004    0.000    0.004    0.000 {built-in method builtins.len}
     2000    0.000    0.000    0.000    0.000 {method 'bit_length' of 'int' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
     2059    0.000    0.000    0.000    0.000 {method 'getrandbits' of '_random.Random' objects}

Oh no. It takes list_comprehension 11.554 seconds compared to update_loop 3.964 seconds.

This is obviously hard to optimize for the interpreter as it cannot predict the effect of the function (add_one). Adding that call in each iteration of the creation of the list add a big overhead in performance.

Conclusion

Can we conclude that list comprehension always beats updating an existing list? Not really. There is the memory dimension. If lists are big or memory is a sparse resource or you want to avoid too much memory cleanup by Python, then updating the list might be a better option.

Sort a Python List with String of Integers or a Mixture

What will we cover in this tutorial?

  • How can you sort a list of strings containing integers by the integer value?
  • Or what if it contains both strings containing integers and integers?
  • Finally, also how if only a substring contains integers?

Why sort on a list of integers represented in strings fails

First of, we need to understand why it is not trivial to solve by just calling sort on the list.

Let’s just try with an example.

l = ['4', '8', '12', '23', '4']
l.sort()
print(l)

Which will result in the following list.

['12', '23', '4', '4', '8']

Where you see the list is sorted lexicographical order and not by the numeric value the strings represent.

How to solve this

Solving this is quite straight forward if you know your way around Python. You look in the documentation and see that it takes a key as argument. Okay, you are new to this, so what does it mean.

key specifies a function of one argument that is used to extract a comparison key from each list element

Python docs.

Still not comfortable about it. Let’s try to figure it out together. If you are new to Python, you might not know that you can send functions as arguments like any other value.

The key argument is a function that will be applied on every item in the list. The output of that function will be used to make a simple comparison and order it by that.

That is great news. Why?

I am glad you asked. If we just use the int() function as argument, it should cast the string to an integer and use that for comparison and our problem is solved.

Let’s try.

l = ['4', '8', '12', '23', '4']
l.sort(key=int)
print(l)

Resulting to the following list.

['4', '4', '8', '12', '23']

How simple is that?

What if my list is a mixture of integers and strings of integers?

What is your wild guess?

l = ['4', '8', 12, '23', 4]
l.sort(key=int)
print(l)

Notice that some integers are not strings any more. Let see the output.

['4', 4, '8', 12, '23']

It works. This is why we love Python!

But what if it is more complex?

A complex examples of sorting

Say we have a list of of strings like this one.

l = ['4 dollars', '8 dollars', '12 dollars', '23 dollars', '4 dollars']

The story is something like this. You ask a lot of providers how much it will cost to give a specific service. The answers are given in the list and you want to investigate them in order of lowest price.

We can just do the same, right?

l = ['4 dollars', '8 dollars', '12 dollars', '23 dollars', '4 dollars']
l.sort(key=int)
print(l)

Wrong!

Traceback (most recent call last):
  File "main.py", line 2, in <module>
    l.sort(key=int)
ValueError: invalid literal for int() with base 10: '4 dollars'

The string is not just an integer. It contains more information.

The good luck is that we can send any function. Let’s try to create one.

def comp(o):
  return int(o.split()[0])
l = ['4 dollars', '8 dollars', '12 dollars', '23 dollars', '4 dollars']
l.sort(key=comp)
print(l)

And the output is as desired.

['4 dollars', '4 dollars', '8 dollars', '12 dollars', '23 dollars']

Too fast? Let’s just analyse our function comp. It contains only one return statement. Try to read it from inside out.

o.split() splits the string up in a list of items contain word by word. Hence, the call of ‘4 dollars’.split() will result in [‘4’, ‘dollars’].

Then o.split()[0] will return the first item of that list, i.e. ‘4’.

Finally, we cast it to an integer by int(o.split()[0]).

Remember that the comparison is done by the output of the function, that is what the function returns, which in this case is the integer represented by the first item in the string.

What about lambda?

Lambda? Yes, lambda functions is also a hot subject.

A lambda function is just a smart way to write simple functions you send as arguments to other functions. Like in this case a sorting function.

Let’s try if we can do that.

l = ['4 dollars', '8 dollars', '12 dollars', '23 dollars', '4 dollars']
l.sort(key=lambda o: int(o.split()[0]))
print(l)

Resulting in the same output.

['4 dollars', '4 dollars', '8 dollars', '12 dollars', '23 dollars']

A bit magic with lambda functions? We advice you to read this tutorial on the subject.