Zoney

5 Functional Programming#

Functions are a form of encapsulation supported natively in Python. By breaking down large blocks of code into functions and calling them layer by layer, we can decompose complex tasks into simpler ones. This decomposition can be referred to as procedural programming. Functions are the basic unit of procedural programming.

Functional programming (note the addition of the character "式") — Functional Programming, while it can also be categorized under procedural programming, its concepts are closer to mathematical computation.

First, we need to clarify the concepts of Computer and Compute.

At the level of computers, the CPU executes instructions for addition, subtraction, multiplication, and division, as well as various conditional judgments and jump instructions. Therefore, assembly language is the closest language to computers.

Computation, on the other hand, refers to mathematical computation, where the more abstract the computation, the further it is from computer hardware.

In terms of programming languages, the lower the level of the language, the closer it is to the computer, with low abstraction and high execution efficiency, such as C language; the higher the level of the language, the closer it is to computation, with high abstraction and low execution efficiency, such as Lisp.

Functional programming is a programming paradigm with a very high level of abstraction. Functions written in pure functional programming languages do not have variables, so for any function, as long as the input is determined, the output is also determined. We refer to such pure functions as having no side effects. In programming languages that allow the use of variables, the internal variable states of functions are uncertain, and the same input may yield different outputs; thus, such functions have side effects.

One characteristic of functional programming is that it allows functions themselves to be passed as parameters to other functions and also allows returning a function!

Python provides partial support for functional programming. Since Python allows the use of variables, Python is not a pure functional programming language.

Higher-order Functions

Variables can point to functions
Take Python's built-in absolute value function abs() as an example. Calling this function is done with the following code:

>>> abs(-10)
10

But what if we just write abs?

>>> abs
<built-in function abs>

It can be seen that abs(-10) is a function call, while abs is the function itself.

To obtain the result of the function call, we can assign the result to a variable:

>>> x = abs(-10)
>>> x
10

But what if we assign the function itself to a variable?

>>> f = abs
>>> f
<built-in function abs>

Conclusion: The function itself can also be assigned to a variable, that is, a variable can point to a function.

If a variable points to a function, can we call this function through that variable? Let's verify with code:

>>> f = abs
>>> f(-10)
10

Success! This indicates that the variable f now points to the abs function itself. Calling the abs() function directly and calling the variable f() are completely the same.

Function names are also variables
So what is a function name? A function name is actually a variable that points to a function! For the abs() function, we can completely consider the function name abs as a variable that points to a function that can compute absolute values!

What happens if we point abs to another object?

>>> abs = 10
>>> abs(-10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable

After pointing abs to 10, we can no longer call the function using abs(-10)! Because the variable abs no longer points to the absolute value function but to an integer 10!

Of course, actual code should never be written this way; this is just to illustrate that function names are also variables. To restore the abs function, please restart the Python interactive environment.

Note: Since the abs function is actually defined in the import builtins module, to make the modification of the abs variable's reference effective in other modules, you need to use import builtins; builtins.abs = 10.

Passing Functions
Since variables can point to functions, and function parameters can accept variables, a function can accept another function as a parameter, and such a function is called a higher-order function.

A very simple higher-order function:

def add(x, y, f):
    return f(x) + f(y)

When we call add(-5, 6, abs), the parameters x, y, and f receive -5, 6, and abs respectively. Based on the function definition, we can deduce the calculation process as:

x = -5
y = 6
f = abs
f(x) + f(y) ==> abs(-5) + abs(6) ==> 11
return 11

Writing higher-order functions allows function parameters to accept other functions.

Summary
Passing functions as parameters makes such functions higher-order functions, and functional programming refers to this highly abstract programming paradigm.

1.1 map/reduce

Python has built-in map() and reduce() functions.

The map() function takes two parameters: one is a function, and the other is an Iterable. map applies the passed function to each element of the sequence in turn and returns the result as a new Iterator.

For example, if we have a function f(x)=x² and want to apply this function to a list [1, 2, 3, 4, 5, 6, 7, 8, 9], we can implement it using map() as follows:

Now, let's implement it with Python code:

>>> def f(x):
...     return x * x
...
>>> r = map(f, [1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> list(r)
[1, 4, 9, 16, 25, 36, 49, 64, 81]

The first parameter passed to map() is f, which is the function object itself. Since the result r is an Iterator, which is a lazy sequence, we use the list() function to compute the entire sequence and return a list.

You might think that we don't need the map() function; we can also compute the result with a loop:

L = []
for n in [1, 2, 3, 4, 5, 6, 7, 8, 9]:
    L.append(f(n))
print(L)

Indeed, this works, but can you immediately see from the loop code that "apply f(x) to each element of the list and generate a new list with the results"?

Thus, as a higher-order function, map() abstracts the operation rules, allowing us to compute not only simple f(x)=x² but also any complex function, such as converting all numbers in this list to strings:

>>> list(map(str, [1, 2, 3, 4, 5, 6, 7, 8, 9]))
['1', '2', '3', '4', '5', '6', '7', '8', '9']

Just one line of code.

Next, let's look at the usage of reduce. Reduce applies a function to a sequence [x1, x2, x3, ...]. This function must accept two parameters, and reduce continues to accumulate the result with the next element of the sequence, resulting in:

reduce(f, [x1, x2, x3, x4]) = f(f(f(x1, x2), x3), x4)
For example, to sum a sequence, we can use reduce:

>>> from functools import reduce
>>> def add(x, y):
...     return x + y
...
>>> reduce(add, [1, 3, 5, 7, 9])
25

Of course, the summation can be directly done using Python's built-in sum() function, so there's no need to use reduce.

However, if we want to transform the sequence [1, 3, 5, 7, 9] into the integer 13579, reduce can come in handy:

>>> from functools import reduce
>>> def fn(x, y):
...     return x * 10 + y
...
>>> reduce(fn, [1, 3, 5, 7, 9])
13579

This example itself is not very useful, but if we consider that strings are also a sequence, with slight modifications to the above example, combined with map(), we can write a function to convert str to int:

>>> from functools import reduce
>>> def fn(x, y):
...     return x * 10 + y
...
>>> def char2num(s):
...     digits = {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9}
...     return digits[s]
...
>>> reduce(fn, map(char2num, '13579'))
13579

Organizing it into a str2int function looks like this:

from functools import reduce

DIGITS = {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9}

def str2int(s):
    def fn(x, y):
        return x * 10 + y
    def char2num(s):
        return DIGITS[s]
    return reduce(fn, map(char2num, s))

It can also be further simplified using a lambda function:

from functools import reduce

DIGITS = {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9}

def char2num(s):
    return DIGITS[s]

def str2int(s):
    return reduce(lambda x, y: x * 10 + y, map(char2num, s))

This means that if Python did not provide the int() function, you could completely write a function to convert strings to integers in just a few lines of code!

1.2 filter
The built-in filter() function in Python is used for filtering sequences.

Similar to map(), filter() also takes a function and a sequence. Unlike map(), filter() applies the passed function to each element in turn and decides whether to keep or discard the element based on whether the return value is True or False.

For example, to remove even numbers from a list and keep only odd numbers, we can write:

def is_odd(n):
    return n % 2 == 1

list(filter(is_odd, [1, 2, 4, 5, 6, 9, 10, 15]))
# Result: [1, 5, 9, 15]

To remove empty strings from a sequence, we can write:

def not_empty(s):
    return s and s.strip()

list(filter(not_empty, ['A', '', 'B', None, 'C', '  ']))
# Result: ['A', 'B', 'C']

It can be seen that using the filter() higher-order function is key to correctly implementing a "filtering" function.

Note that the filter() function returns an Iterator, which is a lazy sequence, so to force filter() to complete the calculation results, we need to use the list() function to obtain all results and return a list.

The role of filter() is to sift out elements that meet the criteria from a sequence. Since filter() uses lazy evaluation, it only filters and returns the next sifted element when the filter() result is taken.

1.3 sorted
Sorting algorithms
Sorting is also an algorithm frequently used in programs. Whether using bubble sort or quick sort, the core of sorting is comparing the sizes of two elements. If they are numbers, we can compare them directly, but what if they are strings or two dicts? Directly comparing their mathematical sizes is meaningless, so the comparison process must be abstracted through a function.

Python's built-in sorted() function can sort lists:

>>> sorted([36, 5, -12, 9, -21])
[-21, -12, 5, 9, 36]

Additionally, the sorted() function is also a higher-order function; it can accept a key function to implement custom sorting, such as sorting by absolute value:

>>> sorted([36, 5, -12, 9, -21], key=abs)
[5, 9, -12, -21, 36]

The function specified by key will be applied to each element of the list and sorting will be done based on the results returned by the key function. Comparing the original list and the list processed by key=abs:

list = [36, 5, -12, 9, -21]
keys = [36, 5,  12, 9,  21]

Then the sorted() function sorts according to keys and returns the corresponding elements of the list:

keys sorted result => [5, 9,  12,  21, 36]
                     |  |    |    |   |
final result     => [5, 9, -12, -21, 36]

Now let's look at an example of sorting strings:

>>> sorted(['bob', 'about', 'Zoo', 'Credit'])
['Credit', 'Zoo', 'about', 'bob']

By default, when sorting strings, it compares based on ASCII values, so since 'Z' < 'a', the result is that the uppercase letter Z will come before the lowercase letter a.

Now, we want to sort while ignoring case. To implement this algorithm, we don't need to make major changes to the existing code; we just need to use a key function to map the strings for case-insensitive sorting. Ignoring case when comparing two strings actually means converting both strings to uppercase (or lowercase) before comparing.

Thus, we can achieve case-insensitive sorting by passing a key function to sorted:

>>> sorted(['bob', 'about', 'Zoo', 'Credit'], key=str.lower)
['about', 'bob', 'Credit', 'Zoo']

To perform reverse sorting, we don't need to modify the key function; we can pass a third parameter reverse=True:

>>> sorted(['bob', 'about', 'Zoo', 'Credit'], key=str.lower, reverse=True)
['Zoo', 'Credit', 'bob', 'about']

From the above examples, it can be seen that the abstraction capability of higher-order functions is very powerful, and the core code can remain very concise.

Summary
sorted() is also a higher-order function. The key to sorting with sorted() is to implement a mapping function.
Returning Functions
Functions as Return Values
Higher-order functions can not only accept functions as parameters but can also return functions as result values.

Let's implement a summation function with variable parameters. Typically, the summation function is defined as follows:

def calc_sum(*args):
    ax = 0
    for n in args:
        ax = ax + n
    return ax

However, if we don't need to sum immediately but want to calculate it later in the code as needed, we can return the summation function instead of the summation result:

def lazy_sum(*args):
    def sum():
        ax = 0
        for n in args:
            ax = ax + n
        return ax
    return sum

When we call lazy_sum(), what is returned is not the summation result but the summation function:

>>> f = lazy_sum(1, 3, 5, 7, 9)
>>> f
<function lazy_sum.<locals>.sum at 0x101c6ed90>

The actual summation result is calculated when we call the function f:

>>> f()
25

In this example, we defined a function sum within the lazy_sum function, and the inner function sum can reference the parameters and local variables of the outer function lazy_sum. When lazy_sum returns the function sum, the relevant parameters and variables are preserved in the returned function. This structure, known as a "closure," has great power.

Please note that when we call lazy_sum(), each call returns a new function, even if the same parameters are passed:

>>> f1 = lazy_sum(1, 3, 5, 7, 9)
>>> f2 = lazy_sum(1, 3, 5, 7, 9)
>>> f1==f2
False

The results of f1() and f2() do not affect each other.

Closure
Conditions

1. An inner function is defined in the outer function.
2. The outer function has a return value.
3. The returned value is the name of the inner function.
4. The inner function references variables from the outer function.

def outer_function():
    ...
    def inner_function():
        ...
    return inner_function

Note that the returned function references the local variable args, so when a function returns another function, its internal local variables are still referenced by the new function. Therefore, closures are easy to use but not easy to implement.

Another issue to note is that the returned function does not execute immediately; it only executes when f() is called.

def count():
    fs = []
    for i in range(1, 4):
        def f():
             return i*i
        fs.append(f)
    return fs

f1, f2, f3 = count()

In the above example, a new function is created in each iteration of the loop, and all three created functions are returned.

You might think that calling f1(), f2(), and f3() should yield 1, 4, and 9, but the actual result is:

>>> f1()
9
>>> f2()
9
>>> f3()
9

All are 9! The reason is that the returned functions reference the variable i, but it does not execute immediately. By the time all three functions are returned, the referenced variable i has already become 3, so the final result is 9.

When returning closures, remember: do not reference any loop variables or variables that may change later.

What if you must reference a loop variable? The method is to create another function that binds the current value of the loop variable to the function's parameters. Regardless of how the loop variable changes later, the value bound to the function parameter remains unchanged:

def count():
    def f(j):
        def g():
            return j*j
        return g
    fs = []
    for i in range(1, 4):
        fs.append(f(i)) # f(i) is executed immediately, so the current value of i is passed to f()
    return fs

Now let's look at the results:

>>> f1, f2, f3 = count()
>>> f1()
1
>>> f2()
4
>>> f3()
9

The downside is that the code is longer, but we can use lambda functions to shorten the code.

nonlocal
Using closures means that the inner function references the local variables of the outer function. If we only read the value of the outer variable, we find that the returned closure function works perfectly:

def inc():
    x = 0
    def fn():
        # Only reading the value of x:
        return x + 1
    return fn

f = inc()
print(f()) # 1
print(f()) # 1

However, if we assign a value to the outer variable, since the Python interpreter treats x as a local variable of the function fn(), it will raise an error:

Error
def inc():
    x = 0
    def fn():
        # nonlocal x
        x = x + 1
        return x
    return fn

f = inc()
print(f()) # 1
print(f()) # 2

The reason is that x, as a local variable, has not been initialized, and directly calculating x + 1 is not possible. But we actually want to reference x from the inc() function, so we need to add a nonlocal x declaration inside the fn() function. With this declaration, the interpreter treats x in fn() as the local variable of the outer function, which has already been initialized, allowing it to correctly compute x + 1.

When using closures, before assigning values to outer variables, you need to declare that the variable is not a local variable of the current function using nonlocal.

Summary

1. A function can return a computed result, or it can return a function.
2. When returning a function, remember that the function has not executed, and do not reference any variables that may change.

Anonymous Functions
When we pass functions, sometimes it is more convenient to directly pass anonymous functions without explicitly defining them.

Python provides limited support for anonymous functions. Taking the map() function as an example, when calculating f(x)=x², we can directly pass an anonymous function instead of defining a function f(x):

>>> list(map(lambda x: x * x, [1, 2, 3, 4, 5, 6, 7, 8, 9]))
[1, 4, 9, 16, 25, 36, 49, 64, 81]

By comparison, the anonymous function lambda x: x * x is essentially:

def f(x):
    return x * x

The keyword lambda indicates an anonymous function, and the x before the colon represents the function parameter.

Anonymous functions have a limitation: they can only have one expression, do not need to write return, and the return value is the result of that expression.

Using anonymous functions has the advantage that since the function has no name, there is no need to worry about function name conflicts. Additionally, anonymous functions are also function objects; we can assign an anonymous function to a variable and then use that variable to call the function:

>>> f = lambda x: x * x
>>> f
<function <lambda> at 0x101c6ef28>
>>> f(5)
25

Similarly, we can also return anonymous functions as return values, for example:

def build(x, y):
    return lambda: x * x + y * y

Python's support for anonymous functions is limited, and they can only be used in some simple cases.

Decorators
Characteristics

1. Function A appears as a parameter (Function B accepts Function A as a parameter).
2. It must have the characteristics of a closure.

# Define a decorator
def decorate(func):
    a = 100
    print('Wrapper outer layer print test')

    def wrapper():
        func()
        print('---------> Painting')
        print('---------> Laying floor', a)
        print('---------> Installing door')

    print('Wrapper loading completed......')
    return wrapper


# Use the decorator
@decorate
def house():
    print('I am a roughcast house....')

'''
The default execution:
1. house is the decorated function,
2. The decorated function is passed as a parameter to the decorator decorate.
3. Execute the decorate function.
4. Assign the return value back to house.
'''

print(house)
house()  # wrapper()
# def house1():
#     house()
#     print('Painting')
#     print('Laying floor')


# Call the house function
# house()

Since functions are also objects and function objects can be assigned to variables, we can also call the function through the variable.

>>> def now():
...     print('2015-3-25')
...
>>> f = now
>>> f()
2015-3-25

Function objects have a name attribute (note: it has two underscores before and after), which can be used to get the function's name:

>>> now.__name__
'now'
>>> f.__name__
'now'

Now, suppose we want to enhance the functionality of the now() function, for example, automatically printing logs before and after the function call, but we do not want to modify the definition of the now() function. This way of dynamically adding functionality during code execution is called a decorator.

Essentially, a decorator is a higher-order function that returns a function.

Multi-layer Decorators
If the decorators are layered, the one closest to the function takes precedence.

# Decorator
def zhuang1(func):
    print('------->1 start')

    def wrapper(*args, **kwargs):
        func()
        print('Painting')

    print('------->1 end')

    return wrapper


def zhuang2(func):  # func=house
    print('------->2 start')

    def wrapper(*args, **kwargs):
        func()
        print('Laying floor, installing door.....')

    print('------->2 end')

    return wrapper


@zhuang2
@zhuang1
def house():  # house = wrapper
    print('I am a roughcast house.....')


house()

Output
------->1 start
------->1 end
------->2 start
------->1 end
I am a roughcast house.....
Painting
Laying floor, installing door.....

Decorators with Parameters
'''
Decorators with parameters are three-layered.
The outermost function is responsible for receiving the decorator parameters, while the inner content remains the same as the original decorator.

# Decorator with parameters
def outer(a):  # First layer: responsible for receiving the decorator parameters
    print('------------1 start')

    def decorate(func):  # Second layer: responsible for receiving the function
        print('------------2 start')

        def wrapper(*args, **kwargs):  # Third layer: responsible for receiving the function parameters
            func(*args)
            print("----> Laying tiles {} pieces".format(a))

        print('------------2 end')
        return wrapper  # What is returned is the third layer

    print('------------1 end')
    return decorate  # What is returned is the second layer


@outer(10)
def house(time):
    print('I received the key to the house on {} date, it is a roughcast house....'.format(time))


# @outer(100)
# def street():
#     print('The name of the newly built street is: Heiquan Road')


house('2019-6-12')

# street()

In object-oriented (OOP) design patterns, decorators are referred to as the decoration pattern. OOP's decoration pattern needs to be implemented through inheritance and composition, while Python supports decorators not only through OOP but also directly at the syntax level. Python decorators can be implemented using functions or classes.

Partial Functions
Python's functools module provides many useful features, one of which is partial functions. Note that the partial function here is different from the mathematical concept of partial functions.

When introducing function parameters, we mentioned that by setting default values for parameters, we can reduce the difficulty of function calls. Partial functions can also achieve this. For example:

The int() function can convert strings to integers, and when only a string is passed, the int() function defaults to converting in decimal:

>>> int('12345')
12345

However, the int() function also provides an additional base parameter, which defaults to 10. If the base parameter is passed, it can perform N-base conversions:

>>> int('12345', base=8)
5349
>>> int('12345', 16)
74565

Suppose we need to convert a large number of binary strings; passing int(x, base=2) each time is very cumbersome. Therefore, we can define a function int2() that defaults to base=2:

def int2(x, base=2):
    return int(x, base)

This way, converting binary becomes very convenient:

>>> int2('1000000')
64
>>> int2('1010101')
85

functools.partial helps us create a partial function. We do not need to define int2() ourselves; we can directly use the following code to create a new function int2:

>>> import functools
>>> int2 = functools.partial(int, base=2)
>>> int2('1000000')
64
>>> int2('1010101')
85

So, in summary, the function of functools.partial is to fix some parameters of a function (i.e., set default values) and return a new function that makes calling this new function simpler.

Note that the new int2 function created above simply sets the default value of the base parameter to 2, but it can also accept other values when calling the function:

>>> int2('1000000', base=10)
1000000

Finally, when creating a partial function, it can actually accept a function object, *args, and **kw as three parameters. When passing:

int2 = functools.partial(int, base=2)

It essentially fixes the keyword parameter base of the int() function, which means:

int2('10010')

is equivalent to:

kw = { 'base': 2 }
int('10010', **kw)

When passing:

max2 = functools.partial(max, 10)
it will automatically add 10 as part of *args to the left, which means:

max2(5, 6, 7)
is equivalent to:

args = (10, 5, 6, 7)
max(*args)
The result is 10.

Summary

When the number of parameters for a function is too large and needs to be simplified, using functools.partial can create a new function that can fix some parameters of the original function, making it simpler to call.

6 Modules#

In Python, a .py file is called a module (Module).

What are the benefits of using modules?

1. The biggest benefit is that it greatly improves the maintainability of the code. Secondly, when writing code, you don't have to start from scratch. Once a module is completed, it can be referenced from other places. When writing programs, we often reference other modules, including Python's built-in modules and those from third-party modules.

2. Using modules can also avoid function name and variable name conflicts. Functions and variables with the same name can completely exist in different modules, so when we write our own modules, we don't have to worry about name conflicts with other modules. However, we should also be careful not to conflict with built-in function names.

You might also think about what happens if modules with the same name are written by different people? To avoid module name conflicts, Python introduces a method of organizing modules by directory, called packages.

For example, a file named abc.py is a module named abc, and a file named xyz.py is a module named xyz. If our abc and xyz module names conflict with other modules, we can organize them through packages to avoid conflicts. The method is to choose a top-level package name, such as mycompany, and store them in the following directory:

mycompany
├─ __init__.py
├─ abc.py
└─ xyz.py

After introducing packages, as long as the top-level package name does not conflict with others, all modules will not conflict with others. Now, the name of the abc.py module becomes mycompany.abc, and similarly, the name of the xyz.py module becomes mycompany.xyz.

mycompany
 ├─ web
 │  ├─ __init__.py
 │  ├─ utils.py
 │  └─ www.py
 ├─ __init__.py
 ├─ abc.py
 └─ utils.py

The module name of the file www.py is mycompany.web.www, and the module names of the two files utils.py are mycompany.utils and mycompany.web.utils.

mycompany.web is also a module; please point out the corresponding .py file for this module.

!! When creating modules, be careful with naming; it should not conflict with Python's built-in module names. For example, since the system has a sys module, your own module cannot be named sys.py, otherwise, you will not be able to import the system's built-in sys module.

Summary

1. A module is a collection of Python code that can be used by other modules and can also use other modules.

2. When creating your own modules, be aware of:
   2.1 Module names should follow Python's variable naming conventions; do not use Chinese or special characters.
   2.2 Module names should not conflict with system module names; it is best to check whether the system already has that module by executing import abc in the Python interactive environment. If successful, it indicates that the system has this module.

Using Modules
Python itself has many very useful built-in modules, and as long as they are installed, these modules can be used immediately.

Taking the built-in sys module as an example, let's write a hello module:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

' a test module '

__author__ = 'Michael Liao'

import sys

def test():
    args = sys.argv
    if len(args)==1:
        print('Hello, world!')
    elif len(args)==2:
        print('Hello, %s!' % args[1])
    else:
        print('Too many arguments!')

if __name__=='__main__':
    test()

The first and second lines are standard comments; the first line comment allows this hello.py file to run directly on Unix/Linux/Mac, and the second line comment indicates that the .py file itself uses standard UTF-8 encoding.

The fourth line is a string that represents the module's documentation comment; any string that is the first in the module code is treated as the module's documentation comment.

The sixth line uses the author variable to write the author's name, so when you publish the source code, others can admire your name.

This is the standard file template for Python modules; of course, you can delete everything, but following the standard is certainly not wrong.

The following part is the actual code.

You might have noticed that the first step to using the sys module is to import it:

import sys

After importing the sys module, we have the variable sys pointing to that module. Using this variable sys, we can access all the functionalities of the sys module.

The sys module has a argv variable, which stores all command line parameters in a list. argv always has at least one element because the first parameter is always the name of the .py file.
For example:
Running python3 hello.py gives sys.argv as ['hello.py'];

Running python3 hello.py Michael gives sys.argv as ['hello.py', 'Michael'].

Finally, note these two lines of code:

if __name__=='__main__':
    test()

When we run the hello module file from the command line, the Python interpreter sets a special variable name to main, while if the hello module is imported elsewhere, the if check will fail. Therefore, this if test allows a module to execute some additional code when run from the command line, the most common of which is running tests.

We can run the command line to see the effect of hello.py:

$ python3 hello.py
Hello, world!
$ python hello.py Michael
Hello, Michael!

If we start the Python interactive environment and then import the hello module:

$ python3
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello
>>>

When importing, Hello, world! is not printed because the test() function is not executed.

Only when calling hello.test() will it print Hello, world!:

>>> hello.test()
Hello, world!

Scope
In a module, we may define many functions and variables, but some functions and variables we want to be accessible to others, while some we want to be used only within the module. In Python, this is achieved through the _ prefix.

1. Normal function and variable names are public and can be referenced directly, such as: abc, x123, PI, etc.;

2. Variables like xxx are special variables that can be referenced directly but have special purposes, such as the author and name mentioned above. The documentation comment defined in the hello module can also be accessed using the special variable doc. We generally should not use such variable names for our own variables;

3. Variables or functions like _xxx and xxx are non-public (private) and should not be referenced directly, such as _abc, __abc, etc.;

The reason we say that private functions and variables "should not" be referenced directly rather than "cannot" is that Python does not have a method to completely restrict access to private functions or variables. However, as a programming habit, private functions or variables should not be referenced.

What use do private functions or variables have? Please see the example:

def _private_1(name):
    return 'Hello, %s' % name

def _private_2(name):
    return 'Hi, %s' % name

def greeting(name):
    if len(name) > 3:
        return _private_1(name)
    else:
        return _private_2(name)

We expose the greeting() function in the module while hiding the internal logic with private functions. This way, when calling the greeting() function, there is no need to worry about the internal details of the private functions. This is also a very useful method of code encapsulation and abstraction, meaning: all functions that do not need to be referenced externally should be defined as private, and only those that need to be referenced externally should be defined as public.

Installing Third-party Modules
In Python, installing third-party modules is done through the package management tool pip.

Generally, third-party libraries are registered on the official Python website pypi.python.org. To install a third-party library, you must first know the name of the library, which can be searched on the official website or pypi. For example, the name of Pillow is Pillow, so the command to install Pillow is:

pip install Pillow

Module Search Path
When we try to load a module, Python searches for the corresponding .py file in specified paths. If it cannot be found, it raises an error:

>>> import mymodule
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named mymodule

By default, the Python interpreter searches the current directory, all installed built-in modules, and third-party modules. The search paths are stored in the sys module's path variable:

>>> import sys
>>> sys.path

If we want to add our own search directory, there are two methods:
One is to directly modify sys.path to add the directory to search:

>>> import sys
>>> sys.path.append('/Users/michael/my_py_scripts')

This method modifies it at runtime and becomes ineffective after the run ends.

The other is to set the environment variable PYTHONPATH, which will automatically add the content of this environment variable to the module search path. The setting method is similar to setting the Path environment variable. Note that you only need to add your own search path; Python's own search paths are not affected.