Mastering Python Linters: A Guide for Developers – AlgoTrading101 Blog

What types of Python linters exist?

Types of Python linters can be categorized into two groups which are Code formatting and style and Error Detection.

The first group checks if the code adheres to the stylistic guidelines, such as PEP 8, which is the style guide for Python code. This includes checking indentation, line spacing, and the use of variables and function names.

The second group uncovers potential errors before the code is executed. They pinpoint issues like undeclared variables, possible syntax errors, and other inconsistencies that could lead to bugs.

Sometimes, some linters fall into both groups and we will cover them in this article.

How to get started with Python linters?

To get started with Python linters, all you need to do is to install them via pip and configure them if needed. Most linters prefer pure Python files (e.g. ruff) while others can also style your notebooks (e.g. black).

The linters that we will take a look at are the linters I personally use for all my projects which are the following:

• Black
• Pylint
• Ruff
• MyPy
• Bandit
• PyDocstyle

We will look into each one of them and I’ll show you how to work with them. Then, we’ll combine them together into a pre-commit hook that will check all of our files and stop the commit if we have linting errors.

How to use Black

Using Black is as simple as running a single command. To format a single Python file:

To format a directory, you can do:

To format all files at your current location do:

For example, here is a small code block before Black is applied:

def my_function(arg1,arg2):
    print( "arg1:",arg1,"arg2:",arg2)

Here is the after:

def my_function(arg1, arg2):
    print("arg1:", arg1, "arg2:", arg2)

How to configure Black?

Black aims to be an opinionated formatter, so configuration options are minimal. However, you can configure line length (default is 88 characters) and exclude specific files. For example, to set a line length of 100 characters:

black your_script.py -l 100

To exclude a directory or a file, use the --exclude parameter. Here’s how to exclude a directory named migrations:

black your_directory --exclude='/migrations/'

Configuration can also be specified in a pyproject.toml file, which Black will automatically detect and use. For example:

[tool.black]
line-length = 100
exclude = '''
/(
    migrations
)/
'''

The code block above combines the two configuration options we ran manually. This way, you can run black without the need to pass extra arguments.

Pylint Python Linter: Why Is It a Game-Changer for Python Developers?

Pylint is a versatile Python linter for static code analysis. It checks Python code against a wide range of programming standards, highlights errors, and enforces a more explicit coding standard.

It offers detailed reports on code quality, potentially problematic areas, code duplication, styling issues, and more. It is quite customizable and supports plugins.

Link to repository: https://github.com/pylint-dev/pylint

How to use Pylint?

To use Pylint, simply run it against a Python file or a module. For example, to lint a single file:

For linting an entire Python package:

Pylint will analyze the code and output a report detailing various issues, categorized by their nature (e.g., errors, warnings, refactor suggestions).

Here is an example code block before abiding by Pylint:

class my_class:
    def func1(self):
        pass

    def anotherFunction(self, arg1):
        self.myvar = arg1
        print(arg1)

obj = my_class()
obj.func1()
obj.anotherFunction(10)

Here is the code block after cleaning out Pylint errors:

class MyClass:
    """Example class demonstrating Pylint improvements."""

    def __init__(self):
        """Initialize the class."""
        self.my_var = None

    def function_one(self):
        """Example method that does nothing."""
        # Previously had 'pass', removed as it's unnecessary here.

    def another_function(self, arg1):
        """Print the provided argument.

        Args:
            arg1 (int): The argument to be printed.
        """
        self.my_var = arg1
        print(arg1)

obj = MyClass()
obj.function_one()
obj.another_function(10)

Sometimes, Pylint might be wrong in its interpretation. In that case, you can ignore specific Pylint errors either in your entire file, specific line, specific function/class, or more.

For instance, if you want to ignore a particular warning, say line-too-long (C0301), on a specific line, you can do the following:

some_really_long_line = '...'  # pylint: disable=line-too-long

You can also just write the code of the error but that makes your ignored error less explicit. To disable a warning for an entire file, add a comment at the top of the file:

# pylint: disable=line-too-long

How to use Ruff?

Using Ruff is straightforward. To analyze a single Python file:

ruff check your_script.py

And to lint an entire project:

Here is an example code block before Ruff:

def calculate_area( length, width ):
    area=length*width
    print("Area:",area)
    return(area)

calculate_area(10,20)

Here is the after:

def calculate_area(length, width):
    area = length * width
    print("Area:", area)
    return area

calculate_area(10, 20)

These corrections align the code with Python’s PEP 8 style guide, improving readability and maintainability. In a real-world scenario, a linter like Ruff would also flag other potential issues like variable naming conventions, line lengths, and more complex stylistic concerns.

Sometimes, some Ruff errors might not make perfect sense for your code implementation or you might be too lazy to fix it. Sometimes, it might reduce some efficiency so you can ignore errors like this:

def legacy_data_formatter(data):
    # Legacy system requires old-style %-formatting, not .format() or f-strings
    # ruff: ignore=modern-string-formatting
    formatted_data = "Name: %s, Age: %d" % (data['name'], data['age'])
    return formatted_data

data = {'name': 'Alice', 'age': 30}
formatted_data = legacy_data_formatter(data)
print(formatted_data)

How to configure Ruff?

Ruff can be configured to suit specific project requirements. Configuration typically involves creating a .ruffrc file in the root directory of your project. Here’s an example of what the configuration file might look like:

[ruff]
ignore = E203, W503
max-line-length = 120
select = C,E,F,W,B,B950
exclude = .venv,.git,__pycache__,old,build,dist

In this .ruffrc file, we’re setting specific rules to ignore, defining the maximum line length, selecting the error codes to check for, and excluding directories from linting.

How to use MyPy?

To use MyPy, simply run it against a Python file or directory. For example, to type check a single file:

Or to check an entire project:

mypy your_project_directory/

MyPy analyzes the annotated types in your Python code, reports discrepancies, and suggests corrections where type mismatches are detected. You might notice that it is quite slower to run when compared to the previous linters. This is because it is written in Python. 😀

Here is an example code block before MyPy:

def square_numbers(numbers):
    return {number: number**2 for number in numbers}

result = square_numbers([1, 2, 3, 4])
print(result)

Here is an example code block after implementing typing and abiding by MyPy suggestions:

from typing import List, Dict

def square_numbers(numbers: List[int]) -> Dict[int, int]:
    return {number: number**2 for number in numbers}

result = square_numbers([1, 2, 3, 4])
print(result)

Sometimes, MyPy might be wrong in its interpretation as Python isn’t a static language and sometimes it doesn’t make sense to force it to be one. In that case, you can ignore specific MyPy errors either in your entire file, specific line, specific function/class, or more.

Suppose you have a function that interacts with a third-party library where the return type of a function is not clearly defined (e.g., it could return different types based on certain conditions). However, you know from the docs that under certain conditions, the return type will be an integer.

from typing import Any
from some_third_party_library import get_dynamic_value

def calculate_value() -> int:
    value: Any = get_dynamic_value()
    # Based on certain conditions, you know 'value' will be an integer
    return value  # MyPy will flag this as an error

We can ignore the error like this:

def calculate_value() -> int:
    value: Any = get_dynamic_value()
    return value  # type: ignore

How to use Bandit?

Once installed, you can run Bandit on your Python files or projects to check for security issues. Here’s how to run it on a single file:

For scanning an entire project directory, use:

bandit -r /path/to/your/project

Bandit will recursively scan all the Python files in the specified directory and output any security warnings.

How to use Python Linters with Pre-commit hooks?

To use Python linters with pre-commit hooks, you first need to install the pre-commit framework. You can do this using pip:

After installation, create a .pre-commit-config.yaml file in the root directory of your project. This file will define which hooks (linters in this case) you want to run.

In your .pre-commit-config.yaml, you can specify various linters (like Black, Pylint, MyPy, Bandit, and others) as individual hooks. Here’s an example configuration:

repos:
  - repo: https://github.com/psf/black
    rev: stable
    hooks:
      - id: black

  - repo: https://github.com/PyCQA/pylint
    rev: pylint-2.9.6
    hooks:
      - id: pylint

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v0.910
    hooks:
      - id: mypy

  - repo: https://github.com/PyCQA/bandit
    rev: 1.7.0
    hooks:
      - id: bandit

Each - repo: section specifies a linter’s repository, the version (rev:), and the id of the hook.

After setting up the configuration file, you need to install the hooks. Run the following command in your project directory:

With the hooks installed, they will automatically run on every git commit. If a hook finds issues, the commit will be blocked until those issues are resolved, ensuring that your code adheres to the standards set by the linters.

You can also manually run all the hooks against all the files in your repository with:

pre-commit run --all-files