[Python] Pytest Advance

程式語言：Python

Package：

pytest

官方文件
範例 Source code

功能：Python test 設定與 Builtin Fixtures

習慣放 __init__ 在每個測試資料夾，以免出現同名的檔案
即使 function 名字不同，個別測試時 ok，但當集體測試時便會出錯

built-in fixtures

• 暫存
• tmpdir: Function Scope
• tmpdir_factory: Session Scope

test_advance.py

def test_tmpdir(tmpdir):
    tmdir_same_part(tmpdir)


def test_tmpdir_factory(tmpdir_factory):
    dir_ = tmpdir_factory.mktemp("mydir")

    # 印出暫存位置
    base_temp = tmpdir_factory.getbasetemp()
    print("base:", base_temp)

    tmdir_same_part(dir_)


def tmdir_same_part(dir_):
    file1 = dir_.join("fileName1.txt")
    sub_dir = dir_.mkdir("dirName")
    file2 = sub_dir.join("fileName2.txt")

    file1.write("一二三")
    file2.write("四五六")

    assert file1.read() == "一二三"
    assert file2.read() == "四五六"

• cli command
• pytestconfig

conftest.py，自定義的 cli command 必須放在這，或是 plugin 中

# pytest_addoption 是特定名字
def pytest_addoption(parser):
    parser.addoption("--mybool", action="store_true", help="boolean option")
    parser.addoption("--name", action="store", default="abc", help="name: abc or ABC")

test_advance.py

import pytest


# pytestconfig fixtures
def test_pytestconfig_option(pytestconfig):
    print('"name" :', pytestconfig.getoption("name"), pytestconfig.option.name)
    print('"mybool" :', pytestconfig.getoption("mybool"), pytestconfig.option.mybool)


# pytestconfig fixtures
def test_pytestconfig(pytestconfig):
    print("args            :", pytestconfig.args)
    print("inifile         :", pytestconfig.inifile)
    print("invocation_dir  :", pytestconfig.invocation_dir)
    print("rootdir         :", pytestconfig.rootdir)
    print("-k EXPRESSION   :", pytestconfig.getoption("keyword"))
    print("-v, --verbose   :", pytestconfig.getoption("verbose"))
    print("-q, --quiet     :", pytestconfig.getoption("--quiet"))
    print("-l, --showlocals:", pytestconfig.getoption("showlocals"))
    print("--tb=style      :", pytestconfig.getoption("tbstyle"))


# request fixtures
def test_pytestconfig_legacy(request):
    print('\n"name" :', request.config.getoption("name"))
    print('"myopt" :', request.config.getoption("mybool"))
    print('"keyword" :', request.config.getoption("keyword"))

輸入 help 可看到已加入

pytest -h

custom options:
  --mybool              boolean option
  --name=NAME           name: abc or ABC

• cache
• 快取運用

test_advance.py

import pytest
import datetime
import random
import time
from collections import namedtuple

Duration = namedtuple("Duration", ["current", "last"])


@pytest.fixture(scope="session")
def duration_cache(request):
    # 存放的資料夾與檔名
    key = "folder/fileName"
    d = Duration({}, request.config.cache.get(key, {}))
    yield d
    request.config.cache.set(key, d.current)


@pytest.fixture()
def check_duration(request, duration_cache):
    d = duration_cache
    nodeid = request.node.nodeid
    start_time = datetime.datetime.now()
    yield
    duration = (datetime.datetime.now() - start_time).total_seconds()
    d.current[nodeid] = duration
    if d.last.get(nodeid, None) is not None:
        errorstring = "測試時間是上次的兩倍"
        assert duration <= (d.last[nodeid] * 2), errorstring


@pytest.mark.parametrize("i", range(5))
def test_cache(i, check_duration):
    time.sleep(random.random())

輸入指令，可看到 cache 存放的值

pytest test_advance.py --cache-clear
pytest test_advance.py --cache-show

folder\fileName contains:
  {'test_advance.py::test_cache[0]': 0.122007,
   'test_advance.py::test_cache[1]': 0.996057,
   'test_advance.py::test_cache[2]': 0.941054,
   'test_advance.py::test_cache[3]': 0.222012,
   'test_advance.py::test_cache[4]': 0.799045}

• capsys
• 獲取 stdout & stdout
• 暫時輸出 stdout

test_advance.py

import sys
import pytest


def capsys_Hi(name):
    print("Hi, {}".format(name))
    print("有錯誤", file=sys.stderr)


def test_capsys_Hi(capsys):
    capsys_Hi("一")
    out, err = capsys.readouterr()
    assert out == "Hi, 一\n"
    assert "錯誤" in err

    capsys_Hi("二")
    capsys_Hi("三")
    out, err = capsys.readouterr()
    assert out == "Hi, 二\nHi, 三\n"
    assert err == ""


def test_capsys_disabled(capsys):
    with capsys.disabled():
        print("\n總是顯示")
    print("有 -s 才顯示")

• monkeypatch
• setattr(target, name, value=, raising=True): Set an attribute. 
• delattr(target, name=, raising=True): Delete an attribute. 
• setitem(dic, name, value): Set a dictionary entry. 
• delitem(dic, name, raising=True): Delete a dictionary entry. 
• setenv(name, value, prepend=None): Set an environmental variable. 
• delenv(name, raising=True): Delete an environmental variable. 
• syspath_prepend(path): Prepend path to sys.path
• chdir(path): Change the current working directory
• 可搭配 unittest.mock

test_advance.py

import math


def test_monkeypatch(monkeypatch):
    d = {"a": 1}
    assert d.get("a") == 1
    monkeypatch.setitem(d, "a", 100)
    assert d.get("a") == 100

    with monkeypatch.context() as m:
        m.setattr(math, "cos", (lambda x: x))
        assert math.cos("a") == "a"
    assert math.cos(math.pi) == -1

Configuration

除了 setup.cfg 開頭略有不同，格式是一致的
pytest.ini

[pytest]
addopts = -rsxX -l --tb=short --strict
xfail_strict = true
; 分號註解

setup.cfg

[tool:pytest]
addopts = -rsxX -l --tb=short --strict
xfail_strict = true
; 分號註解

tox.ini

[pytest]
addopts = -rsxX -l --tb=short --strict
xfail_strict = true
; 分號註解

help 中有定義可設定的值

pytest -h
[pytest] ini-options in the first pytest.ini|tox.ini|setup.cfg file found:
  markers (linelist)       markers for test functions
  empty_parameter_set_mark (string) default marker for empty parametersets
  norecursedirs (args)     directory patterns to avoid for recursion
  testpaths (args)         directories to search for tests when no files or dire
  console_output_style (string) console output: classic or with additional progr
  usefixtures (args)       list of default fixtures to be used with this project
  python_files (args)      glob-style file patterns for Python test module disco
  python_classes (args)    prefixes or glob names for Python test class discover
  python_functions (args)  prefixes or glob names for Python test function and m
  xfail_strict (bool)      default for the strict parameter of xfail markers whe
  junit_suite_name (string) Test suite name for JUnit report
  junit_logging (string)   Write captured log messages to JUnit report: one of n
  doctest_optionflags (args) option flags for doctests
  doctest_encoding (string) encoding used for doctest files
  cache_dir (string)       cache directory path.
  filterwarnings (linelist) Each line specifies a pattern for warnings.filterwar
  log_print (bool)         default value for --no-print-logs
  log_level (string)       default value for --log-level
  log_format (string)      default value for --log-format
  log_date_format (string) default value for --log-date-format
  log_cli (bool)           enable log display during test run (also known as "li
  log_cli_level (string)   default value for --log-cli-level
  log_cli_format (string)  default value for --log-cli-format
  log_cli_date_format (string) default value for --log-cli-date-format
  log_file (string)        default value for --log-file
  log_file_level (string)  default value for --log-file-level
  log_file_format (string) default value for --log-file-format
  log_file_date_format (string) default value for --log-file-date-format
  addopts (args)           extra command line options
  minversion (string)      minimally required pytest version
  mock_traceback_monkeypatch (string) Monkeypatch the mock library to improve re
  mock_use_standalone_module (string) Use standalone "mock" (from PyPI) instead

以下是常用的設定

• addopts
• 預設命令動作

  addopts = -rsxX -l --tb=short --strict

• markers
• 註冊 markers 與 --strict 搭配使用，以免打錯字
  

  markers =
   hot: hot run
   get: get function test
  

• 可用以下指令查詢

  pytest --markers

• minversion
• 指定最小 pytest version

  minversion = 3.0

• norecursedirs
• 不用尋找的檔案與資料夾，* 表示任意長度的值，空格隔開
  

  ; default: .* build dist CVS _darcs {arch} *.egg
  norecursedirs = .* venv src *.egg dist build

• testpaths
• 指定尋找的位置
  

  testpaths = tests

• python_classes
  python_files
  python_functions
• 更改尋找 test 的規則，* 表示任意長度的值，空格隔開
  

  python_classes = *Test Test* *Suite
  python_files = test_* *_test check_*
  python_functions = test_* check_*
  

• xfail_strict
• 指定 xfail 的 function 測試 pass 時，視為 Error
  

  xfail_strict = true

With Other Tools

pdb

pytest -v --lf -x --pdb

• p/print expr
• Prints the value of exp. 
• pp expr
• Pretty prints the value of expr. 
• l/list
• Lists the point of failure and five lines of code above and below.
• l/list begin,end: Lists specific line numbers. 
• a/args
• Prints the arguments of the current function with their values. (This is helpful when in a test helper function.) 
• u/up
• Moves up one level in the stack trace. 
• d/down
• Moves down one level in the stack trace. q/quit: Quits the debugging session.

pytest-cov

pytest --cov=src

# 可輸出 html 格式的報表
pytest --cov=src --cov-report=html

參考

Python Testing with pytest
unittest.mock
pytest-cov’s documentation