[Python] Pytest Advance

程式語言：Python
Package：
pytest
官方文件
範例 Source code

功能：Python test 設定與 Builtin Fixtures

習慣放 __init__ 在每個測試資料夾，以免出現同名的檔案
即使 function 名字不同，個別測試時 ok，但當集體測試時便會出錯

built-in fixtures

  • 暫存
    • tmpdir: Function Scope
    • tmpdir_factory: Session Scope
test_advance.py
  1. def test_tmpdir(tmpdir):
  2.     tmdir_same_part(tmpdir)
  3.  
  4.  
  5. def test_tmpdir_factory(tmpdir_factory):
  6.     dir_ = tmpdir_factory.mktemp("mydir")
  7.  
  8.     # 印出暫存位置
  9.     base_temp = tmpdir_factory.getbasetemp()
  10.     print("base:", base_temp)
  11.  
  12.     tmdir_same_part(dir_)
  13.  
  14.  
  15. def tmdir_same_part(dir_):
  16.     file1 = dir_.join("fileName1.txt")
  17.     sub_dir = dir_.mkdir("dirName")
  18.     file2 = sub_dir.join("fileName2.txt")
  19.  
  20.     file1.write("一二三")
  21.     file2.write("四五六")
  22.  
  23.     assert file1.read() == "一二三"
  24.     assert file2.read() == "四五六"
  • cli command
    • pytestconfig
conftest.py，自定義的 cli command 必須放在這，或是 plugin 中
  1. # pytest_addoption 是特定名字
  2. def pytest_addoption(parser):
  3.     parser.addoption("--mybool", action="store_true", help="boolean option")
  4.     parser.addoption("--name", action="store", default="abc", help="name: abc or ABC")
test_advance.py
  1. import pytest
  2.  
  3.  
  4. # pytestconfig fixtures
  5. def test_pytestconfig_option(pytestconfig):
  6.     print('"name" :', pytestconfig.getoption("name"), pytestconfig.option.name)
  7.     print('"mybool" :', pytestconfig.getoption("mybool"), pytestconfig.option.mybool)
  8.  
  9.  
  10. # pytestconfig fixtures
  11. def test_pytestconfig(pytestconfig):
  12.     print("args            :", pytestconfig.args)
  13.     print("inifile         :", pytestconfig.inifile)
  14.     print("invocation_dir  :", pytestconfig.invocation_dir)
  15.     print("rootdir         :", pytestconfig.rootdir)
  16.     print("-k EXPRESSION   :", pytestconfig.getoption("keyword"))
  17.     print("-v, --verbose   :", pytestconfig.getoption("verbose"))
  18.     print("-q, --quiet     :", pytestconfig.getoption("--quiet"))
  19.     print("-l, --showlocals:", pytestconfig.getoption("showlocals"))
  20.     print("--tb=style      :", pytestconfig.getoption("tbstyle"))
  21.  
  22.  
  23. # request fixtures
  24. def test_pytestconfig_legacy(request):
  25.     print('\n"name" :', request.config.getoption("name"))
  26.     print('"myopt" :', request.config.getoption("mybool"))
  27.     print('"keyword" :', request.config.getoption("keyword"))
輸入 help 可看到已加入
pytest -h
custom options:
  --mybool              boolean option
  --name=NAME           name: abc or ABC
  • cache
    • 快取運用
test_advance.py
  1. import pytest
  2. import datetime
  3. import random
  4. import time
  5. from collections import namedtuple
  6.  
  7. Duration = namedtuple("Duration", ["current", "last"])
  8.  
  9.  
  10. @pytest.fixture(scope="session")
  11. def duration_cache(request):
  12.     # 存放的資料夾與檔名
  13.     key = "folder/fileName"
  14.     d = Duration({}, request.config.cache.get(key, {}))
  15.     yield d
  16.     request.config.cache.set(key, d.current)
  17.  
  18.  
  19. @pytest.fixture()
  20. def check_duration(request, duration_cache):
  21.     d = duration_cache
  22.     nodeid = request.node.nodeid
  23.     start_time = datetime.datetime.now()
  24.     yield
  25.     duration = (datetime.datetime.now() - start_time).total_seconds()
  26.     d.current[nodeid] = duration
  27.     if d.last.get(nodeid, None) is not None:
  28.         errorstring = "測試時間是上次的兩倍"
  29.         assert duration <= (d.last[nodeid] * 2), errorstring
  30.  
  31.  
  32. @pytest.mark.parametrize("i", range(5))
  33. def test_cache(i, check_duration):
  34.     time.sleep(random.random())
輸入指令，可看到 cache 存放的值
pytest test_advance.py --cache-clear
pytest test_advance.py --cache-show
folder\fileName contains:
  {'test_advance.py::test_cache[0]': 0.122007,
   'test_advance.py::test_cache[1]': 0.996057,
   'test_advance.py::test_cache[2]': 0.941054,
   'test_advance.py::test_cache[3]': 0.222012,
   'test_advance.py::test_cache[4]': 0.799045}
  • capsys
    • 獲取 stdout & stdout
    • 暫時輸出 stdout
test_advance.py
  1. import sys
  2. import pytest
  3.  
  4.  
  5. def capsys_Hi(name):
  6.     print("Hi, {}".format(name))
  7.     print("有錯誤", file=sys.stderr)
  8.  
  9.  
  10. def test_capsys_Hi(capsys):
  11.     capsys_Hi("一")
  12.     out, err = capsys.readouterr()
  13.     assert out == "Hi, 一\n"
  14.     assert "錯誤" in err
  15.  
  16.     capsys_Hi("二")
  17.     capsys_Hi("三")
  18.     out, err = capsys.readouterr()
  19.     assert out == "Hi, 二\nHi, 三\n"
  20.     assert err == ""
  21.  
  22.  
  23. def test_capsys_disabled(capsys):
  24.     with capsys.disabled():
  25.         print("\n總是顯示")
  26.     print("有 -s 才顯示")
  • monkeypatch
    • setattr(target, name, value=, raising=True): Set an attribute. 
    • delattr(target, name=, raising=True): Delete an attribute. 
    • setitem(dic, name, value): Set a dictionary entry. 
    • delitem(dic, name, raising=True): Delete a dictionary entry. 
    • setenv(name, value, prepend=None): Set an environmental variable. 
    • delenv(name, raising=True): Delete an environmental variable. 
    • syspath_prepend(path): Prepend path to sys.path
    • chdir(path): Change the current working directory
  • 可搭配 unittest.mock
test_advance.py
  1. import math
  2.  
  3.  
  4. def test_monkeypatch(monkeypatch):
  5.     d = {"a": 1}
  6.     assert d.get("a") == 1
  7.     monkeypatch.setitem(d, "a", 100)
  8.     assert d.get("a") == 100
  9.  
  10.     with monkeypatch.context() as m:
  11.         m.setattr(math, "cos", (lambda x: x))
  12.         assert math.cos("a") == "a"
  13.     assert math.cos(math.pi) == -1

Configuration

除了 setup.cfg 開頭略有不同，格式是一致的
pytest.ini
[pytest]
addopts = -rsxX -l --tb=short --strict
xfail_strict = true
; 分號註解
setup.cfg
[tool:pytest]
addopts = -rsxX -l --tb=short --strict
xfail_strict = true
; 分號註解
tox.ini
[pytest]
addopts = -rsxX -l --tb=short --strict
xfail_strict = true
; 分號註解
help 中有定義可設定的值
pytest -h
[pytest] ini-options in the first pytest.ini|tox.ini|setup.cfg file found:
  markers (linelist)       markers for test functions
  empty_parameter_set_mark (string) default marker for empty parametersets
  norecursedirs (args)     directory patterns to avoid for recursion
  testpaths (args)         directories to search for tests when no files or dire
  console_output_style (string) console output: classic or with additional progr
  usefixtures (args)       list of default fixtures to be used with this project
  python_files (args)      glob-style file patterns for Python test module disco
  python_classes (args)    prefixes or glob names for Python test class discover
  python_functions (args)  prefixes or glob names for Python test function and m
  xfail_strict (bool)      default for the strict parameter of xfail markers whe
  junit_suite_name (string) Test suite name for JUnit report
  junit_logging (string)   Write captured log messages to JUnit report: one of n
  doctest_optionflags (args) option flags for doctests
  doctest_encoding (string) encoding used for doctest files
  cache_dir (string)       cache directory path.
  filterwarnings (linelist) Each line specifies a pattern for warnings.filterwar
  log_print (bool)         default value for --no-print-logs
  log_level (string)       default value for --log-level
  log_format (string)      default value for --log-format
  log_date_format (string) default value for --log-date-format
  log_cli (bool)           enable log display during test run (also known as "li
  log_cli_level (string)   default value for --log-cli-level
  log_cli_format (string)  default value for --log-cli-format
  log_cli_date_format (string) default value for --log-cli-date-format
  log_file (string)        default value for --log-file
  log_file_level (string)  default value for --log-file-level
  log_file_format (string) default value for --log-file-format
  log_file_date_format (string) default value for --log-file-date-format
  addopts (args)           extra command line options
  minversion (string)      minimally required pytest version
  mock_traceback_monkeypatch (string) Monkeypatch the mock library to improve re
  mock_use_standalone_module (string) Use standalone "mock" (from PyPI) instead
以下是常用的設定
  • addopts
    • 預設命令動作 
      addopts = -rsxX -l --tb=short --strict
  • markers
    • 註冊 markers 與 --strict 搭配使用，以免打錯字
      markers =
 hot: hot run
 get: get function test
    • 可用以下指令查詢 
      pytest --markers
  • minversion
    • 指定最小 pytest version 
      minversion = 3.0
  • norecursedirs
    • 不用尋找的檔案與資料夾，* 表示任意長度的值，空格隔開
      ; default: .* build dist CVS _darcs {arch} *.egg
norecursedirs = .* venv src *.egg dist build
  • testpaths
    • 指定尋找的位置
      testpaths = tests
  • python_classes
    python_files
    python_functions
    • 更改尋找 test 的規則，* 表示任意長度的值，空格隔開
      python_classes = *Test Test* *Suite
python_files = test_* *_test check_*
python_functions = test_* check_*
  • xfail_strict
    • 指定 xfail 的 function 測試 pass 時，視為 Error
      xfail_strict = true

With Other Tools

pdb
pytest -v --lf -x --pdb
  • p/print expr
    • Prints the value of exp. 
  • pp expr
    • Pretty prints the value of expr. 
  • l/list
    • Lists the point of failure and five lines of code above and below.
    • l/list begin,end: Lists specific line numbers. 
  • a/args
    • Prints the arguments of the current function with their values. (This is helpful when in a test helper function.) 
  • u/up
    • Moves up one level in the stack trace. 
  • d/down
    • Moves down one level in the stack trace. q/quit: Quits the debugging session.
pytest-cov
pytest --cov=src

# 可輸出 html 格式的報表
pytest --cov=src --cov-report=html

參考

Python Testing with pytest
unittest.mock
pytest-cov’s documentation

留言

張貼留言