[Python] Pytest 基本介紹

程式語言：Python

Package：: pytest

基本探索規則

檔名為 test_*.py 或 *_test.py
method or function 的名字前綴為 test_*
位於 class 中同上命名規則的 method
但 class 的名字前綴為 Test* 且沒有 __init__ method
特殊檔名

conftest.py (可多個)
pytest.ini (唯一)
tox.ini
setup.cfg

基本指令

輸出格式

# test_show.py
import pytest


def test_sample1():
    assert 1 == 1


def test_sample2():
    assert [1, 2, 3] == [3, 2, 1]


@pytest.mark.xfail()
def test_sample3():
    assert 1 != 1


@pytest.mark.xfail()
def test_sample4():
    assert 1 == 1

建立好上面的檔案，輸入以下指令

pytest test_show.py

首先，第一行表示 session 啟動，再來是基本訊息，並收集到四個測試

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 4 items

再來是錯誤訊息，共四個測試，「點」表示 Pass，F 表示 Fail，x 表示預計是 fail，X 表示是預計是 Fail 卻 Pass

PASSED (.): 測試成功
FAILED (F): 測試失敗 (XPASS + strict 也視為失敗)
SKIPPED (s): 跳過測試
xfail (x): 指定失敗正常
XPASS (X): 指定失敗正常，卻成功
ERROR (E): 測試函數以外的錯誤，像是 fixture

test_show.py .FxX                                                        [100%]                                                    [100%]

再來是錯誤的地方

================================== FAILURES ===================================
________________________________ test_sample2 _________________________________

    def test_sample2():
>       assert [1, 2, 3] == [3, 2, 1]
E       assert [1, 2, 3] == [3, 2, 1]
E         At index 0 diff: 1 != 3
E         Use -v to get the full diff

test_show.py:9: AssertionError

最後的總結

========== 1 failed, 1 passed, 1 xfailed, 1 xpassed in 0.08 seconds ===========

若要較詳細的資訊，可加上 -v

pytest test_show.py -v

會直接說明測試的狀態，甚至會協助 debug

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0 -- d:\coding\venv\devlop\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 4 items

test_show.py::test_sample1 PASSED                                        [ 25%]
test_show.py::test_sample2 FAILED                                        [ 50%]
test_show.py::test_sample3 xfail                                         [ 75%]
test_show.py::test_sample4 XPASS                                         [100%]

================================== FAILURES ===================================
________________________________ test_sample2 _________________________________

    def test_sample2():
>       assert [1, 2, 3] == [3, 2, 1]
E       assert [1, 2, 3] == [3, 2, 1]
E         At index 0 diff: 1 != 3
E         Full diff:
E         - [1, 2, 3]
E         ?  ^     ^
E         + [3, 2, 1]
E         ?  ^     ^

test_show.py:9: AssertionError
========== 1 failed, 1 passed, 1 xfailed, 1 xpassed in 0.05 seconds ===========

subset Test

Test Function

一切建立在 assert 即可
```
def test_basic():
    assert 1 == 1
```

exception 測試

def test_exception():
    with pytest.raises(TypeError):
        raise TypeError

mark function

@pytest.mark.fruits
def test_apple():
    assert 1 == 1


@pytest.mark.fruits
@pytest.mark.vegetable
def test_tomato():
    assert 1 == 1

-m

pytest test_sample.py -m "fruits and not vegetable" -v

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0 -- d:\coding\venv\devlop\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 34 items / 33 deselected

test_sample.py::test_apple PASSED                                        [100%]

=================== 1 passed, 33 deselected in 0.11 seconds ===================

可跳過測試，並且可設定特定條件決定是否跳過

@pytest.mark.skip(reason="skip this")
def test_skip():
    assert False


import sys
@pytest.mark.skipif(sys.version_info < (4, 3), reason="requires python4.3")
def test_skipif():
    assert False

加上 -r，輸入以下指令，可印出跳過的原因

pytest test_sample.py -k skip -rxs

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 34 items / 32 deselected

test_sample.py ss                                                        [100%]
=========================== short test summary info ===========================
SKIP [1] test_sample.py:33: skip this
SKIP [1] test_sample.py:41: requires python4.3

================== 2 skipped, 32 deselected in 0.03 seconds ===================

期望會失敗的測試，但不跳過，若成功則會是 XPASS 狀態
也可設定條件，決定是否視為 xfail
當 xfail_strict=true 時，XPASS 也會視為 Fail，可在 pytest.ini 內設定

@pytest.mark.xfail(reason="fail but still test")
def test_xfail():
    assert False


@pytest.mark.xfail(reason="fail but still test maybe pass")
def test_xpass():
    assert True


version = (1, 0, 0)
@pytest.mark.xfail(version < (0, 5, 0), reason="if version over 0.5.0, it should pass")
def test_condition_xfail():
    assert True

加上 -r，輸入以下指令，可印出指定 fail 的原因

pytest test_sample.py -k _x -rxs

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 34 items / 31 deselected

test_sample.py xX.                                                       [100%]
=========================== short test summary info ===========================
XFAIL test_sample.py::test_xfail
  fail but still test

======== 1 passed, 31 deselected, 1 xfailed, 1 xpassed in 0.06 seconds ========

傳入參數測試，list 會逐項代入，可設定 ids 讓報表更好看
但在 ids_func 則是逐元素代入，這點要注意

@pytest.mark.parametrize("a,b,c", [(1, 2, 3), (4, 5, 6)])
def test_para(a, b, c):
    assert (a, b, c) == (1, 2, 3)


@pytest.mark.parametrize("a,b,c", [(1, 2, 3), (4, 5, 6)], ids=["tuple1", "tuple2"])
def test_para_ids(a, b, c):
    assert (a, b, c) == (1, 2, 3)


def ids_func(t):
    return "[{}]".format(t)


@pytest.mark.parametrize("a,b,c", [(1, 2, 3), (4, 5, 6)], ids=ids_func)
def test_para_ids_func(a, b, c):
    assert (a, b, c) == (1, 2, 3)


class Info:
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c


@pytest.mark.parametrize("i", [Info(1, 2, 3), Info(4, 5, 6)])
def test_para_class(i):
    assert i == Info(1, 2, 3)


def ids_class_func(t):
    return "Info({},{},{})".format(t.a, t.b, t.c)


@pytest.mark.parametrize("i", [Info(1, 2, 3), Info(4, 5, 6)], ids=ids_class_func)
def test_para_ids_class_func(i):
    assert i == Info(1, 2, 3)


@pytest.mark.parametrize(
    "i",
    [
        pytest.param(Info(1, 2, 3), id="basic"),
        pytest.param(Info(4, 5, 6), id="advance"),
    ],
)
def test_para_ids_setting(i):
    assert i == Info(1, 2, 3)

pytest test_sample.py -k test_para -v --tb=no

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0 -- d:\coding\venv\devlop\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 34 items / 22 deselected

test_sample.py::test_para[1-2-3] PASSED                                  [  8%]
test_sample.py::test_para[4-5-6] FAILED                                  [ 16%]
test_sample.py::test_para_ids[tuple1] PASSED                             [ 25%]
test_sample.py::test_para_ids[tuple2] FAILED                             [ 33%]
test_sample.py::test_para_ids_func[[1]-[2]-[3]] PASSED                   [ 41%]
test_sample.py::test_para_ids_func[[4]-[5]-[6]] FAILED                   [ 50%]
test_sample.py::test_para_class[i0] FAILED                               [ 58%]
test_sample.py::test_para_class[i1] FAILED                               [ 66%]
test_sample.py::test_para_ids_class_func[Info(1,2,3)] FAILED             [ 75%]
test_sample.py::test_para_ids_class_func[Info(4,5,6)] FAILED             [ 83%]
test_sample.py::test_para_ids_setting[basic] FAILED                      [ 91%]
test_sample.py::test_para_ids_setting[advance] FAILED                    [100%]

============== 9 failed, 3 passed, 22 deselected in 0.14 seconds ==============

Fixture

可獨立放在 confest.py 中

fixture 基本運用，fixture 本身的名字即是參數名

@pytest.fixture
def fixture_fun1():
    return 1


@pytest.fixture
def fixture_fun2():
    print("\nsetup")
    yield 1
    print("\nteardown")


def test_fixture(fixture_fun1, fixture_fun2):
    assert fixture_fun1 == 1
    assert fixture_fun2 == 1

輸入以下指令

pytest test_sample.py::test_fixture -v -s

================================================================== test session starts ===================================================================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0 -- d:\coding\venv\devlop\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 1 item

test_sample.py::test_fixture
setup
PASSED
teardown


================================================================ 1 passed in 0.03 seconds ================================================================

scope 展現，未指定 scope 預設為 function
因 class 無法輸入參數，所以會使用 @pytest.mark.usefixtures
其中的 method 仍可以輸入 fixture

@pytest.fixture(scope="function")
def func_scope():
    pass


@pytest.fixture(scope="module")
def mod_scope():
    pass


@pytest.fixture(scope="session")
def sess_scope():
    pass


@pytest.fixture(scope="class")
def class_scope():
    pass


def test_fixture_scope1(sess_scope, mod_scope, func_scope):
    pass


def test_fixture_scope2(sess_scope, mod_scope, func_scope):
    pass


@pytest.mark.usefixtures("class_scope")
class TestClassFixtures:
    def test_method1(self, mod_scope):
        pass

    def test_method2(self, func_scope):
        pass

輸入以下指令

pytest test_sample.py -k "test_fixture_scope or TestClass" --setup-show

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 34 items / 30 deselected

test_sample.py
SETUP    S sess_scope
    SETUP    M mod_scope
        SETUP    F func_scope
        test_sample.py::test_fixture_scope1 (fixtures used: func_scope, mod_scope, sess_scope).
        TEARDOWN F func_scope
        SETUP    F func_scope
        test_sample.py::test_fixture_scope2 (fixtures used: func_scope, mod_scope, sess_scope).
        TEARDOWN F func_scope
      SETUP    C class_scope
        test_sample.py::TestClassFixtures::test_method1 (fixtures used: class_scope, mod_scope).
        SETUP    F func_scope
        test_sample.py::TestClassFixtures::test_method2 (fixtures used: class_scope, func_scope).
        TEARDOWN F func_scope
      TEARDOWN C class_scope
    TEARDOWN M mod_scope
TEARDOWN S sess_scope

=================== 4 passed, 30 deselected in 0.05 seconds ===================

fixture 可互相呼叫，但只能呼叫本身以上的 scope，session > module > class > function

@pytest.fixture
def func_scope_with_sess_scope(sess_scope):
    pass


def test_fixture_with_fixture(func_scope_with_sess_scope):
    pass


# Error: ScopeMismatch
@pytest.fixture(scope="session")
def sess_scope_with_func_scope(func_scope):
    pass


def test_fixture_with_wrong_fixture(sess_scope_with_func_scope):
    pass

fixture 名字可重新命名

@pytest.fixture(name="shortName")
def fixture_name_too_long():
    pass


def test_fixture_rename(shortName):
    pass

傳入參數，大致上跟 @pytest.mark.parametrize 差不多
但在 idsFunc 的處理上有些不同，在此並未逐元素傳入
request 則是內建的特別 fixtures

@pytest.fixture(params=[(1, 2, 3), [4, 5, 6]])
def func_fixture_para(request):
    return request.param


def test_fixture_para(func_fixture_para):
    assert func_fixture_para == (1, 2, 3)


@pytest.fixture(params=[(1, 2, 3), [4, 5, 6]], ids=["tuple1", "tuple2"])
def func_fixture_para_ids(request):
    return request.param


def test_fixture_para_ids(func_fixture_para_ids):
    assert func_fixture_para_ids == (1, 2, 3)


def ids_func_fixture(t):
    return "{},{},{}".format(t[0], t[1], t[2])


@pytest.fixture(params=[(1, 2, 3), [4, 5, 6]], ids=ids_func_fixture)
def func_fixture_para_idsFunc(request):
    return request.param


def test_fixture_para_ids_func(func_fixture_para_idsFunc):
    assert func_fixture_para_idsFunc == (1, 2, 3)

輸入以下指令

pytest test_sample.py -k test_fixture_para -v --tb=no

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-4.0.2, py-1.7.0, pluggy-0.8.0 -- d:\coding\venv\devlop\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\zps\Desktop\pytest, inifile:
collected 35 items / 29 deselected

test_sample.py::test_fixture_para[func_fixture_para0] PASSED             [ 16%]
test_sample.py::test_fixture_para[func_fixture_para1] FAILED             [ 33%]
test_sample.py::test_fixture_para_ids[tuple1] PASSED                     [ 50%]
test_sample.py::test_fixture_para_ids[tuple2] FAILED                     [ 66%]
test_sample.py::test_fixture_para_ids_func[1,2,3] PASSED                 [ 83%]
test_sample.py::test_fixture_para_ids_func[4,5,6] FAILED                 [100%]

============== 3 failed, 3 passed, 29 deselected in 0.07 seconds ==============

Flask 範例

flask_pytest
├─src
│ ├─main.py
│ │
│ ├─flaskr
│ │ ├─main.py
│ │ ├─models.py
│ │ ├─views.py
│ │ └─__init__.py
│ │
│ └─instance
│ └config.cfg
│
└─tests
├─conftest.py
│
├─func
│ └test_views.py
│
└─unit
└test_models.py

詳細請參考連結的 source code
只列出比較重要的部分，首先必須定義 create_app function，這樣便可以自由載入設定檔
flaskr_pytest/src/flaskr/main.py

"""
main
"""
from flask import Flask, g
from flask_sqlalchemy import SQLAlchemy

#######################
#### Configuration ####
#######################

# Create the instances of the Flask extensions (flask-sqlalchemy, flask-login, etc.) in
# the global scope, but without any arguments passed in.  These instances are not attached
# to the application at this point.
db = SQLAlchemy()


######################################
#### Application Factory Function ####
######################################


def create_app(config_filename=None):
    app = Flask(__name__, instance_relative_config=True)
    print("instance Path:", app.instance_path)
    app.config.from_pyfile(config_filename)

    initialize_extensions(app)
    register_blueprints(app)

    @app.before_request
    def configSetting():
        g.USERNAME = app.config["USERNAME"]
        g.PASSWORD = app.config["PASSWORD"]

    print("DB Path:", app.config["SQLALCHEMY_DATABASE_URI"])

    return app


##########################
#### Helper Functions ####
##########################


def initialize_extensions(app):
    # Since the application instance is now created, pass it to each Flask
    # extension instance to bind it to the Flask application instance (app)
    db.init_app(app)


def register_blueprints(app):
    # Since the application instance is now created, register each Blueprint
    # with the Flask application instance (app)
    from .views import flaskr

    app.register_blueprint(flaskr)

利用 fixture 達到初始化目的，再來就是自由發揮了
flaskr_pytest/tests/conftest.py

import pytest
from flask import g, session
import sys, os

# 若是用 pip install -e，無需下面兩行
myPath = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, myPath + "/../src")
from flaskr import create_app, db, models


@pytest.fixture
def new_User(test_client):
    user = models.User(name="Sun")

    return user


@pytest.fixture(scope="session")
def test_client():
    app = create_app("config.cfg")

    @app.before_request
    def configSetting():
        g.USERNAME = app.config["USERNAME"]
        g.PASSWORD = app.config["PASSWORD"]

    # Flask provides a way to test your application by exposing the Werkzeug test Client
    # and handling the context locals for you.
    client = app.test_client()

    # Establish an application context before running the tests.
    # for SQLAlchemy setting
    ctx = app.app_context()
    ctx.push()

    yield client  # this is where the testing happens!

    ctx.pop()


@pytest.fixture
def init_db(test_client):
    # Create the database and the database table
    db.create_all()

    yield db

    db.session.remove()
    db.drop_all()

參考

Testing Flask Applications
Testing a Flask Application using pytest
Testing Flask SQLAlchemy database with pytest
Python Testing with pytest

子風的知識庫

搜尋此網誌