pytest parameterized session fixtures execute too many times
<p>Consider the following test code, which compares a mock run result with an expected result. The value of the run result depends on a value of a parameterized fixture paramfixture, which provides two values, so there are two possible variants of the run result. Since they are all session fixtures we should expect the run_result fixture execute only two times.</p>

<p>Now, please take a look at the test case test_run_result, which receives the run_result and expected_result fixtures to compare, and also receives the tolerance fixture, which is parameterized with two values. The test case checks if the difference between expected and resulted falls within the tolerance. Note that the run does not depend on the tolerance.</p>

<p>For some reason, which I donât understand Pytest executes the run_result() fixture three times. Can you explain why? </p>

<p>This was tested using pytest vers. 2.9.1</p>

<p>By the way, the run_result fixture would execute only two times if the test case weren't parameterized or were parameterized using a decoractor instead of a fixture i.e.: @pytest.mark.parametrize('tolerance', [1e-8, 1e-11]).</p>

<pre><code>import pytest

runcounter = 0

@pytest.fixture(scope="session", params=[1e-8, 1e-11])
def tolerance(request):
    """Precision in floating point compare."""
    return request.param

@pytest.fixture(scope='session', params=[1, 2])
def paramfixture(request):
    return request.param

@pytest.fixture(scope="session")
def expected_result(paramfixture):
    return 1 + paramfixture

@pytest.fixture(scope='session')
def run_result(paramfixture):
    global runcounter
    runcounter = runcounter + 1
    print "Run #", runcounter, 'param:', paramfixture
    return 1 + paramfixture

def test_run_result(run_result, expected_result, tolerance):
    print "run_result: %d, expected_result: %d" % (run_result, expected_result)
    assert abs(run_result - expected_result) &lt; tolerance
</code></pre>

<p>Pytest screenshot:</p>

<pre><code>$ py.test -vs test/end2end/test_temp.py
===================================================== test session starts ======================================================
platform linux2 -- Python 2.7.11, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /home/f557010/.conda/envs/sfpdev/bin/python
cachedir: .cache
rootdir: /home/f557010/svndev/SFP, inifile: pytest.ini
collected 4 items

test/end2end/test_temp.py::test_run_result[1e-08-1] Run # 1 param: 1
run_result: 2, expected_result: 2
PASSED
test/end2end/test_temp.py::test_run_result[1e-08-2] Run # 2 param: 2
run_result: 3, expected_result: 3
PASSED
test/end2end/test_temp.py::test_run_result[1e-11-2] 
run_result: 3, expected_result: 3
PASSED
test/end2end/test_temp.py::test_run_result[1e-11-1] Run # 3 param: 1
run_result: 2, expected_result: 2
PASSED

=================================================== 4 passed in 0.01 seconds ===================================================
</code></pre>

<p>It appears to be running the test 4 times, not 3, which makes sense if it's doing an all-combinations run.</p>

<ul>
<li>Run #1: Param 1, Tolerance 1</li>
<li>Run #2: Param 2, Tolerance 1</li>
<li>Run #3: Param 1, Tolerance 2</li>
<li>Run #4: Param 2, Tolerance 2</li>
</ul>

<p>Running four times seems to me like a reasonable approach given the test definition.</p>

python py.test