Repository navigation
[FR] Integrate xfail #33
Description
I have a test that SEGFAULTs under random envs (usually only in one or two jobs in the matrix). So I decided to use forked to isolate it and marked it with xfail as follows:
@pytest.mark.xfail(
reason='This test causes SEGFAULT, flakily.',
strict=False,
)
@pytest.mark.forked
def test_exec_command(ssh_channel):
"""Test getting the output of a remotely executed command."""
...But, despite my expectations, when it crashes, pytest shows it as a failure and not as XFAIL.
Could you please clarify whether this behavior is intended or is it a bug? (looks like a bug to me)
Am I doing anything wrong? Am I missing something here?
(Ref: https://github.com/ansible/pylibssh/runs/728770734?check_suite_focus=true#step:9:120)
Activity
RonnyPfannschmidt commentedon Jun 2, 2020
Segfaults are currently not integrated
webknjaz commentedon Jun 2, 2020
Is it hard to integrate them?
[-][BUG?] Clarify `xfail` compatibility[/-][+][FR] Integrate `xfail`[/+]RonnyPfannschmidt commentedon Jun 2, 2020
i have no idea, i haven't worked on it and i don't plan to maintain/work on pytest-forked i only took it out of xdist to enable dropping the boxed feature from xdist itself
webknjaz commentedon Jun 2, 2020
@RonnyPfannschmidt where can I see how xfail is processed? I'm trying to figure out how things work in src/_pytest/mark/structures.py but it seems like it's probably not the place where I can hack into...
I want to contribute but struggling to find a starting point.
RonnyPfannschmidt commentedon Jun 2, 2020
I vaguely recall that xfail is processed in the skipping plugin
webknjaz commentedon Jun 5, 2020
So it turned out that for xfail to work, I need to set
outcome = 'skipped'andwasxfail = 'the reason message'attrs on the report object.webknjaz commentedon Jun 5, 2020
Opened an issue about this pytest-dev/pytest#7327