Skip to content

Correctly expanding Scenario Outline params for pytest BDD (fixes #636)#644

Closed
chasinglogic wants to merge 1 commit intoallure-framework:masterfrom
chasinglogic:master
Closed

Correctly expanding Scenario Outline params for pytest BDD (fixes #636)#644
chasinglogic wants to merge 1 commit intoallure-framework:masterfrom
chasinglogic:master

Conversation

@chasinglogic
Copy link
Copy Markdown

@chasinglogic chasinglogic commented Jan 13, 2022 •
edited
Loading

This fixes #636 by making each scenario outline param a separate param in the
generated allure JSON.

Context

When using a Scenario Outline and pytest-bdd the parameters passed to the allure listener were of the incorrect type. The Parameter class in Allure expects the value field to be a String but since the callspec has _pytest_bdd_example which is a dictionary the value would be of that type. Which would serialise to JSON correctly but cause the following error when Allure generate attempted to deserialise it:

Could not read test result file targetreports/6557fe86-9aff-4b34-9198-2a61facbdff6-result.json
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.lang.String` from Object value (token `JsonToken.START_OBJECT`)
 at [Source: (sun.nio.ch.ChannelInputStream); line: 1, column: 621] (through reference chain: io.qameta.allure.model.TestResult["parameters"]->java.util.ArrayList[0]->io.qameta.allure.model.Parameter["value"])

This change expands the pytest bdd examples so they are each a parameter and brings it in line with other test frameworks and Allure behaviour.

Checklist

@chasinglogic
fix: Correctly expand Scenario Outline params for pytest BDD
a840bf2 
This fixes #636 by making each scenario outline param a separate param in the
generated allure JSON.
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Jan 13, 2022 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@chasinglogic
Copy link
Copy Markdown
Author

chasinglogic commented Jan 13, 2022

Will write unit tests later today and am signing the CLA now.

@chasinglogic
Copy link
Copy Markdown
Author

chasinglogic commented Jan 13, 2022

Example of bad JSON before this change:

{
  "name": "<market> market, the Company logo returns the user to the home page [UK]",
  "status": "passed",
  "steps": [
    {
      "name": "Given the market is UK",
      "status": "passed",
      "start": 1641383124791,
      "stop": 1641383135301
    },
    {
      "name": "And I am on the Website home page",
      "status": "passed",
      "start": 1641383135302,
      "stop": 1641383141638
    },
    {
      "name": "When I click the Company logo",
      "status": "passed",
      "start": 1641383141639,
      "stop": 1641383141811
    },
    {
      "name": "Then I am on the home page in the relevant market",
      "status": "passed",
      "start": 1641383141811,
      "stop": 1641383141814
    }
  ],
  "parameters": [
    {
      "name": "_pytest_bdd_example",
      "value": {
        "market": "UK"
      }
    }
  ],
  "start": 1641383124772,
  "stop": 1641383141814,
  "uuid": "1a3b4903-ffca-bc2b-6df5-84d4a29e1466",
  "historyId": "7af04ec143715c102e88898fbce93f5f",
  "fullName": "header/logo.feature:<market> market, the Company logo returns the user to the home page",
  "labels": [
    {
      "name": "host",
      "value": "Anonymouss-MacBook-Pro.local"
    },
    {
      "name": "thread",
      "value": "4474-MainThread"
    },
    {
      "name": "framework",
      "value": "pytest-bdd"
    },
    {
      "name": "language",
      "value": "cpython3"
    },
    {
      "name": "feature",
      "value": "Website logo header checks"
    }
  ]
}

Good JSON after this change:

{
  "name": "<market> market, desktop size, the header contains the desktop Basket/cart icon [UK]",
  "status": "passed",
  "steps": [
    {
      "name": "Given the market is UK",
      "status": "passed",
      "start": 1642070685922,
      "stop": 1642070691849
    },
    {
      "name": "And the desktop window size is desktop",
      "status": "passed",
      "start": 1642070691854,
      "stop": 1642070691957
    },
    {
      "name": "And I am on the Website home page",
      "status": "passed",
      "start": 1642070691958,
      "stop": 1642070695608
    },
    {
      "name": "Then the site header contains the Basket/cart icon",
      "status": "passed",
      "start": 1642070695609,
      "stop": 1642070695634
    }
  ],
  "parameters": [
    {
      "name": "market",
      "value": "UK"
    }
  ],
  "start": 1642070685913,
  "stop": 1642070695635,
  "uuid": "ea31ba2b-7648-f82f-558c-7ab596d19331",
  "historyId": "6d7ed49e4a7590d0d7c5f1a67e81a8bd",
  "fullName": "header/basket.feature:<market> market, desktop size, the header contains the desktop Basket/cart icon",
  "labels": [
    {
      "name": "host",
      "value": "the-enterprise.local"
    },
    {
      "name": "thread",
      "value": "65605-MainThread"
    },
    {
      "name": "framework",
      "value": "pytest-bdd"
    },
    {
      "name": "language",
      "value": "cpython3"
    },
    {
      "name": "feature",
      "value": "Website basket/cart icon header checks"
    }
  ]
}

@chasinglogic
Copy link
Copy Markdown
Author

chasinglogic commented Jan 17, 2022

So I've been working through the broken tests but would like some confirmation that this solution will be acceptable before I spend much more time on this if that's possible?

@JagadeeshJayachandran
Copy link
Copy Markdown

JagadeeshJayachandran commented Jan 21, 2022

Any update, please. We have updated our feature files as per pytest-bdd 5.0.0, we didnt realize about this allure reporting issue. This issue blocks us from using pytest-bdd 5.0.0

@ofirm93
Copy link
Copy Markdown

ofirm93 commented May 26, 2022

@skhomuti is there a problem with this bug fix?
May someone re-trigger the checks? as this is a real issue which makes tests disappear from the report.

aspenboy
aspenboy approved these changes Jul 21, 2022
View reviewed changes
Copy link
Copy Markdown

@aspenboy aspenboy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

JagadeeshJayachandran
JagadeeshJayachandran approved these changes Jul 27, 2022
View reviewed changes
Copy link
Copy Markdown

@JagadeeshJayachandran JagadeeshJayachandran left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks fine @chasinglogic

@JagadeeshJayachandran
Copy link
Copy Markdown

JagadeeshJayachandran commented Jul 27, 2022

Can we please fix this code merge errors

@chasinglogic
Copy link
Copy Markdown
Author

chasinglogic commented Jul 28, 2022

@aspenboy @JagadeeshJayachandran while I appreciate the reviews, AFAICT the maintainer of this repository is @sseliverstov and they haven't merged any PRs in quite a long time.

I will not be monitoring this PR for further updates.

@skhomuti
Copy link
Copy Markdown
Contributor

skhomuti commented Jul 29, 2022

@chasinglogic hi! Thank you for your contribution and sorry for a little bit inactive period. I'll check this PR and return back for you with feedback. 🙏

@sqandrew
Copy link
Copy Markdown

sqandrew commented Nov 2, 2022

hello guys!
It already has a workaround but having a fix would be really great.

Will that fix will be released this year? @sseliverstov @skhomuti what do you think?

thanks in advance

@chasinglogic chasinglogic closed this by deleting the head repository Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@aspenboy aspenboy aspenboy approved these changes

@JagadeeshJayachandran JagadeeshJayachandran JagadeeshJayachandran approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

theme:pytest-bdd

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

pytest-bdd 5.0.0: error on generating report with failed tests with scenario outlines

7 participants

@chasinglogic @CLAassistant @JagadeeshJayachandran @ofirm93 @skhomuti @sqandrew @aspenboy