Skip to content

[TT-17615] Update the API tests to run in parallel #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
konrad-sol merged 1 commit into from
Jul 27, 2026
+1 −2
Merged

[TT-17615] Update the API tests to run in parallel #146

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/actions/tests/api-tests/action.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@
USER_API_SECRET: ${{ inputs.user_api_secret }}
API_MARKERS: ${{ inputs.api_markers }}
run: |
pytest="pytest --ci --random-order --force-flaky --no-success-flaky-report --maxfail=3 --junitxml=${XUNIT_REPORT_PATH} --cache-clear --ignore=./tests/mdcb -v --log-cli-level=ERROR"
pip install --no-deps -r requirements.txt
cat >pytest.env <<-EOF
TYK_TEST_BASE_URL=http://localhost:3000/
Expand All @@ -44,5 +43,5 @@
GATEWAY_CONTAINER_NAME=tyk
TYK_TEST_KEYCLOAK_URL=http://localhost:8180
TYK_TEST_KEYCLOAK_INTERNAL_URL=http://keycloak:8080
EOF

Check warning on line 46 in .github/actions/tests/api-tests/action.yaml

View check run for this annotation

probelabs / Visor: architecture

architecture Issue 

The action's test configuration has been moved to `pytest_ci.ini`, but this file is not part of the review. This makes it impossible to confirm that critical test execution flags (e.g., `--maxfail=3`, `--force-flaky`, `--random-order`) have been preserved. The absence of these flags could negatively impact CI performance and reliability by changing how test failures and flaky tests are handled.
Raw output 
Ensure that consuming repositories provide a `pytest_ci.ini` that includes the configurations previously hardcoded in this action. To make this dependency clearer, you could add a comment in the action's YAML file listing the expected settings or link to an example configuration file.

Check warning on line 46 in .github/actions/tests/api-tests/action.yaml

View check run for this annotation

probelabs / Visor: architecture

architecture Issue 

The path to the pytest configuration file (`pytest_ci.ini`) is hardcoded within the action. This creates a rigid dependency and limits reusability, as consumers of this action cannot specify a different configuration file if their project structure or needs differ.
Raw output 
To improve flexibility and reusability, make the configuration file path an input parameter for the action. You can set `pytest_ci.ini` as the default value to maintain current behavior for existing users. Example: `inputs: { pytest_config: { default: 'pytest_ci.ini' } }` and then use `${{ inputs.pytest_config }}` in the run step.
env $(cat pytest.env | xargs) $pytest -m "${{ inputs.api_markers }}"
env $(cat pytest.env | xargs) pytest -c pytest_ci.ini --ci --junitxml=${XUNIT_REPORT_PATH} -m "${{ inputs.api_markers }}"

Check failure on line 47 in .github/actions/tests/api-tests/action.yaml

View check run for this annotation

probelabs / Visor: security

security Issue 

The `xunit_report_path` input is used to construct a shell command without proper quoting. An attacker who can control this input could inject arbitrary commands to be executed on the GitHub Actions runner. The unquoted use of `${XUNIT_REPORT_PATH}` allows for shell command injection.
Raw output 
Quote the `${XUNIT_REPORT_PATH}` variable in the `run` step to prevent shell injection. By wrapping the variable in double quotes, you ensure that its value is treated as a single string, even if it contains spaces or shell metacharacters.
Comment thread
konrad-sol marked this conversation as resolved.
Loading