Skip to content

Commit 5659bb7

Browse files
authored
Migrate PostCommit workflows from pull_request_target to pull_request (#40373)
1 parent 086ab9b commit 5659bb7

95 files changed

Lines changed: 360 additions & 292 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ACTIONS.md‎

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -31,15 +31,8 @@ Currently, we have both GitHub-hosted and self-hosted runners for running the Gi
3131
* Ubuntu 24.04 self-hosted runner: `[self-hosted, ubuntu-24.04, main]` (also `small`, `highmem`, or `highmem22` pool labels as needed)
3232
* Windows Server 2019 self-hosted runner: `[self-hosted, windows-server-2019]`
3333
* MacOS GitHub-hosted runner: `macos-latest`
34-
* Every workflow that tests the source code, needs to have the workflow trigger `pull_request_target` instead of `pull_request`.
34+
* Every workflow that tests the source code needs to have the workflow trigger `pull_request`.
3535
* The workflow must have set read permissions for all the available scopes and jobs: `permissions: read-all`. It must be set at the top of the `jobs` directive.
36-
* For those workflows that have the `pull_request_target` trigger, in the checkout step must be added a ref to `${{ github.event.pull_request.head.sha }}`
37-
``` yaml
38-
- name: Checkout code
39-
uses: actions/checkout@v#
40-
with:
41-
ref: ${{ github.event.pull_request.head.sha }}
42-
```
4336
* If your workflow runs successfully in a GitHub-hosted runner but not in the self-hosted runner, it might need a new installation step.
4437
```yaml
4538
- name: Setup Node
@@ -55,7 +48,7 @@ Currently, we have both GitHub-hosted and self-hosted runners for running the Gi
5548
```yaml
5649
name: GitHub Actions Example
5750
on:
58-
pull_request_target:
51+
pull_request:
5952
branches: ['master']
6053
permissions: read-all
6154
jobs:
@@ -64,8 +57,6 @@ jobs:
6457
steps:
6558
- name: Check out repository code
6659
uses: actions/checkout@v2
67-
with:
68-
ref: ${{ github.event.pull_request.head.sha }}
6960
- run: echo "This job is now running on a ubuntu server hosted by Apache Beam!"
7061
- name: Setup Node
7162
uses: actions/setup-node@v3
@@ -85,7 +76,6 @@ jobs:
8576
* A **detailed review** for changes in the workflows is needed due to important **security concerns**.
8677
* **DO NOT** Approve and Run changes in the workflows in the PR Conversation tab, under "Workflow(s) awaiting approval".
8778
* For approving the updates in the workflows, you should go to the Repository Actions and filter All Workflows by `action_required`. The search will display the workflows that need to be reviewed before running. **Please make sure reviewing the file that is referenced by the workflow.**
88-
* Seed job will be emulated using the `Approve and Run` built-in feature of GitHub Actions, since the workflows will use the `pull_request_target` directive; no modifications would be allowed either for new or existent jobs unless a committer explicitly approves the job from GitHub Actions UI.
8979

9080
#### Issue Management
9181
Phrases self-assign, close, or manage labels on an issue:

‎.github/workflows/IO_Iceberg_Integration_Tests.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ name: IcebergIO Integration Tests
1818
on:
1919
schedule:
2020
- cron: '15 4/6 * * *'
21-
pull_request_target:
21+
pull_request:
2222
paths: [ 'release/trigger_all_tests.json', '.github/trigger_files/IO_Iceberg_Integration_Tests.json' ]
2323
workflow_dispatch:
2424

25-
# Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
25+
# Setting explicit permissions for the action
2626
permissions:
2727
actions: write
2828
pull-requests: write
@@ -51,7 +51,7 @@ env:
5151
jobs:
5252
IO_Iceberg_Integration_Tests:
5353
if: |
54-
github.event_name == 'pull_request_target' ||
54+
github.event_name == 'pull_request' ||
5555
github.event_name == 'workflow_dispatch' ||
5656
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
5757
github.event.comment.body == 'Run IcebergIO Integration Test'

‎.github/workflows/IO_Iceberg_Integration_Tests_Dataflow.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ name: IcebergIO Integration Tests on Dataflow
1818
on:
1919
schedule:
2020
- cron: '30 4/6 * * *'
21-
pull_request_target:
21+
pull_request:
2222
paths: [ 'release/trigger_all_tests.json', '.github/trigger_files/IO_Iceberg_Integration_Tests_Dataflow.json' ]
2323
workflow_dispatch:
2424

25-
# Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
25+
# Setting explicit permissions for the action
2626
permissions:
2727
actions: write
2828
pull-requests: write
@@ -51,7 +51,7 @@ env:
5151
jobs:
5252
IO_Iceberg_Integration_Tests_Dataflow:
5353
if: |
54-
github.event_name == 'pull_request_target' ||
54+
github.event_name == 'pull_request' ||
5555
github.event_name == 'workflow_dispatch' ||
5656
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
5757
github.event.comment.body == 'Run IcebergIO Integration Tests on Dataflow'

‎.github/workflows/IO_Iceberg_Managed_Integration_Tests_Dataflow.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ name: IcebergIO Managed Integration Tests on Dataflow
1818
on:
1919
schedule:
2020
- cron: '30 4/6 * * *'
21-
pull_request_target:
21+
pull_request:
2222
paths: [ 'release/trigger_all_tests.json', '.github/trigger_files/IO_Iceberg_Managed_Integration_Tests_Dataflow.json' ]
2323
workflow_dispatch:
2424

25-
# Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
25+
# Setting explicit permissions for the action
2626
permissions:
2727
actions: write
2828
pull-requests: write
@@ -51,7 +51,7 @@ env:
5151
jobs:
5252
IO_Iceberg_Managed_Integration_Tests_Dataflow:
5353
if: |
54-
github.event_name == 'pull_request_target' ||
54+
github.event_name == 'pull_request' ||
5555
github.event_name == 'workflow_dispatch' ||
5656
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
5757
github.event.comment.body == 'Run IcebergIO Managed Integration Tests on Dataflow'

‎.github/workflows/IO_Iceberg_Performance_Tests.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ name: IcebergIO Performance Tests
1818
on:
1919
schedule:
2020
- cron: '10 10/12 * * *'
21-
pull_request_target:
21+
pull_request:
2222
paths: [ '.github/trigger_files/IO_Iceberg_Performance_Tests.json' ]
2323
workflow_dispatch:
2424

25-
#Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
25+
# Setting explicit permissions for the action
2626
permissions:
2727
actions: write
2828
pull-requests: write
@@ -51,7 +51,7 @@ env:
5151
jobs:
5252
IO_Iceberg_Performance_Tests:
5353
if: |
54-
github.event_name == 'pull_request_target' ||
54+
github.event_name == 'pull_request' ||
5555
github.event_name == 'workflow_dispatch' ||
5656
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
5757
github.event.comment.body == 'Run IcebergIO Performance Test'

‎.github/workflows/README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ If you would like to manually trigger a job, you have 2 options:
4444

4545
1) Trigger Phrases: Some jobs have trigger phrases associated with them (e.g. `Run XYZ PreCommit`). These will appear in statuses of previous PR runs of that check. You can trigger the job on any PR by commenting that trigger phrase in the PR.
4646

47-
**Note:** this approach is found not scalable ([#28909](https://github.com/apache/beam/issues/28909)) and currently only enabled for PreCommit workflows. For PostCommit jobs, it is currently replaced by a temporary approach: test suites are configured to trigger whenever a particular trigger file is modified. Test [workflows](https://github.com/apache/beam/tree/master/.github/workflows) have [pull_request_target paths](https://github.com/apache/beam/blob/e33dec69c7cfd01c0b827538e1dad8567e3ff95e/.github/workflows/beam_PreCommit_Whitespace.yml#L25), which include a trigger file. Whenever a trigger file is modified, the test suite will trigger on the pull request. Make any change to this file to trigger the job. The trigger file looks like the following: `.github/trigger_files/<workflow_file_name.json>`.
47+
**Note:** this approach is found not scalable ([#28909](https://github.com/apache/beam/issues/28909)) and currently only enabled for PreCommit workflows. For PostCommit jobs, it is currently replaced by a temporary approach: test suites are configured to trigger whenever a particular trigger file is modified. Test [workflows](https://github.com/apache/beam/tree/master/.github/workflows) have [pull_request paths](https://github.com/apache/beam/blob/e33dec69c7cfd01c0b827538e1dad8567e3ff95e/.github/workflows/beam_PreCommit_Whitespace.yml#L25), which include a trigger file. Whenever a trigger file is modified, the test suite will trigger on the pull request. Make any change to this file to trigger the job. The trigger file looks like the following: `.github/trigger_files/<workflow_file_name.json>`.
4848

4949
2) **Committers only** - Manual triggering: Any committer can start any job with a [workflow_dispatch](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch) trigger defined (all jobs should have these). To do so, navigate to the [Actions tab](https://github.com/apache/beam/actions), click on your desired workflow in the left navigation bar, and then click `Run Workflow`.
5050

@@ -73,7 +73,7 @@ jobs:
7373
job_phrase: [Run Job Phrase]
7474
if: |
7575
github.event_name == 'push' ||
76-
github.event_name == 'pull_request_target' ||
76+
github.event_name == 'pull_request' ||
7777
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
7878
github.event_name == 'workflow_dispatch' ||
7979
github.event.comment.body == 'Run Job Phrase'
@@ -102,7 +102,7 @@ jobs:
102102
python_version: ['3.9','3.10','3.11','3.12']
103103
if: |
104104
github.event_name == 'push' ||
105-
github.event_name == 'pull_request_target' ||
105+
github.event_name == 'pull_request' ||
106106
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
107107
github.event_name == 'workflow_dispatch' ||
108108
(github.event_name == 'issue_comment' && github.event.comment.body == format('{0} {1}', matrix.job_phrase, matrix.python_version))
@@ -124,16 +124,16 @@ GitHub allows workflows to define a set of triggers that dictate when a job shou
124124
For the purposes of Beam, each CI workflow should define the following triggers:
125125

126126
1) A `push` trigger
127-
2) A `pull_request_target` trigger
127+
2) A `pull_request` trigger
128128
3) An issue_comment trigger (for issue created). This is needed for comment triggering support (see section below).
129129
4) A scheduled trigger
130130
5) A workflow_dispatch trigger
131131

132-
The `push`/`pull_request_target` triggers should only run when appropriate paths are modified. See https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Go.yml#L4 for an example (you can copy and paste this into your workflow, you just need to change the paths).
132+
The `push`/`pull_request` triggers should only run when appropriate paths are modified. See https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Go.yml#L4 for an example (you can copy and paste this into your workflow, you just need to change the paths).
133133

134134
## Checkout step
135135

136-
Because we use the `pull_request_target` trigger instead of `pull_request`, we need an explicit checkout of the correct commit. This can be done as a step that uses the `setup-action` action in your workflow. See https://github.com/apache/beam/blob/0ee2dc73ec6f555a5bf1a643dffd37f4927be67e/.github/workflows/beam_PreCommit_Go.yml#L65-L70 for an example (you can copy and paste this into your workflow). Please make sure that you checkout the code before using the composite action.
136+
For comment triggering support, we need an explicit checkout of the correct commit. This can be done as a step that uses the `setup-action` action in your workflow. See https://github.com/apache/beam/blob/0ee2dc73ec6f555a5bf1a643dffd37f4927be67e/.github/workflows/beam_PreCommit_Go.yml#L65-L70 for an example (you can copy and paste this into your workflow). Please make sure that you checkout the code before using the composite action.
137137

138138
## Token Permissions
139139

‎.github/workflows/beam_Infrastructure_UsersPermissions.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ name: Modify the GCP User Roles according to the infra/users.yml file
2424

2525
on:
2626
workflow_dispatch:
27-
pull_request_target:
27+
pull_request:
2828
types: [opened, synchronize, reopened, closed]
2929
paths:
3030
- 'infra/iam/users.yml'
@@ -83,6 +83,7 @@ jobs:
8383
8484
- name: Upload plan as a comment to PR
8585
if: github.event.action == 'opened' || github.event.action == 'synchronize' || github.event.action == 'reopened'
86+
continue-on-error: true
8687
env:
8788
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
8889
GH_REPO: ${{ github.repository }}

‎.github/workflows/beam_Playground_CI_Nightly.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ on:
2020
- cron: '20 */12 * * *'
2121
workflow_dispatch:
2222

23-
#Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
23+
# Setting explicit permissions for the action
2424
permissions:
2525
actions: write
2626
pull-requests: read
@@ -52,7 +52,7 @@ jobs:
5252
beam_Playground_CI_Nightly:
5353
if: |
5454
github.event_name == 'workflow_dispatch' ||
55-
github.event_name == 'pull_request_target' ||
55+
github.event_name == 'pull_request' ||
5656
(github.event_name == 'schedule' && github.repository == 'apache/beam')
5757
runs-on: [self-hosted, ubuntu-24.04, highmem]
5858
name: "beam_Playground_CI_Nightly"

‎.github/workflows/beam_PostCommit_Go_Dataflow_ARM.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@ on:
2020
types: [created]
2121
schedule:
2222
- cron: '30 3/6 * * *'
23-
pull_request_target:
23+
pull_request:
2424
paths: ['release/trigger_all_tests.json', '.github/trigger_files/beam_PostCommit_Go_Dataflow_ARM.json']
2525
workflow_dispatch:
2626

27-
#Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
27+
# Setting explicit permissions for the action
2828
permissions:
2929
actions: write
3030
pull-requests: read
@@ -55,7 +55,7 @@ jobs:
5555
if: |
5656
github.event_name == 'push' ||
5757
github.event_name == 'workflow_dispatch' ||
58-
github.event_name == 'pull_request_target' ||
58+
github.event_name == 'pull_request' ||
5959
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
6060
github.event.comment.body == 'Run Go PostCommit Dataflow ARM'
6161
runs-on: [self-hosted, ubuntu-24.04, main]

‎.github/workflows/beam_PostCommit_Go_VR_Flink.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ name: PostCommit Go VR Flink
1818
on:
1919
schedule:
2020
- cron: '30 3/6 * * *'
21-
pull_request_target:
21+
pull_request:
2222
paths: ['release/trigger_all_tests.json', '.github/trigger_files/beam_PostCommit_Go_VR_Flink.json']
2323
workflow_dispatch:
2424

25-
#Setting explicit permissions for the action to avoid the default permissions which are `write-all` in case of pull_request_target event
25+
# Setting explicit permissions for the action
2626
permissions:
2727
actions: write
2828
pull-requests: read
@@ -52,7 +52,7 @@ jobs:
5252
beam_PostCommit_Go_VR_Flink:
5353
if: |
5454
github.event_name == 'workflow_dispatch' ||
55-
github.event_name == 'pull_request_target' ||
55+
github.event_name == 'pull_request' ||
5656
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
5757
github.event.comment.body == 'Run Go Flink ValidatesRunner'
5858
runs-on: [self-hosted, ubuntu-24.04, main]

0 commit comments

Comments
 (0)