[WIP] TEZ-4476: Move tez-ui testing from PhantomJS to Headless Chrome - #514

💔 -1 overall

This message was automatically generated.

💔 -1 overall

This message was automatically generated.

💔 -1 overall

This message was automatically generated.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/4/console in case of problems.

💔 -1 overall

This message was automatically generated.

@abstractdog @ayushtkn
Could you please review the PR at your convenience?
Thank you !

thanks for this patch @maheshrajus, a few notes:

1. after removing phantomjs, we can try to build tez-ui also on arm64:

   tez/.github/workflows/build.yml

   Line 53 in 17d1549

   && '-pl !tez-ui' || '' }}

2. checked out pr, ran mvn clean install in tez-ui and I saw a browser window running the tests:

3. after install, I got this diff:

-"moment@>= 2.29.4":
-  version "2.29.4"
-  resolved "https://registry.yarnpkg.com/moment/-/moment-2.29.4.tgz#3dbe052889fe7c1b2ed966fcb3a77328964ef108"
+"moment@>= 2.6.0":
+  version "2.30.1"
+  resolved "https://registry.yarnpkg.com/moment/-/moment-2.30.1.tgz#f8c91c07b7a786e30c59926df530b4eac96974ae"

@abstractdog Thank you for the review.
Let me check and address the review comments.

2. checked out pr, ran mvn clean install in tez-ui and I saw a browser window running the tests:

@abstractdog Initially the changes related to headless present in my first commit. After that while fixing ui-tests it got overwritten. Let me fix and run locally and verify it. thanks !

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/5/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/6/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/7/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/8/console in case of problems.

💔 -1 overall

This message was automatically generated.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/9/console in case of problems.

🎊 +1 overall

This message was automatically generated.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/10/console in case of problems.

1. after removing phantomjs, we can try to build tez-ui also on arm64:

@abstractdog Can we handle this in separate JIRA ?

@abstractdog I fixed your review comments and verified locally + in yetus CI env.
Could you please review the PR at your convenience?
Thank you !

🎊 +1 overall

This message was automatically generated.

@abstractdog I fixed your review comments and verified locally + in yetus CI env. Could you please review the PR at your convenience? Thank you !

looks good so far, I have only one concern: on a vanilla ubuntu server, when chromium wasn't installed, the build simply failed by:

Building..Built project successfully. Stored in "/home/abstractdog/apache/tez/tez-ui/src/main/webapp/tmp/class-tests_dist-NhEe6O0O.tmp".
not ok 1 Chrome - Browser "scripts/chrome-launcher.sh --headless --disable-gpu --no-sandbox --disable-dev-shm-usage --no-default-browser-check --no-first-run --ignore-certificate-errors --test-type <url>" exited unexpectedly.

if the build process cannot take care of fetching a chrome or chromium, then a preinstalled chrome is a hard dependency of a mvn clean install command, which doesn't sound good to me
could the build process take care of the same automatically?

if the build process cannot take care of fetching a chrome or chromium, then a preinstalled chrome is a hard dependency of a mvn clean install command, which doesn't sound good to me
could the build process take care of the same automatically?

@abstractdog Thank you for the review. I have not taken care of chromium, Let me check and fix it.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/11/console in case of problems.

@abstractdog I have addressed your review comments and tested the changes in the environments listed below. Everything is working as expected.
Please let me know if I missed anything or if I should verify this in any other environment. Thanks !

macOS with & without system chrome, 
Ubuntu 22.04 – no system Chrome
Ubuntu 22.04 – system Chrome installed
Debian 12 – no system Chrome
Amazon Linux 2023 – no system Chrome

💔 -1 overall

This message was automatically generated.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-514/12/console in case of problems.

🎊 +1 overall

This message was automatically generated.

@abstractdog
Could you please review the PR at your convenience?
Thank you !

@abstractdog Could you please review the PR at your convenience? Thank you !

thanks @maheshrajus , tried it on a vanilla ubuntu, and doesn't seem to work as I expect, let me share what I can see and maybe you can tell if I miss something:

1. run mvn clean install in tez-ui, the doc says "yarn install" (run automatically by Maven) takes care of that

2. I get an error like:

Building...Built project successfully. Stored in "/home/abstractdog/apache/tez/tez-ui/src/main/webapp/tmp/class-tests_dist-oQZlTw0S.tmp".
not ok 1 Chrome - Browser "scripts/chrome-launcher.sh --headless --disable-gpu --no-sandbox --disable-dev-shm-usage --no-default-browser-check --no-first-run --ignore-certificate-errors --test-type <url>" exited unexpectedly.

1..1
# tests 1
# pass  0
# fail  1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

3. running it manually: empty response, not even an echo I would expect

abstractdog@ubuntu-8gb-hel1-3:~/apache/tez/tez-ui$ ./src/main/webapp/scripts/chrome-launcher.sh --headless --disable-gpu --no-sandbox --disable-dev-shm-usage --no-default-browser-check --no-first-run --ignore-certificate-errors "fake"
abstractdog@ubuntu-8gb-hel1-3:~/apache/tez/tez-ui$ echo $?
127

so I'm not sure anymore if mvn install automatically takes care of yarn install, or I have to do it, becuase even if I saw the echo message, it would tell "Alternatively, run 'yarn install' inside src/main/webapp to let puppeteer download Chromium automatically.", but doc says "# "yarn install" (run automatically by Maven) will download a bundled # Chromium, so the build works on a vanilla machine with no pre-installed"

thanks @maheshrajus , tried it on a vanilla ubuntu, and doesn't seem to work as I expect, let me share what I can see and maybe you can tell if I miss something:

I checked it locally with ubuntu and worked for me. Let check it if i am missing anything over here. need to check if already yarn package installed in this vm or not. Let me check and run in freshly created vanilla ubuntu vm. thanks !