[HWORKS-3185] Fix cves branch 3.5.5.2 - #63
Open
gibchikafa wants to merge 2 commits into
Open
gibchikafa wants to merge 2 commits into
gibchikafa wants to merge 2 commits into
Conversation
…kListenerEvent ### What changes were proposed in this pull request? JsonProtocol tidy up. Only parse JSON relating to Spark events. https://issues.apache.org/jira/browse/SPARK-52381 ### Why are the changes needed? Tidier code and https://lists.apache.org/thread/9zwkdo85wcdfppgqvbhjly8wdgf595yp ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Unit test ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#51323 from pjfanning/SPARK-52381-br3.5. Authored-by: PJ Fanning <pjfanning@users.noreply.github.com> Signed-off-by: yangjie01 <yangjie01@baidu.com> (cherry picked from commit a53a9c4)
…eper up Companion to the cherry-pick above, which is the important half of this pair: SPARK-52381 is the fix for CVE-2025-54920, the deserialization RCE in the Spark History Server, and this branch is what builds sparkhistoryserver:3.5.5.2 - the image the 5.0.x line actually runs. Scope was taken from the jar list of the published spark-3.5.5.2-bin-without-hadoop-with-hive.tgz rather than from dev/deps, which disagrees with what ships. 18 shipped artifacts carry 36 Critical or High advisories; this closes 23 of them. - netty 4.1.96 -> 4.1.137.Final. 17 advisories across codec, codec-http, codec-http2 and handler. Within the 4.1 line, which Netty maintains for binary compatibility, and the version the Hive fork already runs on branch-4.1. - jackson 2.15.2 -> 2.18.8 (both the core and databind properties). CVE-2026-54512 and -54513, the PolymorphicTypeValidator bypasses, plus GHSA-r7wm-3cxj-wff9. 2.15.x has no fix, so this has to change lines; jackson-module-scala_2.12 2.18.8 exists and Spark 4.0 shipped 2.18.x, so the API surface is not new ground. - ivy 2.5.1 -> 2.5.3 (CVE-2022-46751), matching what upstream v3.5.9 pins. - lz4-java 1.8.0 -> 1.8.1 (CVE-2025-12183). CVE-2025-66566 stays open on the same jar - no fixed release exists. - jdom2 pinned to 2.0.6.1 (CVE-2021-33813). Nothing declares it; it comes in through aliyun-sdk-oss, which pins 2.0.6 outright. - zookeeper 3.6.3 -> 3.8.6 (CVE-2023-44981, Critical). Not in this dist - Hadoop is provided - but it is in the spark-operator image, and 3.8.6 is the version the Hive fork settled on. - derby moves to `provided`, which takes CVE-2022-46337 out of the assembly. Nothing to upgrade to: the advisory names 10.14.3, never released, and 10.17.1.0 needs Java 21 against these Java 17 images. Derby only ever backed the local metastore, which Hopsworks never uses. Deliberately not touched, all of them Hive 3.0.0.14.6 transitives or otherwise not a version bump: calcite-core 1.16.0 CVE-2022-39135, Critical. Needs 1.32.0; comes from the Hive fork, and spark-hive references it. jackson-mapper-asl CVE-2019-10202, Critical, Jackson 1.x is EOL and 1.9.13 no fix will ever ship. Already excluded from one path, still reaches the assembly by another. libthrift 0.12.0 Three advisories; 0.14.0 covers two, 0.23.0 all three. Hive's generated Thrift code constrains it. okhttp 3.12.12 Real here, unlike on branch-4.1 where 4.12.0 ships. It backs the fabric8 Kubernetes client (kubernetes-httpclient-okhttp), so 4.9.2+ is a client-compatibility question, not a bump. ini4j 0.5.4, gson 2.2.4, aircompressor 0.27. Upstream's 3.5 line is in maintenance and has moved none of these except ivy, so there is no upstream validation to lean on - the bumps above were chosen for line-compatibility and for agreeing with the Hive fork. NOT BUILT. `build/mvn -pl common/network-common -am dependency:resolve` passes and every target artifact resolves, but a real compile needs io.hops hadoop-client-api 3.4.3.1-EE-RC4 and hive-exec 3.0.0.14.6, neither of which is in the local repository, and nexus returns 401 without credentials. For the same reason dev/deps was hand-edited rather than regenerated: run dev/test-dependencies.sh --replace-manifest on a credentialed agent, and expect derby to drop out of the manifest as well. JsonProtocolSuite, which the cherry-pick extends, has not been run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015B2RucKxNKWyW6z4ScQyA6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.