update n5 component versions

[bogovicj]

Leaving this as a draft for now, because some artifacts depending on n5 will need updating

PRs are open for some of them:

• bigwarp
• bigdataviewer-core
• bigdataviewer-n5

Details

• n5-4.0.0
• n5-aws-s3-5.0.0
• n5-blosc-2.0.0
• n5-google-cloud-6.0.0
• n5-hdf5-3.0.0
• n5-ij-5.0.0
• n5-imglib2-8.0.0
• n5-universe-3.0.0
• n5-zarr-2.0.0
• n5-zstandard-2.0.0

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/ngff-weekly-dev-update-thread/110810/95

[ctrueden]

Thanks, @bogovicj! FYI, I'm in the process of improving the pom-scijava CI—once that work makes it onto the mainline branch, we can rebase this PR onto the latest to see more clearly what else breaks due to this update.

[bogovicj]

Hi @ctrueden ,
Any news about this pom-scijava CI? We're eager to merge this in.

[ctrueden]

@bogovicj Thanks for checking in. I've been working on this a lot in recent days.

There is a new command line tool, pombast, that serves as a more robust and powerful replacement for the previous "mega-melt" shell scripts here in pom-scijava. I've been addressing the pom-scijava version skew issues one by one in conjunction with pombast development. If you visit https://status.scijava.org and click the Smelt column twice to sort it descending, you can see which components are currently validating versus having some version-skew-related issues. As of this writing, here's what's left to address:

• net.imagej:imagej-legacy:2.0.3 - Pain due to the jump from Java 8 to 11 in dependencies, notably SCIFIO. Probably can work around this by configuring pombast to build and run imagej-legacy with Java 11 rather than 8...
• net.imagej:imagej-server:0.2.0 - Same: need to test this with Java 11 now, due to SCIFIO Java version bump
• net.imagej:imagej-tensorflow:1.1.9 - Same: need to test this with Java 11 now, due to SCIFIO Java version bump
• org.janelia.saalfeldlab:n5-universe:2.3.0 - N5IO No container exists at...
• org.janelia.saalfeldlab:n5-zarr:1.5.1 - zarrCacheBehaviorHelper:383 arrays first differed at element [0]; expected:<[a]> but was:<[b]>
• org.janelia.saalfeldlab:n5:3.5.1 - N5IO No container exists at...
• org.morphonets:SNT:5.0.9 - needs adjustment to account for jgrapht API breakage
• org.scijava:scijava-search:3.0.0 - needs adjustment to account for forum.image.sc Discourse REST API changes (?)
• org.scijava:scripting-kotlin:0.2.1 - needs adjustment to match pom-scijava-base's current kotlin version
• sc.fiji:TrackMate-Skeleton:2.0.3 - needs updating to match TrackMate 8.x API breakage
• sc.fiji:TrackMate-StarDist:1.2.1 - needs updating to match TrackMate 8.x API breakage
• sc.fiji:labkit-ui:0.4.0 - one failing test with NPE; probably not a version skew issue (flaky test?) but needs confirmation

@bogovicj Do you know what the N5 test errors mean? Are the tests themselves broken, or did some remote access mechanism change in a backwards-incompatible way? Should I work around it by skipping these tests for the mega-melt CI?

For what it's worth, a few days ago I made an attempt to smelt everything with all the latest N5 component releases across the board, but there were many more problems. If you want to try doing it yourself, you can:

1. Clone pom-scijava repo.
2. Edit all n5<*>.version properties to the values you want to test. (For unreleased changes, you can set the version to 999-test or some such in a local working copy, mvn install it, then set the version in pom-scijava to match.)
3. uv tool install --from git+https://github.com/scijava/pombast pombast
4. cd /path/to/pom-scijava && pombast smelt .

The first time it runs it takes much longer, because it needs to clone repositories for all components being tested, and it has no successes cached from prior runs. But subsequent runs should be much faster, re-testing only those components that have not already succeeded with that particular dependency configuration.

Once I get the rest of the issues above under control (hopefully in the next couple of days—although grant proposal submission deadline also draws nigh... 👻), I'll give another go at smelting all the latest n5 stuff. Or if you do it and want to report back here, that's cool too.

Edit: I forgot to mention: the version of the BOM currently in testing is on the pombast branch here, not master. I could merge it anytime, although the CI build will still be failing if I do that. I was hoping to get everything perfect and then merge it and watch the CI turn green... 😇

[bogovicj]

Awesome, thanks for the run down @ctrueden !

I expect I'll be able to figure out what going on with the n5 artifacts. Will investigate and report back. May have some questions about pombast on the way.

[bogovicj]

The n5 artifacts, and bigwarp appear to be under control now. Locally:

Details

$ pombast smelt -r "scijava.public=https://maven.scijava.org/content/groups/public" -i "org.janelia.*:*" -i "sc.fiji:bigwarp_fiji" -v .

...

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Component                                     ┃ Binary ┃ Source  ┃ Duration ┃ Note          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ org.janelia.saalfeldlab:n5:4.0.1              │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-aws-s3:5.0.0       │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-blosc:2.0.0        │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-google-cloud:6.0.0 │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-hdf5:3.0.0         │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-ij:5.0.0           │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-imglib2:8.0.0      │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-universe:3.0.0     │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-viewer_fiji:6.2.0  │ -      │ SKIPPED │        - │ prior success │
│ org.janelia.saalfeldlab:n5-zarr:2.0.0         │ -      │ SKIPPED │        - │ prior success │
│ sc.fiji:bigwarp_fiji:9.4.0                    │ -      │ SKIPPED │        - │ prior success │
└───────────────────────────────────────────────┴────────┴─────────┴──────────┴───────────────┘

I needed to add this -r "scijava.public=https://maven.scijava.org/content/groups/public" for bigwarp, otherwise it would skip with "No SCM found" or similar.

This issue in n5-core seemed to have been the root cause. This was "just" an issue with test compilation, so I'm confident this didn't actually need another release, but whatever, it was easy to do.

[ctrueden]

@bogovicj Does anything remain to be resolved before merging this PR? Are there other downstream components that break due to these version updates?

[bogovicj]

@ctrueden ,
I'm confident that all the downstream components depending on n5 build with tests pass, so this can be merged from my pov.

[ctrueden]

@bogovicj The following components now fail the smelt action due to the n5 updates:

n5-aws-s3 broken tests:

• org.janelia.saalfeldlab.n5.s3.AmazonS3KeyValueAccessTest
• org.janelia.saalfeldlab.n5.s3.AmazonS3UtilsTest
• org.janelia.saalfeldlab.n5.s3.N5AmazonS3MockTests

All of the form: SdkClient Unable to execute HTTP request: Connect to localhost:9000 [localhost/127.0.0.1] failed: Connection refused (Connection refused)

n5-universe broken tests:

• org.janelia.saalfeldlab.n5.universe.storage.n5.N5AmazonS3FactoryTest$N5AmazonS3MockTest
• org.janelia.saalfeldlab.n5.universe.storage.zarr.zarr2.Zarr2AmazonS3FactoryTest$ZarrAmazonS3MockTest
• org.janelia.saalfeldlab.n5.universe.storage.zarr.zarr3.Zarr3AmazonS3FactoryTest$ZarrAmazonS3MockTest

All of the form: N5IO Inferred Backend was HTTP but could not create KeyValueAccess

multiview-reconstruction:

package org.janelia.saalfeldlab.n5.universe.metadata.ome.ngff.v04 does not exist
package org.janelia.saalfeldlab.n5.universe.metadata.ome.ngff.v04.OmeNgffMultiScaleMetadata does not exist
package org.janelia.saalfeldlab.n5.universe.metadata.ome.ngff.v04.coordinateTransformations does not exist

labkit-ui:

NoSuchMethod 'void org.janelia.saalfeldlab.n5.DatasetAttributes.<init>(long[], int[], org.janelia.saalfeldlab.n5.DataType, org.janelia.saalfeldlab.n5.Compression)'

You can see the exact error messages by visiting https://status.scijava.org and clicking the ❌ in the Smelt column of each component. Let me know if you need any help with anything.

[ctrueden]

FWIW, I think the reason your testing didn't catch multiview-reconstruction or labkit-ui is due to -i "org.janelia.*:*" -i "sc.fiji:bigwarp_fiji" arguments to pombast—that excluded most components from the smelt. The better way to test only affected components would be to use the "change" and "prune" flags. Something like:

pombast smelt \
-c 'org.janelia.saalfeldlab:n5:4.0.1' \
-c 'org.janelia.saalfeldlab:n5-aws-s3:5.0.0' \
-c 'org.janelia.saalfeldlab:n5-blosc:2.0.0' \
-c 'org.janelia.saalfeldlab:n5-google-cloud:6.0.0' \
-c 'org.janelia.saalfeldlab:n5-hdf5:3.0.0' \
-c 'org.janelia.saalfeldlab:n5-ij:5.0.0' \
-c 'org.janelia.saalfeldlab:n5-imglib2:8.0.0' \
-c 'org.janelia.saalfeldlab:n5-universe:3.0.1' \
-c 'org.janelia.saalfeldlab:n5-viewer_fiji:6.2.0' \
-c 'org.janelia.saalfeldlab:n5-zarr:2.0.0' \
-c 'org.janelia:n5-zstandard:2.0.0' \
-p .

The intent of the -c flags is to note specific changes you are focused on testing (which then don't have to be edited in the pom.xml itself locally), while -p combos with that to limit included components being smelted to only those with a dependency on at least one of the changed components.

However, this functionality was ported by AI from a melting-pot.sh feature I haven't tested or touched in many years, so it might still be buggy. (I haven't been using it yet personally—I always just smelt everything, and let the caching take care of optimizing what gets tested.) We definitely should make this work robustly, though, to save everyone time when they need to test disruptive changes like this!

As for the n5-aws-s3 and n5-universe test failures on my system: I have no clue. 🤷

[ctrueden]

Here's some more detail about the n5-universe build failures.

Stack trace:

org.janelia.saalfeldlab.n5.N5Exception$N5IOException: Inferred Backend was HTTP but could not create KeyValueAccess
	at org.janelia.saalfeldlab.n5.universe.KeyValueAccessBackend.getKeyValueAccess(KeyValueAccessBackend.java:116)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.getKeyValueAccess(N5Factory.java:241)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.openN5Container(N5Factory.java:755)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.openWriter(N5Factory.java:651)
	at org.janelia.saalfeldlab.n5.universe.N5Factory.openWriter(N5Factory.java:589)
	at org.janelia.saalfeldlab.n5.universe.StorageSchemeWrappedN5Test.getWriter(StorageSchemeWrappedN5Test.java:29)
	at org.janelia.saalfeldlab.n5.universe.storage.n5.N5StorageTests$N5FactoryTest.createN5Writer(N5StorageTests.java:65)
	at org.janelia.saalfeldlab.n5.universe.storage.n5.N5StorageTests$N5FactoryTest.createN5Writer(N5StorageTests.java:60)
	at org.janelia.saalfeldlab.n5.AbstractN5Test.createTempN5Writer(AbstractN5Test.java:106)
	at org.janelia.saalfeldlab.n5.AbstractN5Test.createTempN5Writer(AbstractN5Test.java:99)
	at org.janelia.saalfeldlab.n5.AbstractN5Test.createTempN5Writer(AbstractN5Test.java:91)
	at org.janelia.saalfeldlab.n5.AbstractN5Test.testWriteReadIntBlock(AbstractN5Test.java:381)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	... snip ...
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
Caused by: java.lang.UnsupportedOperationException: HttpKeyValueAccess only supports reader actions
	at org.janelia.saalfeldlab.n5.universe.KeyValueAccessBackend.newHttpKeyValueAccess(KeyValueAccessBackend.java:351)
	at org.janelia.saalfeldlab.n5.universe.KeyValueAccessBackend.apply(KeyValueAccessBackend.java:81)
	at org.janelia.saalfeldlab.n5.universe.KeyValueAccessBackend.getKeyValueAccess(KeyValueAccessBackend.java:113)

And testing n5-universe on my system took 5174.2 seconds! It's 550 test errors times 2 (once for binary test, one for rebuild-from-source)... so each maybe has a 5-second timeout being hit?

[bogovicj]

Argh, thanks for this info @ctrueden , investigating now...