Some cases are currently always failing at the fieldcompare step, as they include values too close to machine precision. There are currently defined in a separate test suite:
expected-to-fail:
tutorials:
- *elastic-tube-3d_fluid-openfoam_solid-fenics # too small values to compare
- *two-scale-heat-conduction_macro-dumux_micro-dumux # too small values to compare
It would be useful to have a per-test option, similar to the max_time, to override the tolerance used by fieldcompare, or skip the step altogether. For example:
- &elastic-tube-3d_fluid-openfoam_solid-fenics
path: elastic-tube-3d
case_combination:
- fluid-openfoam
- solid-fenics
max_time_windows: 1
reference_result: ./elastic-tube-3d/reference-results/fluid-openfoam_solid-fenics.tar.gz
+ tolerance: 1e-2
Alternatively, we could revisit these cases to see why they produce such values, and avoid them.
Some cases are currently always failing at the fieldcompare step, as they include values too close to machine precision. There are currently defined in a separate test suite:
It would be useful to have a per-test option, similar to the
max_time, to override the tolerance used by fieldcompare, or skip the step altogether. For example:- &elastic-tube-3d_fluid-openfoam_solid-fenics path: elastic-tube-3d case_combination: - fluid-openfoam - solid-fenics max_time_windows: 1 reference_result: ./elastic-tube-3d/reference-results/fluid-openfoam_solid-fenics.tar.gz + tolerance: 1e-2Alternatively, we could revisit these cases to see why they produce such values, and avoid them.