Skip to content

Overriding the fieldcompare tolerance (or skipping) #837

@MakisH

Description

@MakisH

Some cases are currently always failing at the fieldcompare step, as they include values too close to machine precision. There are currently defined in a separate test suite:

  expected-to-fail:
    tutorials:
      - *elastic-tube-3d_fluid-openfoam_solid-fenics # too small values to compare
      - *two-scale-heat-conduction_macro-dumux_micro-dumux # too small values to compare

It would be useful to have a per-test option, similar to the max_time, to override the tolerance used by fieldcompare, or skip the step altogether. For example:

      - &elastic-tube-3d_fluid-openfoam_solid-fenics
         path: elastic-tube-3d
         case_combination:
           - fluid-openfoam
           - solid-fenics
         max_time_windows: 1
         reference_result: ./elastic-tube-3d/reference-results/fluid-openfoam_solid-fenics.tar.gz
+        tolerance: 1e-2

Alternatively, we could revisit these cases to see why they produce such values, and avoid them.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Started

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions