[All] Introduce Snapshot feature

[Lucas-TJ]

Based on #6129 and #6130

What is a Snapshot ?

A snapshot is an object that stores all information required to save the state of a simulation, so that it can be later restored.

Concept

General

A snapshot is a class structured like this :

• Data and Links are stored in dedicated structures called DataInfo and LinkInfo
• If the object is a node, then it creates a SnapshotNode is created. It contains data, links, components (SnapshotObject) and child nodes (also SnapshotNode instances).
• If the object is a component, a SnapshotObject is created. It contains only data and links, and is stored inside a SnapshotNode.

Once the snapshot has been built, it can be exported to the format chosen by the user. For example, calling saveSnapshot with the JSON exporter serializes the snapshot into a JSON file. Loading works the same way in reverse: the snapshot is first imported from the chosen format, then used to restore the simulation state.

Step-by-step

Structure of a Snapshot

public:
    struct DataInfo
    {
        std::string name;
        std::string type;
        std::string value;
    };

    struct LinkInfo
    {
        std::string name;
        std::string type;
        std::string value;
    };

    struct SnapshotObject
    {
        std::string m_name;
        std::vector<DataInfo> m_dataContainer;
        std::vector<LinkInfo> m_linkContainer;
        void* m_internalState { nullptr };

        SnapshotObject() = default;
        explicit SnapshotObject(const std::string& name) : m_name(name){}

        virtual ~SnapshotObject() = default;
    };

    struct SnapshotNode : public SnapshotObject
    {
        std::vector<SnapshotObject> components;
        std::vector<std::shared_ptr<SnapshotNode>> children;
        
        SnapshotNode() = default;
        SnapshotNode(const std::string& name) : SnapshotObject(name) {}
        SnapshotNode(const SnapshotObject& obj) : SnapshotObject(obj) {}

        ~SnapshotNode() noexcept = default;
    };

    std::shared_ptr<SnapshotNode> m_graphRoot { nullptr };

A Snapshot is organized as a graph. The root object, m_graphRoot, is the object that is exported to or imported from a file (or stored directly in memory). The graph is composed of nodes.

• A node corresponds to the struct SnapshotNode. Each node contains a name, a data container, a link container, a list of components, and a list of child nodes.
• A component corresponds to the struct SnapshotObject. Each component contains a name, a data container, and a link container.
• Data and links are represented in the same way: each entry stores its name, type, and value

For example,

{
    "name" : "root",

    "data" : [{},{}],
    "links" : [{},{}],
    
    "components" : [
        {
            "data" : [{},{}],
            "links" : [{},{}]
        },
        {
            "data" : [{},{}],
            "links" : [{},{}]
        }
    ],

    "children": []
}

Saving and loading snapshots rely on two visitors: SaveSnapshotVisitor and LoadSnapshotVisitor. These visitors traverse the scene graph to collect or restore data and links. Once collected, a snapshot can either be kept in memory or exported to a JSON file using SnapshotJSONExporter. The reverse process is used when loading a snapshot.

Save

Saving a snapshot relies on the SaveSnapshotVisitor, which traverses the scene graph. For each visited node or component, it calls:

Base::saveSnapshot(std::vector<std::shared_ptr<SnapshotNode>>& ).

Inside Base::saveSnapshot , the function

Base::createSnapshotObject(std::vector<std::shared_ptr<Snapshot::SnapshotNode>>&)

creates either a SnapshotObject or a SnapshotNode, depending on the type of the current object.

he snapshot object is then populated with:

• snapshotObject->m_name = this->getName() to store the object's name.
• a part to serialize the object's data.
• a part to serialize the object's links.
• saveInternalStateIn(*snapshotObject) to serialize the object's Internal State.

Finally, the newly created SnapshotObject (or SnapshotNode) is inserted into the snapshot graph.

Load

Loading a snapshot relies on the LoadSnapshotVisitor, which traverses the scene graph and restores the saved state by calling:
Base::loadDataSnapshot(const std::shared_ptr<Snapshot::SnapshotObject>& snapshotObject)

Base::loadLinkSnapshot(const std::shared_ptr<Snapshot::SnapshotObject>& snapshotObject)

Base::loadInternalStateFrom(const Snapshot::SnapshotObject& snapshot)

loadDataSnapshot use : BaseData::read(const std::string& value) to restore data values from the snapshot.

loadLinkSnapshot use : BaseLink::readFromSnapshot(const std::string& value) to restore links from the snapshot.

SnapshotManager

Overview

SnapshotManager is responsible for storing and managing snapshots, whether they are kept in memory or stored on disk.

API

Snapshots are stored in two separate containers: m_snapshotsFromMemory for in-memory snapshots and m_snapshotsFromFiles for snapshots stored on disk. The addSnapshotFromMemory and addSnapshotFromFile functions are used to add snapshots to these containers.

Although in-memory and on-disk snapshots are handled separately, their APIs follow the same overall design.

• doMemorySave and doMemoryLoad save and restore snapshots in memory.
• doSaveTo and doLoadTo save and load snapshots from disk. doSaveTo allows the output file format to be specified.

The doSaveTo function also provides an isGroup parameter, and the doLoadToGroup function is available for loading a collection of snapshots. These features are intended for saving and restoring groups of snapshots within a single file.

SnapshotJSONExporter

Overview

SnapshotJSONExporter provides the functionality required to export a a snapshot to JSON and import it back from a JSON file. It relies on the nlohmann/json library.

API

The implementation is split in two parts : one for exporting snapshots and one for importing them.

• For exporting, the exportToJSON function serializes a Snapshot into a JSON object by calling the appropriate to_json overloads before writing the result to a file.

• For importing, the importFrom function reads a JSON file and reconstructs a Snapshot by calling the corresponding from_json overloads.

• Two additional helper functions, fileToString and snapshotToString, provide serialization and deserialization to and from strings.

[with-all-tests]

---

By submitting this pull request, I acknowledge that
I have read, understand, and agree SOFA Developer Certificate of Origin (DCO).

---

Reviewers will merge this pull-request only if

• it builds with SUCCESS for all platforms on the CI.
• it does not generate new warnings.
• it does not generate new unit test failures.
• it does not generate new scene test failures.
• it does not break API compatibility.
• it is more than 1 week old (or has fast-merge label).

[alxbilger]

if you only use static methods, I am not sure the non-static data members are used. So I would remove them.

Also, I am not sure about the design of the class. I don't think that this class manages anything. You seem to store all the data elsewhere.

[Lucas-TJ]

This class is here to store snapshots when I save in memory. I use it in SofaImGUI.
I will work on it again to make it clearer.

[bakpaul]

Nice work ! I have two main comments :

1. This PR seems to be based on #6129 and #6130. If so, please stfate it in the PR description and add the right tag to the PR.
2. I saw many places where you didn't use constness indicator but the object was never modified. I know this might not be the most heavy load work of all time, but it might help some optimization when dealing with strings in loops.

[bakpaul]

I've got an issue with this test, it creates a new object, snapshot it and verifies the values. The issue being that those values are the default ones. This would work if snapshot took the default values and not the actual ones. Could you modify a set of values before the snapshot ?

[Lucas-TJ]

In fact, the goal of this test is just to verify that saveSnapshot() correctly saves the data into a snapshot.
At the beginning, the snapshot is completely empty. After calling saveSnapshot(), the data values are stored in the snapshot. That is what I want to verify.
Of course, I can also change values before saveSnapshot(), and verify if the snapshot takes the changes into account.
I add some assertion macros to verify if the snapshot exist and is empty.

[bakpaul]

Maybe adding another test to test this might be better then.

[bakpaul]

Why replacing // by nothing ?

[Lucas-TJ]

At the beginning, this was a design choice related to how snapshots were saved. I chose to save paths without the // (for example, @object1 instead of @//Object1). However, maybe it's now fine to keep the //, so I'll update it and try if it works correctly.

[damienmarchal]

Hi all,

This discussion remind me a vague issue in sofa where the "root" node is not properly serialize by all code path, because the name of the root node and the initiale "/" are actually the same thing.

Regarding the PR, I'm curious to know in which circumstance there could be multiple "//" in path serialization (expect the corner case of the root node) ? To me this indicates that some name are "empty", removing the double // to / change the "meaning" of the path. So yes, please keep it.

[bakpaul]

No error or warning when a link doesn't have @ and is ignored ?

[bakpaul]

MAybe a stupid question, but I am wondering what happnds to slave objects ? Are they also children of the node of the owner ? Or they are skipped by this design ?

[damienmarchal]

To be confirmed but, if slaves are not children of a node, then the snapshot system must handle them with a dedicated code path. In PR description slaves are not mentionned... but I have see other @bakpaul comment about the slave topic, so I will have the solution in a subsequent file.

[Lucas-TJ]

Actually the design doesn't take slave objects account. I will modify and apply your suggestions.
Thank you for your remarks !

[bakpaul]

Regarding my comment on the slave object, it could be solved here. But it might require a little modification of the objectSnapshot structure.