docs.rs CDN: set up tracing & configure datadog logging for application logs

[syphar]

I think this should be it.

Related links:

• https://www.fastly.com/documentation/guides/integrations/streaming-logs/setting-up-remote-log-streaming-for-compute/

Generally:
This just uses the standard tracing json log format, depending on what we want to do in datadog we either need to pre-process these logs in datadog, or rename fields in here.

I'm not sure about the api-key / secret thing, feel free to tell me what should be different.

Also: these are just error / info logs from the wasm module itself. We don't have any access log yet.

Future improvements:

• start adding request-id, create a surrounding span that includes it so we can see it in the logs.

[syphar]

r? @marcoieni

[syphar]

Some research showed we're missing some fields:

• https://docs.datadoghq.com/api/latest/logs/send-logs/?code-lang=rust
• https://www.fastly.com/documentation/guides/integrations/logging-endpoints/log-management-and-observability/log-streaming-datadog/#recommended-log-format

will improve this and re-request a review, perhaps include the request id.

[syphar]

So, "small" update here.

For now I'm using the help of the tracing-serde crate to build my own tracing layer, and a custom struct for the log format that datadog expects.

Generally I would like to extract that stuff into a separate small crate (tracing-fastly or so), so we (and others) can reuse it.

[syphar]

that's because our test-suite now only runs with the fastly test-runner, not as normal tests any more.

[marcoieni]

do we need to explicitly exclude the package? If yes, should we also exclude terragrunt/modules/crates-io/compute-static?

[syphar]

it seems like our package needs to be excluded, but the compute-static one not. didn't dig into why yet.

[syphar]

so the main reason for all the code in crate::logging is

• provide the datadog json format
• collect fields from tracing spans, also emit them on the log.

[marcoieni]

ok, I will apply this now. You can squash your commits if you want, otherwise I can merge like this.