diff --git a/README.md b/README.md index d378571..ed158d5 100644 --- a/README.md +++ b/README.md @@ -18,8 +18,9 @@ TraceClient trace = TraceClient.builder("https://trace.danielstephenson.dev", "M // Say so, every startup, on the program's own logger. if (trace.isEnabled()) { - getLogger().info("Usage reporting is on: MyPlugin sends its name, version and command names to " - + "https://trace.danielstephenson.dev - nothing about players or the server. " + getLogger().info("Usage reporting is on: MyPlugin sends its name, version, command names and a " + + "random server ID to https://trace.danielstephenson.dev - nothing about players, and " + + "nothing that identifies the server's owner or address. " + "Turn it off with usage-reporting.enabled: false in this plugin's config.yml, " + "or for every plugin with enabled: false in plugins/trace/config.yml. " + "Details: https://github.com/Stephenson-Software/trace#usage-reporting"); @@ -47,6 +48,61 @@ Before 0.4.0, `builder` took two arguments and only events tagged by hand carried a version. Upgrading is one argument: in a Bukkit plugin, `getDescription().getVersion()`. +## Every event carries a random server ID + +Since 0.5.0, every event also carries the tag `install`: a random ID for +the installation, so the trace server can count **distinct servers** ("active +servers in the last 30 days") rather than raw events. This is the same idea +as bStats' `serverUuid`, and it is said out loud here because it is the one +thing the client sends that is the same from one event to the next. + +**What it is.** A `UUID.randomUUID()`. It is not derived from anything — not +a hostname, an IP address, a MAC address, a player, an account or a path. It +identifies no person and no address; all it can say is "these events came +from the same server". (The trace server still sees the IP address of every +HTTP request, as every web server does.) + +**Where it lives.** On a Spigot server, as the `server-id:` line of +`plugins/trace/config.yml`, shared by every plugin on that server. The first +time an *enabled* client finds no `server-id:` there, it appends one, under a +comment that says what it is: + +```yaml +# +# server-id: a random ID made on first run and sent as the tag "install", so +# trace can count servers, not events. It identifies no person and no IP +# address. Delete the server-id line to get a new one. +server-id: 0f8b6c1e-3a52-4c8e-9a0d-6e2f1b7c4d90 +``` + +Nothing else in the file is touched: comments, `enabled:` and `tags:` are +kept byte for byte. If the file cannot be read or written, a fresh ID is used +in memory for that run only — `build()` still never throws. + +**Resetting it.** Delete the `server-id:` line; the next start writes a new +one. Or set your own value (letters, digits, `_`, `.`, `-`; at most 255 +characters). + +**Opting out.** Every [opt-out](#opting-out) below also stops the ID: a +disabled client never generates one, never writes one, and sends nothing. +There is no way to send events without it short of turning reporting off; +that is deliberate, so "how many servers" is a number that can be trusted. + +**Programs that are not plugins.** Without `serverWideConfig(...)`, no ID is +made up and no hidden file is written anywhere. A program that wants to be +counted passes one it stores itself, e.g. a UUID it generated on first run +and keeps in its own config: + +```java +TraceClient.builder(url, "MyCli", version).key(key) + .installId(config.getString("usage-reporting.install-id")) // null or blank: none sent + .build(); +``` + +An explicit `installId(...)` wins over `server-id:`. An event that passes its +own `install` tag keeps it. `trace.installId()` returns the ID in use (`null` +when disabled or when there is none), so a program can print it. + ## What `report` promises | Property | Meaning | @@ -65,7 +121,7 @@ last word. `build()` checks these in order; the first match wins and is what | Switch | `disabledReason()` | |---|---| | Environment: `TRACE_USAGE_REPORTING=off` (or `false`, `0`, `no`) or `DO_NOT_TRACK=1` (or `true`, `yes`), case-insensitive. Always checked. | `environment` | -| Server-wide, when `serverWideConfig(pluginsDirectory)` was given: `enabled: false` in `plugins/trace/config.yml`. `build()` creates the file with `enabled: true` (and a commented-out [`tags:`](#server-wide-tags) example) if it is missing and never rewrites it afterwards; it is read with a line regex, no YAML library. An IO failure is logged at `FINE` and counts as enabled. | `server-wide config: plugins/trace/config.yml` | +| Server-wide, when `serverWideConfig(pluginsDirectory)` was given: `enabled: false` in `plugins/trace/config.yml`. `build()` creates the file with `enabled: true` (and a commented-out [`tags:`](#server-wide-tags) example) if it is missing; afterwards the only change it ever makes is appending a [`server-id:`](#every-event-carries-a-random-server-id) line when an enabled client finds none. It is read with a line regex, no YAML library. An IO failure is logged at `FINE` and counts as enabled. | `server-wide config: plugins/trace/config.yml` | | The program's own setting: `enabled(false)`. | `config.yml` | | No key, or a blank one. | `no key` | @@ -127,7 +183,7 @@ plugins already vendor bStats' `Metrics.java`. com.github.Stephenson-Software trace-client-java - 0.4.0 + 0.5.0 ``` @@ -138,10 +194,11 @@ Shade it into a plugin jar; it is one class. `POST {baseUrl}/api/metrics` with `Authorization: Bearer ` and a body of ```json -{"application":"MyPlugin","name":"command","value":1.0,"tags":{"name":"home","version":"1.4.0"}} +{"application":"MyPlugin","name":"command","value":1.0,"tags":{"name":"home","version":"1.4.0","install":"0f8b6c1e-3a52-4c8e-9a0d-6e2f1b7c4d90"}} ``` -`value` is omitted when not given; `tags` always holds at least `version`. The server assigns the +`value` is omitted when not given; `tags` always holds at least `version`, and +`install` whenever the client has an [installation ID](#every-event-carries-a-random-server-id). The server assigns the timestamp. A `201` is success; anything else is logged at `FINE` and dropped. ## Keys diff --git a/pom.xml b/pom.xml index 1db6945..414a666 100644 --- a/pom.xml +++ b/pom.xml @@ -6,7 +6,7 @@ software.stephenson trace-client - 0.4.0 + 0.5.0 jar trace-client diff --git a/src/main/java/software/stephenson/trace/TraceClient.java b/src/main/java/software/stephenson/trace/TraceClient.java index 26cad2a..dbd9132 100644 --- a/src/main/java/software/stephenson/trace/TraceClient.java +++ b/src/main/java/software/stephenson/trace/TraceClient.java @@ -1,5 +1,5 @@ /* - * trace-client 0.4.0 -- https://github.com/Stephenson-Software/trace-client-java + * trace-client 0.5.0 -- https://github.com/Stephenson-Software/trace-client-java * * One call to report that a program was used. Copy this file into a project as * is, or depend on the artifact; either way there is nothing else to add. @@ -17,10 +17,12 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; +import java.nio.file.StandardOpenOption; import java.util.Collections; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; +import java.util.UUID; import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; @@ -77,6 +79,17 @@ * {@code command} event can be tied to a release as well as a * {@code startup} one. An event's own {@code version} tag wins over it. * + *

Every event also carries a random per-installation ID as the tag + * {@code install}, so the trace server can count distinct servers rather + * than raw events. On a Spigot server it is the {@code server-id:} line of + * {@code plugins/trace/config.yml}, generated with + * {@link UUID#randomUUID()} and appended to that file the first time an + * enabled client finds none -- the same idea as bStats' {@code serverUuid}. + * Anything else may pass one with {@link Builder#installId(String)}; + * without either, no {@code install} tag is sent. The ID is random: it + * names no person, account or address. A disabled client never generates + * or writes one. An event's own {@code install} tag wins over it. + * *

A disabled client is a no-op that costs nothing. Programs that run on * other people's machines should expose their own switch in their * configuration and say on startup whether reporting is on. @@ -106,7 +119,7 @@ public final class TraceClient { /** This client's version, as sent in the User-Agent. */ - public static final String VERSION = "0.4.0"; + public static final String VERSION = "0.5.0"; /** How many reports may wait to be sent before new ones are dropped. */ public static final int QUEUE_CAPACITY = 256; @@ -131,6 +144,16 @@ public final class TraceClient { /** The server-wide switch, relative to the plugins directory. */ static final String SERVER_WIDE_CONFIG_PATH = "trace" + File.separator + "config.yml"; + /** + * Explains the {@code server-id:} line. Part of a freshly created file, + * and written above the line when it is appended to an existing one. + */ + static final String SERVER_ID_COMMENT = + "#\n" + + "# server-id: a random ID made on first run and sent as the tag \"install\", so\n" + + "# trace can count servers, not events. It identifies no person and no IP\n" + + "# address. Delete the server-id line to get a new one.\n"; + /** Exactly what a missing server-wide switch file is created with. */ static final String SERVER_WIDE_CONFIG_CONTENT = "# Server-wide switch for usage reporting by plugins that report to trace\n" @@ -144,10 +167,15 @@ public final class TraceClient { + "# own tag of the same name wins. On a test or CI server, uncomment the two\n" + "# lines below so its events are left out of real-installation figures.\n" + "# tags:\n" - + "# ci: \"true\"\n"; + + "# ci: \"true\"\n" + + SERVER_ID_COMMENT; + + /** The tag every event carries the installation's ID as. */ + static final String INSTALL_TAG = "install"; private static final Pattern ENABLED_LINE = Pattern.compile("^\\s*enabled\\s*:\\s*(\\S+)"); private static final Pattern TAGS_LINE = Pattern.compile("^tags\\s*:\\s*(#.*)?$"); + private static final Pattern SERVER_ID_LINE = Pattern.compile("^server-id\\s*:(.*)$"); // What the trace server accepts in a report's tags (MetricDto): at most // MAX_TAGS pairs, keys not blank, keys and values at most MAX_TAG_LENGTH @@ -168,6 +196,7 @@ public final class TraceClient { private final Logger logger; private final String disabledReason; // null when enabled private final Map serverWideTags; // never null; read once, at build() + private final String installId; // null when disabled or when there is none private final ThreadPoolExecutor executor; // null when disabled private TraceClient(Builder builder) { @@ -181,6 +210,9 @@ private TraceClient(Builder builder) { : readServerWideConfig(builder.pluginsDirectory); this.disabledReason = disabledReason(builder, serverWide); this.serverWideTags = serverWide.tags; + // After the opt-outs, never before: a disabled client neither makes + // up an ID nor writes one to disk. + this.installId = disabledReason == null ? resolveInstallId(builder, serverWide) : null; if (disabledReason == null) { this.executor = new ThreadPoolExecutor( 1, 1, 30, TimeUnit.SECONDS, @@ -228,6 +260,62 @@ public String disabledReason() { return disabledReason; } + /** + * The random per-installation ID every event carries as the tag + * {@code install}, or {@code null} when the client is disabled or has + * none (no {@link Builder#serverWideConfig(File)} and no + * {@link Builder#installId(String)}). + */ + public String installId() { + return installId; + } + + /** + * The installation's ID: the builder's, else the server-wide file's + * {@code server-id:}, else a new random one appended to that file. If the + * file could not be read or written, the new ID lives in memory for this + * process only. Never throws. + */ + private String resolveInstallId(Builder builder, ServerWideConfig serverWide) { + if (builder.installId != null) { + return builder.installId; + } + if (builder.pluginsDirectory == null) { + return null; + } + if (serverWide.serverId != null) { + return serverWide.serverId; + } + String fresh = UUID.randomUUID().toString(); + if (!serverWide.read) { + // A file that could not be read is not appended to: one that is + // there but unreadable would otherwise gain a line every start. + log("using an in-memory server-id for this process: server-wide config was not readable"); + return fresh; + } + File location = new File(builder.pluginsDirectory, SERVER_WIDE_CONFIG_PATH); + try { + Path file = location.toPath(); + byte[] existing = Files.readAllBytes(file); + StringBuilder block = new StringBuilder(); + if (existing.length > 0 && existing[existing.length - 1] != '\n') { + block.append('\n'); + } + if (!serverWide.created) { + block.append(SERVER_ID_COMMENT); // a fresh file already has it + } + block.append("server-id: ").append(fresh).append('\n'); + Files.write(file, block.toString().getBytes(StandardCharsets.UTF_8), StandardOpenOption.APPEND); + // Read back, so two clients racing on one file agree on the first line. + String written = parseServerWideConfig(Files.readAllLines(file, StandardCharsets.UTF_8)).serverId; + return written != null ? written : fresh; + } catch (IOException | RuntimeException failure) { + log("could not write server-id to server-wide config " + location + + ", using an in-memory one for this process: " + failure); + return fresh; + } + } + private static String disabledReason(Builder builder, ServerWideConfig serverWide) { if (environmentDisables()) { return REASON_ENVIRONMENT; @@ -264,16 +352,31 @@ private static boolean isYes(String value) { return v.equals("1") || v.equals("true") || v.equals("yes"); } - /** What {@code plugins/trace/config.yml} says: the switch and the server-wide tags. */ + /** What {@code plugins/trace/config.yml} says: the switch, the server-wide tags and the server-id. */ static final class ServerWideConfig { - static final ServerWideConfig NONE = new ServerWideConfig(false, Collections.emptyMap()); + /** No file was read: none was asked for, or it could not be read. */ + static final ServerWideConfig NONE = new ServerWideConfig(false, Collections.emptyMap(), null); final boolean disables; final Map tags; + final String serverId; // null when the file has no usable server-id: line + final boolean read; // the file was read successfully + final boolean created; // ... and build() had just created it + + ServerWideConfig(boolean disables, Map tags, String serverId) { + this(disables, tags, serverId, false, false); + } - ServerWideConfig(boolean disables, Map tags) { + private ServerWideConfig(boolean disables, Map tags, String serverId, boolean read, boolean created) { this.disables = disables; this.tags = tags; + this.serverId = serverId; + this.read = read; + this.created = created; + } + + ServerWideConfig readFromDisk(boolean created) { + return new ServerWideConfig(disables, tags, serverId, true, created); } } @@ -291,12 +394,14 @@ private ServerWideConfig readServerWideConfig(File pluginsDirectory) { // Inside the try: toPath() throws InvalidPathException for a path // the file system cannot represent, and build() never throws. Path file = location.toPath(); + boolean created = false; if (!Files.exists(file)) { Files.createDirectories(file.getParent()); Files.write(file, SERVER_WIDE_CONFIG_CONTENT.getBytes(StandardCharsets.UTF_8)); // just written: enabled: true, and the tags example commented out + created = true; } - return parseServerWideConfig(Files.readAllLines(file, StandardCharsets.UTF_8)); + return parseServerWideConfig(Files.readAllLines(file, StandardCharsets.UTF_8)).readFromDisk(created); } catch (IOException | RuntimeException failure) { log("could not read server-wide config " + location + ": " + failure); return ServerWideConfig.NONE; @@ -312,10 +417,14 @@ private ServerWideConfig readServerWideConfig(File pluginsDirectory) { * lines indented differently from its first entry. Values may be bare, * double- or single-quoted. Entries the trace server would reject -- and * anything this reader does not understand -- are dropped, one by one, - * and at most {@link #MAX_TAGS} are kept; nothing here throws. + * and at most {@link #MAX_TAGS} are kept. The first {@code server-id:} + * line at column 0, outside a {@code tags:} block, whose value is a valid + * tag value made of {@code [A-Za-z0-9_.-]} is the installation's ID; any + * other is ignored. Nothing here throws. */ static ServerWideConfig parseServerWideConfig(List lines) { Boolean disables = null; + String serverId = null; Map tags = new LinkedHashMap<>(); boolean inTags = false; int entryIndent = -1; @@ -345,6 +454,16 @@ static ServerWideConfig parseServerWideConfig(List lines) { entryIndent = -1; continue; } + if (serverId == null) { + Matcher matcher = SERVER_ID_LINE.matcher(line); + if (matcher.matches()) { + String value = scalar(matcher.group(1).trim()); + if (value != null && value.length() <= MAX_TAG_LENGTH && TAG_KEY.matcher(value).matches()) { + serverId = value; + } + continue; + } + } if (disables == null) { Matcher matcher = ENABLED_LINE.matcher(line); if (matcher.find()) { @@ -353,7 +472,8 @@ static ServerWideConfig parseServerWideConfig(List lines) { } } return new ServerWideConfig(disables != null && disables, - tags.isEmpty() ? Collections.emptyMap() : Collections.unmodifiableMap(tags)); + tags.isEmpty() ? Collections.emptyMap() : Collections.unmodifiableMap(tags), + serverId); } private static void addServerWideTag(Map tags, String entry) { @@ -493,6 +613,19 @@ static Map withVersion(Map tags, String version) return merged; } + /** + * The tags plus {@code install}, unless they already carry one, there is + * no ID, or adding it would pass {@link #MAX_TAGS}. A copy when it adds. + */ + static Map withInstall(Map tags, String installId) { + if (installId == null || tags.containsKey(INSTALL_TAG) || tags.size() >= MAX_TAGS) { + return tags; + } + Map merged = new LinkedHashMap<>(tags); + merged.put(INSTALL_TAG, installId); + return merged; + } + /** Reports that {@code name} happened. */ public void report(String name) { report(name, null, null); @@ -507,7 +640,7 @@ public void report(String name, Double value, Map tags) { return; } final String body = json(application, name, value, - withServerWideTags(withVersion(tags, version), serverWideTags)); + withServerWideTags(withInstall(withVersion(tags, version), installId), serverWideTags)); executor.execute(() -> send(body)); } @@ -644,6 +777,7 @@ public static final class Builder { private String key; private boolean enabled = true; private File pluginsDirectory; + private String installId; private Logger logger; private Builder(String baseUrl, String application, String version) { @@ -682,14 +816,19 @@ public Builder enabled(boolean enabled) { * in a Bukkit plugin), {@link #build()} makes sure * {@code plugins/trace/config.yml} exists -- creating it with * {@code enabled: true} if it is missing -- and honours - * {@code enabled: false} in it. The file is never rewritten once it - * exists. Its optional {@code tags:} block is added to every event - * this client reports, below the event's own tags: + * {@code enabled: false} in it. Its optional {@code tags:} block is + * added to every event this client reports, below the event's own + * tags. Its {@code server-id:} line is sent as the tag + * {@code install}; when an enabled client finds none, it appends one + * -- a random UUID under a comment saying what it is -- and otherwise + * never changes the file. If that write fails, the ID is kept in + * memory for this process only: * *

          * enabled: true
          * tags:
          *   ci: "true"
+         * server-id: 0f8b6c1e-...
          * 
* *

Both are read once, here. Optional; programs that are not @@ -700,6 +839,27 @@ public Builder serverWideConfig(File pluginsDirectory) { return this; } + /** + * The installation's ID, for programs that are not Spigot plugins: + * sent as the tag {@code install} on every event, so the trace + * server can count installations. It should be random -- e.g. a + * {@link UUID#randomUUID()} the program stores in its own + * configuration -- and never derived from a person, account or + * address. Takes precedence over the server-wide {@code server-id:}. + * Trimmed; {@code null} or blank means none. Without it, and without + * {@link #serverWideConfig(File)}, no {@code install} tag is sent. + * + * @throws IllegalArgumentException when longer than {@value TraceClient#MAX_TAG_LENGTH} characters + */ + public Builder installId(String installId) { + String trimmed = installId == null ? null : installId.trim(); + if (trimmed != null && trimmed.length() > MAX_TAG_LENGTH) { + throw new IllegalArgumentException("installId is longer than " + MAX_TAG_LENGTH + " characters"); + } + this.installId = trimmed == null || trimmed.isEmpty() ? null : trimmed; + return this; + } + /** Where dropped reports are mentioned, at {@link Level#FINE}. Optional. */ public Builder logger(Logger logger) { this.logger = logger; diff --git a/src/test/java/software/stephenson/trace/TraceClientTest.java b/src/test/java/software/stephenson/trace/TraceClientTest.java index e3aa781..4203cec 100644 --- a/src/test/java/software/stephenson/trace/TraceClientTest.java +++ b/src/test/java/software/stephenson/trace/TraceClientTest.java @@ -37,6 +37,8 @@ */ class TraceClientTest { + private static final String UUID_PATTERN = "[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}"; + private HttpServer server; private final List received = new CopyOnWriteArrayList<>(); private volatile int replyStatus = 201; @@ -588,8 +590,16 @@ void serverWideConfig_isCreatedWithTheExactContentWhenMissing(@TempDir Path plug + "# own tag of the same name wins. On a test or CI server, uncomment the two\n" + "# lines below so its events are left out of real-installation figures.\n" + "# tags:\n" - + "# ci: \"true\"\n"; - assertEquals(expected, new String(Files.readAllBytes(file), StandardCharsets.UTF_8)); + + "# ci: \"true\"\n" + + "#\n" + + "# server-id: a random ID made on first run and sent as the tag \"install\", so\n" + + "# trace can count servers, not events. It identifies no person and no IP\n" + + "# address. Delete the server-id line to get a new one.\n"; + String actual = new String(Files.readAllBytes(file), StandardCharsets.UTF_8); + assertTrue(actual.startsWith(expected), actual); + String idLine = actual.substring(expected.length()); + assertTrue(idLine.matches("server-id: " + UUID_PATTERN + "\n"), "the template is followed by one server-id line: " + idLine); + assertEquals(idLine.substring("server-id: ".length()).trim(), client.installId()); assertTrue(client.isEnabled(), "a freshly created switch file means enabled"); assertNull(client.disabledReason()); client.close(); @@ -709,14 +719,14 @@ void serverWideTags_areMergedIntoEveryEvent(@TempDir Path plugins) throws Except String body = reportedBody(plugins, "startup", tags); // Assert - assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"ci\":\"true\"}}", body); + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"test-server\",\"ci\":\"true\"}}", body); } @Test void serverWideTags_areAddedToAnEventWithNoTagsOfItsOwn(@TempDir Path plugins) throws Exception { writeServerWideConfig(plugins, "tags:\n ci: \"true\"\n"); - assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"ci\":\"true\"}}", + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"test-server\",\"ci\":\"true\"}}", reportedBody(plugins, "startup", null)); } @@ -733,7 +743,7 @@ void serverWideTags_neverOverwriteTheEventsOwnTag(@TempDir Path plugins) throws // Assert assertEquals("{\"application\":\"MyPlugin\",\"name\":\"command\",\"tags\":" - + "{\"version\":\"1.2.3\",\"name\":\"home\",\"ci\":\"true\"}}", body); + + "{\"version\":\"1.2.3\",\"name\":\"home\",\"install\":\"test-server\",\"ci\":\"true\"}}", body); } @Test @@ -831,7 +841,7 @@ void serverWideTags_dropEntriesTheServerWouldRejectOrThatAreNotUnderstood() { @Test void serverWideTags_areCappedSoTheEventStaysWithinTheServersTagLimit(@TempDir Path plugins) throws Exception { - // Arrange: 40 server-wide tags, 30 event tags. + // Arrange: 40 server-wide tags, 29 event tags. StringBuilder file = new StringBuilder("tags:\n"); for (int i = 0; i < 40; i++) { file.append(" s").append(i).append(": v\n"); @@ -839,7 +849,7 @@ void serverWideTags_areCappedSoTheEventStaysWithinTheServersTagLimit(@TempDir Pa assertEquals(TraceClient.MAX_TAGS, tagsOf(file.toString()).size(), "at most MAX_TAGS are read"); writeServerWideConfig(plugins, file.toString()); Map tags = new LinkedHashMap<>(); - for (int i = 0; i < 30; i++) { + for (int i = 0; i < 29; i++) { tags.put("e" + i, "v"); } @@ -848,8 +858,9 @@ void serverWideTags_areCappedSoTheEventStaysWithinTheServersTagLimit(@TempDir Pa // Assert int pairs = body.split("\":\"v\"", -1).length - 1; - assertEquals(TraceClient.MAX_TAGS - 1, pairs, "30 event tags + version + 1 server-wide: " + body); - assertTrue(body.contains("\"e29\":\"v\"") && body.contains("\"version\":\"1.2.3\"") + assertEquals(TraceClient.MAX_TAGS - 2, pairs, "29 event tags + version + install + 1 server-wide: " + body); + assertTrue(body.contains("\"e28\":\"v\"") && body.contains("\"version\":\"1.2.3\"") + && body.contains("\"install\":\"test-server\"") && body.contains("\"s0\":\"v\"") && !body.contains("\"s1\""), body); } @@ -878,7 +889,7 @@ void serverWideTags_areNoneWithoutAServerWideConfigOrAFileThatHasNone(@TempDir P // A file with only the switch in it. arrived = new CountDownLatch(1); writeServerWideConfig(plugins, "enabled: true\n"); - assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\"}}", reportedBody(plugins, "startup", null)); + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"test-server\"}}", reportedBody(plugins, "startup", null)); } @Test @@ -887,12 +898,15 @@ void serverWideTags_aFreshlyCreatedFileHasNoActiveTags(@TempDir Path plugins) th String body = reportedBody(plugins, "startup", null); // ... and nothing in it is live: the example is commented out. + // (The install tag is the server-id build() appended, not a tags: entry.) assertTrue(Files.exists(plugins.resolve("trace").resolve("config.yml"))); - assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\"}}", body); + assertTrue(body.matches("\\{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":" + + "\\{\"version\":\"1\\.2\\.3\",\"install\":\"" + UUID_PATTERN + "\"}}"), body); TraceClient.ServerWideConfig config = TraceClient.parseServerWideConfig( Arrays.asList(TraceClient.SERVER_WIDE_CONFIG_CONTENT.split("\n"))); assertTrue(config.tags.isEmpty()); assertFalse(config.disables); + assertNull(config.serverId, "the template's server-id explanation is a comment, not a value"); } @Test @@ -910,7 +924,7 @@ void serverWideTags_malformedFileNeverThrowsAndReportingStillWorks(@TempDir Path arrived = new CountDownLatch(1); writeServerWideConfig(plugins, content); String body = assertDoesNotThrow(() -> reportedBody(plugins, "startup", null), content); - assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\"}}", body, content); + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"test-server\"}}", body, content); } // Bytes that are not UTF-8 at all. @@ -918,8 +932,12 @@ void serverWideTags_malformedFileNeverThrowsAndReportingStillWorks(@TempDir Path arrived = new CountDownLatch(1); Path file = plugins.resolve("trace").resolve("config.yml"); Files.write(file, new byte[] {'t', 'a', 'g', 's', ':', '\n', ' ', ' ', 'c', 'i', ':', ' ', (byte) 0xC3, (byte) 0x28, '\n'}); - assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\"}}", reportedBody(plugins, "startup", null), - "a file that is not UTF-8 counts as enabled with no tags"); + byte[] notUtf8 = Files.readAllBytes(file); + String body = reportedBody(plugins, "startup", null); + assertTrue(body.matches("\\{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":" + + "\\{\"version\":\"1\\.2\\.3\",\"install\":\"" + UUID_PATTERN + "\"}}"), + "a file that is not UTF-8 counts as enabled with no tags, and an in-memory install ID: " + body); + assertArrayEquals(notUtf8, Files.readAllBytes(file), "a file that cannot be read is never appended to"); } @Test @@ -980,7 +998,190 @@ void serverWideConfig_aPathThatCannotBeConvertedIsLoggedFineAndTreatedAsEnabled( client.close(); } + @Test + void installId_isGeneratedPersistedOnceAndReusedAcrossBuilds(@TempDir Path plugins) throws Exception { + // Arrange + Path file = plugins.resolve("trace").resolve("config.yml"); + + // Act: two starts of the same server, two plugins each time. + TraceClient first = TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(plugins.toFile()).build(); + String afterFirst = new String(Files.readAllBytes(file), StandardCharsets.UTF_8); + TraceClient second = TraceClient.builder(baseUrl(), "OtherPlugin", "4.5.6").key("k").serverWideConfig(plugins.toFile()).build(); + TraceClient third = TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(plugins.toFile()).build(); + first.report("startup"); + assertTrue(arrived.await(5, TimeUnit.SECONDS)); + first.close(); + second.close(); + third.close(); + + // Assert + String id = first.installId(); + assertNotNull(id); + assertTrue(id.matches(UUID_PATTERN), id); + assertEquals(id, second.installId(), "every plugin on the server shares the one ID"); + assertEquals(id, third.installId(), "and it survives a restart"); + assertEquals(afterFirst, new String(Files.readAllBytes(file), StandardCharsets.UTF_8), "only the first build writes"); + assertEquals(1, afterFirst.split("\nserver-id: ", -1).length - 1, afterFirst); + assertTrue(afterFirst.endsWith("\nserver-id: " + id + "\n"), afterFirst); + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"" + id + "\"}}", + received.get(0).body); + } + + @Test + void installId_isAppendedWithoutDisturbingTheOperatorsFile(@TempDir Path plugins) throws Exception { + // Arrange: hand-edited, comments everywhere, no trailing newline. + String operatorsFile = "# my notes -- keep me\nenabled: true # on, for now\ntags:\n ci: \"true\" # test box\n# the end"; + Path file = writeServerWideConfigExactly(plugins, operatorsFile); + + // Act + TraceClient client = TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(plugins.toFile()).build(); + client.close(); + + // Assert + String after = new String(Files.readAllBytes(file), StandardCharsets.UTF_8); + assertEquals(operatorsFile + "\n" + TraceClient.SERVER_ID_COMMENT + "server-id: " + client.installId() + "\n", after, + "the operator's text is kept byte for byte; one commented block is appended"); + TraceClient.ServerWideConfig reread = TraceClient.parseServerWideConfig(Arrays.asList(after.split("\n", -1))); + assertEquals(Collections.singletonMap("ci", "true"), reread.tags, "the appended line does not join the tags block"); + assertEquals(client.installId(), reread.serverId); + } + + @Test + void installId_anExistingServerIdIsUsedAndAnUnusableOneIsIgnored() { + assertEquals("abc-123", TraceClient.parseServerWideConfig(Arrays.asList("server-id: \"abc-123\" # mine")).serverId); + assertEquals("first", TraceClient.parseServerWideConfig(Arrays.asList("server-id: first", "server-id: second")).serverId); + assertEquals("good", TraceClient.parseServerWideConfig(Arrays.asList("server-id:", "server-id: has space", "server-id: good")).serverId); + assertNull(TraceClient.parseServerWideConfig(Arrays.asList(" server-id: indented")).serverId, "column 0 only"); + assertNull(TraceClient.parseServerWideConfig(Arrays.asList("# server-id: commented")).serverId); + TraceClient.ServerWideConfig inTags = TraceClient.parseServerWideConfig(Arrays.asList("tags:", " server-id: x")); + assertNull(inTags.serverId, "inside tags: it is a tag"); + assertEquals(Collections.singletonMap("server-id", "x"), inTags.tags); + } + + @Test + void installId_unwritableConfigFallsBackToAnInMemoryIdWithoutThrowing(@TempDir Path scratch) throws Exception { + // Arrange: a "plugins directory" that is a regular file, so nothing + // under it can be created (permission bits are no use as root). + Path notADirectory = scratch.resolve("plugins"); + Files.write(notADirectory, "not a directory".getBytes(StandardCharsets.UTF_8)); + + // Act + TraceClient first = assertDoesNotThrow(() -> TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k") + .serverWideConfig(notADirectory.toFile()).build()); + TraceClient second = TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(notADirectory.toFile()).build(); + first.report("startup"); + assertTrue(arrived.await(5, TimeUnit.SECONDS)); + first.close(); + second.close(); + + // Assert + assertTrue(first.isEnabled()); + assertTrue(first.installId().matches(UUID_PATTERN), first.installId()); + assertNotEquals(first.installId(), second.installId(), "nothing persisted: each process gets its own"); + assertTrue(received.get(0).body.contains("\"install\":\"" + first.installId() + "\""), received.get(0).body); + assertEquals("not a directory", new String(Files.readAllBytes(notADirectory), StandardCharsets.UTF_8)); + } + + @Test + void installId_aDisabledClientNeitherGeneratesNorWritesOne(@TempDir Path plugins) throws Exception { + Path file = plugins.resolve("trace").resolve("config.yml"); + + // Environment: the file is not even created. + environment.put("DO_NOT_TRACK", "1"); + TraceClient byEnvironment = TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(plugins.toFile()).build(); + assertNull(byEnvironment.installId()); + assertFalse(Files.exists(file), "an environment opt-out touches nothing on disk"); + environment.clear(); + environment.put("TRACE_USAGE_REPORTING", "off"); + assertNull(TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(plugins.toFile()).build().installId()); + assertFalse(Files.exists(file)); + environment.clear(); + + // The server-wide switch. + writeServerWideConfigExactly(plugins, "enabled: false\n"); + assertNull(TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").serverWideConfig(plugins.toFile()).build().installId()); + assertEquals("enabled: false\n", new String(Files.readAllBytes(file), StandardCharsets.UTF_8)); + + // The plugin's own switch, and no key. + writeServerWideConfigExactly(plugins, "enabled: true\n"); + assertNull(TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").enabled(false).serverWideConfig(plugins.toFile()).build().installId()); + assertNull(TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").serverWideConfig(plugins.toFile()).build().installId()); + assertEquals("enabled: true\n", new String(Files.readAllBytes(file), StandardCharsets.UTF_8)); + + // A file created by a disabled client has no server-id line in it. + Files.delete(file); + TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").enabled(false).serverWideConfig(plugins.toFile()).build(); + assertEquals(TraceClient.SERVER_WIDE_CONFIG_CONTENT, new String(Files.readAllBytes(file), StandardCharsets.UTF_8)); + + // An explicit ID is not sent either. + assertNull(TraceClient.builder(baseUrl(), "MyCli", "1.2.3").key("k").enabled(false).installId("abc").build().installId()); + } + + @Test + void installId_canBeGivenExplicitlyByAProgramThatIsNotAPlugin(@TempDir Path plugins) throws Exception { + // Arrange + TraceClient client = TraceClient.builder(baseUrl(), "MyCli", "1.2.3").key("k").installId(" cli-install-1 ").build(); + + // Act + client.report("startup"); + + // Assert + assertTrue(arrived.await(5, TimeUnit.SECONDS)); + client.close(); + assertEquals("cli-install-1", client.installId()); + assertEquals("{\"application\":\"MyCli\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"cli-install-1\"}}", + received.get(0).body); + + // None given, none sent; blank is none. + assertNull(TraceClient.builder(baseUrl(), "MyCli", "1.2.3").key("k").build().installId()); + assertNull(TraceClient.builder(baseUrl(), "MyCli", "1.2.3").key("k").installId(" ").build().installId()); + assertNull(TraceClient.builder(baseUrl(), "MyCli", "1.2.3").key("k").installId(null).build().installId()); + StringBuilder overlong = new StringBuilder(); + for (int i = 0; i <= TraceClient.MAX_TAG_LENGTH; i++) { + overlong.append('x'); + } + assertThrows(IllegalArgumentException.class, + () -> TraceClient.builder(baseUrl(), "MyCli", "1.2.3").installId(overlong.toString())); + + // An explicit ID wins over the server-wide file, which then gains no server-id. + TraceClient both = TraceClient.builder(baseUrl(), "MyPlugin", "1.2.3").key("k").installId("explicit") + .serverWideConfig(plugins.toFile()).build(); + both.close(); + assertEquals("explicit", both.installId()); + assertEquals(TraceClient.SERVER_WIDE_CONFIG_CONTENT, + new String(Files.readAllBytes(plugins.resolve("trace").resolve("config.yml")), StandardCharsets.UTF_8)); + } + + @Test + void installId_anEventsOwnInstallTagWins(@TempDir Path plugins) throws Exception { + // Arrange + writeServerWideConfig(plugins, "tags:\n install: from-tags-block\n"); + Map tags = new LinkedHashMap<>(); + tags.put("install", "the-plugins-own"); + + // Act + String body = reportedBody(plugins, "startup", tags); + + // Assert + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"install\":\"the-plugins-own\",\"version\":\"1.2.3\"}}", body); + assertEquals(Collections.singletonMap("install", "the-plugins-own"), tags, "the caller's map is not modified"); + + // Without its own, the event gets the server-id, not the tags: block's entry. + arrived = new CountDownLatch(1); + assertEquals("{\"application\":\"MyPlugin\",\"name\":\"startup\",\"tags\":{\"version\":\"1.2.3\",\"install\":\"test-server\"}}", + reportedBody(plugins, "startup", null)); + } + + /** + * Writes {@code content} as the server-wide file, followed by a fixed + * {@code server-id: test-server} line so the tag tests get a predictable + * {@code install} tag rather than a random one. + */ private static Path writeServerWideConfig(Path plugins, String content) throws java.io.IOException { + return writeServerWideConfigExactly(plugins, content + "\nserver-id: test-server\n"); + } + + private static Path writeServerWideConfigExactly(Path plugins, String content) throws java.io.IOException { Path file = plugins.resolve("trace").resolve("config.yml"); Files.createDirectories(file.getParent()); Files.write(file, content.getBytes(StandardCharsets.UTF_8));