Testing merge

CHANGELOG.md

@@ -1,5 +1,79 @@
 # Changelog
 
+## 2.72.0
+
+### New Features ✨
+
+#### Asgi
+
+- Set `user_agent.original` for inbound filters by @sentrivana in [#7794](https://github.com/getsentry/sentry-python/pull/7794)
+- Add `sentry.is_localhost` for inbound filters by @sentrivana in [#7781](https://github.com/getsentry/sentry-python/pull/7781)
+
+#### Boto3
+
+- Add `ServiceExtension` class by @pabloDeputter in [#7536](https://github.com/getsentry/sentry-python/pull/7536)
+- Add response, retry, and error span attributes by @pabloDeputter in [#7499](https://github.com/getsentry/sentry-python/pull/7499)
+- Add common OTel AWS client attributes by @pabloDeputter in [#7481](https://github.com/getsentry/sentry-python/pull/7481)
+
+#### Wsgi
+
+- Add `user_agent.original` attr for inbound filters by @sentrivana in [#7793](https://github.com/getsentry/sentry-python/pull/7793)
+- Add `sentry.is_localhost` for inbound filters by @sentrivana in [#7780](https://github.com/getsentry/sentry-python/pull/7780)
+
+#### Other
+
+- (aiohttp) Add `sentry.is_localhost` for inbound filters by @sentrivana in [#7785](https://github.com/getsentry/sentry-python/pull/7785)
+- (aiohttp,sanic,tornado) Add `user_agent.original` attr for inbound filters by @sentrivana in [#7795](https://github.com/getsentry/sentry-python/pull/7795)
+- (sanic) Add `sentry.is_localhost` for inbound filters by @sentrivana in [#7783](https://github.com/getsentry/sentry-python/pull/7783)
+- (tornado) Add `sentry.is_localhost` for inbound filters by @sentrivana in [#7787](https://github.com/getsentry/sentry-python/pull/7787)
+- (types) Export DataCollectionUserOptions from sentry_sdk.types by @sentrivana in [#7957](https://github.com/getsentry/sentry-python/pull/7957)
+- Better IPv6 localhost detection by @sentrivana in [#7786](https://github.com/getsentry/sentry-python/pull/7786)
+
+### Bug Fixes 🐛
+
+#### Boto3
+
+- Trace the complete botocore client-call lifecycle by @pabloDeputter in [#7538](https://github.com/getsentry/sentry-python/pull/7538)
+- Finish `StreamingBody` span correctly by @pabloDeputter in [#7540](https://github.com/getsentry/sentry-python/pull/7540)
+
+#### Other
+
+- (crons) Return `None` from `capture_checkin` when nothing is sent by @HardMax71 in [#7868](https://github.com/getsentry/sentry-python/pull/7868)
+- (scrubber) Scrub request bodies that are JSON arrays by @Om-singhaI in [#7561](https://github.com/getsentry/sentry-python/pull/7561)
+- Fix span, metrics, log client reports by @sentrivana in [#7933](https://github.com/getsentry/sentry-python/pull/7933)
+
+### Internal Changes 🔧
+
+#### Deps
+
+- Lock file maintenance by @renovate in [#7848](https://github.com/getsentry/sentry-python/pull/7848)
+- Lock file maintenance by @renovate in [#7484](https://github.com/getsentry/sentry-python/pull/7484)
+
+#### Openai Agents
+
+- Bump version for type checking to 0.23.1 by @alexander-alderman-webb in [#7935](https://github.com/getsentry/sentry-python/pull/7935)
+- Use `model_dump()` instead of deprecated `dict()` by @intertermux-code in [#7862](https://github.com/getsentry/sentry-python/pull/7862)
+
+#### Starlette
+
+- Drop direct `anyio` dependency by @alexander-alderman-webb in [#7724](https://github.com/getsentry/sentry-python/pull/7724)
+- Expect parametrized route in `starlette>=1.7` by @alexander-alderman-webb in [#7789](https://github.com/getsentry/sentry-python/pull/7789)
+
+#### Other
+
+- (boto3) Split integration into internal modules by @pabloDeputter in [#7537](https://github.com/getsentry/sentry-python/pull/7537)
+- (huggingface_hub) Use `httpx2-pytest` for `huggingface_hub>=2` by @alexander-alderman-webb in [#7790](https://github.com/getsentry/sentry-python/pull/7790)
+- (mypy) Bump version to 2.4.0 by @alexander-alderman-webb in [#7937](https://github.com/getsentry/sentry-python/pull/7937)
+- (openai) Stop directly calling `_calculate_*_token_usage()` by @alexander-alderman-webb in [#7875](https://github.com/getsentry/sentry-python/pull/7875)
+- (strawberry-graphql) Bump version for type checking to 0.332.0 by @alexander-alderman-webb in [#7936](https://github.com/getsentry/sentry-python/pull/7936)
+- Deprecate `ai.monitoring.record_token_usage()` by @alexander-alderman-webb in [#7918](https://github.com/getsentry/sentry-python/pull/7918)
+- Deprecate `send_default_pii` by @sentrivana in [#7694](https://github.com/getsentry/sentry-python/pull/7694)
+- Move FastAPI active thread tracking into Starlette by @faizansaiyed123 in [#7825](https://github.com/getsentry/sentry-python/pull/7825)
+- Upgrade ubuntu 20.04/22.04 runners to 24.04 by @joshuarli in [#7843](https://github.com/getsentry/sentry-python/pull/7843)
+- 🤖 Update test matrix with new releases (10/05) by @github-actions in [#7849](https://github.com/getsentry/sentry-python/pull/7849)
+- 🤖 Update test matrix with new releases (09/28) by @github-actions in [#7715](https://github.com/getsentry/sentry-python/pull/7715)
+- Deprecate transaction profiling by @sentrivana in [#7751](https://github.com/getsentry/sentry-python/pull/7751)
+
 ## 2.71.0
 
 ### New Features ✨

docs/conf.py

@@ -31,7 +31,7 @@
 copyright = "2019-{}, Sentry Team and Contributors".format(datetime.now().year)
 author = "Sentry Team and Contributors"
 
-release = "2.71.0"
+release = "2.72.0"
 version = ".".join(release.split(".")[:2])  # The short X.Y version.
 
 

sentry_sdk/_batcher.py

@@ -20,6 +20,7 @@ class Batcher(Generic[T]):
     FLUSH_WAIT_TIME = 5.0
 
     TYPE = ""
+    CLIENT_REPORT_TYPE = ""
     CONTENT_TYPE = ""
 
     def __init__(
@@ -123,13 +124,16 @@ def add(self, item: "T") -> None:
                 return None
 
             with self._lock:
-                if len(self._buffer) >= self.MAX_BEFORE_DROP:
-                    self._record_lost(item)
-                    return None
-
-                self._buffer.append(item)
-                if len(self._buffer) >= self.MAX_BEFORE_FLUSH:
-                    self._flush_event.set()
+                dropped = len(self._buffer) >= self.MAX_BEFORE_DROP
+                if not dropped:
+                    self._buffer.append(item)
+                    if len(self._buffer) >= self.MAX_BEFORE_FLUSH:
+                        self._flush_event.set()
+
+            # Record the loss outside of the lock, since this might involve
+            # serializing the item to report lost bytes
+            if dropped:
+                self._record_lost(item)
         finally:
             self._active.flag = False
 
@@ -149,25 +153,25 @@ def flush(self) -> None:
         finally:
             self._active.flag = was_active
 
-    def _add_to_envelope(self, envelope: "Envelope") -> None:
-        envelope.add_item(
-            Item(
-                type=self.TYPE,
-                content_type=self.CONTENT_TYPE,
-                headers={
-                    "item_count": len(self._buffer),
-                },
-                payload=PayloadRef(
-                    json={
-                        "version": 2,
-                        "items": [
-                            self._to_transport_format(item) for item in self._buffer
-                        ],
-                    }
-                ),
-            )
+    @classmethod
+    def _to_envelope_item(cls, items: "list[T]") -> "Item":
+        return Item(
+            type=cls.TYPE,
+            content_type=cls.CONTENT_TYPE,
+            headers={
+                "item_count": len(items),
+            },
+            payload=PayloadRef(
+                json={
+                    "version": 2,
+                    "items": [cls._to_transport_format(item) for item in items],
+                }
+            ),
         )
 
+    def _add_to_envelope(self, envelope: "Envelope") -> None:
+        envelope.add_item(self._to_envelope_item(self._buffer))
+
     def _flush(self) -> "Optional[Envelope]":
         envelope = Envelope(
             headers={"sent_at": format_timestamp(datetime.now(timezone.utc))}
@@ -183,7 +187,17 @@ def _flush(self) -> "Optional[Envelope]":
         return envelope
 
     def _record_lost(self, item: "T") -> None:
-        pass
+        envelope_item = None
+        with capture_internal_exceptions():
+            envelope_item = self._to_envelope_item([item])
+
+        if envelope_item:
+            self._record_lost_func(
+                reason="queue_overflow",
+                data_category=self.CLIENT_REPORT_TYPE,
+                item=envelope_item,
+                quantity=1,
+            )
 
     @staticmethod
     def _to_transport_format(item: "T") -> "Any":

sentry_sdk/_log_batcher.py

@@ -1,7 +1,6 @@
 from typing import TYPE_CHECKING
 
 from sentry_sdk._batcher import Batcher
-from sentry_sdk.envelope import Item, PayloadRef
 from sentry_sdk.utils import serialize_attribute
 
 if TYPE_CHECKING:
@@ -16,6 +15,7 @@ class LogBatcher(Batcher["Log"]):
     FLUSH_WAIT_TIME = 5.0
 
     TYPE = "log"
+    CLIENT_REPORT_TYPE = "log_item"
     CONTENT_TYPE = "application/vnd.sentry.items.log+json"
 
     @staticmethod
@@ -41,26 +41,3 @@ def _to_transport_format(item: "Log") -> "Any":
             res["span_id"] = item["span_id"]
 
         return res
-
-    def _record_lost(self, item: "Log") -> None:
-        # Construct log envelope item without sending it to report lost bytes
-        log_item = Item(
-            type=self.TYPE,
-            content_type=self.CONTENT_TYPE,
-            headers={
-                "item_count": 1,
-            },
-            payload=PayloadRef(
-                json={
-                    "version": 2,
-                    "items": [self._to_transport_format(item)],
-                }
-            ),
-        )
-
-        self._record_lost_func(
-            reason="queue_overflow",
-            data_category="log_item",
-            item=log_item,
-            quantity=1,
-        )

sentry_sdk/_metrics_batcher.py

@@ -15,6 +15,7 @@ class MetricsBatcher(Batcher["Metric"]):
     FLUSH_WAIT_TIME = 5.0
 
     TYPE = "trace_metric"
+    CLIENT_REPORT_TYPE = "trace_metric"
     CONTENT_TYPE = "application/vnd.sentry.items.trace-metric+json"
 
     @staticmethod
@@ -39,10 +40,3 @@ def _to_transport_format(item: "Metric") -> "Any":
             res["unit"] = item["unit"]
 
         return res
-
-    def _record_lost(self, item: "Metric") -> None:
-        self._record_lost_func(
-            reason="queue_overflow",
-            data_category="trace_metric",
-            quantity=1,
-        )

sentry_sdk/_types.py

@@ -413,6 +413,7 @@ class DataCollection(TypedDict):
         "log_item",
         "log_byte",
         "trace_metric",
+        "trace_metric_byte",
     ]
     SessionStatus = Literal["ok", "exited", "crashed", "abnormal"]
 

sentry_sdk/client.py

@@ -555,15 +555,15 @@
                 except Exception:
                     exception_raised_in_before_send = True
                     raise
             if new_event is None:
                 logger.info("before send dropped event")
                 if self.transport:
                     reason = (
                         "callback_error"
                         if exception_raised_in_before_send
                         else "before_send"
                     )
                     self.transport.record_lost_event(reason, data_category="error")
 
             event = new_event
 
@@ -764,6 +764,27 @@
 
         return return_value
 
+    def _record_lost_telemetry(
+        self, reason: str, ty: str, telemetry: "Union[Log, Metric]"
+    ) -> None:
+        if self.transport is None:
+            return
+
+        batcher_cls: "Union[type[LogBatcher], type[MetricsBatcher]]"
+        if ty == "log":
+            data_category: "EventDataCategory" = "log_item"
+            batcher_cls = LogBatcher
+        else:
+            data_category = "trace_metric"
+            batcher_cls = MetricsBatcher
+
+        # Construct envelope item without sending it to report lost bytes
+        item = None
+        with capture_internal_exceptions():
+            item = batcher_cls._to_envelope_item([telemetry])  # type: ignore[list-item]
+
+        self.transport.record_lost_event(reason, data_category=data_category, item=item)
+
     def _capture_telemetry(
         self,
         telemetry: "Optional[Union[Log, Metric, Span]]",
@@ -804,32 +825,12 @@
                     exception_raised_in_before_send_func = True
                     raise
 
-            if ty == "log":
-                if exception_raised_in_before_send_func:
-                    if self.transport:
-                        self.transport.record_lost_event(
-                            "callback_error", data_category="log_item"
-                        )
-                    return
-                if serialized is None:
-                    if self.transport:
-                        self.transport.record_lost_event(
-                            "before_send", data_category="log_item"
-                        )
-                    return
-
-            elif ty == "metric":
+            if ty in ("log", "metric"):
                 if exception_raised_in_before_send_func:
-                    if self.transport:
-                        self.transport.record_lost_event(
-                            "callback_error", data_category="trace_metric"
-                        )
+                    self._record_lost_telemetry("callback_error", ty, telemetry)  # type: ignore[arg-type]
                     return
                 if serialized is None:
-                    if self.transport:
-                        self.transport.record_lost_event(
-                            "before_send", data_category="trace_metric"
-                        )
+                    self._record_lost_telemetry("before_send", ty, telemetry)  # type: ignore[arg-type]
                     return
 
             elif ty == "span" and isinstance(telemetry, Span):

sentry_sdk/consts.py

@@ -1667,10 +1667,10 @@
 
             This can be used, for instance, for manual PII stripping before sending.
 
             By the time `before_send` is executed, all scope data has already been applied to the event. Further
             modification of the scope won't have any effect.
 
         :param before_breadcrumb: This function is called with an SDK-specific breadcrumb object before the breadcrumb
             is added to the scope.
 
             When nothing is returned from the function, the breadcrumb is dropped.
@@ -1873,4 +1873,4 @@
 del _get_default_options
 
 
-VERSION = "2.71.0"
+VERSION = "2.72.0"

sentry_sdk/integrations/gql.py

@@ -38,7 +38,9 @@
 
     from sentry_sdk._types import Event, EventProcessor
 
-    EventDataType = Dict[str, Union[str, Tuple[VariableDefinitionNode, ...]]]
+    EventDataType = Union[
+        Dict[str, Union[str, Tuple[VariableDefinitionNode, ...], None]]
+    ]
 
 
 class GQLIntegration(Integration):

sentry_sdk/integrations/openai_agents/patches/agent_run.py

@@ -65,7 +65,7 @@ def _maybe_start_agent_span(
     context_wrapper._sentry_current_agent = agent  # type: ignore[attr-defined]
     span = invoke_agent_span(context_wrapper, agent, span_kwargs)
     context_wrapper._sentry_agent_span = span  # type: ignore[attr-defined]
-    agent._sentry_agent_span = span  # type: ignore[attr-defined]
+    agent._sentry_agent_span = span
 
     if not is_streaming:
         return span

sentry_sdk/integrations/openai_agents/patches/models.py

@@ -71,7 +71,7 @@ def _get_model(
 
     # Capture the request model name for spans (agent.model can be None when using defaults)
     request_model_name = model.model if hasattr(model, "model") else str(model)
-    agent._sentry_request_model = request_model_name  # type: ignore[attr-defined]
+    agent._sentry_request_model = request_model_name
 
     # Wrap _fetch_response if it exists (for OpenAI models) to capture response model
     if hasattr(model, "_fetch_response"):
@@ -81,7 +81,7 @@ def _get_model(
         async def wrapped_fetch_response(*args: "Any", **kwargs: "Any") -> "Any":
             response = await original_fetch_response(*args, **kwargs)
             if hasattr(response, "model") and response.model:
-                agent._sentry_response_model = str(response.model)  # type: ignore[attr-defined]
+                agent._sentry_response_model = str(response.model)
             return response
 
         model._fetch_response = wrapped_fetch_response

sentry_sdk/integrations/openai_agents/patches/runner.py

@@ -45,7 +45,7 @@ async def on_tool_start(
         context: "ToolContext[TContext]",
         agent: "Agent[TContext]",
         tool: "Tool",
-    ) -> "None":
+    ) -> None:
         if not isinstance(tool, FunctionTool):
             return
 
@@ -65,7 +65,7 @@ async def on_tool_end(
         agent: "Agent[TContext]",
         tool: "Tool",
         result: "object",
-    ) -> "None":
+    ) -> None:
         if not isinstance(tool, FunctionTool):
             return
 
@@ -94,7 +94,7 @@ def _patch_run_hooks(hooks: "RunHooks[TContext]") -> None:
     @wraps(original_on_tool_start)
     async def on_tool_start(
         context: "ToolContext[TContext]", agent: "Agent[TContext]", tool: "Tool"
-    ) -> "None":
+    ) -> None:
         with capture_internal_exceptions():
             await sentry_hooks.on_tool_start(context, agent, tool)
         await original_on_tool_start(context, agent, tool)
@@ -105,7 +105,7 @@ async def on_tool_end(
         agent: "Agent[TContext]",
         tool: "Tool",
         result: "object",
-    ) -> "None":
+    ) -> None:
         with capture_internal_exceptions():
             await sentry_hooks.on_tool_end(context, agent, tool, result)
         await original_on_tool_end(context, agent, tool, result)