Rewrite SDK for .NET 7/9 with async-first API and enhanced examples#4
Open
ilyazub wants to merge 9 commits into
Open
Rewrite SDK for .NET 7/9 with async-first API and enhanced examples#4ilyazub wants to merge 9 commits into
ilyazub wants to merge 9 commits into
Conversation
Complete rewrite of serpapi-dotnet from scratch: - Multi-target net7.0 + net9.0 - Async-first API with CancellationToken (SearchAsync, HtmlAsync, etc.) - Sync convenience wrappers for non-async contexts - System.Text.Json (zero external runtime deps for core) - IDisposable with proper HttpClient lifecycle - Exception hierarchy: SerpApiException, HttpException, KeyException, TimeoutException - SerpApiResponse wrapper with SearchMetadata, OrganicResults, pagination - IAsyncEnumerable pagination (SearchPagesAsync) on net8.0+ - DI integration: services.AddSerpApi() with IHttpClientFactory - Dictionary<string, string> params (replaces Hashtable) - PascalCase API surface per .NET conventions - xUnit test suite (36 tests, mocked HTTP) - CI: GitHub Actions with net7.0/net9.0 matrix - Release pipeline: tag-triggered NuGet publish - Full README with quickstart, all engines, error handling, DI, pagination
Six self-contained buildable example projects: - BasicSearch: minimal sync usage - AsyncSearch: concurrent requests with CancellationToken - Pagination: IAsyncEnumerable page iteration - MultipleEngines: Google, Bing, YouTube, Maps, Locations - ErrorHandling: all exception types demonstrated - DependencyInjection: IServiceCollection + IHttpClientFactory
Examples: - Split ResearchFanOut into focused examples (88 + 75 lines) - Add ProgressiveRefinement example (narrow → broad → time-filtered) - Remove flawed verification loop (overclaimed statistical rigor) - Accept custom query via CLI arg for evergreen usage - Add examples run step to CI workflow Security and correctness: - Fix SSRF: NextPageAsync validates pagination URL origin before sending API key - Fix disposal leak: error paths now dispose SerpApiResponse - Fix pagination contract: SearchPagesAsync captures nextUrl before yielding so consumers can safely dispose each page - Fix AccountAsync: now calls ThrowIfError like other methods - Fix path injection: SearchArchiveAsync escapes searchId - Fix HttpResponseMessage leak in GetStringAsync - Fix options mutation: DI constructor copies options defensively - Add maxPages validation (ArgumentOutOfRangeException if <= 0) - 5 new tests (41 total)
Unit gap tests (23 new): SSRF scheme validation, cancellation tokens, SearchPagesAsync iteration/max-pages, all sync wrappers, HttpRequestException wrapping, non-JSON error bodies, non-key API errors, response null-safety, dispose idempotency, client ownership semantics. Integration tests (10 new): Google, Bing, Google Maps, YouTube, Location, Account, Archive round-trip, Pagination, SearchPagesAsync — all skip unless SERPAPI_API_KEY env var is set (CI has secrets).
Addresses issues reported across SerpApi SDKs (Go #6, Ruby #16/#26, Python #19/#52/#53): - Document Polly retry pattern for 429 rate limiting - Document corporate proxy configuration via HttpClientHandler
New examples map directly to SerpApi customer use cases: - LeadFinder: local business discovery via Google Maps - CompetitorTracker: brand vs competitor SERP positions - RankTracker: keyword position monitoring with pagination - PriceMonitor: cross-platform price comparison (Shopping + Walmart) - AiResearchAgent: multi-source RAG context (web + news + scholar) - ContentDiscovery: trending topics, PAA, content gaps Kept: ErrorHandling, DependencyInjection (DX essentials) Removed: BasicSearch, AsyncSearch, Pagination, MultipleEngines, ResearchFanOut, ProgressiveRefinement (generic/feature-focused)
Multi-target: netstandard2.0, net7.0, net8.0, net9.0, net10.0. Polyfills for netstandard2.0: Microsoft.Bcl.AsyncInterfaces (IAsyncEnumerable), System.Text.Json. Full API surface available on all targets including DI extensions, async pagination, and typed exceptions. Follows Stripe/Azure/AWS SDK pattern for maximum reach (.NET Framework 4.6.2+, Xamarin, Unity, MAUI).
Shared HttpClient via IClassFixture (HTTP/2 multiplexing). xunit.runner.json enables parallel collections. 44.8s → 980ms.
- Replace string.Contains(StringComparison) with IndexOf (MissingMethodException on .NET Framework) - Sync wrappers use Task.Run to avoid SynchronizationContext deadlocks - Integration tests accept both SERPAPI_API_KEY and API_KEY env vars - Examples marked IsPackable=false via Directory.Build.props - Release workflow: target .NET 10, scope pack to SDK project, create GitHub Release - CI pack step scoped to serpapi.csproj only
There was a problem hiding this comment.
Pull request overview
This PR modernizes the SerpApi .NET SDK by replacing the legacy synchronous/Newtonsoft implementation with a multi-targeted, async-first HttpClient-based client, adding DI support, and significantly expanding runnable examples and automated test coverage. It also refreshes CI/release automation to build/test across multiple .NET versions and to package/publish to NuGet.
Changes:
- Replaces the legacy SerpApi client with a new async-first SDK (
SerpApiClient,SerpApiResponse, typed exceptions) targetingnetstandard2.0throughnet10.0. - Migrates tests from MSTest to xUnit, adds extensive unit tests + optional live integration tests.
- Adds/updates CI and release workflows and introduces multiple runnable example projects.
Reviewed changes
Copilot reviewed 69 out of 70 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| test/xunit.runner.json | Enables xUnit parallel execution settings. |
| test/test.csproj | Migrates test project to xUnit + adds dependencies and runner config. |
| test/SerpApiResponseTests.cs | Adds unit tests for SerpApiResponse convenience APIs. |
| test/SerpApiExceptionTests.cs | Adds unit tests for new exception types. |
| test/SerpApiClientTests.cs | Adds comprehensive unit tests for client behavior and edge cases. |
| test/serpapi_search_test.cs | Removes legacy MSTest-based test. |
| test/MockHttpHandler.cs | Adds mock HttpMessageHandler for unit tests. |
| test/location_test.cs | Removes legacy MSTest-based test. |
| test/IntegrationTests.cs | Adds optional live integration tests using SkippableFact. |
| test/google_search_test.cs | Removes legacy MSTest-based test. |
| test/GlobalUsings.cs | Adds global xUnit import for the test project. |
| test/example_search_youtube_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_yahoo_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_walmart_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_naver_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_home_depot_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_scholar_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_reverse_image_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_product_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_play_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_maps_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_local_services_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_jobs_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_images_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_events_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_google_autocomplete_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_ebay_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_duckduckgo_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_bing_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_baidu_test.cs | Removes legacy MSTest-based example test. |
| test/example_search_apple_app_store_test.cs | Removes legacy MSTest-based example test. |
| test/DependencyInjectionTests.cs | Adds tests validating DI registration behavior. |
| test/archive_search_test.cs | Removes legacy MSTest-based test. |
| test/account_test.cs | Removes legacy MSTest-based test. |
| serpapi/SerpApiServiceCollectionExtensions.cs | Adds IHttpClientFactory / DI registration extension. |
| serpapi/SerpApiResponse.cs | Introduces SerpApiResponse wrapper over JsonDocument. |
| serpapi/SerpApiExceptions.cs | Introduces typed exception hierarchy for SDK errors. |
| serpapi/SerpApiClientOptions.cs | Introduces options for API key, base URL, and timeout. |
| serpapi/SerpApiClient.cs | Introduces new async-first client with pagination helpers and sync wrappers. |
| serpapi/serpapi.csproj | Updates package metadata, multi-targeting, dependencies, and build settings. |
| serpapi/serpapi.cs | Removes legacy Newtonsoft-based implementation. |
| serpapi.sln | Removes legacy solution file. |
| serpapi-dotnet.slnx | Adds new solution definition including examples, library, and tests. |
| README.md | Rewrites documentation with new API, usage, and examples listing. |
| Makefile | Removes legacy Makefile automation. |
| examples/README.md | Adds examples overview and run instructions. |
| examples/RankTracker/RankTracker.csproj | Adds RankTracker runnable example project. |
| examples/RankTracker/Program.cs | Implements RankTracker example using pagination. |
| examples/PriceMonitor/Program.cs | Implements PriceMonitor example with parallel searches. |
| examples/PriceMonitor/PriceMonitor.csproj | Adds PriceMonitor runnable example project. |
| examples/LeadFinder/Program.cs | Implements LeadFinder example (Google Maps lead extraction). |
| examples/LeadFinder/LeadFinder.csproj | Adds LeadFinder runnable example project. |
| examples/ErrorHandling/Program.cs | Implements ErrorHandling example demonstrating exception types. |
| examples/ErrorHandling/ErrorHandling.csproj | Adds ErrorHandling runnable example project. |
| examples/Directory.Build.props | Adds examples build props (non-packable) inheriting repo props. |
| examples/DependencyInjection/Program.cs | Implements DI example using AddSerpApi. |
| examples/DependencyInjection/DependencyInjection.csproj | Adds DI runnable example project + dependencies. |
| examples/ContentDiscovery/Program.cs | Implements ContentDiscovery example (news → PAA → competitors). |
| examples/ContentDiscovery/ContentDiscovery.csproj | Adds ContentDiscovery runnable example project. |
| examples/CompetitorTracker/Program.cs | Implements CompetitorTracker example running searches in parallel. |
| examples/CompetitorTracker/CompetitorTracker.csproj | Adds CompetitorTracker runnable example project. |
| examples/AiResearchAgent/Program.cs | Implements AI/RAG context gathering example. |
| examples/AiResearchAgent/AiResearchAgent.csproj | Adds AiResearchAgent runnable example project. |
| Directory.Packages.props | Switches centralized package versions to new dependency set (xUnit, Extensions.*, etc.). |
| Directory.Build.props | Updates global build properties (roll-forward, TFM warnings). |
| .gitignore | Replaces narrow ignores with standard .NET/IDE ignores. |
| .github/workflows/release.yml | Adds release automation for packing and NuGet publishing on tags. |
| .github/workflows/dotnet-core.yml | Removes legacy CI workflow. |
| .github/workflows/ci.yml | Adds multi-SDK CI build/test + runs example projects. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| run: dotnet build --no-restore --configuration Release | ||
|
|
||
| - name: Test | ||
| run: dotnet test --no-build --configuration Release |
| run: dotnet build --configuration Release --no-restore | ||
|
|
||
| - name: Test | ||
| run: dotnet test --configuration Release --no-build |
Comment on lines
+8
to
+9
| /// Skipped at runtime unless SERPAPI_API_KEY env var is set. | ||
| /// Run with: dotnet test --filter "Category=Integration" |
|
|
||
| private SerpApiResponse GetResult(string key) | ||
| { | ||
| Skip.If(_f.Client is null, "SERPAPI_API_KEY not set"); |
| var title = item.TryGetProperty("title", out var t) ? t.GetString() : "?"; | ||
| var price = item.TryGetProperty("extracted_price", out var p) ? $"${p}" : "N/A"; | ||
| var source = item.TryGetProperty("source", out var s) ? s.GetString() : ""; | ||
| Console.WriteLine($" ${price,-8} {source,-15} {title}"); |
Comment on lines
+444
to
+448
| ```bash | ||
| export SERPAPI_KEY=your_key_here | ||
| cd examples/BasicSearch | ||
| dotnet run | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes:
.github/workflows/ci.ymlfor supporting .NET 7, 8, 9, and 10. Also runs example projects as part of CI..github/workflows/release.ymlto automate NuGet publishing on version tag pushes.