test(etl): stub Groups::Connector in sync_all rake test#139
Open
billie-clawford-stacksbot[bot] wants to merge 1 commit into
Open
test(etl): stub Groups::Connector in sync_all rake test#139billie-clawford-stacksbot[bot] wants to merge 1 commit into
billie-clawford-stacksbot[bot] wants to merge 1 commit into
Conversation
… CI failure) sync_all's sync_all test didn't stub Stacks::Etl::Groups::Connector when stacks:etl:sync_google_groups was wired into sync_all (50bd261). CI's own test run for stacks:etl:sync_all was therefore invoking the REAL connector, which fetched real Google Group threads and tried to index chunks — creating an Embedding record with the 'embedding' vector column. That column is added outside db/schema.rb specifically so schema:load works on Postgres without pgvector (Heroku CI's in-dyno Postgres), so the raw column write raised: ActiveModel::UnknownAttributeError: unknown attribute 'embedding' for Embedding. (surfaced in the ops thread via /app/lib/stacks/etl/connector.rb:54 → etl.rake:38 → test/lib/tasks/etl_rake_test.rb:37). Fix: stub Stacks::Etl::Groups::Connector#new/#run in the sync_all test, mirroring how the Meet sweeps are already stubbed via Stacks::Etl::Meet.expects(:sweep_all_users!) in the same test — so the test asserts sync_all's three-source wiring/order without touching Google or Embedding at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
Flagged in Twist thread stacks.garden3d.net:
The backtrace bottoms out in
test/lib/tasks/etl_rake_test.rb:37— this is Heroku CI's own test run, not a live prod ETL failure.When
stacks:etl:sync_google_groupsgot wired intostacks:etl:sync_all(50bd261), thesync_allrake test wasn't updated to stubStacks::Etl::Groups::Connector. So CI's test forsync_allactually invoked the real connector, which hit real Google Group threads and tried to chunk+index them — creating anEmbeddingrow with theembeddingvector column. That column is intentionally added outsidedb/schema.rb(see the pgvector note at the top of the file) soschema:loadworks on Postgres without pgvector — which is exactly what Heroku CI's in-dyno Postgres is. Hence the raw-column write blew up.(Every other connector test in the suite already knows about this and calls
skip_without_pgvector— e.g.test/lib/stacks/etl/groups/connector_test.rb:5,test/lib/stacks/etl/connector_test.rb:24— but thesync_allrake test never exercises a real connector for Meet, only for Groups, since Meet is invoked via the already-stubbedStacks::Etl::Meet.sweep_all_users!helper.)Fix
Stub
Stacks::Etl::Groups::Connector#new/#runin thesync_alltest, mirroring how the Meet sweeps are already stubbed viaStacks::Etl::Meet.expects(:sweep_all_users!)in the same test. The test now assertssync_all's three-source invocation order (Meet API → Gemini notes → Google Groups) without touching Google or Embedding.No production code changed — this is a test-only fix for a CI-only failure mode.
Testing
Could not run the Rails/Minitest suite directly from this sandbox (no persistent Ruby 3.1.7 + Postgres toolchain here — same limitation noted on PR #122). Requesting Heroku CI run on this PR to confirm green; will monitor and squash-merge on green per standing instruction.