Skip to content

clickhouse: append column number for each line - #6420

Open
jSherz wants to merge 1 commit into
parca-dev:mainfrom
jSherz:jsj/clickhouse-column-number-panic
Open

clickhouse: append column number for each line#6420
jSherz wants to merge 1 commit into
parca-dev:mainfrom
jSherz:jsj/clickhouse-column-number-panic

Conversation

@jSherz

@jSherz jSherz commented Oct 7, 2026

Copy link
Copy Markdown

The Line struct has six child columns. The ClickHouse querier did not write to ColumnNumber. The column stayed shorter than the Line struct, and Arrow panicked with "index out of range" when it built the record.

Append 0 to ColumnNumber for each line, as the parcacol writers do. Add a test that builds a record from stored function data.

James Sherwood-Jones @claude
pkg/clickhouse: append column number for each line
d10bf9f 
The Line struct has six child columns. The ClickHouse querier did not
write to ColumnNumber. The column stayed shorter than the Line struct,
and Arrow panicked with "index out of range" when it built the record.

Append 0 to ColumnNumber for each line, as the parcacol writers do.
Add a test that builds a record from stored function data.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jSherz
jSherz requested a review from a team as a code owner October 7, 2026 13:50
@jSherz

jSherz commented Oct 7, 2026

Copy link
Copy Markdown
Author

Example stack trace:

panic: arrow/array: index out of range

goroutine 20580 [running]:
github.com/apache/arrow-go/v18/arrow/array.NewSliceData({0x86ef5d0, 0x1406c1669730}, 0x0, 0x6dcf)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/data.go:232 +0x367
github.com/apache/arrow-go/v18/arrow/array.(*Struct).setData(0x140748f3da80, 0x1406c1669c00)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:195 +0x12f
github.com/apache/arrow-go/v18/arrow/array.NewStructData(...)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:120
github.com/apache/arrow-go/v18/arrow/array.(*StructBuilder).NewStructArray(0x376c96?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:406 +0x3f
github.com/apache/arrow-go/v18/arrow/array.(*StructBuilder).NewArray(0x140745812d20?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:399 +0x13
github.com/apache/arrow-go/v18/arrow/array.(*baseListBuilder).newData(0x140745812d20)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/list.go:558 +0x7c
github.com/apache/arrow-go/v18/arrow/array.(*ListBuilder).NewListArray(0x140698ec6b40?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/list.go:539 +0x15
github.com/apache/arrow-go/v18/arrow/array.(*ListBuilder).NewArray(0x1406ccdffba8?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/list.go:527 +0x13
github.com/apache/arrow-go/v18/arrow/array.(*StructBuilder).newData(0x1406bcbdd380)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:414 +0xb2
github.com/apache/arrow-go/v18/arrow/array.(*StructBuilder).NewStructArray(0x140695742028?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:405 +0x13
github.com/apache/arrow-go/v18/arrow/array.(*StructBuilder).NewArray(0x140745812d90?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/struct.go:399 +0x13
github.com/apache/arrow-go/v18/arrow/array.(*baseListBuilder).newData(0x140745812d90)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/list.go:558 +0x7c
github.com/apache/arrow-go/v18/arrow/array.(*ListBuilder).NewListArray(0x424a01?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/list.go:539 +0x15
github.com/apache/arrow-go/v18/arrow/array.(*ListBuilder).NewArray(0x140695742170?)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/list.go:527 +0x13
github.com/apache/arrow-go/v18/arrow/array.(*RecordBuilder).NewRecordBatch(0x1406bf92e080)
	github.com/apache/arrow-go/v18@v18.5.2/arrow/array/record.go:371 +0x14a
github.com/parca-dev/parca/pkg/clickhouse.(*Querier).rowsToArrowRecords(0x14068f890f50, {0x86a7dc8, 0x14069a17bdd0}, {0x7ebdd4e816b0, 0x140695a80480}, 0x0)
	github.com/parca-dev/parca/pkg/clickhouse/querier.go:1069 +0x1b66
github.com/parca-dev/parca/pkg/clickhouse.(*Querier).QueryMerge(0x14068f890f50, {0x86a7dc8?, 0x14069a17bdd0?}, {0x1406968add60, 0x49}, {_, _, _}, {0x1ab3f00, 0xee25841ef, ...}, ...)
	github.com/parca-dev/parca/pkg/clickhouse/querier.go:661 +0xbb8
github.com/parca-dev/parca/pkg/query.(*ColumnQueryAPI).selectMerge(0x14068fb909a0, {0x86a7dc8, 0x14069a17bdd0}, 0x140697154aa0, {0x14069f06c4e0, 0x2, 0x2}, 0x0, {0x0, 0x0})
	github.com/parca-dev/parca/pkg/query/columnquery.go:1358 +0x1af
github.com/parca-dev/parca/pkg/query.(*ColumnQueryAPI).Query(0x14068fb909a0, {0x86a7dc8, 0x14069a17bdd0}, 0x14069a073900)
	github.com/parca-dev/parca/pkg/query/columnquery.go:333 +0x127b
github.com/parca-dev/parca/gen/proto/go/parca/query/v1alpha1._QueryService_Query_Handler.func1({0x86a7dc8?, 0x14069a17bdd0?}, {0x7d53620?, 0x14069a073900?})
	github.com/parca-dev/parca/gen/proto/go/parca/query/v1alpha1/query_vtproto.pb.go:235 +0xcb
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/logging.UnaryServerInterceptor.UnaryServerInterceptor.func2({0x86a7dc8, 0x140695d2e5a0}, {0x7d53620, 0x14069a073900}, 0x21?, 0x14069c31bb78)
	github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.3.3/interceptors/server.go:22 +0x283
google.golang.org/grpc.getChainUnaryHandler.func1({0x86a7dc8?, 0x140695d2e5a0?}, {0x7d53620?, 0x14069a073900?})
	google.golang.org/grpc@v1.84.0/server.go:1238 +0x153
github.com/parca-dev/parca/pkg/server.(*Server).ListenAndServe.(*ServerMetrics).UnaryServerInterceptor.UnaryServerInterceptor.func16({0x86a7dc8, 0x140695d2e5a0}, {0x7d53620, 0x14069a073900}, 0x14069f06c480?, 0x140695a9c600)
	github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.3.3/interceptors/server.go:22 +0x283
google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1({0x86a7dc8, 0x140695d2e5a0}, {0x7d53620, 0x14069a073900}, 0x14069f06c480, 0x18?)
	google.golang.org/grpc@v1.84.0/server.go:1229 +0x7c
github.com/parca-dev/parca/gen/proto/go/parca/query/v1alpha1._QueryService_Query_Handler({0x7c2aa80, 0x14068fb909a0}, {0x86a7dc8, 0x140695d2e5a0}, 0x14069682b758, 0x14068f9048c0)
	github.com/parca-dev/parca/gen/proto/go/parca/query/v1alpha1/query_vtproto.pb.go:237 +0x143
google.golang.org/grpc.(*Server).register.(*Server).wrapUnaryHandler.func1({0x7c2aa80, 0x14068fb909a0}, {0x86c9400, 0x140698ca7c00})
	google.golang.org/grpc@v1.84.0/server.go:1284 +0xbb
google.golang.org/grpc.(*Server).processRPC(0x14068f851208, {0x86a7dc8, 0x140695d2e4b0}, 0x140696d68540, 0x14068f5c9530, 0x14068f904b60, 0x0)
	google.golang.org/grpc@v1.84.0/server.go:1450 +0x120b
google.golang.org/grpc.(*Server).handleStream(0x14068f851208, {0x86aa330, 0x14069f2faea0}, 0x140696d68540)
	google.golang.org/grpc@v1.84.0/server.go:1595 +0xa70
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	google.golang.org/grpc@v1.84.0/server.go:1096 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 20441
	google.golang.org/grpc@v1.84.0/server.go:1107 +0x11d

Looking forward to the evolutions of Parca on ClickHouse 🎉

metalmatze
metalmatze approved these changes Oct 9, 2026
View reviewed changes

@metalmatze metalmatze left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@jSherz jSherz mentioned this pull request Oct 12, 2026
Open
@jSherz

jSherz commented Oct 12, 2026

Copy link
Copy Markdown
Author

@metalmatze I can't merge this - do the Vercel jobs need triggering or please could you merge it?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@metalmatze metalmatze metalmatze approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jSherz @metalmatze