Search existing issues
Describe the bug
HI! I like your app very much, thank you a lot! But, I don't know nothing about GitHub and app development, so, please just take a bug report from Claude
Describe the bug
With recordly.export.experimentalNvidiaCuda enabled ("Lightning export"), every MP4 export attempts the nvidia-cuda-compositor backend first, but the native helper (recordly-nvidia-cuda-compositor.exe) fails during NVENC session creation and exits with code 1. Recordly then silently falls back to windows-d3d11-compositor with no error/toast shown to the user — the export dialog's "Path:" line just changes mid-export from (nvidia-cuda-compositor) to (windows-d3d11-compositor).
This is 100% reproducible: checked native-export-diagnostics logs going back to Aug 3, and every single CUDA attempt (6 out of 6, across 3 different days) fails with the exact same error, at the exact same point:
NvEncoder::CreateEncoder : m_nvenc.nvEncInitializeEncoder(m_hEncoder, &m_initializeParams) returned error 8
at E:\Sach\Sua\test\record\Recordly-release-v1.3.1-20260525\.tmp\video-sdk-samples\Samples\NvCodec\NvEncoder\NvEncoder.cpp:273
Error 8 in the NVIDIA Video Codec SDK is NV_ENC_ERR_INVALID_PARAM. Notably, CUDA context creation, the decoder session, and encoder object construction all succeed first (stdout logs Session Initialization Time: ~100-150ms) — it's specifically the nvEncInitializeEncoder call for the encoder session that's rejected as invalid, every time, regardless of source video (confirmed on a tiny 375-frame test clip and a 24073-frame/~13-minute recording).
The export args being passed are unremarkable and don't look malformed (even dimensions, standard settings):
--width 1920 --height 1080 --fps 30 --bitrate-mbps 27 --encoding-mode quality
--content-width 1764 --content-height 992 ...
Because the app doesn't surface this failure to the user, and Task Manager only shows the integrated GPU working during export, it's easy to assume (as I initially did) that this is a graphics-preference/driver-routing issue rather than the CUDA encoder path erroring out and being silently swallowed.
Expected behavior
Expected behavior
- NVENC should actually initialize on supported hardware, or
- If encoder init fails, the user should see a visible error/notice explaining the fallback to
windows-d3d11-compositor, instead of it happening silently.
Desktop:
- Device: Dell Precision 5560
- GPU: NVIDIA RTX A2000 Laptop GPU, 4GB (+ Intel integrated GPU, hybrid/Optimus)
- NVIDIA driver: 596.86, Release branch 595 (released 2026-07-30)
- OS: Windows 11
- Recordly version: built from
Recordly-release-v1.3.1-20260525 per the path embedded in the helper's error output.
To Reproduce
To Reproduce
- Laptop with NVIDIA Optimus/hybrid graphics,
recordly.export.experimentalNvidiaCuda: true.
- Record anything, open editor, Export → MP4.
- Watch "Path:" during "Preparing export...": shows
nvidia-cuda-compositor.
- A few % into rendering, "Path:" silently changes to
windows-d3d11-compositor.
- Check
%APPDATA%\Recordly\native-export-diagnostics\ — the corresponding *-recordly-static-layout-*.stderr.log for that export will contain the nvEncInitializeEncoder ... error 8 failure, exitCode: 1 in the matching .manifest.json.
Screenshots
OS
Windows
OS Version
Windows 11
Other OS
No response
Additional context
Additional context
Windows Graphics Settings was already set to "High performance" for Recordly.exe, which correctly moved the WebGL/PixiJS preview and editor rendering onto the discrete GPU — so this isn't a GPU-routing/preference issue for the main process. The failure is specifically in the separate native recordly-nvidia-cuda-compositor.exe helper's NVENC encoder session init.
Given NV_ENC_ERR_INVALID_PARAM happens consistently at encoder init (not decoder, not CUDA context) regardless of resolution/content, my guess (not confirmed) is either:
- an
NV_ENC_INITIALIZE_PARAMS/encode-config field being set to a value the driver rejects on this specific hardware, or
- a Video Codec SDK header version compiled into the helper that doesn't match what driver branch 595.x expects.
RTX A2000 is the workstation-branded Ampere (GA107) mobile line rather than a GeForce card, in case that's relevant to whatever encoder config/preset GUID is being requested.
Happy to run a debug build, capture NVENC API tracing, or test other export settings if that helps narrow it down — logs above are from 6 independent export attempts, all failing identically, so this looks fully deterministic on this hardware/driver combo rather than a flaky timing issue.
Search existing issues
Describe the bug
Describe the bug
With
recordly.export.experimentalNvidiaCudaenabled ("Lightning export"), every MP4 export attempts thenvidia-cuda-compositorbackend first, but the native helper (recordly-nvidia-cuda-compositor.exe) fails during NVENC session creation and exits with code 1. Recordly then silently falls back towindows-d3d11-compositorwith no error/toast shown to the user — the export dialog's "Path:" line just changes mid-export from(nvidia-cuda-compositor)to(windows-d3d11-compositor).This is 100% reproducible: checked
native-export-diagnosticslogs going back to Aug 3, and every single CUDA attempt (6 out of 6, across 3 different days) fails with the exact same error, at the exact same point:Error 8 in the NVIDIA Video Codec SDK is
NV_ENC_ERR_INVALID_PARAM. Notably, CUDA context creation, the decoder session, and encoder object construction all succeed first (stdout logsSession Initialization Time: ~100-150ms) — it's specifically thenvEncInitializeEncodercall for the encoder session that's rejected as invalid, every time, regardless of source video (confirmed on a tiny 375-frame test clip and a 24073-frame/~13-minute recording).The export args being passed are unremarkable and don't look malformed (even dimensions, standard settings):
Because the app doesn't surface this failure to the user, and Task Manager only shows the integrated GPU working during export, it's easy to assume (as I initially did) that this is a graphics-preference/driver-routing issue rather than the CUDA encoder path erroring out and being silently swallowed.
Expected behavior
Expected behavior
windows-d3d11-compositor, instead of it happening silently.Desktop:
Recordly-release-v1.3.1-20260525per the path embedded in the helper's error output.To Reproduce
To Reproduce
recordly.export.experimentalNvidiaCuda: true.nvidia-cuda-compositor.windows-d3d11-compositor.%APPDATA%\Recordly\native-export-diagnostics\— the corresponding*-recordly-static-layout-*.stderr.logfor that export will contain thenvEncInitializeEncoder ... error 8failure,exitCode: 1in the matching.manifest.json.Screenshots
OS
Windows
OS Version
Windows 11
Other OS
No response
Additional context
Additional context
Windows Graphics Settings was already set to "High performance" for
Recordly.exe, which correctly moved the WebGL/PixiJS preview and editor rendering onto the discrete GPU — so this isn't a GPU-routing/preference issue for the main process. The failure is specifically in the separate nativerecordly-nvidia-cuda-compositor.exehelper's NVENC encoder session init.Given
NV_ENC_ERR_INVALID_PARAMhappens consistently at encoder init (not decoder, not CUDA context) regardless of resolution/content, my guess (not confirmed) is either:NV_ENC_INITIALIZE_PARAMS/encode-config field being set to a value the driver rejects on this specific hardware, orRTX A2000 is the workstation-branded Ampere (GA107) mobile line rather than a GeForce card, in case that's relevant to whatever encoder config/preset GUID is being requested.
Happy to run a debug build, capture NVENC API tracing, or test other export settings if that helps narrow it down — logs above are from 6 independent export attempts, all failing identically, so this looks fully deterministic on this hardware/driver combo rather than a flaky timing issue.