-
Notifications
You must be signed in to change notification settings - Fork 2
Improve e2e auto paste performance #30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
ecf01f7
add more telemetry after transcribe path
sumerc 9eea25a
feat(clipboard): replace pbcopy/pbpaste with native NSPasteboard to c…
sumerc bab3e05
perf: eliminate blocking beep on macOS and drop keybd_event dependency
sumerc 54425b3
docs: clarify that CHANGELOG.md should only be edited at release time
sumerc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,22 +1,40 @@ | ||
| #import <AVFoundation/AVFoundation.h> | ||
|
|
||
| // Feedback tones via AVAudioPlayer: the OS owns the audio machinery, so a beep | ||
| // is fire-and-forget with no app-managed device to init, keep warm, or | ||
| // serialize against capture. Players are built once from in-memory WAV bytes; | ||
| // each play restarts from the top so a rapid start-then-end beep both sound. | ||
| // needs no app-managed device to init, keep warm, or serialize against capture. | ||
| // Players are built once from in-memory WAV bytes; each play restarts from the | ||
| // top so a rapid start-then-end beep both sound. | ||
|
|
||
| static AVAudioPlayer *_players[8]; | ||
|
|
||
| // Serial, so two beeps never touch one player's currentTime concurrently — | ||
| // AVAudioPlayer is not documented thread-safe. | ||
| static dispatch_queue_t beepQueue(void) { | ||
| static dispatch_queue_t q; | ||
| static dispatch_once_t once; | ||
| dispatch_once(&once, ^{ | ||
| q = dispatch_queue_create("dev.zee.beep", DISPATCH_QUEUE_SERIAL); | ||
| }); | ||
| return q; | ||
| } | ||
|
|
||
| void zeeBeepLoad(int idx, const void *wav, int len) { | ||
| NSData *data = [NSData dataWithBytes:wav length:(NSUInteger)len]; | ||
| AVAudioPlayer *p = [[AVAudioPlayer alloc] initWithData:data error:nil]; | ||
| [p prepareToPlay]; | ||
| _players[idx] = p; | ||
| } | ||
|
|
||
| // zeeBeepPlay dispatches instead of playing inline: -play blocks while it starts | ||
| // the audio hardware — prepareToPlay does not prevent it, and the hardware powers | ||
| // back down between beeps — and that landed on the push-to-talk path twice per | ||
| // dictation, at press and at release. The user hears the tone at the same moment; | ||
| // only the caller's wait is gone. Measured cost in docs/design-notes.md. | ||
| void zeeBeepPlay(int idx) { | ||
| AVAudioPlayer *p = _players[idx]; | ||
| if (p == nil) return; | ||
| p.currentTime = 0; | ||
| [p play]; | ||
| dispatch_async(beepQueue(), ^{ | ||
| p.currentTime = 0; | ||
| [p play]; | ||
| }); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,7 @@ | ||
| package clipboard | ||
|
|
||
| import cb "github.com/atotto/clipboard" | ||
| // Read returns the clipboard's current text, or "" when it holds none. | ||
| func Read() (string, error) { return read() } | ||
|
|
||
| func Read() (string, error) { | ||
| return cb.ReadAll() | ||
| } | ||
|
|
||
| func Copy(text string) error { | ||
| return cb.WriteAll(text) | ||
| } | ||
| // Copy replaces the clipboard contents with text. | ||
| func Copy(text string) error { return write(text) } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| #import <AppKit/AppKit.h> | ||
| #import <ApplicationServices/ApplicationServices.h> | ||
| #include <string.h> | ||
|
|
||
| // NSPasteboard + CGEvent, called from clipboard_darwin.go. (Objective-C can't | ||
| // live in a cgo preamble — it is compiled as C — so it goes here, mirroring | ||
| // permissions/permissions_darwin.m.) | ||
| // | ||
| // Both replace things that cost real felt latency: pbcopy/pbpaste fork(), and | ||
| // fork freezes every thread for O(resident memory), which is significant once a | ||
| // local model is resident — while keybd_event held Cmd+V down for a hardcoded | ||
| // 100 ms sleep. Measurements in docs/design-notes.md. | ||
|
|
||
| // clipCopy replaces the pasteboard with one UTF-8 text item. Returns 1 on | ||
| // success. No locale involved, unlike the pbcopy child process it replaces. | ||
| int clipCopy(const char *utf8) { | ||
| @autoreleasepool { | ||
| NSString *s = [NSString stringWithUTF8String:utf8]; | ||
| if (s == nil) { | ||
| return 0; | ||
| } | ||
| NSPasteboard *pb = [NSPasteboard generalPasteboard]; | ||
| [pb clearContents]; | ||
| return [pb setString:s forType:NSPasteboardTypeString] ? 1 : 0; | ||
| } | ||
| } | ||
|
|
||
| // clipRead returns the pasteboard's text, malloc'd for the caller to free, or | ||
| // NULL when it holds no text (empty, or an image). | ||
| char *clipRead(void) { | ||
| @autoreleasepool { | ||
| NSString *s = [[NSPasteboard generalPasteboard] stringForType:NSPasteboardTypeString]; | ||
| if (s == nil) { | ||
| return NULL; | ||
| } | ||
| return strdup([s UTF8String]); | ||
| } | ||
| } | ||
|
|
||
| // clipPaste synthesizes Cmd+V into whichever app has focus. Deliberately the | ||
| // same event mechanism as the keybd_event call it replaces — NULL source, | ||
| // annotated session tap, flags set explicitly so a physically-held modifier | ||
| // cannot leak in — minus the sleep between down and up. Requires Accessibility; | ||
| // without it macOS drops the events silently. | ||
| void clipPaste(void) { | ||
| const CGKeyCode kVK_V = 0x09; | ||
| CGEventRef down = CGEventCreateKeyboardEvent(NULL, kVK_V, true); | ||
| CGEventRef up = CGEventCreateKeyboardEvent(NULL, kVK_V, false); | ||
| CGEventSetFlags(down, kCGEventFlagMaskCommand); | ||
| CGEventSetFlags(up, kCGEventFlagMaskCommand); | ||
| CGEventPost(kCGAnnotatedSessionEventTap, down); | ||
| CGEventPost(kCGAnnotatedSessionEventTap, up); | ||
| CFRelease(down); | ||
| CFRelease(up); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| package clipboard | ||
|
|
||
| import "testing" | ||
|
|
||
| // Round-trips through the real pasteboard, restoring whatever was there. The | ||
| // Turkish characters are the point: the pbcopy backend mangled them unless | ||
| // LC_CTYPE was forced to UTF-8, and NSPasteboard takes an NSString directly, so | ||
| // the locale hack could go. | ||
| func TestCopyReadRoundTrip(t *testing.T) { | ||
| prev, err := Read() | ||
| if err != nil { | ||
| t.Fatalf("Read: %v", err) | ||
| } | ||
| t.Cleanup(func() { Copy(prev) }) | ||
|
|
||
| const want = "zee ğşıİçöü — naïve 日本語" | ||
| if err := Copy(want); err != nil { | ||
| t.Fatalf("Copy: %v", err) | ||
| } | ||
| got, err := Read() | ||
| if err != nil { | ||
| t.Fatalf("Read: %v", err) | ||
| } | ||
| if got != want { | ||
| t.Errorf("round trip: got %q, want %q", got, want) | ||
| } | ||
| } | ||
|
|
||
| func BenchmarkCopy(b *testing.B) { | ||
| prev, _ := Read() | ||
| b.Cleanup(func() { Copy(prev) }) | ||
| for b.Loop() { | ||
| Copy("the quick brown fox jumps over the lazy dog") | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 Warning — A NULL event-create result crashes the app instead of dropping the paste.
clipPaste never checks the results of CGEventCreateKeyboardEvent. When the function returns NULL (event source cannot be created under resource pressure or certain TCC/sandbox states), CGEventSetFlags(down, ...) is undefined and CFRelease(down)/CFRelease(up) on a NULL ref is a documented crash — so a failed event creation terminates the app on the paste path instead of a silently-dropped keystroke. Guard each ref (return early / skip CFRelease when NULL) as the replaced keybd_event path did internally.