Skip to content

Add runPyAsync - #41

Merged
Shimuuar merged 15 commits into
masterfrom
runAsync
Aug 23, 2026
Merged

Add runPyAsync#41
Shimuuar merged 15 commits into
masterfrom
runAsync

Conversation

@Shimuuar

@Shimuuar Shimuuar commented Jul 27, 2026

Copy link
Copy Markdown
Owner

This variant of runPy which executes python code in separate OS thread. Mostly useful for async in python

At the moment I'm not quite sure that API is right but it works with example in #40

See #40

@qwbarch

qwbarch commented Aug 7, 2026

Copy link
Copy Markdown

If I'm understanding this right, a new OS thread is created every time runPyAsync is called right? Would it be better to instead have some sort of rpc-like communication and have the Python side run its tasks only on the event handler? I feel like this would be more similar in performance in calling a bunch of async functions in python compared to spawning a new OS thread for each call.

Just curious though, I don't know much about this so I'm interested in what your thoughts are!

@Shimuuar

Shimuuar commented Aug 7, 2026

Copy link
Copy Markdown
Owner Author

Yes. runPyAsync creates new OS thread. But since full haskell IO is allowed it's possible to construct any sort of thread pool. But only primitives are provided and drawing rest of the owl is left to library user

I think approach is workable but current implementation deadlocks readily. One have to be very careful about locking: liftIO should release GIL to allow other threads make progress. There's need to distinguish threads spawned by runPyAsync and rest. I didn't get around to actually implement that

@Shimuuar

Copy link
Copy Markdown
Owner Author

I've been very-very wrong. I played a bit with runPyAsync and have small program that deadlocks reliably (branch deadlock). I'll have to investigate what exactly deadlock further

As it turns out it's possible to cancel python threads asynchronously
We must to try interrupt thread concurrently by throwing haskell and python
thread
@Shimuuar

Copy link
Copy Markdown
Owner Author

I fixed at very least most of deadlocks. But I tested extensively with python 3.13 and it looks test deadlock reliably with 3.10 & 3.11

@Shimuuar
Shimuuar marked this pull request as ready for review August 23, 2026 08:28
@Shimuuar
Shimuuar merged commit a8a73f5 into master Aug 23, 2026
11 checks passed
@Shimuuar
Shimuuar deleted the runAsync branch August 23, 2026 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants