Move postgres to pytest

[AndrewJackson2020]

Actions [0]. Implements parallel pytest testing. This is accomplished by moving the responsibility of tearing down and setting up a postgres instance from xpg to pytest. This allows pytest-xdist to setup one postgres instance per pytest worker. They still share the same ngnix instance, though we may (or may not) want to move this to pytest as well at some point.

In the long term I propose moving this into a dedicated library that can be reused across projects since the whole process of setup, teardown, etc of postgres instances is pretty common.

timing with (2 workers)

> time net-with-nginx python -m pytest  -n 2 test/
real	0m53.335s
user	0m2.209s
sys	0m1.060s

timing on master branch (1 worker)

> time xpg test
real	1m46.212s
user	0m1.295s
sys	0m0.521s

[0] #289

[imor]

No objections to the direction @AndrewJackson2020 if you could make the tests in CI pass. One thing I'm not sure of is if our current tests are parallel safe. But even if they are not, making them safe should be the way to go.

[AndrewJackson2020]

No objections to the direction @AndrewJackson2020 if you could make the tests in CI pass. One thing I'm not sure of is if our current tests are parallel safe. But even if they are not, making them safe should be the way to go.

I believe they are with one exception, I believe this is due to the shared nginx instance, there may be a way of getting this to work without moving nginx orchestration into python. I will get the tests to pass here and then switch this PR to "ready to review" for a more in depth review.

[AndrewJackson2020]

@imor, I got the tests working. I'm really not happy about the blurred boundary between xpg and the extension project though. This PR is basically taking more and more responsibility from xpg into the pg_net. I think that these constructs (starting and stopping postgres server, reserving port, providing API for pytest for all of this, etc) should probably be available for other projects to use without vendoring a bunch of code between them. I almost wonder if we should move some of the pytest orchestration into xpg. Perhaps xpg could provide test helpers for rust, pytest, etc.

@steve-chavez Really curious what you think about this.

[steve-chavez]

@steve-chavez Really curious what you think about this.

@AndrewJackson2020 Comparing this PR to master, it seems this isn't adding too much speed to tests (both finish in ~2m). And this adds a good amount of code; so the benefit is not that clear.

---

IMO the ideal way for parallel testing would be to reuse pg_regress schedule (see parallel_schedule).

Some backstory, pytest was added to pg_net more as a workaround because we couldn't get the collect function (http_collect_response IIRC) to work since pg_regress ran it on the same transaction so the test waited forever; but I believe there was something else going on here. (a COMMIT once the request was done should have fixed it)

So if we provide nginx externally and we can convert tests to pg_regress, I believe that schedule feature should work for parallel.

I'm really not happy about the blurred boundary between xpg and the extension project though. This PR is basically taking more and more responsibility from xpg into the pg_net. I think that these constructs (starting and stopping postgres server, reserving port, providing API for pytest for all of this, etc) should probably be available for other projects to use without vendoring a bunch of code between them. I almost wonder if we should move some of the pytest orchestration into xpg. Perhaps xpg could provide test helpers for rust, pytest, etc

Just read that after writing the above and yes I think that's the ideal way 🚀

xpg is clunky but it can be improved. My intention was to overall improve pg_regress so we don't need python or any extra tooling. Since xpg can do custom postgres builds we can do that without blocking us, while also upstreaming patches where possible.

[steve-chavez]

Some backstory, pytest was added to pg_net more as a workaround because we couldn't get the collect function (http_collect_response IIRC) to work since pg_regress ran it on the same transaction so the test waited forever; but I believe there was something else going on here. (a COMMIT once the request was done should have fixed it)

pg_regress does work, see #296

My intention was to overall improve pg_regress so we don't need python or any extra tooling.

Of course pg_regress is not enough to provision other components, so to be more precise the end result would be a Nix/pg_regress test framework.