Add grid module, .grid_finder, and .grid_monitor tools - #169
Open
Baharis wants to merge 12 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Instamatic has already some tools to deal with the "grid" i.e. the copper scaffolding used to hold thin carbon film with the samples. Most of the routines deal with its presence in a occasional rather than exhaustive manner: by looking at the grid in imaging mode, one can quickly realize that some part of detector is shadowed and simply ignore it from further analysis. However, the grid is never treated as an object of investigation itself, as none of the methods currently present require this.
This PR introduces a new small module,
instamatic.grid. It exists to collect all functionality, where the grid rather than a crystal sample is a target of investigation itself. The new module stores functionality used to: find grid position (currently using two methods: beam scanning/shadowing and manually), refine grid parameters against points on the edge, read or write its geometry from file, draw current grid model, predict available area and new window positions, and many more.instamatic.grid_finderNew functionality is available via two scripts:
instamatic.grid_finderand.grid_monitor. The finder can be ran inmanualorautomode.instamatic.grid_finder manualopens aVideoStreamFrameand asks user to travel around the window edge while clicking left mouse button. Each click adds current stage position to the list of "intercepts". Inautomode, the same "intercepts" are found by traveling from the origin in star shape and binary-searching the edge based on total counts on the camera. The grid is then fit to said "intercepts". Everything is saved live in a--file, by defaultgrid.yaml:This grid file can be reused by other experiments, further improved by running
instamatic.grid_finder, or plotted on some provided ax object usingplot_gridfrominstamatic.grid.artist, which for the file above produces:instamatic.grid_monitorSince monitoring the quality of grid determination live can be useful not only in an complex pipeline, but also by itself, the plotting function is granted to the user also directly via a simple GUI, where a path to grid
--filecan be given:In this particular case, the grid geometry is fitted against intercept points from four windows, indexed 0 to 3. Position of window 0 has been determined manually by traversing along the stage edge while regularly adding new points. Windows 1-3 were fit from 2nd-order 3-arms star search (3+3 points per window), with some-angular offset per window.
The new functionality may find some use by itself, but it is predominantly great as a prerequisite for long automated experiments that are a matter of ongoing development. Since the automated grid geometry determination relies on scanning with a beam only, it can be performed by a experiment that has no information about "imaging" config. The continuous-serial electron diffraction routine implemented in incoming patches uses this information to easily define areas where it can scan for new data, without any need to change detector config or any imaging data.
Additions
docs/programs.md: Added new programs to list, added new programs' documentation;camera_serval.py: Fixed a small config bug introduced in Makeget_moviesetup eager, acquisition lazy; optimize serval #166;grid/__init__.py: A few common type hints and utils used in the module;grid/artist.py: Functionplot_gridthat draws grid details onto provided axes.grid/finder.py: High-level tools to locate, read, write grid & intercept information;grid/grid.py: Definitions, registry, fitting of grids: square, rectangular, hexagonal;grid/monitor.py: GUI utility, standalone or embed-able, auto-plots grid geometry;grid/sweeping.py:Sweepers used to find grid edges within thestar_sweepfunction;grid/window.py: Definition of individual windows used by higher-levelgrid.py;pyproject.toml: Addedinstamatic.grid_finderand.grid_monitorentry points;tests/test_grid.py: Test suite covering 70% of new functionality (not GUI, scripts).Notes
This is a direct prerequisite to a new experiment routine that is already ready, functional, tested, proved to give good 3D ED data. In order to merge this new routine before October, I'd like to merge this PR before the end of the next week.