Fix comparison-star duplicates from cross-site astrometry - #133
Conversation
A multi-telescope series disagrees on catalog positions by one to two arcsec frame to frame, which the 1.0 arcsec cross-match radius read as different stars. One star became several candidates, all of which centroided back onto it, so the ensemble filled with copies reporting identical counts, and each fragment made the others look unisolated. On a 13-frame Kleopatra set spanning five telescopes, 8 of the 10 selected comparison stars were four duplicated pairs. Widen the radius to 2.5 arcsec, match each row to its nearest cluster rather than the first within range, carry the cluster position as the mean of its rows, and merge clusters closer together than one aperture radius, which centroiding cannot separate in any case. The same set now selects 10 distinct stars, all catalogued on every frame, with isolation distances of 9.6 to 54.7 arcsec rather than 1.0 to 1.8. Also report when the target falls outside the comparison ensemble's magnitude range, where its calibrated magnitude is extrapolated from the zero point rather than interpolated within it.
jnation3406
left a comment
jnation3406
left a comment
There was a problem hiding this comment.
Looks fine overall to me. One day when we aren't working on different branches we should try to unify the catalog matching / nearest source code so we don't have so many different versions.
| for row in rows: | ||
| nearest = -1 | ||
| if cluster_ra.size: | ||
| separations = angular_distances_arcsec(row["ra_deg"], row["dec_deg"], cluster_ra, cluster_dec) |
There was a problem hiding this comment.
This function is very similar to the find_nearest_source function in catalog_utils.py, but that is in my hr_diagram branch for now. We may want to refactor later to re-use code for all the things doing nearest matching on catalogs.
This function also looks a lot like minimum_neighbor_distances_arcsec from geometry.py. Claude says that can be sped up a ton by using astropy's match_to_catalog_sky to do the MxN catalog nearest neighbor matching using a KD-tree instead of the full MxN looping. But maybe you've already considered these things.
There was a problem hiding this comment.
Swapped the cluster merge to search_around_sky, which is the natural fit there since it wants all pairs inside a radius rather than nearest neighbours. Verified identical output against the old sweep - same groups, same positions, and the Kleopatra light curve is unchanged.
Left two alone for now. _assign_rows_to_clusters needs conflict resolution to go KD-tree (a cluster takes one row per frame, and two rows can be nearest to the same one) . Worth doing, since mutual-nearest would also remove the current dependence on row order, but it's a behaviour change rather than a speedup. And minimum_neighbor_distances_arcsec runs on the same cluster list immediately after the merge and is the same O(N²), 9 s at 20000, so it's really the other half of this - match_to_catalog_sky(nthneighbor=2) drops straight in. Both feel like they belong in the unification you're describing rather than here.
2 participants
A multi-telescope series disagrees on catalog positions by one to two arcsec frame to frame, which the 1.0 arcsec cross-match radius read as different stars. One star became several candidates, all of which centroided back onto it, so the ensemble filled with copies reporting identical counts, and each fragment made the others look unisolated. On a 13-frame Kleopatra set spanning five telescopes, 8 of the 10 selected comparison stars were four duplicated pairs.
Widen the radius to 2.5 arcsec, match each row to its nearest cluster rather than the first within range, carry the cluster position as the mean of its rows, and merge clusters closer together than one aperture radius, which centroiding cannot separate in any case. The same set now selects 10 distinct stars, all catalogued on every frame, with isolation distances of 9.6 to 54.7 arcsec rather than 1.0 to 1.8.
Also report when the target falls outside the comparison ensemble's magnitude range, where its calibrated magnitude is extrapolated from the zero point rather than interpolated within it.