canonmn_int: round converted focal lengths - #9475
Conversation
On i686, excess floating-point precision can make products such as 100.0F * 1.4F slightly lower than 140. Converting that value directly to int truncates it to 139, so teleconverter lenses do not match their EXIF focal lengths. Round the products before converting them to int to make lens matching independent of excess floating-point precision.
kevinbackhouse
left a comment
There was a problem hiding this comment.
Thanks! This is an improvement.
By the way, do you know whether there's a possibility that the number could be too big, causing the static_cast<int> to overflow? If so, somebody is inevitably going to send us a vulnerability report about it so it might be good to fix it proactively. I recently wrote some code to do a cast with overflow checking in #9433.
|
Tick the box to add this pull request to the merge queue (same as
|
The values passed to |
On i686, x87 excess floating-point precision can make products such as
100.0F * 1.4Fslightly lower than 140. Converting that value directly tointtruncates it to 139, so teleconverter lens entries do not match the focal lengths decoded from metadata.This rounds the focal lengths after applying the teleconverter factor and before converting them to
int. The existing lens tests cover the affected cases.Verification: