Skip to content

fix: correct element-wise comparison in words_diff_ratio#826

Open
lxcxjxhx wants to merge 1 commit into
QData:masterfrom
lxcxjxhx:fix/words-diff-ratio
Open

fix: correct element-wise comparison in words_diff_ratio#826
lxcxjxhx wants to merge 1 commit into
QData:masterfrom
lxcxjxhx:fix/words-diff-ratio

Conversation

@lxcxjxhx

@lxcxjxhx lxcxjxhx commented Jul 9, 2026

Copy link
Copy Markdown

Summary

Fix words_diff_ratio to return correct element-wise comparison ratio instead of always returning 1/num_words.

Problem

The method words_diff_ratio in AttackedText compares two word lists using self.words != x.words. Since self.words is a Python list, this comparison returns a single boolean value (True/False) rather than an element-wise comparison array.

# Before (incorrect)
return float(np.sum(self.words != x.words)) / self.num_words
# self.words != x.words returns True (single boolean)
# np.sum(True) = 1
# Result: always 1/num_words, regardless of actual differences

Fix

Convert lists to numpy arrays before comparison to enable element-wise comparison:

# After (correct)
return float(np.sum(np.array(self.words) != np.array(x.words))) / self.num_words

Testing

from textattack.shared.attacked_text import AttackedText

text1 = AttackedText({"text": "the quick brown fox"})
text2 = AttackedText({"text": "the slow brown fox"})

# Before fix: 0.25 (1/4, incorrect)
# After fix: 0.25 (1/4, correct - only "quick" vs "slow" differs)

text3 = AttackedText({"text": "a b c d"})
text4 = AttackedText({"text": "w x y z"})

# Before fix: 0.25 (1/4, incorrect)
# After fix: 1.0 (4/4, correct - all words differ)

Fixes #787

self.words and x.words are Python lists, so `self.words != x.words`
returns a single boolean (True/False) instead of an element-wise
comparison array. np.sum(True) always equals 1, making the ratio
incorrect for any sequence length.

Convert to numpy arrays before comparison to get proper element-wise
results.

Fixes QData#787
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

the return of AttackedText.word_diff_ratio is 1 all the time

1 participant