Skip to content

Match direct-child <br> and <em> in _stringify#133

Open
c-tonneslan wants to merge 1 commit into
opencivicdata:masterfrom
c-tonneslan:fix-stringify-direct-br
Open

Match direct-child <br> and <em> in _stringify#133
c-tonneslan wants to merge 1 commit into
opencivicdata:masterfrom
c-tonneslan:fix-stringify-direct-br

Conversation

@c-tonneslan

Copy link
Copy Markdown

_stringify turns <br> inside a table cell into a newline (and marks <em>), but the XPath *//br expands to child::*/descendant-or-self::node()/br — it only matches a <br> nested inside a child element of the cell. A <br> that is a direct child of the <td> is never matched, so the newline isn't inserted and two logical lines get glued together.

scraper._stringify(lxml.html.fromstring("<td>Line1<br/>Line2</td>"))
# before: 'Line1Line2'
# after:  'Line1\nLine2'

.//br is descendant-or-self::node()/br, a strict superset that also covers the nested case, so cells whose <br> sits inside a <font>/other wrapper (as in the current fixtures) behave exactly as before. Same fix for *//em.//em.

Added a parametrized test in tests/test_parsing.py covering both the direct-child and nested cases (the direct-child case fails on the current XPath).

Signed-off-by: c-tonneslan <cst0520@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant