Followup to #158
Per XML specification, all XML readers should perform line-ending normalization as a pre-processing step (or functionally behaving as if it were a pre-processing step - XML 1.0 Section 2.11)
This is both an ergonomic issue and also a minor correctness issue, because EOL normalization is blind to document contents and therefore also applies within elements, e.g. <?xml\r?> ought to be parsed as <?xml\n?> - which libxml2 does in fact do.
It's an ergonomic issue in the sense that a user should not need to care about it - the .xml_content(), .xml10_content(), .xml11_content() functions etc. ought to be superfluous and are potentially error prone. The user also shouldn't need to track the version declared in the declaration - they likely just want the document to be processed according to the correct specification per the declaration or according to the one they specify, but that need not change on a per-element basis.
Performing universal EOL normalization as a pre-processor pass through a bufffer would reduce total allocations and likely improve performance on average, while improving ergonomics and compliance.
The mechanism could also likely be used as a base upon which line/column tracking (#109) could be added.
Potentially troublesome aspects:
- For users who process noncompliant "almost-XML" documents that contain binary blobs, we would need a way to disable it, possibly document-wide, or perhaps just during the usage of
stream().
- We might want to make the current Reader constructors
from_reader(), from_str(), from_bytes() methods buffered by-default, and perhaps add a new constructor which takes only a pre-processed XmlDoc type or similar where decoding + normalization has already happened, allowing proper zero-copy. If we keep zero-copy (which isn't really zero-copy in practice) at all - maybe worth doing some benchmarking to see how much time is actually saved vs. just having a buffered path. XML parsing is easily 25-50x slower than copying the bytes around once or twice anyway, and the XmlSource code + macros are all quite complex.
Followup to #158
Per XML specification, all XML readers should perform line-ending normalization as a pre-processing step (or functionally behaving as if it were a pre-processing step - XML 1.0 Section 2.11)
This is both an ergonomic issue and also a minor correctness issue, because EOL normalization is blind to document contents and therefore also applies within elements, e.g.
<?xml\r?>ought to be parsed as<?xml\n?>- whichlibxml2does in fact do.It's an ergonomic issue in the sense that a user should not need to care about it - the
.xml_content(),.xml10_content(),.xml11_content()functions etc. ought to be superfluous and are potentially error prone. The user also shouldn't need to track the version declared in the declaration - they likely just want the document to be processed according to the correct specification per the declaration or according to the one they specify, but that need not change on a per-element basis.Performing universal EOL normalization as a pre-processor pass through a bufffer would reduce total allocations and likely improve performance on average, while improving ergonomics and compliance.
The mechanism could also likely be used as a base upon which line/column tracking (#109) could be added.
Potentially troublesome aspects:
stream().from_reader(),from_str(),from_bytes()methods buffered by-default, and perhaps add a new constructor which takes only a pre-processedXmlDoctype or similar where decoding + normalization has already happened, allowing proper zero-copy. If we keep zero-copy (which isn't really zero-copy in practice) at all - maybe worth doing some benchmarking to see how much time is actually saved vs. just having a buffered path. XML parsing is easily 25-50x slower than copying the bytes around once or twice anyway, and theXmlSourcecode + macros are all quite complex.