Elegant support for DOS or posix newlines

form xref

The different line endings are one of those little technical things that seem custom-made to confuse and frustrate non-technical users. They are literally invisible!

But at least the workaround is easy:

Could we make this easier on our users?

Would it be possible to transparently support flat text files with either dos or posix newlines?


P.S. Where would I open an issue for this? Maybe https://github.com/qiime2/q2cli/issues
P.P.S. Everyone has the same problem with the em-dash

1 Like

I was pretty certain that we’re using universal newline handling everywhere in QIIME 2 (it’s the default in Python 3, so we’d have to very specifically ask the computer to not transparently handle newlines), but it’s possible there’s an issue somewhere.

Are we certain that it is newlines causing the issue?

1 Like

I’m not sure what’s causing this issue. @Nicholas_Bokulich tried to troubleshoot this too, but I’m not sure we made much progress.

I’m glad that these sorts of issues are being handled in Qiime 2! I’ll close this thread for now why we investigate this specific case.

Turns out it was a byte-order mark:

We definitely need to fix that! It comes up surprisingly often, although this is the first case of a UTF-8 BOM. Most of the BOMs are only 2 bytes and from UTF-16.