# UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
**URL:** https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385
**Category:** User Support
**Tags:** import
**Created:** [March 12, 2018, 1:23pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385 "2018-03-12T13:23:31Z")
**Posts on this page:** 18
**Page:** 1
### Author:  [@Patilism](https://forum.qiime2.org/u/Patilism)
#### Post date: [March 12, 2018, 1:23pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/1 "2018-03-12T13:23:32Z")
Hello, I saw there was a similar topic created by somebody but wasn't resolved perhaps. ([Manifest File: Invalid text encoding](https://forum.qiime2.org/t/manifest-file-invalid-text-encoding/1827))
In my case, I am using a demultiplexed fastq file, which I zipped using gzip, and then tried to import using the following command, but couldn't succeed...
**qiime tools import --type 'SampleData[SequencesWithQuality]' --input-path ATCC/PGM052\_QIIME2.fastq.gz --output-path demux.qza --source-format SingleEndFastqManifestPhred33**
Traceback (most recent call last):
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/tools.py", line 116, in import\_data
view\_type=source\_format)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/sdk/result.py", line 214, in import\_data
return cls._from\_view(type_, view, view\_type, provenance\_capture)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/sdk/result.py", line 239, in \_from\_view
result = transformation(view)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/core/transform.py", line 57, in transformation
self.validate(view)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/core/transform.py", line 131, in validate
view.validate('min')
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/plugin/model/file\_format.py", line 32, in validate
if not self.sniff():
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2\_types/per\_sample\_sequences/\_format.py", line 35, in sniff
line = fh.readline()
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self.\_buffer\_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
An unexpected error has occurred:
'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
# See above for debug info.
Also since you have asked in the previous unresolved topic, I have ran the following commands too and got respective outputs (see below).
**echo $LC\_ALL**
**echo $LANG**
en\_US.UTF-8
Please help. Thanks in advance.
---
### Author:  [@Patilism](https://forum.qiime2.org/u/Patilism)
#### Post date: [March 12, 2018, 1:24pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/2 "2018-03-12T13:24:16Z")
I'm getting a similar error... but this is not related to metadata file. I'm trying to import a demultiplexed fastq file in QIIME2 using following command. Please help me resolve this issue.
**qiime tools import --type 'SampleData[SequencesWithQuality]' --input-path PGM052\_ATCC\_Std.fastq.gz --output-path demux.qza --source-format SingleEndFastqManifestPhred33**
Traceback (most recent call last):
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/tools.py", line 116, in import\_data
view\_type=source\_format)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/sdk/result.py", line 214, in import\_data
return cls._from\_view(type_, view, view\_type, provenance\_capture)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/sdk/result.py", line 239, in \_from\_view
result = transformation(view)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/core/transform.py", line 57, in transformation
self.validate(view)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/core/transform.py", line 131, in validate
view.validate('min')
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/qiime2/plugin/model/file\_format.py", line 32, in validate
if not self.sniff():
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2\_types/per\_sample\_sequences/\_format.py", line 35, in sniff
line = fh.readline()
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self.\_buffer\_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
An unexpected error has occurred:
'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
See above for debug info.
---
### Author:  [@thermokarst](https://forum.qiime2.org/u/thermokarst)
#### Post date: [March 12, 2018, 1:52pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/3 "2018-03-12T13:52:40Z")
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 13, 2018, 9:20pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/4 "2018-03-13T21:20:52Z")
Hey @Patilism,
> [@Patilism](#):
>
> echo $LC\_ALL
>
> echo $LANG
>
> en\_US.UTF-8
Based on that, I think you need to also set `LC_ALL`:
```auto
export LC_ALL=en_US.utf8
```
Does that fix the issue?
If so you can add it to your `$HOME/.bashrc` file so you don't have to set it every time.
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 13, 2018, 9:20pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/5 "2018-03-13T21:20:54Z")
---
### Author:  [@Patilism](https://forum.qiime2.org/u/Patilism)
#### Post date: [March 14, 2018, 2:51pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/6 "2018-03-14T14:51:52Z")
Hi Evan,
Thanks for your response. But it didn't fix the issue. I got following error...
Traceback (most recent call last):
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in
sys.exit(qiime())
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in **call**
return self.main(\*args, \*\*kwargs)
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 676, in main
\_verify\_python3\_env()
File "/Users/rohanpatil/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/\_unicodefun.py", line 118, in \_verify\_python3\_env
'for mitigation steps.' + extra)
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult [http://click.pocoo.org/python3/for](http://click.pocoo.org/python3/for) mitigation steps.
This system lists a couple of UTF-8 supporting locales that
you can pick from. The following suitable locales where
discovered: af\_ZA.UTF-8, am\_ET.UTF-8, be\_BY.UTF-8, bg\_BG.UTF-8, ca\_ES.UTF-8, cs\_CZ.UTF-8, da\_DK.UTF-8, de\_AT.UTF-8, de\_CH.UTF-8, de\_DE.UTF-8, el\_GR.UTF-8, en\_AU.UTF-8, en\_CA.UTF-8, en\_GB.UTF-8, en\_IE.UTF-8, en\_NZ.UTF-8, en\_US.UTF-8, es\_ES.UTF-8, et\_EE.UTF-8, eu\_ES.UTF-8, fi\_FI.UTF-8, fr\_BE.UTF-8, fr\_CA.UTF-8, fr\_CH.UTF-8, fr\_FR.UTF-8, he\_IL.UTF-8, hr\_HR.UTF-8, hu\_HU.UTF-8, hy\_AM.UTF-8, is\_IS.UTF-8, it\_CH.UTF-8, it\_IT.UTF-8, ja\_JP.UTF-8, kk\_KZ.UTF-8, ko\_KR.UTF-8, lt\_LT.UTF-8, nl\_BE.UTF-8, nl\_NL.UTF-8, no\_NO.UTF-8, pl\_PL.UTF-8, pt\_BR.UTF-8, pt\_PT.UTF-8, ro\_RO.UTF-8, ru\_RU.UTF-8, sk\_SK.UTF-8, sl\_SI.UTF-8, sr\_YU.UTF-8, sv\_SE.UTF-8, tr\_TR.UTF-8, uk\_UA.UTF-8, zh\_CN.UTF-8, zh\_HK.UTF-8, zh\_TW.UTF-8
Click discovered that you exported a UTF-8 locale
but the locale system could not pick up from it because
it does not exist. The exported locale is "en\_US.utf8" but it
is not supported
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 14, 2018, 3:09pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/7 "2018-03-14T15:09:38Z")
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 14, 2018, 9:51pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/8 "2018-03-14T21:51:53Z")
Hey @Patilism,
My bad, I think your system spells it: `en_US.UTF-8` so if you try that instead for `LC_ALL` it should work.
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 14, 2018, 9:51pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/9 "2018-03-14T21:51:55Z")
---
### Author:  [@Patilism](https://forum.qiime2.org/u/Patilism)
#### Post date: [March 16, 2018, 1:13pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/10 "2018-03-16T13:13:01Z")
> [@ebolyen](#):
>
> en\_US.UTF-8
Hi Evan,
Thanks again for your prompt reply. I did **LC\_ALL=en\_US.UTF-8** and it resolved above error, but didn't fix the previous/very first error. I'm getting the same error message.
I also cross-checked that my demultiplexed fastq file is in unix / utf-8 format. Please feel free to have a look at the test file ([Error](https://exchangelabsgmu-my.sharepoint.com/:u:/g/personal/rpatil8_masonlive_gmu_edu/EZ_JVw07_NdFj9_yED7oMXcBwYEy0murf-TxhzF-O7GCPw?e=wXvSPy)), and let me know.
Thanks and Regards,
Rohan
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [March 16, 2018, 1:18pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/11 "2018-03-16T13:18:57Z")
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 19, 2018, 3:47pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/12 "2018-03-19T15:47:37Z")
Thanks for the file @Patilism,
Looking back at your original post I was too distracted by the decode error to notice that your file is the wrong kind for that `--source-format`.
I _am_ able to reproduce your error, and I can confirm that it is a confusing one in context. What _should_ have happened is the error would explain that the file was compressed and the wrong type. Instead it tried to read the gzip file as if it was plaintext causing the codec error. [I created an issue](https://github.com/qiime2/qiime2/issues/385) to improve that.
To fix your problem you just need a different kind of format. To help you with that, how many samples do you have? Is it just the one, or are there more?
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 19, 2018, 3:47pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/13 "2018-03-19T15:47:42Z")
---
### Author:  [@Patilism](https://forum.qiime2.org/u/Patilism)
#### Post date: [March 20, 2018, 1:01pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/14 "2018-03-20T13:01:05Z")
Thanks again Evan.
Please let me know what different kind of format is needed. For now, I'm having just this one sample for testing purpose, but later I'll be having multiple samples with much bigger datasets to deal with.
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [March 20, 2018, 1:26pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/15 "2018-03-20T13:26:03Z")
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 20, 2018, 5:34pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/16 "2018-03-20T17:34:07Z")
Hey @Patilism,
Since you are using SingleEndFastqManifestPhred33 as your source-format, let's go with that and create a manifest that contains your `ATCC/PGM052_QIIME2.fastq.gz` file. Since you do have multiple samples, I would recommend starting with at least 2 samples. While 1 _should_ work in principle, I'm not 100% certain as some visualizations are trying to show a distribution (or summarize one), and a distribution of 1 doesn't always work the way we expect it to.
In any case, you want to create a file [as described in this section](https://docs.qiime2.org/2018.2/tutorials/importing/#fastq-manifest-formats). Which means you'll have something that looks roughly like:
```plaintext
sample-id,absolute-filepath,direction
PGM052,/some/filepath/ATCC/PGM052_QIIME2.fastq.gz,forward
```
You can create this file in Excel as it's really just a spreadsheet, but you need to export as a CSV for QIIME 2 to understand it.
It is this file that you want to provide to `--input-path` instead of your fastq file as it lets QIIME 2 know which samples exist, and where to find the reads for them (and what direction the reads are).
Hope that makes sense!
---
### Author:  [@ebolyen](https://forum.qiime2.org/u/ebolyen)
#### Post date: [March 20, 2018, 5:34pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/17 "2018-03-20T17:34:09Z")
---
### Author:  [@system](https://forum.qiime2.org/u/system)
#### Post date: [April 20, 2018, 11:34pm UTC](https://forum.qiime2.org/t/unicodedecodeerror-utf-8-codec-cant-decode-byte-0x8b-in-position-1-invalid-start-byte/3385/18 "2018-04-20T23:34:09Z")
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.