# RESCRIPT dereplicate uniq mode UnicodeEncodeError

**URL:** <https://forum.qiime2.org/t/rescript-dereplicate-uniq-mode-unicodeencodeerror/25144>\
**Category:** Library Support\
**Created:** [January 5, 2023, 6:36am UTC](https://forum.qiime2.org/t/rescript-dereplicate-uniq-mode-unicodeencodeerror/25144 "2023-01-05T06:36:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rhlampe](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/r/f6c823/32.png) [@rhlampe](https://forum.qiime2.org/u/rhlampe)\
**Post date:** [January 5, 2023, 6:36am UTC](https://forum.qiime2.org/t/rescript-dereplicate-uniq-mode-unicodeencodeerror/25144/1 "2023-01-05T06:36:33Z")

</div>

Hi,

I'm trying to run through the silva example of RESCRIPt ([Processing, filtering, and evaluating the SILVA database (and other reference sequence data) with RESCRIPt](https://forum.qiime2.org/t/processing-filtering-and-evaluating-the-silva-database-and-other-reference-sequence-data-with-rescript/15494)) and am encountering an error at the dereplication step. I'm currently using QIIME2 2022.11 and installed RESCRIPt via pip as suggested:

```auto
pip install git+https://github.com/bokulich-lab/RESCRIPt.git

```

Here are the commands I ran:

```auto
qiime rescript get-silva-data \
    --p-version '138.1' \
    --p-target 'SSURef_NR99' \
    --p-include-species-labels \
    --o-silva-sequences silva-138.1-ssu-nr99-rna-seqs.qza \
    --o-silva-taxonomy silva-138.1-ssu-nr99-tax.qza

qiime rescript reverse-transcribe \
    --i-rna-sequences silva-138.1-ssu-nr99-rna-seqs.qza 
    --o-dna-sequences silva-138.1-ssu-nr99-seqs.qza

qiime rescript cull-seqs \
    --i-sequences silva-138.1-ssu-nr99-seqs.qza \
    --o-clean-sequences silva-138.1-ssu-nr99-seqs-cleaned.qza

qiime rescript filter-seqs-length-by-taxon \
    --i-sequences silva-138.1-ssu-nr99-seqs-cleaned.qza \
    --i-taxonomy silva-138.1-ssu-nr99-tax.qza \
    --p-labels Archaea Bacteria Eukaryota \
    --p-min-lens 900 1200 1400 \
    --o-filtered-seqs silva-138.1-ssu-nr99-seqs-filt.qza \
    --o-discarded-seqs silva-138.1-ssu-nr99-seqs-discard.qza 

qiime rescript dereplicate \
    --i-sequences silva-138.1-ssu-nr99-seqs-filt.qza \
    --i-taxa silva-138.1-ssu-nr99-tax.qza \
    --p-rank-handles 'silva' \
    --p-mode 'uniq' \
    --o-dereplicated-sequences silva-138.1-ssu-nr99-seqs-derep-uniq.qza \
    --o-dereplicated-taxa silva-138.1-ssu-nr99-tax-derep-uniq.qza

```

The dereplicate command produced the following error:

```auto
$ cat /scratch/rlampe/30545035.tscc-mgr7.local/qiime2-q2cli-err-tij6a2zb.log
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: vsearch --derep_fulllength /scratch/rlampe/30545035.tscc-mgr7.local/qiime2/rlampe/data/dab83010-1036-49df-9f40-b75d7c4d9da7/data/dna-sequences.fasta --output /scratch/rlampe/30545035.tscc-mgr7.local/tmpcfgsrr46 --uc /scratch/rlampe/30545035.tscc-mgr7.local/tmp00vq71bj --xsize --threads 1

vsearch v2.22.1_linux_x86_64, 1007.2GB RAM, 64 cores
https://github.com/torognes/vsearch

Dereplicating file /scratch/rlampe/30545035.tscc-mgr7.local/qiime2/rlampe/data/dab83010-1036-49df-9f40-b75d7c4d9da7/data/dna-sequences.fasta 100%
699161179 nt in 477562 seqs, min 900, max 3983, avg 1464
Sorting 100%
435502 unique sequences, avg cluster 1.1, median 1, max 893
Writing FASTA output file 100%
Writing uc file, first part 100%
Writing uc file, second part 100%
/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/rescript/dereplicate.py:115: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  uc['Taxon'] = uc['seqID'].apply(lambda x: taxa.loc[x])
/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/rescript/dereplicate.py:116: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  uc['centroidtaxa'] = uc['centroidID'].apply(lambda x: taxa.loc[x])
Traceback (most recent call last):
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/q2cli/commands.py", line 352, in __call__
    results = action(**arguments)
  File "<decorator-gen-490>", line 2, in dereplicate
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/sdk/action.py", line 234, in bound_callable
    outputs = self._callable_executor_(scope, callable_args,
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/sdk/action.py", line 408, in _callable_executor_
    artifact = qiime2.sdk.Artifact._from_view(
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/sdk/result.py", line 356, in _from_view
    artifact._archiver = archive.Archiver.from_data(
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/core/archive/archiver.py", line 408, in from_data
    Format.write(rec, type, format, data_initializer,
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/core/archive/format/v5.py", line 20, in write
    super().write(archive_record, type, format, data_initializer,
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/core/archive/format/v1.py", line 25, in write
    provenance_capture.finalize(
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/core/archive/provenance.py", line 320, in finalize
    self.write_citations_bib()
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/core/archive/provenance.py", line 311, in write_citations_bib
    self.citations.save(str(self.path / self.CITATION_FILE))
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/qiime2/core/cite.py", line 71, in save
    bp.dump(db, f, writer=writer)
  File "/projects/ps-allenlab/rlampe/bin/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages/bibtexparser/ __init__.py", line 108, in dump
    bibtex_file.write(writer.write(bib_database))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0161' in position 2884: ordinal not in range(256)

```

Thanks in advance!  
Rob

---

<div class="post-metadata">

**Author:** ![Nicholas\_Bokulich](https://forum.qiime2.org/user_avatar/forum.qiime2.org/nicholas_bokulich/32/19937_2.png) [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)\
**Post date:** [January 5, 2023, 6:55am UTC](https://forum.qiime2.org/t/rescript-dereplicate-uniq-mode-unicodeencodeerror/25144/2 "2023-01-05T06:55:48Z")

</div>

Hi @rhlampe ,

This error is indicating that you need to set another language encoding, as the default that you are using (`latin-1`) cannot encode some special characters (in the citations specifically! So nothing to do with RESCRIPt itself). See this topic for troubleshooting steps:

> [@Plugin error from diversity: 'latin-1' codec can't encode character '\\u03b2' (...)](https://forum.qiime2.org/t/plugin-error-from-diversity-latin-1-codec-cant-encode-character-u03b2/11779/3):
>
> Hey @efratm! I think this is the first time I’ve seen someone with the latin-1 encoding (which is weird as it used to be a super common standard). There’s not necessarily any bad characters here, rather latin-1 just cannot express some symbols. What strikes me as odd is that QIIME 2 is attempting to use latin-1 at all. It looks like perhaps we did not specify utf-8 (which can encode all characters) to our bibtex library, and so it must be using a default? Where it is getting that default is l…

(you can also search the forum for "codec can't encode character" to see quite a few similar topics)

Good luck!

---

<div class="post-metadata">

**Author:** ![rhlampe](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/r/f6c823/32.png) [@rhlampe](https://forum.qiime2.org/u/rhlampe)\
**Post date:** [January 5, 2023, 6:49pm UTC](https://forum.qiime2.org/t/rescript-dereplicate-uniq-mode-unicodeencodeerror/25144/3 "2023-01-05T18:49:53Z")

</div>

Thanks so much! Fixed it following that thread.
