reference file in manifest could not be found

Hi.
I’m trying to import a set of forward and reverse files with the following command:

PS C:\Users\hzq128> docker run -t -i -v ${pwd}:/data qiime2/core:2020.6 qiime tools import --type ‘SampleData[PairedEndSequencesWithQuality]’ --input-path manifest.txt --output-path paired.gza --input-format PairedEndFastqManifestPhred33

I’m using docker as the Virtualbox, and I think I’m using the latest version of Qiime2.
however, I get the following error:

manifest.txt is not a(n) PairedEndFastqManifestPhred33 file:
File referenced on line 2 could not be found (C:\Users\hzq128\s-13_R1.fastq.gz).

the manifest file is in the same directory (C:\Users\hzq128) with the .fastq.gz files.

and here’s is my manifest file:

sample-id,absolute-filepath,direction
s-13,C:\Users\hzq128\s-13_R1.fastq.gz,forward
s-13,C:\Users\hzq128\s-13_R2.fastq.gz,reverse

I looked into similar issues. I’m using the absolute path and have the files in the same folder. I don’t know what I’m missing since qiime can’t find the reference file.

Hello @Sina ,

I believe this is about file format.

Here is how I make my import manifest.

sample-id,forward-absolute-filepath,reverse-absolute-filepath

Input format is PairedEndFastqManifestPhred33V2 and also type is SampleData[PairedEndSequencesWithQuality].

I have no experience with virtual box. Maybe the error is related to that, I don’t know. But you can try making your manifest as I mentioned. Maybe that would work.

Best wishes.

Hi @Sina.

The format you specified (PairedEndFastqManifestPhred33) is fine, there is no need to change that.

The issue is this:

Know why it can’t be found? Because QIIME 2 is running in a docker container (according to your first command shared), which uses an entirely different file system.

Good, that means you have an easy workaround:

sample-id,absolute-filepath,direction
s-13,$PWD/s-13_R1.fastq.gz,forward
s-13,$PWD/s-13_R2.fastq.gz,reverse

The $PWD is a linux environment variable that contains the absolute path to the current working directory - this should work for you - let us know how it goes.

Thank you Matthew.

I'm still having the same issue (attached image).
I'm using PowerShell could that be a problem?

this is the manifest file. manifest.txt (330 Bytes)

Thanks,
Sina.

No.

sample-id,absolute-filepath,direction
s-13,$PWD/s-13_R1.fastq.gz,forward
s-13,$PWD/s-13_R2.fastq.gz,reverse

Please note the direction of the slashes.

manifest.txt is not a(n) PairedEndFastqManifestPhred33 file:

File referenced on line 2 could not be found (PWD/s-13_R1.fastq.gz).

Maybe I should try VirtualBox or Amazon Web Services.
I’ll try to see if that fixes the problem.

Thanks,
Sina.

You’re missing the $ before PWD.

There was a problem importing manifest.txt:

  manifest.txt is not a(n) PairedEndFastqManifestPhred33 file:

  File referenced on line 2 could not be found ($PWD/s-13_R1.fastq.gz).

PS C:\Users\hzq128> docker run -t -i -v ${pwd}:/data qiime2/core:2020.6 qiime tools import  --type 'SampleData[PairedEndSequencesWithQuality]'  --input-path manifest.txt  --output-path paired.gza  --input-format PairedEndFastqManifestPhred33
There was a problem importing manifest.txt:

  manifest.txt is not a(n) PairedEndFastqManifestPhred33 file:

  File referenced on line 2 could not be found ($pwd/s-13_R1.fastq.gz).

It’s the same thing again!

Again, you need to use $PWD, not $pwd, or PWD, or any other combo - the env var is pretty specific.

Let’s take a step back though - let’s confirm your files are all loaded in docker correctly:

docker run -t -i -v ${pwd}:/data qiime2/core:2020.6 ls -lah

Please copy-and-paste the results here. Put the results in a triple-backtick block to format correctly:

```
PASTE HERE
```

okay. here is what I see.

total 988M
drwxrwxrwx 1 root root  20K Aug  4 17:06  .
drwxr-xr-x 1 root root 4.0K Aug  4 17:28  ..
drwxrwxrwx 1 root root 4.0K Aug  4 17:06  .VirtualBox
drwxrwxrwx 1 root root    0 Jul 31 16:05  .conda
drwxrwxrwx 1 root root 4.0K Jul 20 22:24  .docker
drwxrwxrwx 1 root root    0 Jan 18  2020  .matplotlib
-rwxr-xr-x 1 root root 3.7K Aug  3 15:26  .test.tsv.un~
-rwxr-xr-x 1 root root 5.9K Aug  3 15:31  .test.txt.un~
drwxrwxrwx 1 root root    0 Jul  2 19:52 '3D Objects'
drwxrwxrwx 1 root root    0 Jun 13  2019  AppData
drwxrwxrwx 1 root root    0 Nov 12  2019  BentleyDownloads
drwxrwxrwx 1 root root    0 Jul  2 19:52  Contacts
drwxrwxrwx 1 root root  48K Aug  4 17:24  Desktop
-rwxr-xr-x 1 root root 5.9M Aug  4 17:17  DesktopActivator.log
-rwxr-xr-x 1 root root 9.8M Feb  1  2020  DesktopActivator.log.1
drwxrwxrwx 1 root root 4.0K Jul 20 16:29  Documents
drwxrwxrwx 1 root root  96K Aug  4 17:24  Downloads
drwxrwxrwx 1 root root    0 Jul  2 19:52  Favorites
drwxrwxrwx 1 root root    0 Aug  3 17:32  Harvey
drwxrwxrwx 1 root root    0 Jul  2 19:52  Links
drwxrwxrwx 1 root root    0 Jun 13  2019  MicrosoftEdgeBackups
drwxrwxrwx 1 root root    0 Jul 22 16:18  Music
drwxrwxrwx 1 root root    0 Jun 14  2019  ODBA
drwxrwxrwx 1 root root    0 Jun 13  2019  OneDrive
drwxrwxrwx 1 root root 4.0K Jul  2 19:52  Pictures
drwxrwxrwx 1 root root    0 Jul  2 19:52 'Saved Games'
drwxrwxrwx 1 root root 4.0K Jul  2 19:52  Searches
drwxrwxrwx 1 root root 4.0K Jul 30 16:54 'University of Texas at San Antonio'
-rwxr-xr-x 1 root root 370M Sep 26  2018  VA_R1.fastq
-rwxr-xr-x 1 root root 369M Sep 26  2018  VA_R2.fastq
drwxrwxrwx 1 root root    0 Jul  2 19:52  Videos
-rwxr-xr-x 1 root root 1.9K Aug  3 16:26  _viminfo
-rwxr-xr-x 1 root root    0 Jul 31 16:20  anaconda
-rwxr-xr-x 1 root root  145 Aug  3 17:32  manifest.csv
-rwxr-xr-x 1 root root  332 Aug  4 17:20  manifest.txt
-rwxr-xr-x 1 root root 101M Sep 25  2018  s-13_R1.fastq.gz
-rwxr-xr-x 1 root root 133M Sep 25  2018  s-13_R2.fastq.gz
-rwxr-xr-x 1 root root  130 Aug  3 15:26  test.tsv
-rwxr-xr-x 1 root root  119 Aug  3 23:40  test.txt
-rwxr-xr-x 1 root root  129 Aug  3 15:30  test.txt~
-rwxr-xr-x 1 root root  278 Aug  2 19:21  testManifest.txt
drwxrwxrwx 1 root root 4.0K Aug  3 15:27  vimfiles

Thank you again for walking me through this process.

Hi again,
I switched to VirtualBox and I was able to import my file and it looks easier to work with. thank you for your help and comments.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.