how to access files with docker?

User Support
1

HI

I am new to using QIIME2 on docker. What I don't understand is I have my raw files on desktop of my computer. I am using a Mac. I access QIIME2 from the mac terminal by using this command.

docker run --rm -it \
    -u $(id -u):$(id -g) \
    -v $(pwd):/data \
    qiime2/core:2018.11 \
        /bin/bash

WHat I don't understand is how to access my raw files which I need to analyse. how to read them in to QIIME2 not he docker and once I analyse them where is the docker saving them, how can I access them,

please help

3

Hi @Huda_Ghori!

I strongly recommend using the native installation on mac (link), it will be much less work for you.

See this line?

You're telling docker, "Hey, load up all of the files in my current working directory, and put them into the "/data" directory inside of the docker container."

So, when you run this command, the /bin/bash at the end of the command tells docker to launch a bash shell. You can cd /data, and if you run ls, you will see all of your files there. You can now work on those files, and any edits or changes will be synchronized on your host machine (the mac).

Hope that helps!

:qiime2:

6

HI
I have installed QIIME2 natively on my Mac and I am running it on an instance. However, on the very first step i am getting the Errno 28 error that say no space left on the device. The error is as follows. Please help to resolve it. :frowning:

(qiime2-2020.2) ubuntu@microbiome:/analysis$ qiime tools import --type 'SampleData[PairedEndSequencesWithQuality]' --input-path /analysis/tomove --input-format CasavaOneEightSingleLanePerSampleDirFmt --output-path SM_16S.qza



Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/shutil.py", line 122, in copyfile
    copyfileobj(fsrc, fdst)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/shutil.py", line 82, in copyfileobj
    fdst.write(buf)
OSError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/builtin/tools.py", line 158, in import_data
    view_type=input_format)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/result.py", line 241, in import_data
    validate_level='max')
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/result.py", line 267, in _from_view
    result = transformation(view, validate_level)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/core/transform.py", line 70, in transformation
    new_view = transformer(view)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py", line 87, in _4
    dirfmt, SingleLanePerSamplePairedEndFastqDirFmt)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_types/per_sample_sequences/_transformer.py", line 56, in _single_lane_per_sample_fastq_helper
    read_number=read_number)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 87, in write_data
    result.path._move_or_copy(self.path_maker(**kwargs))
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/core/path.py", line 35, in _move_or_copy
    return shutil.copy(str(self), str(other))
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/shutil.py", line 241, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/shutil.py", line 122, in copyfile
    copyfileobj(fsrc, fdst)
OSError: [Errno 28] No space left on device

An unexpected error has occurred:

  [Errno 28] No space left on device

See above for debug info.
8

Hi @Huda_Ghori!

Unfortunately there is not much we can do on our end - the error message is saying that your hard drive has run out of available space. Options for resolving this include cleaning up old files, or upgrading to a larger disk drive.

I am curious though:

These two things seem incompatible --- the error message you provided is showing a linux filepath. Are you running linux on your mac? If so, how? Via VirtualBox? Docker?

The answers to these questions might help guide us with some additional alternatives for addressing your disk space issues.

Keep us posted!

1 Like
10

HI Thank you for replying

I have installed it natively but I am using it through a cloud server and I log in to QIIME2 this way:

source activate qiime2-2020.2

and then I work with the data uploaded in my instance

11

This is my df -Th output

(base) ubuntu@microbiome:~$ df -Th
Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 16G 0 16G 0% /dev
tmpfs tmpfs 3.2G 756K 3.2G 1% /run
/dev/vda1 ext4 39G 36G 3.3G 92% /
tmpfs tmpfs 16G 0 16G 0% /dev/shm
tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/vda15 vfat 105M 3.6M 101M 4% /boot/efi
/dev/vdd ext4 32G 18G 13G 60% /data
tmpfs tmpfs 3.2G 0 3.2G 0% /run/user/1000

Why is QIIME saving my output to / and to not /data folder which is where i want it to save it

13

QIIME 2 will write the output to whatever directory you ask it (via the filepath you specify in your command invocation) - but I suspect what you're asking about has to do with temporary files, rather than output. Since the QIIME 2 framework uses python, you can control the temporary directory used by setting the TMPDIR environment variable - if you search for that here on the forum you should find plenty of discussion on the topic. So, you could set that tmp dir to some location on your /data directory, which looks like it has a little more available space on it.

Keep us posted!

:qiime2:

15

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.