Permission errors running my developed plugin on a Google Drive FUSE mount

Community Plugin Support
1

Hello. I am currently a rotation student in the knight lab developing a plugin that creates a visualizer. Some strange things are happening that I believe are a result of abstract Google Drive permission errors. I believe the error pops up when I am trying to use the q2templates.render function. I get the following error when running my plugin from the directory I'm currently developing it in, which is in a google drive folder:
/Users/daniperry/Google Drive/My Drive/UCSD/q-winter22/Knight/projects/q2-katharoseq/q2_katharoseq

Traceback (most recent call last):
  File "/Users/daniperry/anaconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/q2cli/commands.py", line 339, in __call__
    results = action(**arguments)
  File "<decorator-gen-547>", line 2, in read_count_threshold
  File "/Users/daniperry/anaconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    outputs = self._callable_executor_(scope, callable_args,
  File "/Users/daniperry/anaconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/qiime2/sdk/action.py", line 453, in _callable_executor_
    ret_val = self._callable(output_dir=temp_dir, **view_args)
  File "/Volumes/GoogleDrive/My Drive/UCSD/q-winter22/Knight/projects/q2-katharoseq/q2_katharoseq/_methods.py", line 106, in read_count_threshold
    q2templates.render(index, output_dir, context=context)
  File "/Users/daniperry/anaconda3/envs/qiime2-2021.11/lib/python3.8/site-packages/q2templates/_templates.py", line 64, in render
    shutil.copy2(source_file, temp_dir.name)
  File "/Users/daniperry/anaconda3/envs/qiime2-2021.11/lib/python3.8/shutil.py", line 436, in copy2
    copystat(src, dst, follow_symlinks=follow_symlinks)
  File "/Users/daniperry/anaconda3/envs/qiime2-2021.11/lib/python3.8/shutil.py", line 396, in copystat
    lookup("chflags")(dst, st.st_flags, follow_symlinks=follow)
PermissionError: [Errno 1] Operation not permitted: '/var/folders/pv/hg29j3j512d8wd_qw_8g3msw0000gn/T/tmp_lx1kfxs/index.html'

Plugin error from katharoseq:

  [Errno 1] Operation not permitted: '/var/folders/pv/hg29j3j512d8wd_qw_8g3msw0000gn/T/tmp_lx1kfxs/index.html'

See above for debug info.

When I move the whole directory outside the google drive it works seamlessly. When I specify an --o-visualization parameter outside the development directory the problem is NOT solved.

To reproduce the error you may be able to simply create two files in the same directory INSIDE a google drive folder:

test_error.py:

# test_error.py

import q2templates

index = 'index.html'
output_dir = '.'
q2templates.render(index, output_dir)

index.html:

{% extends 'base.html' %}

{% block content %}
  <h1>Katharoseq Protocol</h1>

  <div class="row">
    <div class="col-lg-12">
      <div class="text-center">
        <div>
          <p>Here is a piece of text</p>
        </div>

      </div>
    </div>
  </div>

{% endblock %}

Running python test_error.py inside a google drive directory will throw the above error and running it outside a google drive directory will throw no errors.

Mainly I just want to be able to keep the development within my google drive and am unsure why exactly this is posing a problem.

Thanks so much for your help!

1 Like
2

Hi @Daniela_Perry,

Thank you for the traceback and comprehensive description of the problem, it seems that the Google drive FUSE mount is probably missing some operations that make it work like a normal filesystem. This of course isn't ideal, but there isn't really an easy solution. The fact that changing the output doesn't correct this makes sense.

What is happening, is your template assets (the source) are being copied to the destination (some visualization), but to do that, Python's stdlib does quick check as to what the file is and what operations can be performed (this is the part that this specific FUSE mount hasn't implemented for whatever reason). So no matter where you put the output, Python is still going to ask what kinds of operations it can do on the source file.

Considering your goals, I would actually recommend using git (and GitHub) to backup your plugin as you develop it. There should be many people in the Knight lab who can assist with this, as you'll likely find it is the common methodology.

1 Like