Jupyter lab/notebook in Qiime 2 (v. 2021.4)

[Shannon_Wilson]

This is likely a stupid question, but I have spent hours now trolling the forum and the web for answers and nothing seems to work for me.

I am VERY new to coding and trying to figure out how to use jupyter notebooks in conjunction with Qiime2.

I have natively installed Qiime2 (v. 2021.4) on my mac via conda.

In terminal, I am running the following:

" (base) shannonwilson@shannons-mbp ~ % conda activate qiime2-2021.4
(qiime2-2021.4) shannonwilson@shannons-mbp ~ % cd Desktop/qiime_files
(qiime2-2021.4) shannonwilson@shannons-mbp qiime_files % jupyter notebook
[I 09:55:17.816 NotebookApp] Serving notebooks from local directory: /Users/shannonwilson/Desktop/qiime_files
[I 09:55:17.816 NotebookApp] Jupyter Notebook 6.3.0 is running at:
[I 09:55:17.817 NotebookApp] http://localhost:8888/?token=3917a6c50b78741ee9d61792cffb2872307ec9a6679853dc
[I 09:55:17.817 NotebookApp] or http://127.0.0.1:8888/?token=3917a6c50b78741ee9d61792cffb2872307ec9a6679853dc
[I 09:55:17.817 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 09:55:17.824 NotebookApp]

To access the notebook, open this file in a browser:
    file:///Users/shannonwilson/Library/Jupyter/runtime/nbserver-5571-open.html
Or copy and paste one of these URLs:
    http://localhost:8888/?token=3917a6c50b78741ee9d61792cffb2872307ec9a6679853dc
 or http://127.0.0.1:8888/?token=3917a6c50b78741ee9d61792cffb2872307ec9a6679853dc "

I run the command "Import qiime2" and there is no response to the code in the notebook. I however tried to proceed.

I am then trying to run the "moving pictures" tutorial in the jupyter notebook browser. However, I get the following when I try to make a directory for it (e.g. mkdir qiime2-moving-pictures-tutorial
cd qiime2-moving-pictures-tutorial)

I received the error:
"File "", line 1
mkdir qiime2-moving-pictures-tutorial
^
SyntaxError: invalid syntax"

After backing up, resetting my Kernal and terminal completely, I then tried running:

"jupyter serverextension enable --py qiime2 --sys-prefix" in terminal after activating the environment and changing my directory to my desktop (I like to have the files logged there as well) and before launching jupyter as well as in jupyter itself. The same errors are still happening.

So, after that long-winded explination, here is my question:

1. Obviously, its a user error. Do you guys have some guidance on how best to run Jupyter notebook with qiime2? The only tutorials I have found are using CyVerse, and they are only using outdated versions/I know theres a better way.
2. Could I be having these issues because I am trying to change the directory to have all files and copies saved to my desktop? this has not been an issue when I have been running qiime alone.
3. Would this be a better question for the jupyter folks? I figure I would try here first as you guys are FAR more helpful!

Thanks so much for any guideance you can provide to a brand new user!

[jwdebelius (Justine Debelius)]

Hi @Shannon_Wilson,

Welcome to the forum!

We're happy to try and and help! I'm super excited you're using Jupyter Notebooks. There are a couple of tricks, but they're generally fabulous. It also sounds like you've already done a lot of heavy lifting.

@Shannon_Wilson

Obviously, its a user error. Do you guys have some guidance on how best to run Jupyter notebook with qiime2? The only tutorials I have found are using CyVerse, and they are only using outdated versions/I know theres a better way.

The number of error messages on my system that come from the inferface between and is pretty high. You're in good company! I'm not sure about the CyVerse tutorials, so IDK. I think many of the principles are the same, although since I haven't seen them, IDK.

I suspect, without checking your notebook, that you're running on a python or python 3 kernel. You can check your kernel in the upper right hand corner.

(Sorry for my Jupyter Lab display, it's in the same position int he classic notebook view.)

I'd recommend getting a bash kernel. It essentilly lets you operate the notebook like a terminal. Even if you dont use it here explicitly, it can be nice to have and is relatively easy to install. When you want to use it, just select the "bash" or "shell" from your lists of kernels when you start your notebook. (There are bunch of other kernels avaliable. I've had mixed successs with R and QIIME 2, for instance but haven't tried it recently, so ymmv).

If you want to continue with the python kernel, then you need to prefix any shell commands with the ! character. So, for example,

!mkdir qiime2-moving-pictures-tutorial

This triggers some behind the scenes Jypyter magic and it interprets the command as a shell command verses a python function.

I tend to use this approach because I often do a hybrid of python and bash in my notebooks, particularly if I'm working with a remote server and don't want to move visualizations around. So, i'll run a python kernel and have commands like

!mkdir qiime2-moving-pictures-tutorial

But then, i might view my artifact using the python cell which will open up the visualiztion in the notebook. (Thus avoiding a bunch of scp-ing and ssh-ing and different versions of files between my server and laptop.

import Visualization
Visualization.load('my-pcoa.qza')

You might also want to look into the python API, although if you're a novice, it can be a challenge. I find the documentation difficult to access sometimes.

@Shannon_Wilson

Could I be having these issues because I am trying to change the directory to have all files and copies saved to my desktop? this has not been an issue when I have been running qiime alone.

I find it easiest to run my notebooks where my files are. I usually have a folder in my project called ipynb and then a subfolder called data. Then, I start my notebook in the ipynb or project folder and my file system can see everything. Occasionally I start it in the home directory so it can see everything. You can use absloute file paths with the notebooks, but I find it easier to have a closed file structure with everything in a sub directory.

@Shannon_Wilson

Would this be a better question for the jupyter folks? I figure I would try here first as you guys are FAR more helpful!

Thank you , we try?

Best,
Justine

[vheidrich (Vitor Heidrich)]

Hi,

Just complementing @jwdebelius suggestions, you can either switch the kernel to bash or begin all cells containing qiime commands with the flag "%%bash" (which I prefer, because this way you can keep the python kernel). For example:

%%bash

qiime demux summarize \
  --i-data demux.qza \
  --o-visualization demux.qzv

Hope this helps!

Vitor