Jupyter Notebook Tunneling is a convenient trick to have in your back pocket, especially when the resources on your personal computer are limiting your progress. Tunneling allows you to run your notebook from a remote host such that your processes utilize the resources available on your host, rather than your personal computer. In this brief tutorial, we show you how to tunnel your notebook from a Windows machine.

 

Part 1: Steps to setup a conda environment on an IU HPC from a Windows 10 machine.

1. Search for PuTTY in your search bar and open.
2. On PuTTy’s configuration page, type in your IU username followed by an @ symbol and the hostname (as shown in the picture below). Give this address a name in the “Saved Session” bar and click “Save”. This way, next time you connect to the server, you can simply click the server’s hostname you want to log into and click “Load”. As a side note, you can change the appearance (font, size, etc.) on Appearance tab on the left banner for your PuTTY terminal. You can save your setting for future session as well.

 

3. After you’ve entered the hostname in PuTTY, click “Open”.
4. A PuTTY security alert will likely appear. Click “Yes”.
5. A terminal will open for you to login. The terminal will ask for your password. Type in you IU password and click enter. Note that your password characters are hidden, so you will not be able to see them as you type them.

 

6. You should see a screen that looks like this when you’re logged in.

 

7. Now we can prepare our conda environment to tunnel our Jupyter notebook. You will first have to load the anaconda module that is already installed on the server. You can see all of the module available by typing module avail on the command line and clicking enter. You can load a module by typing module load <name of module>. Type module load anaconda and click enter. To create a conda environment and give name it, we use the conda create command with the -n flag. In the command line type conda create -n <give your env a name here>. Conda will ask you to verify the path this environment is saved, type “y” and click enter.

 

8. Now that you’ve started an environment, you can activate it and start adding tools, software, etc. to it. First, make sure you’ve activated your environment by typing source activate <name of your conda env here>. If you don’t remember what you named you environment, you can look at all the environments you’ve created by typing conda env list and clicking enter (it may take a second to load). Notice that once you’ve activated you conda environment, the command line will preface your username with the name of your conda environment.

 

9. Now we can install any other necessary packages into our conda environment. If you should get logged-out between installations, no worries. Just make sure when you log back in, you have to load the anaconda module and activate your conda environment again before continuing installations.

 

Part 2: Move files from your personal computer to the HPC with WinSCP on a Windows 10 machine.

1. WinSCP is just a tool that allows you to transfer files between your personal computer and another computer, like Carbonate. Win stands for Windows and SCP stands for secure copy protocol. Tools like WinSCP are known as file transfer clients. In your Windows search bar at the bottom-left of your screen, search for WinSCP. If it’s not already installed, go ahead and install it. If the installation asks you to specify preferences, just accept the generic options.
2. Open WinSCP once installed. A login configuration screen should automatically appear. Fill in the hostname of the computer you would like to transfer files to. Enter you IU login information and your IU password. Click “Save” so you can simply load this information next time you login. Make sure the Port and File Protocol are the same as in the example below. Then click “Login” and the bottom.

 

3. You will likely see a prompt asking if you’re sure you’d like to connect to the host. Click “yes”. You will then be prompted to re-enter your IU password and to proceed with password authentication.
4. Once you’re logged in, you should see a portal like in the image below. The file structure on the left is your personal computer. You can click directly on the file path in gray and browse your personal computer. The file structure of your home directory on the server you’ve logged into. Be mindful of the paths in case you left lost. Transferring files is as simple as clicking and dragging. You can move files from your personal computer to the server and vice versa! Make sure you click the green, spinny arrows next to the home icon to update the screen after you transfer files or make changes. This is where you can transfer you Jupyter notebook to the server.

 

Part 3: Tunneling your Jupyter notebook from the server.

1. If your PuTTY terminal was idle for too long and logged you out, go ahead and open PuTTY again. This time you should be able to select your server from the “saved session” box and click “Load”. Next click “Open” and follow the prompts to log-in.
2. On you command line, you can type ls to see a list of everything in your current directory. You can type pwd to print working directory (this will show you your current path). You can change directories by typing cd <directory to navigate to>. You can go one directory up (aka back) by typing cd ../ . If you get stuck in a weird command line carrot or would like to cancel an operation, simply type CTRL + C. A couple tricks to remember: you can click the tab key to automatically complete a directory’s name and you can use the up arrow on your keyboard to scroll through your history of commands (or you can simply type history in the command line).

 

3. Now, remember to module load anaconda and source activate <environment name here>. Then on your command line type jupyter notebook –no-browser –port=8080 and click enter.

 

4. Open a Command Prompt terminal by searching “Command Prompt” in the Window search bar at the bottom left of your screen. In the command line, type ssh -N -L 8080:localhost:8080 <IU username here>@<host name here> and click enter. See the example below. When asked if you’re sure you’d like to connect, type yes and click enter to continue.

 

5. Navigate back to your PuTTY terminal. Highlight the Jupyter notebook link – highlighting will automatically copy. Remember CTRL + C is the command to stop a process on the command line (not to copy). If you accidentally interrupt the notebook, not to worry, Jupyter will double check with you before shutting down the notebook.

 

6. Open a new browser and right click to paste the link into the address search bar and voila! It may take a few seconds to load but you should see server’s home directory loaded like the example below.

 

7. One last thing, keep in mind that conda environments can get big quickly! This shouldn’t be an issue on Carbonate since you have 100G storage allocation in your home directory, but it could easily be an issue on a server like burrow where you are only allowed 10G. It is good practice to use the quota command when you’re logged into your account in PuTTY to make sure you haven’t exceeded your storage allocation.