Notebooks have always been a tool for the incremental development of software ideas. Data scientists use Jupyter to journal their work, explore and experiment with novel algorithms, quickly sketch new approaches and immediately observe the outcomes.

Moreover, JupyterLab is moving towards becoming a full-fledged IDE; just not an IDE you are used to. With its great extensions and libraries like kale and nbdev, it is certainly capable of doing much more than just drafting an idea. Check the story below to find out more.

However, once every blue moon, we may want to edit a .py file. This file may hold some utility functions we import in the Notebook or define our models' classes. It's a good practice to work like that, so we don't pollute the Notebooks with many definitions. But the text editor bundled with JupyterLab is just that: a simple, featureless text editor.

So, what can we do? There are efforts like this one, which tries to integrate the monaco editor (the code editor which powers VS Code) into JupyterLab. Still, as the contributors explicitly mention in the README file, "this extension is merely a 'proof-of-concept' implementation and nowhere near production status." Also, the last commit was 3 years ago (at the time of writing), so it doesn't seem like a very active project.

But we do not need any extension. We have a terminal. Thus, we can have ViM. And ViM has everything we need; it just takes some time to master.

If you love ViM, you know what I'm talking about. If you don't, don't shy away yet! This story shows you how to turn ViM into a Python IDE and use it side-by-side with Jupyter.

Learning Rate is a newsletter for those who are curious about the world of AI and MLOps. You'll hear from me every Friday with updates and thoughts on the latest AI news and articles. Subscribe here!

ViM for Python

Let's get to the point. The first step is actually to install ViM if it's not already there. In this story, I assume that we are working in a hoster JupyterLab environment. If you're working locally, there's nothing to stop you from firing up VS Code if you prefer that over ViM.

So, my guess is that you're working inside a VM on the Cloud. That usually means a Linux environment. To install ViM on Debian distributions, run the following commands:

sudo apt-get remove vim-tiny
sudo apt-get update
sudo apt-get install vim

Note that some Linux distributions come with vim-tiny pre-installed. Thus, we need to remove it first and then install the full version.

Then, let's verify we have everything we need. Run vim --version and pay attention to two things:

1. You should have installed VIM > 7.3
2. Check for a +python or +python3 feature

We are ready. Next, we need a good plugin manager.

Vundle

I find that Vundle is an excellent and simple to use plugin manager. Thus, we will use this for installing everything we need. To install Vundle we need two things. First, git clone the project in an appropriate directory:

git clone https://github.com/gmarik/Vundle.vim.git ~/.vim/bundle/Vundle.vim

Then, we need to create a .vimrc file in our home directory and add some lines on top. So, first, create the file:

touch ~/.vimrc

Then, add the following lines:

set nocompatible              " required
filetype off                  " required

" set the runtime path to include Vundle
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()

" let Vundle manage Vundle, required
Plugin 'gmarik/Vundle.vim'

" add all your plugins here between the vundle#begin() and "vundle#end() calls

" All of your Plugins must be added before the following line
call vundle#end()            " required
filetype plugin indent on    " required

See the line that calls vundle#begin()? Between this and the vundle#end() call we can add any plugin we want. As a matter of fact, we have done this already: we have added the gmarik/Vundle.vim plugin. Installing a new plugin is as easy as copying and pasting its unique GitHub path. We will see how later.

For now, launch ViM, enter the command mode by pressing : and execute PluginInstall. Let's move.

A Python IDE

The first plugin we will install enables folding. Have you ever seen the numpy source code? Usually, the docstring of a function takes up the whole space. Let's fold it by default. Install the plugin by adding this line between the begin and end calls we saw earlier:

Plugin 'tmhedberg/SimpylFold'

A reminder: every time you add a new plugin, don't forget to install it with the PluginInstall command we saw.

To enable the docstring folding by default, add the following setting, outside the begin-end block.

let g:SimpylFold_docstring_preview=1

Next, let's enable auto-indentation. Install the following plugin (now you know how):

Plugin 'vim-scripts/indentpython.vim'

Moreover, you can tell ViM how you want it to treat .py files by adding the following options in the vimrc file:

au BufNewFile,BufRead *.py
    \ set tabstop=4 |
    \ set softtabstop=4 |
    \ set shiftwidth=4 |
    \ set textwidth=79 |
    \ set expandtab |
    \ set autoindent |
    \ set fileformat=unix

The names of these settings are pretty much self-explanatory. The autoindent setting does most of the things right, but install the vim-scripts/indentpython.vim plugin, which is python specific, to keep your peace of mind.

Last but not least, you need auto-complete. The best tool for this job is YouCompleteMe. However, its installation is a bit more involved. First, install the plugin with the following line:

Bundle 'Valloric/YouCompleteMe'

It will most probably show you an error at the end. Don't worry. Proceed and install the necessary dependencies:

apt install build-essential cmake python3-dev
apt install mono-complete golang nodejs default-jdk npm

Finally, compile the plugin:

cd ~/.vim/bundle/YouCompleteMe
python3 install.py --all

That's it! You have most of the things you need to turn ViM into a Python IDE. Other plugins you might want to consider are:

• vim-syntastic/syntastic: Python syntax highlighting
• nvie/vim-flake8: PEP8 checking
• scrooloose/nerdtree: folder structure explorer
• tpope/vim-fugitive: git integration

Here is a complete but not exhaustive .vimrc configuration. If you have more goodies to add, please comment!

Conclusion

Notebooks have always been a tool for the incremental development of software ideas. Moreover, JupyterLab is moving towards becoming a full-fledged IDE. However, once every blue moon, we may want to edit a .py file, and the integrated text editor is just a text editor.

This story examines how to transform ViM into a Python IDE and use it as our main code editor through the terminal. If you want more info on using ViM, just run vimtutor in your terminal and follow the instructions!

About the Author

My name is Dimitris Poulopoulos, and I'm a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.

If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter. Also, visit the resources page on my website, a place for great books and top-rated courses, to start building your own Data Science curriculum!