Running Ollama On Google Colab

Introduction

In this article, I will show you how to run Ollama on Google Colab. It's simple and easy to replicate, as this is quite useful if you don't feel like downloading Ollama onto your local machine for simple tasks.

Setup Scripts

You need to create two simple scripts. Create a file called "ollama_install.sh", and "ollama_thread.py". Copy the following code into those files

ollama_install.sh: this will simply create a curl request to install Ollama within your notebook.

echo "Installing Ollama on your environment"
curl -fsSL https://ollama.com/install.sh | sh

ollama_thread.py this will keep the Ollama service alive. This is crucial as it allows you to interact with the given models in realtime.

import subprocess
from concurrent.futures import ThreadPoolExecutor

# Function to run Ollama service
def run_ollama():
    subprocess.Popen(["ollama", "serve"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# Function to start the Ollama service in a thread
def start_service():
    with ThreadPoolExecutor() as executor:
        executor.submit(run_ollama)

# Start the service in parallel
start_service()

Running The Setup Scripts

Create two new code cells and run the following code to execute these scripts

!chmod +x /content/ollama_install.sh
!/content/ollama_install.sh

%run '/content/ollama_thread.py'

Installations and Imports

Copy the following python installations and imports and paste them into your notebook.

!ollama pull llama3.2 #pull the model of your choice (https://ollama.com/search).
!pip install langchain-ollama
import ollama
from IPython.display import display, Markdown
        

Interacting With Our Model

We need to first a process question method first. This will allow us to interact with our model of choice.

def process_question(question, model='llama3.2'):
  messages = [
      {
            'role': 'user',
            'content': f'{question}'
        }
  ]
  response = ollama.chat(model=model, messages=messages)
  return response['message']['content']

Let's now interact with our model by asking it a very simple question.

display(Markdown(process_question("Give me details behind the Python programming language")))

Output:

Our model provided a comprehensive overview of what Python is.

Conclusion

In a few lines of code we simply accomplished getting Ollama to run on Google Colab. This approach provides an easy way to work with large language models without needing specialized hardware or managing your own local server.