Skip to main content

Ollama

Ollama allows you to run open-source large language models, such as Llama 2, locally.

Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile.

It optimizes setup and configuration details, including GPU usage.

For a complete list of supported models and model variants, see the Ollama model library.

Setupโ€‹

First, follow these instructions to set up and run a local Ollama instance:

  • Download
  • Fetch a model via ollama pull <model family>
  • e.g., for Llama-7b: ollama pull llama2 (see full list here
  • This will download the most basic version of the model typically (e.g., smallest # parameters)
  • On Mac, it will download to

~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latest

  • And we specify a particular version, e.g., for ollama pull vicuna:13b-v1.5-16k-q4_0
  • The file is here with the model version in place of latest

~/.ollama/models/manifests/registry.ollama.ai/library/vicuna/13b-v1.5-16k-q4_0

You can easily access models in a few ways:

1/ if the app is running:

  • All of your local models are automatically served on localhost:11434
  • Select your model when setting llm = Ollama(..., model="<model family>:<version>")
  • If you set llm = Ollama(..., model="<model family") withoout a version it will simply look for latest

2/ if building from source or just running the binary:

  • Then you must run ollama serve
  • All of your local models are automatically served on localhost:11434
  • Then, select as shown above

Usageโ€‹

You can see a full list of supported parameters on the API reference page.

from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain_community.llms import Ollama

llm = Ollama(model="llama2")

Optionally, pass StreamingStdOutCallbackHandler to stream tokens:

llm = Ollama(
    model="llama2",
    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
)
llm("Tell me about the history of AI")
' Artificial intelligence (AI) has a rich and varied history that spans several decades. paลบdziernik 1950s and has evolved significantly over time. Here is a brief overview of the major milestones in the history of AI:\n\n1. 1950s: The Dartmouth Conference - Considered the birthplace of AI, this conference brought together computer scientists, mathematicians, and cognitive scientists to discuss the possibilities of creating machines that could simulate human intelligence. Attendees included John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon.\n2. 1951: The Turing Test - Alan Turing proposed a test to measure a machine\'s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. The Turing Test has since become a benchmark for measuring the success of AI systems.\n3. 1956: The First AI Program - John McCarthy created the first AI program, called the Logical Theorist, which was designed to reason and solve problems using logical deduction.\n4. 1960s: Rule-Based Expert Systems - Researchers developed rule-based expert systems, which used a set of rules to reason and make decisions. These systems were widely used in industries such as banking and healthcare.\n5. 1970s: Machine Learning -Machine learning, a subfield of AI, emerged as a way for machines to learn from data without being explicitly programmed. This led to the development of algorithms such as decision trees and neural networks.\n6. 1980s: Expert Systems - The development of expert systems, which were designed to mimic the decision-making abilities of human experts, reached its peak in the 1980s. These systems were widely used in industries such as banking and healthcare.\n7. 1990s: AI Winter - Despite the progress made in AI research, the field experienced a decline in funding and interest in the 1990s, known as the "AI winter."\n8. 2000s: AI Resurgence - The resurgence of AI began in the early 2000s with the development of new algorithms and techniques, such as support vector machines and deep learning. This led to a renewed interest in AI research and applications.\n9. 2010s: Rise of Deep Learning - The development of deep learning algorithms, which are capable of learning and improving on their own by analyzing large amounts of data, has been a major factor in the recent progress made in AI. These algorithms have been used in applications such as image recognition, natural language processing, and autonomous vehicles.\n10. Present Day: AI Continues to Advance - AI is continuing to advance at a rapid pace, with new techniques and applications emerging all the time. Areas of research include natural language processing, computer vision, robotics, and more.\n\nSome notable people who have made significant contributions to the field of AI include:\n\n1. Alan Turing - Considered one of the pioneers of AI, Turing proposed the Turing Test and developed the concept of a universal machine.\n2. John McCarthy - McCarthy is known as the "father of AI" for his work in developing the field of AI. He coined the term "Artificial Intelligence" and was instrumental in organizing the Dartmouth Conference.\n3. Marvin Minsky - Minsky was a pioneer in the field of neural networks and co-founder of the MIT AI Laboratory.\n4. Nathaniel Rochester - Rochester was a computer scientist and cognitive scientist who worked on early AI projects, including the development of the Logical Theorist.\n5. Claude Shannon - Shannon was a mathematician and electrical engineer who is known for his work on information theory, which has had a significant impact on the field of AI.\n6. Yann LeCun - LeCun is a computer scientist and the director of AI Research at Facebook. He is also the Silver Professor of Computer Science at New York University, and a professor at the Courant Institute of Mathematical Sciences.\n7. Geoffrey Hinton - Hinton is a computer scientist and cognitive psychologist who is known for his work on artificial neural networks. He is a pioneer in the field of deep learning and has made significant contributions to the development of convolutional neural networks (CNNs).\n8. Yoshua Bengio - Bengio is a computer scientist and a pioneer in the field of deep learning. He is known for his work on recurrent neural networks (RNNs) and has made significant contributions to the development of CNNs and RNNs.\n9. Andrew Ng - Ng is a computer scientist and entrepreneur who has made significant contributions to the field of AI. He is known for his work on deep learning and has worked at Google, where he founded the Google Brain deep learning project, and at Baidu, where he led the company\'s AI group.\n10. Demis Hassabis - Hassabis is a computer scientist and entrepreneur who is known for his work on deep learning and artificial intelligence. He is the co-founder of DeepMind, which was acquired by Alphabet in 2014, and has made significant contributions to the field of AI.\n\nThese are just a few examples of notable people who have made significant contributions to the field of AI. There are many other researchers and scientists who have also made important advancements in the field.'

Multi-modalโ€‹

Ollama has support for multi-modal LLMs, such as bakllava and llava.

ollama pull bakllava

Be sure to update Ollama so that you have the most recent version to support multi-modal.

from langchain_community.llms import Ollama

bakllava = Ollama(model="bakllava")
import base64
from io import BytesIO

from IPython.display import HTML, display
from PIL import Image


def convert_to_base64(pil_image):
    """
    Convert PIL images to Base64 encoded strings

    :param pil_image: PIL image
    :return: Re-sized Base64 string
    """

    buffered = BytesIO()
    pil_image.save(buffered, format="JPEG")  # You can change the format if needed
    img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
    return img_str


def plt_img_base64(img_base64):
    """
    Disply base64 encoded string as image

    :param img_base64:  Base64 string
    """
    # Create an HTML img tag with the base64 string as the source
    image_html = f'<img src="data:image/jpeg;base64,{img_base64}" />'
    # Display the image by rendering the HTML
    display(HTML(image_html))


file_path = "/Users/rlm/Desktop/Eval_Sets/multi_modal_presentations/DDOG/img_23.jpg"
pil_image = Image.open(file_path)
image_b64 = convert_to_base64(pil_image)
plt_img_base64(image_b64)
llm_with_image_context = bakllava.bind(images=[image_b64])
llm_with_image_context.invoke("What is the dollar based gross retention rate:")
'90%'