Skip to main content

Hugging Face

All functionality related to the Hugging Face Platform.

LLMs​

Hugging Face Hub​

The Hugging Face Hub is a platform with over 350k models, 75k datasets, and 150k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together. The Hub works as a central place where anyone can explore, experiment, collaborate, and build technology with Machine Learning.

To use, we should have the huggingface_hub python package installed.

pip install huggingface_hub

See a usage example.

from langchain_community.llms import HuggingFaceHub

Hugging Face Local Pipelines​

Hugging Face models can be run locally through the HuggingFacePipeline class.

We need to install transformers python package.

pip install transformers

See a usage example.

from langchain_community.llms.huggingface_pipeline import HuggingFacePipeline

Hugging Face TextGen Inference​

Text Generation Inference is a Rust, Python and gRPC server for text generation inference. Used in production at HuggingFace to power LLMs api-inference widgets.

We need to install text_generation python package.

pip install text_generation

See a usage example.

from langchain_community.llms import HuggingFaceTextGenInference

Document Loaders​

Hugging Face dataset​

Hugging Face Hub is home to over 75,000 datasets in more than 100 languages that can be used for a broad range of tasks across NLP, Computer Vision, and Audio. They used for a diverse range of tasks such as translation, automatic speech recognition, and image classification.

We need to install datasets python package.

pip install datasets

See a usage example.

from langchain_community.document_loaders.hugging_face_dataset import HuggingFaceDatasetLoader

Embedding Models​

Hugging Face Hub​

The Hugging Face Hub is a platform with over 350k models, 75k datasets, and 150k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together. The Hub works as a central place where anyone can explore, experiment, collaborate, and build technology with Machine Learning.

We need to install the sentence_transformers python package.

pip install sentence_transformers

HuggingFaceEmbeddings​

See a usage example.

from langchain_community.embeddings import HuggingFaceEmbeddings

HuggingFaceInstructEmbeddings​

See a usage example.

from langchain_community.embeddings import HuggingFaceInstructEmbeddings

HuggingFaceBgeEmbeddings​

BGE models on the HuggingFace are the best open-source embedding models. BGE model is created by the Beijing Academy of Artificial Intelligence (BAAI). BAAI is a private non-profit organization engaged in AI research and development.

See a usage example.

from langchain_community.embeddings import HuggingFaceBgeEmbeddings

Tools​

Hugging Face Hub Tools​

Hugging Face Tools support text I/O and are loaded using the load_huggingface_tool function.

We need to install several python packages.

pip install transformers huggingface_hub

See a usage example.

from langchain.agents import load_huggingface_tool