# Petals – Run LLMs at home, BitTorrent-style

![](https://petals.dev/logo.svg)

Run large language models at home, BitTorrent‑style

- Generate text with **Llama 3.1** (up to 405B), **Mixtral** (8x22B), **Falcon** (40B+) or **BLOOM** (176B) and fine‑tune them for your tasks — using a consumer-grade GPU or Google Colab.
- You load a part of the model, then join a [network](https://health.petals.dev) of people serving its other parts. Single‑batch inference runs at up to **6 tokens/sec** for **Llama 2** (70B) and up to **4 tokens/sec** for **Falcon** (180B) — enough for [chatbots](https://chat.petals.dev) and interactive apps.
- Beyond classic LLM APIs — you can employ any fine-tuning and sampling methods, execute custom paths through the model, or see its hidden states. You get the comforts of an API with the flexibility of **PyTorch** and 🤗 **Transformers**.

**Top contributors** right now:

Loading...

Follow development in [Discord](https://discord.gg/D9MwApKgWa) or via email:

We send updates once a few months. No spam.

We sent you an email to confirm your address. Click it and you're in!

Featured on:

[![](https://petals.dev/techcrunch.png)](https://techcrunch.com/2022/12/20/petals-is-creating-a-free-distributed-network-for-running-text-generating-ai/)

This project is a part of the [BigScience](https://bigscience.huggingface.co/) research workshop.

[![](https://petals.dev/bigscience.png)](https://bigscience.huggingface.co/)