Welcome to local_llm_kit’s documentation!

local_llm_kit is a Python package that provides an OpenAI-like interface for local language models. It allows you to run language models locally while maintaining compatibility with OpenAI’s API structure.

Getting Started:

User Guide:

Advanced Topics:

Development:

Features

  • OpenAI-compatible API interface

  • Support for multiple model backends (Transformers, llama.cpp)

  • Chat completion API

  • Function calling capability

  • Streaming responses

  • JSON mode output

  • Memory management

  • Custom prompt formatting

  • Extensive documentation and tutorials

  • Performance optimization guides

  • Community-driven development

Quick Installation

pip install local-llm-kit

Quick Example

from local_llm_kit import LLMClient

# Initialize the client
client = LLMClient(model="llama2")

# Chat completion
response = client.chat.completions.create(
    model="llama2",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

Getting Help

If you need help using local_llm_kit, you have several options:

  1. Check the Tutorials for step-by-step guides

  2. Read the API Reference for detailed API documentation

  3. Look through Examples for common use cases

  4. Visit our GitHub Issues page

  5. Join our community discussions

Contributing

We welcome contributions! Please see our Contributing guide for details on how to:

  • Report bugs

  • Suggest features

  • Submit pull requests

  • Improve documentation

License

local_llm_kit is released under the MIT License. See the LICENSE file for more details.

Indices and tables