Two ways to get a free LLM into a Python script
When people search for free llms for python, they usually mean one of two things. The first is an open-weight model that you download and run yourself, using something like Ollama, llama.cpp or the Hugging Face transformers library. The second is a hosted endpoint that costs so little, or comes with enough trial credit, that it feels free for a prototype. Both are legitimate, and they suit different jobs.
Running locally means no per-token bill and no data leaving your machine. It also means you need enough RAM or GPU memory for the model you pick, you wait for downloads, and small models that fit on a laptop answer worse than large ones. A hosted API removes the hardware question, but you pay per token once the trial is gone. This page covers both honestly, then shows the Python code for the hosted option on this site, which is OpenAI-compatible and works with the standard openai package.
Local models and hosted API side by side
| Question | Run it yourself | Hosted prepaid API |
|---|---|---|
| Cost per token | None, but you pay for hardware and power | $0.15 per 1M input, $0.50 per 1M output |
| Setup time | Install runtime, download weights, tune settings | Install one package, paste a key |
| Context length | Limited by your memory | 100,000 tokens, prompt plus completion |
| Concurrency | Whatever your machine handles | 8 requests at once on a full key, 2 on a trial key |
| Refusals | Depends on the weights you choose | Tuned to answer without content refusals for lawful adult use |
| Offline use | Yes | No |
If you have a capable GPU and your work is private or offline, local is a fine answer and you can stop reading here. If you want a bigger model without owning the machine, or you want your script to behave the same on a laptop, a server and a CI runner, the hosted route is simpler. Many projects use both: a small local model while developing, the hosted one in production, switched by changing one base URL.
Set up the hosted endpoint in five minutes
You need a key first. Sign up with Google or with an email and password, then claim the $0.50 trial credit with one click on the API key page. The trial is valid for 7 days, one per person, and needs no payment. Then install the SDK and keep the key out of your source code.
pip install openai
export NOTRACK_API_KEY="sk-your-key-here"
The client needs two changes from a stock OpenAI setup: the base URL and the key. The model ID is uncensored, and only one model is offered.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.nsfwvideogenerator.cc/v1",
api_key=os.environ["NOTRACK_API_KEY"],
)
resp = client.chat.completions.create(
model="uncensored",
messages=[
{"role": "system", "content": "You are a concise writing assistant."},
{"role": "user", "content": "Give me three title ideas for a noir short story."},
],
temperature=0.8,
)
print(resp.choices[0].message.content)
print(resp.usage)
Auth is a standard Authorization: Bearer header, which the SDK adds for you. Two methods exist: POST /v1/chat/completions and GET /v1/models. You can confirm the connection with client.models.list(), which returns the available model ID. The full parameter list is on the docs page.
Streaming responses
For anything a person watches, stream. Set stream=True and iterate over chunks as server-sent events arrive. Usage is reported at the end of the stream, so you can still log token counts.
stream = client.chat.completions.create(
model="uncensored",
messages=[{"role": "user", "content": "Write a four-line poem about rain on a tin roof."}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
usage = getattr(chunk, "usage", None)
if usage:
print("\n", usage)
One detail catches people out: a streamed request holds its concurrency slot until the stream finishes. On a trial key with 2 slots, two open streams mean a third call gets a 429 with the code concurrency. Always consume or close the stream.
Tool calling and JSON mode
The endpoint supports tools, tool_choice, streamed tool_calls and results sent back as role: "tool". Here is a complete round trip with one function.
import json
tools = [{
"type": "function",
"function": {
"name": "word_count",
"description": "Count the words in a piece of text.",
"parameters": {
"type": "object",
"properties": {"text": {"type": "string"}},
"required": ["text"],
},
},
}]
messages = [{"role": "user", "content": "How many words are in: the quick brown fox jumps"}]
first = client.chat.completions.create(
model="uncensored", messages=messages, tools=tools, tool_choice="auto"
)
msg = first.choices[0].message
if msg.tool_calls:
messages.append(msg)
for call in msg.tool_calls:
args = json.loads(call.function.arguments)
result = len(args["text"].split())
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": str(result),
})
final = client.chat.completions.create(model="uncensored", messages=messages, tools=tools)
print(final.choices[0].message.content)
Always parse arguments defensively; the model returns it as a JSON string and any model can occasionally produce a malformed one. For structured output without tools, use JSON mode.
resp = client.chat.completions.create(
model="uncensored",
messages=[
{"role": "system", "content": "Reply with a JSON object with keys title and tags (list of strings)."},
{"role": "user", "content": "Describe a lighthouse keeper's diary."},
],
response_format={"type": "json_object"},
temperature=0.2,
)
data = json.loads(resp.choices[0].message.content)
print(data["title"], data["tags"])
Say the word JSON and name the keys in your prompt, and keep temperature low. Wrap json.loads in a try block for production code.
Rate limits, errors and retries
A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 and 60. Go over and you get HTTP 429 with a code of rate_limited (too many per minute) or concurrency (too many at once). On temporary overload the API also returns 429, with a Retry-After: 5 header. The SDK retries some errors on its own; for a script that fans out, a small explicit wrapper is clearer.
import time
from concurrent.futures import ThreadPoolExecutor
from openai import RateLimitError
def ask(prompt, tries=4):
for attempt in range(tries):
try:
r = client.chat.completions.create(
model="uncensored",
messages=[{"role": "user", "content": prompt}],
max_tokens=300,
)
return r.choices[0].message.content
except RateLimitError:
time.sleep(5 * (attempt + 1))
raise RuntimeError("still rate limited")
prompts = [f"Summarize idea number {i} in one sentence." for i in range(20)]
with ThreadPoolExecutor(max_workers=2) as pool: # 8 on a full key
results = list(pool.map(ask, prompts))
Matching max_workers to your key's concurrency limit prevents most 429s before they happen. Each response also carries X-Request-Id, X-Balance-USD and two rate limit headers, which you can read through client.chat.completions.with_raw_response.create(...) when you need them for debugging or support.
How to choose between local and hosted
Skip the ideology and answer four questions about your project.
- Does the data have to stay on your machine? If a contract or a habit says yes, run locally. Hosted prompts are not used for training here, but they do travel over the network.
- How much memory do you have? Model size drives quality. A laptop without a dedicated GPU limits you to small models, and the answers will show it. A hosted model gives you a large one with a 100,000 token context window, which is hard to match at home.
- How many requests will you send? A few hundred a week is trivial on either side. Hundreds of thousands per month turns the hosted bill into a real number, and a local box starts to look attractive if it sits idle otherwise. Work the sum out: 100,000 requests of 1,000 input and 500 output tokens costs 100,000 x $0.0004 = $40.
- Do you need refusals handled a certain way? Many open-weight models ship with built-in refusal behaviour, and others are tuned to drop it. The hosted model here is tuned to answer without content refusals for lawful adult use, so you do not need to hunt for the right weights.
If your answers point both ways, start hosted for the first week. The trial costs nothing, and you will learn what your prompts need before you buy hardware or download another multi-gigabyte file.
Common mistakes in Python scripts
Most problems people hit are small and fixable.
- Wrong base URL. It must end in
/v1. A missing suffix gives a 404, not a helpful message. Printclient.base_urlif in doubt. - Key in the code. Read it from an environment variable, as above, and add your
.envfile to.gitignore. - Wrong model name. The only model ID is
uncensored. Names from other providers will not work, so change them when you port code. - Unbounded loops. A tool-calling loop that never ends is an expensive bug. Cap it at, say, five rounds.
- Ignoring the context budget. Prompt and completion share the 100,000 tokens. If you append every turn forever, you will eventually hit the limit. Trim or summarize old messages.
- Assuming JSON is always valid. Even in JSON mode, check the keys you need before using them.
When something looks wrong on the server side, quote the X-Request-Id value from the response when you write to support. It lets the operators find your exact call.
What the hosted option costs for Python work
A typical script call with 1,000 input and 500 output tokens costs $0.00015 plus $0.00025, so $0.0004. The $0.50 trial covers about 1,250 of them. A $20 top-up, the minimum, covers 50,000. Credit never expires, failed or refused requests are not charged, and prompts are not used for training. Top-ups go up to $500 with a bonus from $50, and payment is in crypto, USDT on TRC20 or USDC on Base. See the pricing page for the details.
It is text only: no images, audio, video, embeddings or fine-tuning. And the one hard rule is no sexual content involving minors. For a Python project that needs text generation with fewer refusals, that is the whole surface.
