Use with AI agents

Agents need two things from the web: what's out there (search results) and what it says (page content). SerpHubAPI returns both in a single call, in a format you can put straight into a prompt.

The two parameters that matter

Parameter	What it does
include_content: N (1–5)	Fetches the top N result pages and returns their main content as Markdown in content[]. +3 credits per page fetched; pages that fail are free and come back with error: "fetch_failed".
format: "markdown"	Returns the whole response — results and page content — as a single Markdown document instead of JSON.

Combined:

curl -X POST https://api.serphubapi.com/v1/search \
  -H "X-API-Key: shapi_xxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"q": "recursive self improvement agents", "include_content": 3, "format": "markdown"}'

You get one Markdown text: a results list, then a ## Content: <url> section for each page. Cost: 10 + 3 × 3 = 19 credits (less if a page couldn't be fetched).

JSON for RAG pipelines

If you chunk and embed content yourself, keep JSON and read content[]:

import os, httpx

r = httpx.post(
    "https://api.serphubapi.com/v1/search",
    headers={"X-API-Key": os.environ["SERPHUBAPI_API_KEY"]},
    json={"q": "EU AI Act obligations for general-purpose models", "include_content": 5},
    timeout=60,
)
r.raise_for_status()
data = r.json()

titles = {o["link"]: o["title"] for o in data["results"].get("organic", [])}
docs = [
    {"source": c["link"], "title": titles.get(c["link"], ""), "text": c["markdown"]}
    for c in data.get("content", [])
    if c["markdown"]
]
# -> split docs into chunks, embed, store; keep data["meta"]["id"] for traceability

Keep meta.id with your chunks: for 30 days you can retrieve exactly what the agent saw with GET /v1/searches/{id}.

A minimal agent tool (LangChain-style)

import os
import httpx
from langchain_core.tools import tool

API = "https://api.serphubapi.com"
HEADERS = {"X-API-Key": os.environ["SERPHUBAPI_API_KEY"]}


@tool
def web_search(query: str, read_pages: int = 0) -> str:
    """Search the web. Returns results as Markdown.
    Set read_pages (0-5) to also include the full text of the top pages."""
    r = httpx.post(
        f"{API}/v1/search",
        headers=HEADERS,
        json={"q": query, "include_content": read_pages, "format": "markdown"},
        timeout=60,
    )
    if r.status_code != 200:
        return f"Search failed: {r.json()['error']['code']}"
    return r.text


@tool
def read_page(url: str) -> str:
    """Read a web page and return its main content as Markdown."""
    r = httpx.post(f"{API}/v1/fetch", headers=HEADERS, json={"url": url}, timeout=60)
    if r.status_code != 200:
        return f"Could not read page: {r.json()['error']['code']}"
    return r.json()["markdown"]


tools = [web_search, read_page]
# llm.bind_tools(tools) or pass to your agent executor

The same two functions work with any framework that accepts Python callables as tools. If your agent runs in an MCP client, use the MCP server instead — no code needed.

Keeping cost and context under control

• Start without content. Let the agent search first (10 credits), then call read_page on the one or two links it actually needs (3 credits each). Use include_content when you know you'll read the top results anyway — it saves a round trip.
• Use mode: "cache_first" for repeated questions. Agents often re-run the same query. With cache_first, a search costs 5 credits, and 0 if it was answered in the last hour.
• Lower num. The price is the same, but fewer results means fewer tokens in context.
• Cap each agent's key. Give every agent its own key with a daily credit limit at /dashboard/keys.
• Empty searches are free, so it's fine to let the agent try narrow queries.
• Handle errors as tool output. Return the error code to the model as text (as above) so it can rephrase or move on rather than crash. 502 all_upstreams_failed is free and safe to retry.