Knowledge Base

AI Fundamentals
Knowledge Wiki

A visual reference covering AI API formats, model serving, token economics, MCP protocol, and building model-agnostic applications.

18 Topics Concept Level April 2026

Core Concepts

01

API Payload Formats

OpenAI, Claude, and Gemini payload formats side-by-side.

02

Message Roles

system, user, and assistant roles — how conversation state works.

03

Temperature

0.0 to 2.0 — when to use deterministic vs creative output.

04

Tokens & Cost

Context windows, input/output pricing, and cost math.

Model Serving

05

vLLM vs Ollama

Dev-friendly vs production-grade local model servers.

06

LiteLLM

One Python library to talk to any AI provider.

07

Multiple Models

Run 4–6 models in parallel on your GPU server.

Architecture

08

Model-Agnostic App

Swap models with one string change — LiteLLM router pattern.

09

API Keys

.env for local, K8s Secrets for production, OAuth for services.

10

Claude.ai vs Code

Browser chat vs terminal agent — when to use each.

MCP

MCP·01

What is MCP

Open standard for connecting AI to external tools and data.

MCP·02

MCP Server

The middleman that translates MCP calls to real service APIs.

MCP·03

Tools, Resources & Prompts

Tools (actions), Resources (data), Prompts (templates).

MCP·04

Who Hosts What

Vendor-hosted vs self-hosted — who owns the data decides.

MCP·05

Connecting to MCP

Claude.ai connectors, Desktop config, or programmatic client.

MCP·06

MCP Credentials

Credential storage patterns for local dev and production.

MCP·07

Writing MCP Servers

pip install mcp — three decorators, done.

MCP·08

MCP & Token Cost

Tool definitions cost tokens on every call — optimise early.

01

API Payload Formats

Each AI provider has its own API structure. OpenAI's format became the de facto standard — most providers copied it. Gemini is the most different. Claude is close but has one key quirk.

🤖 OpenAI — De Facto Standard

POST /v1/chat/completions · system prompt lives inside messages array

          # endpoint

          POST api.openai.com/v1/chat/completions

          {

            "model": "gpt-4o",

            "messages": [

              {"role":"system", "content":"..."},

              {"role":"user",   "content":"..."}

            ],

            "temperature": 0.7,

            "max_tokens": 1000

          }

🧠 Claude — Key Difference

POST /v1/messages · system prompt is a TOP-LEVEL field, not inside messages

          # endpoint

          POST api.anthropic.com/v1/messages

          {

            "model": "claude-sonnet-4...",

            "max_tokens": 1000, ← required!

            "system": "...", ← top level ⚠️

            "messages": [

              {"role":"user", "content":"..."}

            ]

          }

🌐 Gemini — Most Different

POST /v1beta/models/gemini:generateContent · uses "contents" not "messages", "parts" not "content", "model" not "assistant"

        POST generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent

        {

          "contents": [ ← NOT "messages"

            { "role": "user", "parts": [{"text": "Hello"}] }, ← "parts" NOT "content"

            { "role": "model", "parts": [{"text": "Hi!"}] }  ← "model" NOT "assistant"

          ],

          "generationConfig": { "temperature": 0.7, "maxOutputTokens": 1000 }

        }

Provider	Endpoint	Messages Key	System Prompt	AI Role Name	Compatibility
OpenAI	/v1/chat/completions	messages[]	Inside messages	assistant	Standard
Claude	/v1/messages	messages[]	Top-level field	assistant	Has adapter
Gemini	/v1beta/…:generateContent	contents[]	systemInstruction	model	Most different
Mistral	/v1/chat/completions	messages[]	Inside messages	assistant	OpenAI clone
Grok	/v1/chat/completions	messages[]	Inside messages	assistant	OpenAI clone
Ollama	/api/chat	messages[]	Inside messages	assistant	OpenAI clone

02

Message Roles

Every message has a role — who is speaking. The AI reads all messages in order on every call. There is no memory between calls — you must resend the full history every time.

SYS

system — The Director

Sets rules, personality, and constraints before conversation starts. User never sees this. Runs first on every call. Think of it as the AI's job description.

            { "role": "system", "content": "You are a warehouse AI for Zahir's factory.
                                  Be concise. Always include bay location." }
          

USR

user — The Human

The actual human input — questions, requests, commands. What the person types.

{ "role": "user", "content": "How many units of SKU-001 do we have?" }

AST

assistant — The AI's Past Replies

The AI's own previous responses. You include these to give the AI conversation memory — because each call is stateless. No history = no context.

{ "role": "assistant", "content": "SKU-001 has 450 units in Bay 3." }

⚠️Every API call is stateless. You must send the complete conversation history — system + all user + all assistant messages — on every single request. The AI has no memory between calls.

03

Temperature

Controls how creative or deterministic the AI's response is. Range: 0.0 to 2.0. Low = focused and consistent. High = creative and unpredictable.

Deterministic Balanced Creative Chaotic

0.00.51.01.52.0

0.0

Exact

SQL queries, data extraction, classifications, defect detection

0.3

Focused

Summaries, structured reports, consistent analysis

0.7

Balanced

Chatbots, general Q&A, customer support

1.0

Creative

Brainstorming, marketing copy, ideation

1.5+

Avoid

Unpredictable, often nonsensical. Not for production.

💡Rule of thumb: Use low temperature when you want the RIGHT answer. Use high temperature when you want variety. Your warehouse defect classifier → 0.0. Your chatbot → 0.5. Your marketing generator → 0.9.

04

Tokens & Cost

A token is roughly ¾ of a word (English). Everything sent and received is counted. Input and output tokens are billed separately — output costs more.

📏 What is a Token

1 token ≈ 4 characters ≈ ¾ of a word. Non-English (Bahasa, Arabic, Chinese) uses more tokens per word than English.

📥 Input Tokens

System prompt + all message history + tool definitions + attached documents. Counted and billed per call.

📤 Output Tokens

Only the AI's response. Billed separately — typically 3–5× more expensive than input tokens.

Context Window — Everything Must Fit

System

MCP Tools

Conversation History

User

AI Output

System prompt

Tool definitions (every call!)

Conversation history

User message

AI output (billed 3–5× more)

Model	Context Window	Input Price	Output Price
Claude Sonnet 4	200,000 tokens	$3 / 1M tokens	$15 / 1M tokens
GPT-4o	128,000 tokens	$2.50 / 1M tokens	$10 / 1M tokens
Gemini 2.0 Flash	1,000,000 tokens	$0.10 / 1M tokens	$0.40 / 1M tokens
Llama 3 (Ollama/vLLM)	8K–128K tokens	FREE	FREE

🔥Tool definitions cost tokens on every call — even if the tool is never used. 20 tools × ~300 tokens = 6,000 tokens per call. Only load tools relevant to the current task.

05

vLLM vs Ollama

Both are model servers — they run AI models locally on your GPU and serve them via API. Same job, different target audience. Think Nginx vs Apache.

── Web World Analogy ──

Apache

Easy setup

=

Ollama

Dev friendly

Nginx

Production grade

=

vLLM

High performance

── Both serve OpenAI-compatible API ──

Ollama

localhost:11434

≡

vLLM

localhost:8000

≡

OpenAI API

api.openai.com

Feature	Ollama	vLLM
Setup	2 commands, done	Multiple steps, GPU config
Target	Dev / Learning	Production / Scale
Concurrent requests	Limited, queued	Optimized batching
GPU utilization	Basic	Maximum (paged attention)
Multi-model	Swaps in/out	True parallel, multiple ports
Model format	GGUF (quantized)	Full precision or quantized
Docker feel	ollama pull / run / list	Manual, more control
API keys needed	None	None
Cost	Free	Free (your hardware)
GB10 recommendation	Dev & demos	Production workloads

      OLLAMA
      # Install, pull, run — 3 commands

      curl -fsSL https://ollama.com/install.sh | sh

      ollama pull llama3

      ollama run llama3   # API ready at localhost:11434

      vLLM
      # Install and serve

      pip install vllm

      vllm serve meta-llama/Llama-3-8B-Instruct --port 8000   # API ready at localhost:8000

06

LiteLLM

A client-side Python library that translates between all AI provider APIs. You write code once in OpenAI format — LiteLLM routes to any provider. It does NOT run models itself.

✅ What LiteLLM Does

Translates API formats between providers. Manages API keys per provider. Handles retries and fallbacks. Tracks token usage and cost. Routes to cheapest/fastest model.

❌ What LiteLLM Does NOT Do

Does not run any model itself. Does not host anything. Does not replace actual AI providers. It is just a library in your code — nothing to deploy.

      LITELLM — One Interface, Any Model
      import litellm

      # Same code — just change the model string

      litellm.completion(model="claude-sonnet-4-20250514", messages=msgs) # → Anthropic

      litellm.completion(model="gpt-4o",                   messages=msgs) # → OpenAI

      litellm.completion(model="gemini/gemini-2.0-flash",   messages=msgs) # → Google

      litellm.completion(model="ollama/llama3",             messages=msgs) # → Local GB10

🎯Mental model: LiteLLM is a universal remote control. One remote, controls all TVs. You don't buy a new remote for every TV brand — you use one remote that speaks all languages.

07

Running Multiple Models

Just like Docker — you run multiple containers on different ports. Your GB10 with 128GB unified memory can run 4–6 models simultaneously with no swapping.

🐳 Docker vs Model Servers

          # Docker              Model Server

          docker pull nginx  = ollama pull llama3

          docker run -p 8001 = vllm serve --port 8001

          docker ps         = ollama list

          docker stats      = nvidia-smi

          docker-compose    = multiple vLLM instances

🖥️ Your GB10 Memory Budget

          # GB10 = 128GB unified memory

          llama3   8B  = ~8GB

          mistral  7B  = ~8GB

          qwen2.5  7B  = ~8GB

          codellama 7B = ~8GB

          ─────────────────────

          Total used   = 32GB

          Remaining    = 96GB free ✅

      vLLM — Multiple Models on Different Ports
      # Terminal 1 — general tasks

      vllm serve meta-llama/Llama-3-8B-Instruct --port 8001

      # Terminal 2 — coding tasks

      vllm serve mistralai/Mistral-7B-Instruct  --port 8002

      # Terminal 3 — multilingual (Bahasa friendly)

      vllm serve Qwen/Qwen2.5-7B-Instruct       --port 8003

08

Model-Agnostic Architecture

Build your app so the model is a pluggable component. Your tools, prompts, and business logic stay fixed. Only one string changes when you swap providers.

Your Application

MCP Client + Business Logic

↓

LiteLLM Router

Unified interface

↓ swap = change one string ↓

Claude

Anthropic API

GPT-4o

OpenAI API

Llama3

Local GB10

Mistral

Mistral API

── All models share the same tools below ──

📧 Gmail

Pre-built MCP

🗄️ Warehouse

Custom MCP

👁️ YOLO

Custom MCP

📹 Cameras

Custom MCP

      ONE LINE CHANGE
      # Development — free, local

      ACTIVE_MODEL = "ollama/llama3"

      # Production — paid cloud

      ACTIVE_MODEL = "claude-sonnet-4-20250514"

      # Everything else stays identical ↓

      response = litellm.completion(

        model=ACTIVE_MODEL,   # ← only this changes

        tools=your_mcp_tools, # ← stays the same

        messages=history      # ← stays the same

      )

09

API Keys Management

You only set up keys for models you actually use. Never hardcode credentials in code. Store in environment variables locally, Kubernetes Secrets in production.

🖥️ Local Dev
.env file

          # .env — never commit to Git

          ANTHROPIC_API_KEY=sk-ant-xxx

          OPENAI_API_KEY=sk-xxx

          # Ollama = no key needed

☸️ Production
K8s Secret

          kind: Secret

          metadata:

            name: ai-credentials

          stringData:

            anthropic_key: "xxx"

🔐 OAuth
Best practice

          # For Gmail, Drive etc.

          You → OAuth → Google

          Google → Token → You

          # Password never exposed

          # Token revocable anytime

🚫
        Golden Rules:

        ❌ Never hardcode keys in source code  ·  ❌ Never commit .env to Git

        ✅ Environment variables for local  ·  ✅ K8s Secrets for production  ·  ✅ OAuth where possible

10

Claude.ai vs Claude Code

Two different tools for two different jobs. Claude.ai is for thinking and planning. Claude Code runs on your machine and can actually touch your systems.

💬 Claude.ai (this conversation)

Runs in the browser. No access to your local machine or network. Good for learning, planning, writing code and configs.

          ✅ Write code & configs

          ✅ Search the web

          ✅ Plan architecture

          ✅ Explain concepts

          ❌ Cannot SSH to GB10

          ❌ Cannot run commands

          ❌ Cannot access 192.168.x.x

⌨️ Claude Code (terminal tool)

Runs inside your terminal on your machine. Has access to your filesystem, local network, and can execute commands.

          ✅ SSH into GB10

          ✅ Run vLLM / Ollama commands

          ✅ Edit files directly

          ✅ Debug live errors

          ✅ Access local network

          ✅ Deploy to Kubernetes

          npm install -g @anthropic-ai/claude-code

🗺️Recommended workflow: Use Claude.ai for the planning session — learning concepts, designing architecture, writing configs. Switch to Claude Code for the build session — actually deploying, running, and debugging on your GB10 lab.

MCP·01

What is MCP

Model Context Protocol — an open standard that defines how AI models connect to external tools and data sources. Started by Anthropic in Nov 2024. Now governed by the Linux Foundation with backing from OpenAI, Google, and Microsoft.

🔌 MCP is a Protocol

An agreed set of rules for how two parties communicate — just like HTTP, SMTP, HDMI, or USB. MCP defines how AI models talk to tools.

          HTTP   → Browser ↔ Web Server

          SMTP   → Email ↔ Email Server

          HDMI   → Display ↔ Device

          MCP    → AI Model ↔ Tools/Data

🏛️ Standardization

MCP started at Anthropic, then the whole industry joined. Now it belongs to nobody — and everybody.

          Nov 2024 Anthropic launches MCP

          Mar 2025 OpenAI adopts MCP

          May 2025 Microsoft + GitHub join

          Dec 2025 Donated to Linux Foundation

          → Neutral, vendor-independent

🔑 The Key Insight — Two Separate Problems

Layer 1 — Calling the AI

Each vendor has own API format. NOT standardized. Claude=/v1/messages, OpenAI=/v1/chat/completions

Layer 2 — Connecting Tools

THIS is standardized by MCP. Any AI discovers and calls Gmail, Postgres, cameras — same protocol regardless of AI model.

💡MCP solves tool connectivity, not model interoperability. Those are two different problems. MCP only tackles one — but it tackles it well, and the whole industry is behind it.

MCP·02

MCP Server

The MCP Server does all the dirty work — just like Nginx serves HTTP. Claude speaks MCP. The MCP Server translates that into whatever the actual service needs (Gmail API, Postgres queries, RTSP streams).

── Web World ──

Browser

HTTP Client

→

Nginx / Apache

HTTP Server — dirty work

→

Your App / DB

Actual service

── AI World ──

Claude

MCP Client

→

MCP Server

MCP Server — dirty work

→

Gmail / Postgres / YOLO

Actual service

📋 Wire Format

JSON-RPC 2.0 over transport layer. Every message is a structured JSON envelope with a defined schema.

🚌 Transports

STDIO — local process, Claude spawns it. Simple, good for local/lab use.

HTTP/SSE — runs as web server. Better for production and Kubernetes.

🔍 Discovery

Claude asks the MCP server "what tools do you have?" on connection. The server lists all available tools, resources, and prompts automatically.

🎯Write the MCP server once — expose your tools cleanly behind it — and any MCP-compatible AI (Claude, GPT, Copilot) can use it immediately. No custom integration per AI model needed.

MCP·03

Tools, Resources & Prompts

An MCP Server can expose three types of capabilities. Tools are the most common — they let the AI execute actions and query data.

⚙️ Tools

Functions the AI can call — execute actions, query data, trigger events. Most commonly used.

          @mcp.tool()

          def query_stock(sku: str):

            """Get stock level for SKU"""

            # hit your Postgres

            return result

📂 Resources

Data the AI can read — files, feeds, live records, camera streams.

          @mcp.resource(

            "camera://{cam_id}")

          def get_feed(cam_id: str):

            """Get RTSP stream info"""

            return rtsp_info

💬 Prompts

Reusable prompt templates the AI can load for specific tasks.

          @mcp.prompt()

          def defect_report(sku):

            """Analysis template"""

            return f"Analyse defects

            for {sku} last 7d"

🏭 Your Manufacturing Stack as MCP Tools

        @mcp.tool() def query_warehouse(sku: str) → dict:   """Query warehouse stock by SKU"""

        @mcp.tool() def get_defect_rate(line_id: str) → float: """Get YOLO defect rate for line"""

        @mcp.tool() def get_camera_status(cam_id: str) → str:  """Check RTSP feed health"""

        # Now any AI model can call all of this just by talking naturally

MCP·04

Who Hosts What

Only self-host when the data is private to you. Pre-built MCP servers exist for most common cloud services — install and configure, no coding needed.

☁️ You DON'T Host

Vendor hosts the MCP server. You just connect and authenticate.

📧 Gmail — hosted by Google

📁 Google Drive — hosted by Google

📅 Google Calendar — hosted by Google

🐙 GitHub — hosted by GitHub

💬 Slack — hosted by Slack

🏭 You DO Host (Your Lab)

Your private data lives inside your network. You write and host the MCP server.

🗄️ Warehouse DB — your Postgres

👁️ YOLO Inference — your GB10

📹 RTSP Cameras — your cameras

⚙️ Business Logic — custom rules

📊 SKU / Inventory — your data

💡The deciding factor: If the data is yours and private → you host the MCP server. If it's someone else's cloud service → they host it, you just connect.

MCP·05

Connecting to MCP

Three ways to connect — depends on where you're running Claude. The pattern is always the same: get the server, give it credentials, point your client at it.

A

Claude.ai — Zero Code

Settings → Connectors → Gmail → Connect → OAuth with Google. Anthropic hosts the MCP server. You never touch credentials.

Zero code OAuth handled Anthropic hosts

B

Claude Desktop — Local Config

Install MCP server via npm, edit claude_desktop_config.json with credentials, restart.

            { "mcpServers": {

              "gmail": {

                "command": "npx",

                "args": ["-y", "@modelcontextprotocol/server-gmail"],

                "env": { "GMAIL_CLIENT_ID": "xxx" }

              } }}

C

Your Own App — Full Control

Your app acts as the MCP client. Point it to MCP server endpoints. Route tool calls through the AI API programmatically. Model agnostic.

Full control Model agnostic Production ready

MCP·06

MCP Credentials

Where you store credentials depends on where your MCP server runs. The golden rule never changes — never in code, always in environment.

🖥️ Local Dev
.env file

          # never commit to Git

          GMAIL_CLIENT_ID=xxx

          GMAIL_CLIENT_SECRET=xxx

          GMAIL_REFRESH_TOKEN=xxx

☸️ Production
K8s Secret

          kind: Secret

          metadata:

            name: gmail-credentials

          stringData:

            client_id: "xxx"

🔐 OAuth
Best practice

          You → OAuth → Google

          Google → Token → You

          # Password never exposed

          # Token revocable anytime

🚫
        ❌ Never hardcode credentials in source code  ·  ❌ Never commit .env to Git

        ✅ Environment variables for local  ·  ✅ K8s Secrets for production  ·  ✅ OAuth where possible
      

MCP·07

Writing MCP Servers

You only write a custom MCP server for things that are unique to you. Pre-built servers already exist for Gmail, Drive, GitHub, Slack, Postgres. Use pip/npm to install those. Write only what nobody else has.

📦 Pre-built — Just Install

Like npm/pip — don't write what already exists

          # Install pre-built servers

          npm install @modelcontextprotocol/server-gmail

          npm install @modelcontextprotocol/server-gdrive

          npm install @modelcontextprotocol/server-github

          # Then just configure credentials

✏️ Custom — You Write This

Unique to your system — nobody else has these

          # Your warehouse system

          # Your YOLO inference engine

          # Your RTSP camera feeds

          # Your custom business logic

      PYTHON — Simplest MCP Server (3 steps)
      from mcp.server.fastmcp import FastMCP

      # Step 1 — Create the server

      mcp = FastMCP("WarehouseServer")

      # Step 2 — Decorate your functions as tools

      @mcp.tool()

      def query_stock(sku: str) → dict:

        """Query warehouse stock by SKU. Returns units, bay, last updated."""

        # your Postgres query here

        return {"units": 450, "bay": "Bay-3"}

      # Step 3 — Run it

      mcp.run()   # listening for MCP connections

Language	SDK	Install
Python	mcp + FastMCP	pip install mcp
TypeScript/JS	@modelcontextprotocol/sdk	npm install @modelcontextprotocol/sdk
Java/Kotlin	Official MCP SDK	Maven / Gradle
C# (.NET)	Microsoft.McpServer	NuGet

MCP·08

MCP & Token Cost

Every tool definition in your MCP server gets sent as part of the input on every API call — even if the tool is never used. This is important to manage at scale.

Every API Call Carries ALL Tool Definitions

System

MCP Tool Defs ← always sent

Conversation

User

Output

Even if Claude answers "What is 2+2?" — it still reads all your Gmail, Warehouse, YOLO, and Camera tool definitions first. You paid for those tokens.

      TOKEN MATH — 20 Tools Example
      System prompt          ~500 tokens

      20 MCP tool defs      ~4,000 tokens  ← always loaded

      Conversation history  ~2,000 tokens

      User message            ~50 tokens

      ─────────────────────────────────

      Total input           ~6,550 tokens per call

      At 1,000 calls/day  ≈  $0.027/day  ≈  $0.81/month  ← cheap

      At 1M calls/day     ≈  $27/day     ← worth optimising

🎯 Strategy 1
Load selectively

          # Only load relevant tools

          if task == "inventory":

            tools = [warehouse, sku]

          elif task == "email":

            tools = [gmail, calendar]

✂️ Strategy 2
Short descriptions

          # Bad — verbose, ~200 tokens

          """Query the warehouse mgmt
system to retrieve current
stock, history, defects..."""

          # Good — concise, ~30 tokens

          """Get stock, bay, defects."""

🔗 Strategy 3
Group related tools

          # 3 tools = 3× token cost

          get_stock(sku)

          get_bay(sku)

          get_defects(sku)

          # 1 tool = 1× token cost

          query_sku(sku, fields=[...])

AI FundamentalsKnowledge Wiki

API Payload Formats

Message Roles

Temperature

Tokens & Cost

vLLM vs Ollama

LiteLLM

Multiple Models

Model-Agnostic App

API Keys

Claude.ai vs Code

What is MCP

MCP Server

Tools, Resources & Prompts

Who Hosts What

Connecting to MCP

MCP Credentials

Writing MCP Servers

MCP & Token Cost

API Payload Formats

🤖 OpenAI — De Facto Standard

🧠 Claude — Key Difference

🌐 Gemini — Most Different

Message Roles

system — The Director

user — The Human

assistant — The AI's Past Replies

Temperature

Tokens & Cost

📏 What is a Token

📥 Input Tokens

📤 Output Tokens

Context Window — Everything Must Fit

vLLM vs Ollama

Apache

Ollama

Nginx

vLLM

Ollama

vLLM

OpenAI API

LiteLLM

✅ What LiteLLM Does

❌ What LiteLLM Does NOT Do

Running Multiple Models

🐳 Docker vs Model Servers

🖥️ Your GB10 Memory Budget

Model-Agnostic Architecture

Your Application

LiteLLM Router

Claude

GPT-4o

Llama3

Mistral

📧 Gmail

🗄️ Warehouse

👁️ YOLO

📹 Cameras

API Keys Management

🖥️ Local Dev.env file

☸️ ProductionK8s Secret

🔐 OAuthBest practice

Claude.ai vs Claude Code

💬 Claude.ai (this conversation)

⌨️ Claude Code (terminal tool)

What is MCP

🔌 MCP is a Protocol

🏛️ Standardization

🔑 The Key Insight — Two Separate Problems

Layer 1 — Calling the AI

Layer 2 — Connecting Tools

MCP Server

Browser

Nginx / Apache

Your App / DB

Claude

MCP Server

Gmail / Postgres / YOLO

📋 Wire Format

🚌 Transports

AI Fundamentals
Knowledge Wiki

🖥️ Local Dev
.env file

☸️ Production
K8s Secret

🔐 OAuth
Best practice

🖥️ Local Dev
.env file

☸️ Production
K8s Secret

🔐 OAuth
Best practice

🎯 Strategy 1
Load selectively

✂️ Strategy 2
Short descriptions

🔗 Strategy 3
Group related tools