Subscribe

Building a Skill Ecosystem for AI Agents on Kubernetes

My AI agents went from 8 ready skills to 16 in one session. Here's how I built out their toolbox.

Building a Skill Ecosystem for AI Agents on Kubernetes

My AI agents went from 8 ready skills to 16 in one session. Here's how I built out their toolbox and what I learned about the difference between "installed" and "useful."

The Starting Point

OpenClaw ships with 52 bundled skills. Impressive on paper. In practice, when I checked the skill status, most of them showed "missing." The reason: each skill wraps a CLI tool, and most of those tools aren't in the container image. The gateway knows about gh and gog and gemini, but if the binary isn't on disk, the skill stays grayed out.

Out of the box, I had 8 working skills: the ones that either need no external tooling (like weather, healthcheck, node-connect) or ship with Node.js built-ins. Everything else was dead weight.

The Constraint

This runs on Talos Linux. Immutable OS. No SSH. No package manager. The container image is upstream OpenClaw. I'm not forking it to bake in tools, and I don't control the build pipeline. apt-get install isn't an option because there's no apt. Even if I exec'd into the pod and installed something, it would vanish on the next restart.

So I needed a pattern that:

  • Works without modifying the container image
  • Survives pod restarts and upgrades
  • Doesn't require Talos-level changes

The Solution: PVC as a Toolbox

OpenClaw mounts a 5Gi Longhorn PVC at /home/node/.openclaw. This is where config, sessions, and workspace data live. It persists across pod restarts by design. I created a bin/ directory inside it and added it to PATH via an environment variable in the deployment manifest:

env:
  - name: PATH
    value: "/home/node/.openclaw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

Now anything I drop in /home/node/.openclaw/bin/ is available to the gateway process and to every agent session. Install once, use forever, or at least until I delete it.

Three Install Patterns

Not all tools install the same way. I ended up with three distinct patterns depending on what the tool is built with.

1. Static Go Binaries

The easiest. Go compiles to a single static binary with no runtime dependencies. Download, extract, make executable, done.

# GitHub CLI
wget -qO- https://github.com/cli/cli/releases/download/v2.74.0/gh_2.74.0_linux_amd64.tar.gz \
  | tar xz -C /tmp && mv /tmp/gh_2.74.0_linux_amd64/bin/gh /home/node/.openclaw/bin/

# gog (Google Workspace CLI)
wget -qO- https://github.com/snaviaux/gog/releases/download/v0.12.0/gog_linux_amd64.tar.gz \
  | tar xz -C /tmp && mv /tmp/gog /home/node/.openclaw/bin/

# blogwatcher
wget -qO- https://github.com/nicholasgasior/blogwatcher/releases/latest/download/blogwatcher_linux_amd64.tar.gz \
  | tar xz -C /home/node/.openclaw/bin/

chmod +x /home/node/.openclaw/bin/{gh,gog,blogwatcher}

These are the gold standard for this pattern. No dependencies, no runtimes, no library paths to worry about.

2. npm Packages

OpenClaw's image has Node.js, so npm is available. But npm install -g writes to system directories that don't persist. The fix: install to a PVC-backed prefix and symlink the binary.

# mcporter — MCP bridge tool
mkdir -p /home/node/.openclaw/mcporter
cd /home/node/.openclaw/mcporter && npm install mcporter
ln -sf /home/node/.openclaw/mcporter/node_modules/.bin/mcporter /home/node/.openclaw/bin/

# gemini-cli
mkdir -p /home/node/.openclaw/gemini-cli
cd /home/node/.openclaw/gemini-cli && npm install @anthropic-ai/gemini-cli
ln -sf /home/node/.openclaw/gemini-cli/node_modules/.bin/gemini /home/node/.openclaw/bin/

# Claude Code
mkdir -p /home/node/.openclaw/claude-code
cd /home/node/.openclaw/claude-code && npm install @anthropic-ai/claude-code
ln -sf /home/node/.openclaw/claude-code/node_modules/.bin/claude /home/node/.openclaw/bin/

Slightly messier (each tool gets its own directory for the node_modules tree), but it works reliably. The symlink into bin/ means the gateway finds the binary on PATH just like the Go tools.

3. Auth-Required Tools

Having the binary is necessary but not sufficient. Several tools need credentials, and each one handles auth differently:

  • gh (GitHub CLI): needs a Personal Access Token. Set GH_TOKEN as an env var in the deployment manifest, sourced from a SOPS-encrypted secret. Stateless auth, no persistent config needed.
  • gog (Google Workspace): OAuth2 flow. Needs XDG_CONFIG_HOME pointed to the PVC so the token cache persists. First auth requires an interactive browser flow via kubectl exec.
  • gemini: Google OAuth, same XDG_CONFIG_HOME trick as gog. Set GOOGLE_GENAI_USE_GCA=true to use the Google Cloud Application credentials flow.
  • Claude Code: Anthropic subscription OAuth. Credentials exported from macOS Keychain and placed at a known path on the PVC. CLAUDE_CONFIG_DIR env var points there. One gotcha: ANTHROPIC_API_KEY in the environment shadows the OAuth credentials, so the agent has to unset it when spawning Claude Code sessions.

The XDG_CONFIG_HOME Trick

This was the single biggest unlock. Multiple tools (gog, gh, gemini) respect the XDG Base Directory Specification. They all look at XDG_CONFIG_HOME to decide where to store their config and credentials.

One env var in the deployment manifest:

- name: XDG_CONFIG_HOME
  value: "/home/node/.openclaw"

Now gog writes its OAuth tokens to /home/node/.openclaw/gogcli/, gh writes to /home/node/.openclaw/gh/, and so on. All on the PVC. All surviving restarts. Without this, every tool would write to ~/.config/ inside the ephemeral container filesystem and lose credentials on every pod cycle.

Safety Assessment

Before installing any tool, I checked:

  • Who maintains it? Is it a known project or a random npm package?
  • What does it do at install time? Postinstall scripts can run arbitrary code.
  • What data does it send, and where? Tools that phone home to unknown endpoints don't get installed.
  • What permissions does it need? A tool that wants cluster-admin isn't going in.

One tool I explicitly blocked: clawhub. It's an OpenClaw skill marketplace that lets agents install arbitrary code from a community repository. No guardrails, no review process, no sandboxing. In a production cluster, even a homelab one, that's a non-starter. The agents don't get to decide what code runs on my infrastructure. I do.

Installed vs. Useful

Here's the thing nobody tells you about agent skills: having the tool installed is step 1. Step 2 is the agent knowing how to use it.

Each OpenClaw skill has a SKILL.md file that teaches the agent what commands are available, what the expected patterns are, and what the tool is for. Without that file, the agent has a binary on PATH but no idea what to do with it. It's like giving someone a wrench without telling them what they're building.

The bundled skills handle this automatically: they ship with their own SKILL.md. But for custom tooling, you write it yourself. I built two custom skills as Claude Code skill files:

  • openclaw-ops: a 400+ line infrastructure operations guide. Teaches the agent how to check cluster health, read Flux status, query Longhorn volumes, and troubleshoot common issues. All read-only operations.
  • openclaw-chat: a communication assistant. Helps the agent draft responses, manage Telegram messages, and maintain conversational context across sessions.

The skill files are the real product. The binaries are just prerequisites.

Current State: 16 Ready Skills

After the build-out:

| Skill | What It Does | Install Pattern |

|---|---|---|

| coding-agent | Delegates coding tasks to Claude Code | npm |

| gog | Google Calendar, Gmail, Drive, Docs, Sheets | Go binary + OAuth |

| github | GitHub repo operations | Go binary + PAT |

| gh-issues | GitHub Issues management | Go binary + PAT |

| session-logs | Search past agent sessions | Built-in |

| blogwatcher | RSS feed monitoring | Go binary |

| gemini | Google Gemini AI queries | npm + OAuth |

| mcporter | MCP bridge for external tool servers | npm |

| xurl | URL expansion and metadata | npm |

| healthcheck | HTTP endpoint monitoring | Built-in |

| node-connect | Connect to other OpenClaw nodes | Built-in |

| openai-image-gen | DALL-E image generation | Built-in (API) |

| openai-whisper-api | Audio transcription | Built-in (API) |

| skill-creator | Create new custom skills | Built-in |

| slack | Slack messaging (legacy, migrated to Telegram) | Built-in |

| weather | Weather forecasts | Built-in |

That's 16 out of 52 bundled skills working, plus the 2 custom Claude Code skills on top. Not bad for a constrained environment.

What's Still Blocked

Some skills I couldn't get running, and I'm fine with that for now:

  • summarize: requires a Linux binary that doesn't have a published release
  • model-usage: macOS only, reads local billing data that doesn't exist in a container
  • goplaces: needs a Google Places API key that I haven't provisioned
  • nano-banana-pro: needs an API key for a service I don't use

Not every skill needs to exist. The question isn't "how many can I enable?" but "which ones does the agent actually reach for?"

What I Learned

PVCs are underrated as extension points. The pattern of "mount persistent storage, put binaries there, add to PATH" is simple and powerful. It works for any containerized application where you can control environment variables but not the image.

XDG compliance matters. Tools that respect XDG_CONFIG_HOME are dramatically easier to manage in containers. Tools that hardcode ~/.config or ~/.local make you fight the ephemeral filesystem on every restart.

Agents need documentation, not just tools. A binary without a skill file is like an API without docs. The agent can technically invoke it, but it won't know when to, or how to handle the output. The SKILL.md files are what turn a binary into a capability.

Security review before install, always. When your AI agents can reach the internet and your internal services, every tool you give them expands the attack surface. Check the source, check the maintainer, check what data flows where. Block anything that gives the agent unsupervised install capabilities.

Next Steps

The skill ecosystem is built. Now comes the harder question: which skills actually get used?

I'm going to let this run for a couple of weeks and watch the session logs. Usage patterns will tell me which tools the agent naturally reaches for versus which ones sit idle. Skills that don't get used will get pruned: they're just attack surface without value.

The goal isn't 52 green checkmarks. It's a tight, well-understood toolbox where every skill earns its place.

*Tags: Kubernetes, AI Agents, OpenClaw, Self-Hosting, Homelab*