How I build and how deep I go
Agentic systems, LLM inference, and data infrastructure — and the path that led here.
Most “AI agents” are demos. Production systems need the whole stack underneath.
I build across the whole stack — the agentic flows that run a product, and the inference, proxy, and data infrastructure underneath them. Here's what that looks like at the orchestration layer; the infrastructure proof is below.
- 01
Multi-model orchestration
Routing requests between local models (Qwen, Gemma) and cloud APIs (Claude, GPT) based on task complexity + cost. Production cost savings of 60–80% vs. pure cloud setup.
→ Built into Discury's high-volume agent tasks — owned Qwen 3.6, with frontier cloud models only where task quality demanded the premium.
- 02
Hermes-style tool calling
No brittle prompt chains. Agent receives a tool set, decides on its own. Requires a strong reasoning model + correct tool granularity. Lessons learned from production deployment.
→ Built into Margly for autonomous multi-step orchestration over merchant order, cost, and ad data.
- 03
MCP-native architecture
Model Context Protocol as foundation for tool integration. Practical patterns for context management, error recovery, and debugging multi-step agents.
- 04
Production failure modes
Tool calling loops, hallucinated calls, context window poisoning, infinite retry loops. What I've seen in production and how to fix it.
→ Patterns from building and running multiple production AI-powered web apps.
- 05
Token economics of agentic systems
Prefill vs generation cost. KV cache reuse. Speculative decoding for agent loops. Practical ROI analyses.
→ Why Advanty and Discury were built on owned inference — measured ROI on M3 Ultra vs. cloud API per task class; local inference automatically failed over to public cloud when unavailable.
I write about this regularly. If you have a production agentic workflow that's bleeding tokens or has failure mode issues, get in touch →
How far down the stack I go — when the economics demand it.
When the unit economics demand it, I go all the way down — to the inference layer and to the data/access layer. vllm-mlx (58 merged PRs) and a self-built LTE proxy pool are the two ends of that story: owned inference that made products affordable to run, and my own data-collection infrastructure that keeps public web data reliably reachable. I do MLX out of efficiency necessity, not as a research specialty.
vllm-mlx core contributor
58 merged PRs to open-source LLM inference for Apple Silicon (1,600+ GitHub stars). Primary implementor of Anthropic Messages API (/v1/messages) — the compatibility layer that makes vllm-mlx work with Claude Code and OpenCode.
Main areas of work:
- ·KV cache quantization: QuaRot live inference, asymmetric K/V bit quantization for prefix cache, TurboQuant R1 Hadamard rotation for outlier-free MoE weight quantization
- ·Constrained decoding: JSON schema enforcement, thinking suppression, preamble handling, array-of-objects fixes
- ·MLLM infrastructure: logits processor context, token duplication fixes, tools/tool_choice in chat templates
- ·Production reliability: client disconnect detection, in-flight token credit on request abort, generation_tps batch stats
- ·Streaming: UTF-8-safe incremental decode, tool calls with reasoning parser, leak fixes for Anthropic streaming
Splash (Inco AI) — co-author of Qwen3.8-Flash-Next support
An open-source LLM inference engine for Apple Silicon (C++, Objective-C++, Metal; 1,500+ GitHub stars). Qwen3.8-Flash-Next support has been in main since 20 Oct 2026, built on the model computation from my port — the maintainer credits me as co-author on 8 commits: hyper-connection, n-gram embedding and block-indexer kernels, 512-expert routing, and loading GGUFs in Q2_0, BF16, F16, Q8_0 and split files. My pull requests #255 (live request progress and priority in /status) and #354 (lone surrogates as U+FFFD) are merged too.
→ github.com/incoai/splash — co-authorship confirmedData-access infrastructure (the other end)
A self-built LTE proxy pool — Raspberry Pis plus consumer MiFi modems on rotating CGNAT residential IPs — that puts scraping traffic on organically residential addresses, with a commercial proxy as hot fallback. Real throughput from production pipelines (10k+ requests/day).
→ Read: the LTE proxy poolProduction batch inference
Apple M3 Ultra 256GB as primary inference machine. Workloads with 9:1 prefill/generation ratio (image classification, content tagging, structured extraction). 274 tok/s sustained throughput on Gemma 4 26B-A4B at concurrency 8.
Hardware economics
Real ROI analyses: M3 Ultra vs RTX PRO 6000 Blackwell for different workload types. Cost-per-token calculations across cloud providers vs. owned infrastructure. Payback period modeling for hardware investments.
Local LLM deployment patterns
vLLM, SGLang, llama.cpp, MLX. When to use which stack. Quantization tradeoffs. Multi-model serving. Auto-scaling on bare metal vs Kubernetes.
The journey from the start
- TODAY
Current focus
My own products: Self-Healing Parser (data.hilgard.cz), Google SERP & AI Overviews API (serp.hilgard.cz), URL to Markdown API (reader.hilgard.cz) and llamat.com, now in final development. Edge AI on Raspberry Pi + Hailo with my own trained age model. vllm-mlx core contributor, co-author of Qwen3.8-Flash-Next support in Splash. I provide strategic consulting on AI architecture, scraping and system scaling to selected technology companies.
- EARLY 2025
Launch of Advanty, Margly & Discury + vllm-mlx
Official deployment and launch of three end-to-end AI products as CTO and AI architect: Margly (profitability analytics for online merchants), Discury (customer intelligence platform), and Advanty (AI competitive intelligence). The architectures combine local inference with frontier cloud APIs. In parallel, active role as core contributor to the open-source framework vllm-mlx (LLM inference on Apple Silicon, 58 merged PRs, author of the compatible Messages API layer for tools like Claude Code).
- 2024
R&D and AI stack development
Intensive development, testing of local models, and design of hybrid infrastructure for the upcoming AI products. Work on reliability of agentic flow orchestration and on token cost optimisation (inference economics).
- 2023
Development and co-founding of Lobot.chat
Development of an advanced e-commerce chatbot with integrations into platforms like Shopify, WooCommerce, and Magento. As co-founder I continue to actively contribute technically.
- 2022
Full shift to the AI/ML stack
Deep dive into local LLM models, agentic workflows, and inference optimisation. Bridging the gap between academic research and infrastructure a solo founder can actually run in production.
- 2021
Co-founding of GuruWatch
Development and launch of a robust scraping and data pipeline for tracking pricing trends across hundreds of e-shops (brands like Lenovo, Infinix, Niceboy). I still act as the project's technical consultant.
- SEPTEMBER 2020
Successful Hosting90 exit
Full sale of Hosting90 systems s.r.o. to the international WY Group (operator of the Ignum brand). The transaction was publicly announced.
→ hostingy.net - 2002
Founding of Hosting90
Founding of a hosting and infrastructure company and 18 years of building it from scratch — from a garage to a 25-person team running its own servers.
Have an interesting project?
I run my own data infrastructure (serp.hilgard.cz, reader.hilgard.cz, data.hilgard.cz). I provide strategic consulting on AI architecture, scraping and system scaling to selected technology companies.
Get in touch →