Skip to content
TECHNICAL DEPTH

How I build and how deep I go

Agentic systems, LLM inference, and data infrastructure — and the path that led here.

01—AGENTIC SYSTEMS

Most “AI agents” are demos. Production systems need the whole stack underneath.

I build across the whole stack — the agentic flows that run a product, and the inference, proxy, and data infrastructure underneath them. Here's what that looks like at the orchestration layer; the infrastructure proof is below.

  1. 01

    Multi-model orchestration

    Routing requests between local models (Qwen, Gemma) and cloud APIs (Claude, GPT) based on task complexity + cost. Production cost savings of 60–80% vs. pure cloud setup.

    → Built into Discury's high-volume agent tasks — owned Qwen 3.6, with frontier cloud models only where task quality demanded the premium.

  2. 02

    Hermes-style tool calling

    No brittle prompt chains. Agent receives a tool set, decides on its own. Requires a strong reasoning model + correct tool granularity. Lessons learned from production deployment.

    → Built into Margly for autonomous multi-step orchestration over merchant order, cost, and ad data.

  3. 03

    MCP-native architecture

    Model Context Protocol as foundation for tool integration. Practical patterns for context management, error recovery, and debugging multi-step agents.

  4. 04

    Production failure modes

    Tool calling loops, hallucinated calls, context window poisoning, infinite retry loops. What I've seen in production and how to fix it.

    → Patterns from building and running multiple production AI-powered web apps.

  5. 05

    Token economics of agentic systems

    Prefill vs generation cost. KV cache reuse. Speculative decoding for agent loops. Practical ROI analyses.

    → Why Advanty and Discury were built on owned inference — measured ROI on M3 Ultra vs. cloud API per task class; local inference automatically failed over to public cloud when unavailable.

I write about this regularly. If you have a production agentic workflow that's bleeding tokens or has failure mode issues, get in touch →

02—PROOF OF DEPTH

How far down the stack I go — when the economics demand it.

When the unit economics demand it, I go all the way down — to the inference layer and to the data/access layer. vllm-mlx (58 merged PRs) and a self-built LTE proxy pool are the two ends of that story: owned inference that made products affordable to run, and my own data-collection infrastructure that keeps public web data reliably reachable. I do MLX out of efficiency necessity, not as a research specialty.

  • vllm-mlx core contributor

    58 merged PRs to open-source LLM inference for Apple Silicon (1,600+ GitHub stars). Primary implementor of Anthropic Messages API (/v1/messages) — the compatibility layer that makes vllm-mlx work with Claude Code and OpenCode.

    Main areas of work:

    • ·KV cache quantization: QuaRot live inference, asymmetric K/V bit quantization for prefix cache, TurboQuant R1 Hadamard rotation for outlier-free MoE weight quantization
    • ·Constrained decoding: JSON schema enforcement, thinking suppression, preamble handling, array-of-objects fixes
    • ·MLLM infrastructure: logits processor context, token duplication fixes, tools/tool_choice in chat templates
    • ·Production reliability: client disconnect detection, in-flight token credit on request abort, generation_tps batch stats
    • ·Streaming: UTF-8-safe incremental decode, tool calls with reasoning parser, leak fixes for Anthropic streaming
    → github.com/waybarrios/vllm-mlx
  • Splash (Inco AI) — co-author of Qwen3.8-Flash-Next support

    An open-source LLM inference engine for Apple Silicon (C++, Objective-C++, Metal; 1,500+ GitHub stars). Qwen3.8-Flash-Next support has been in main since 20 Oct 2026, built on the model computation from my port — the maintainer credits me as co-author on 8 commits: hyper-connection, n-gram embedding and block-indexer kernels, 512-expert routing, and loading GGUFs in Q2_0, BF16, F16, Q8_0 and split files. My pull requests #255 (live request progress and priority in /status) and #354 (lone surrogates as U+FFFD) are merged too.

    → github.com/incoai/splash — co-authorship confirmed
  • Data-access infrastructure (the other end)

    A self-built LTE proxy pool — Raspberry Pis plus consumer MiFi modems on rotating CGNAT residential IPs — that puts scraping traffic on organically residential addresses, with a commercial proxy as hot fallback. Real throughput from production pipelines (10k+ requests/day).

    → Read: the LTE proxy pool
  • Production batch inference

    Apple M3 Ultra 256GB as primary inference machine. Workloads with 9:1 prefill/generation ratio (image classification, content tagging, structured extraction). 274 tok/s sustained throughput on Gemma 4 26B-A4B at concurrency 8.

  • Hardware economics

    Real ROI analyses: M3 Ultra vs RTX PRO 6000 Blackwell for different workload types. Cost-per-token calculations across cloud providers vs. owned infrastructure. Payback period modeling for hardware investments.

  • Local LLM deployment patterns

    vLLM, SGLang, llama.cpp, MLX. When to use which stack. Quantization tradeoffs. Multi-model serving. Auto-scaling on bare metal vs Kubernetes.

03—TIMELINE

The journey from the start

  1. TODAY

    Current focus

    My own products: Self-Healing Parser (data.hilgard.cz), Google SERP & AI Overviews API (serp.hilgard.cz), URL to Markdown API (reader.hilgard.cz) and llamat.com, now in final development. Edge AI on Raspberry Pi + Hailo with my own trained age model. vllm-mlx core contributor, co-author of Qwen3.8-Flash-Next support in Splash. I provide strategic consulting on AI architecture, scraping and system scaling to selected technology companies.

  2. EARLY 2025

    Launch of Advanty, Margly & Discury + vllm-mlx

    Official deployment and launch of three end-to-end AI products as CTO and AI architect: Margly (profitability analytics for online merchants), Discury (customer intelligence platform), and Advanty (AI competitive intelligence). The architectures combine local inference with frontier cloud APIs. In parallel, active role as core contributor to the open-source framework vllm-mlx (LLM inference on Apple Silicon, 58 merged PRs, author of the compatible Messages API layer for tools like Claude Code).

  3. 2024

    R&D and AI stack development

    Intensive development, testing of local models, and design of hybrid infrastructure for the upcoming AI products. Work on reliability of agentic flow orchestration and on token cost optimisation (inference economics).

  4. 2023

    Development and co-founding of Lobot.chat

    Development of an advanced e-commerce chatbot with integrations into platforms like Shopify, WooCommerce, and Magento. As co-founder I continue to actively contribute technically.

  5. 2022

    Full shift to the AI/ML stack

    Deep dive into local LLM models, agentic workflows, and inference optimisation. Bridging the gap between academic research and infrastructure a solo founder can actually run in production.

  6. 2021

    Co-founding of GuruWatch

    Development and launch of a robust scraping and data pipeline for tracking pricing trends across hundreds of e-shops (brands like Lenovo, Infinix, Niceboy). I still act as the project's technical consultant.

  7. SEPTEMBER 2020

    Successful Hosting90 exit

    Full sale of Hosting90 systems s.r.o. to the international WY Group (operator of the Ignum brand). The transaction was publicly announced.

    → hostingy.net
  8. 2002

    Founding of Hosting90

    Founding of a hosting and infrastructure company and 18 years of building it from scratch — from a garage to a 25-person team running its own servers.

Have an interesting project?

I run my own data infrastructure (serp.hilgard.cz, reader.hilgard.cz, data.hilgard.cz). I provide strategic consulting on AI architecture, scraping and system scaling to selected technology companies.

Get in touch →