LLM Threat Modeling for Self-Hosted Open Source Models with CAST AI and kimchi.dev

Self-hosting changes the ownership model only when the infrastructure is actually yours

LLM threat modeling is useful because running an LLM application is not mainly a question of whether the model can answer requests. A demo proves the model can respond. A reviewable production design has to prove something harder: access is controlled, data paths are understood, runtime behavior is observable, cost is bounded, and someone owns rollback when the system misbehaves.

Self-hosting open source LLMs can give an enterprise more direct control over deployment choices, model endpoints, infrastructure, and data paths. That statement only holds for the parts the enterprise actually operates. If model artifacts are deployed to vendor-hosted infrastructure, or if requests pass through a vendor gateway, the architecture becomes hosted or hybrid for those paths. The threat model has to say which mode is in use before it assigns controls.

For this review, use four deployment modes:

1. Enterprise-managed Kubernetes. The enterprise runs model-serving infrastructure in its own Kubernetes clusters. Kubernetes controls, runtime patching, networking, secrets, observability, and incident response are enterprise responsibilities.
2. Vendor-hosted kimchi.dev infrastructure. The supplied kimchi.dev custom-model documentation describes taking model artifacts stored in Google Cloud Storage and deploying them to kimchi.dev hosted infrastructure via Deploy a Custom Model with Kimchi. In this mode, vendor data handling, runtime ownership, network paths, retention, audit, and rollback behavior require direct validation.
3. Hybrid or proxy-to-provider. The supplied kimchi.dev documentation describes Kimchi Proxy as routing requests to different LLM providers through an OpenAI-style request/response contract, with provider coverage described in Supported LLM providers. In this mode, the enterprise may operate some application and Kubernetes layers while model access or provider routing passes through kimchi.dev and/or external providers.
4. Unverified self-hosted kimchi.dev-on-Kubernetes mode. The supplied evidence does not verify kimchi.dev components running inside the reader’s Kubernetes cluster, a kimchi.dev control-plane/data-plane split, cluster permissions, tenancy model, or a self-hosted kimchi.dev deployment topology. Treat this as unknown until approved architecture evidence exists.

This article assumes enterprise-managed Kubernetes for the infrastructure layer and evaluates kimchi.dev only for the roles supported by the supplied public documentation: LLM routing/provider integration, an OpenAI-style API contract, and custom-model deployment to kimchi.dev hosted infrastructure. It evaluates CAST AI for Kubernetes automation, optimization, autoscaling, cost monitoring, and governance-support surfaces supported by CAST AI public documentation, including Getting Started, the Autoscaler Setup Guide, GPU Instances, Workload Autoscaling Overview, Cost Monitoring Overview, and governance-related documentation such as Audit log, Single Sign-On (SSO), Role-Based Access Control (RBAC), Service Accounts, and Kubernetes permissions.

It does not claim that either vendor supplies the whole enterprise LLM governance model.

Public risk frameworks help with vocabulary. The framing here borrows from public sources such as OWASP guidance for LLM application risks, NIST AI risk-management guidance, MITRE ATLAS, and OpenTelemetry’s Generative AI semantic conventions. Because approved link IDs were not supplied for those external sources, they are referenced as background vocabulary rather than cited as outbound links.

The method is simple:

1. Identify the failure mode.
2. Decide where the control belongs.
3. Define the evidence artifact a reviewer can inspect.
4. Assign an owner.

Evidence legend: separate verified capabilities from recommended controls and open questions

Use this legend throughout the review:

Label	Meaning	How to use it
Verified product capability	Supported by supplied public documentation.	Describe narrowly, without expanding beyond the source.
Recommended enterprise control	A control pattern the enterprise should consider, but not a verified kimchi.dev or CAST AI capability.	Put it in the architecture review, assign an owner, and implement it in the appropriate layer.
Validation question	A required answer before production because supplied evidence does not verify the capability or topology.	Ask the vendor or internal owner; do not publish as a product claim until approved.
Needs review	Security, privacy, legal, partner, or product-sensitive area.	Route for product/partner, security, privacy, and legal review before publication or production use.

This framing matters because LLM architectures often blur product capability, policy aspiration, and operational responsibility. The model is not the control plane. Vendors are diligence inputs. The enterprise still owns the operating model unless a contract and validated architecture say otherwise.

Review architecture: separate the LLM workflow layer from the Kubernetes operating layer

This is an architecture-review map, not a deployable kimchi.dev-on-Kubernetes reference architecture. It deliberately avoids implying that kimchi.dev has a verified self-hosted Kubernetes deployment mode.

User / client
   |
   v
Enterprise app entry point
(identity, app auth, coarse authorization, request classification)
   |
   v
LLM gateway or service layer
(endpoint policy, tenant routing, request shaping, tool policy, logging decisions)
   |
   +--> kimchi.dev documented roles, when used
   |       - Kimchi Proxy provider routing
   |       - OpenAI-style request/response contract
   |       - custom model deployment to kimchi.dev hosted infrastructure
   |
   +--> enterprise-managed model-serving runtime, when self-hosted
           - inference server
           - model artifacts
           - GPU/runtime dependencies
           - rollback path

Kubernetes operating layer
   - namespaces, service accounts, network policy, pod security
   - requests/limits, HPA, node pools, GPU scheduling, secrets references
   - observability collectors and audit/cost integrations

CAST AI documented roles, when used
   - Kubernetes automation and optimization
   - cluster autoscaling and GPU-optimized instance support
   - workload request scaling and HPA management
   - Kubernetes cost visibility by concepts such as cluster and namespace

Evidence systems
   - traces, logs, metrics, eval artifacts, deployment records
   - access reviews, policy decisions, cost records, incident tickets

Layer	Role in the architecture review	Evidence status
1. User and application entry point	Establish user identity, application identity, authentication, coarse authorization, request classification, and rate-limit intent before traffic reaches model-serving components.	Recommended enterprise control.
2. LLM gateway or service layer	Enforce endpoint selection, request shaping, tenant-aware routing, policy checks, logging decisions, and tool permissions outside the model. Inference: for many enterprises, this is where the “golden path” for LLM applications should be packaged, consistent with platform engineering operating model guidance.	Recommended enterprise control.
3. kimchi.dev workflow/model-access layer	Supplied public documentation describes Kimchi Proxy as routing requests to different LLM providers and using an OpenAI-style request/response contract. Supplied documentation also describes custom-model deployment from Google Cloud Storage to kimchi.dev hosted infrastructure. See <a href=”https://kimchi.dev/docs/getting-started” title=”Getting Started	Kimchi Docs”>Getting Started, <a href=”https://kimchi.dev/docs/supported-providers” title=”Supported LLM providers
4. Model serving and runtime	Inference servers, model artifacts, GPU requirements, image provenance, patching, and rollback strategy.	Recommended enterprise control when enterprise-managed; validation question when vendor-hosted.
5. Kubernetes infrastructure	Namespaces, node pools, workload identity, network policy, pod security, secret references, resource requests and limits, admission controls, and cluster permissions. For primitives, see Kubernetes core concepts.	Recommended enterprise control.
6. CAST AI optimization layer	CAST AI documentation describes Kubernetes automation, optimization, autoscaling, GPU-optimized instances, workload request scaling, HPA management, and cost breakdowns by Kubernetes concepts such as cluster and namespace. See <a href=”https://docs.cast.ai/docs/autoscaler” title=”Autoscaler Setup Guide	Cast AI Docs”>Autoscaler Setup Guide, <a href=”https://docs.cast.ai/docs/gpu” title=”GPU Instances
7. Observability and audit	Correlate application traces, model IDs, runtime metrics, retrieval events, tool calls, policy decisions, Kubernetes events, GPU signals, cost records, and audit logs.	Recommended enterprise control. Vendor audit schemas and prompt-retention behavior require validation.
8. Secrets and data	Model artifact stores, vector stores, document systems, provider keys, service accounts, key management, and retention rules.	Recommended enterprise control plus needs security/privacy/legal review for sensitive data.

Some Kubernetes operating-layer controls are not automatically present because an enterprise uses Kubernetes or CAST AI. Network policy depends on the CNI and policy configuration. Admission control depends on the chosen admission controller or policy engine. Workload identity depends on cloud-provider IAM, Kubernetes service accounts, and integration design. Secret handling depends on the selected secret manager, encryption configuration, access policy, and rotation process. Image provenance depends on registry, signing, scanning, and promotion choices. Treat CAST AI as an infrastructure automation and optimization input; do not treat it as a substitute for these platform design decisions.

Concrete request-flow example: what reviewers should be able to reconstruct

A useful review exercise is to trace one request from user action to cost record. The example below is intentionally caveated: it is a design pattern, not a claim that kimchi.dev or CAST AI provides every control.

1. User request enters the enterprise app. The app authenticates the user, resolves tenant or workspace context, applies coarse authorization, and classifies the request. These are enterprise controls.
2. The app calls the LLM gateway or service layer. The gateway applies endpoint policy: allowed model IDs, environment, tenant route, logging decision, rate-limit intent, and tool policy. If the organization uses kimchi.dev, the gateway may call Kimchi Proxy using the documented OpenAI-style request/response contract and provider-routing model described in Supported LLM providers. Data-handling behavior for prompts, outputs, metadata, and logs must be validated directly before production.
3. RAG retrieval is authorized before context is attached. The retrieval service checks document-level or tenant-level authorization before adding chunks to the prompt. Prompt instructions are not accepted as the authorization boundary.
4. The request reaches a model endpoint. In a self-hosted path, the endpoint is an enterprise-managed model-serving runtime on Kubernetes. In a kimchi.dev hosted custom-model path, the supplied documentation describes deployment to kimchi.dev hosted infrastructure from artifacts stored in Google Cloud Storage via Deploy a Custom Model with Kimchi. The review must identify which path applies.
5. Kubernetes schedules and runs the workload. For enterprise-managed serving, the platform team owns namespace boundaries, pod security, network policy, service accounts, secrets, runtime patching, requests/limits, and node isolation. CAST AI may be used for cluster autoscaling, GPU instance support, workload request scaling, HPA management, and cost visibility based on its documentation.
6. Observability and audit evidence are joined. The team should be able to correlate request ID, user or app identity, tenant, model ID/version, retrieval sources, tool calls, policy decision, Kubernetes deployment version, autoscaling events, and Kubernetes cost allocation where available.
7. Rollback has a named owner. If the model, prompt template, retrieval source, routing rule, or Kubernetes runtime misbehaves, the runbook identifies who can freeze routing, roll back, disable a tool, rebuild an index, or change capacity policy.

If the team cannot reconstruct this path for a representative request, the system is not ready for production governance even if the model endpoint works.

Threat-modeling table: risk area → failure mode → control → audit artifact

The table below is the reusable unit. It is not a compliance guarantee and it is not a statement that kimchi.dev or CAST AI provides each control. Unless marked otherwise, controls are recommended enterprise controls.

Risk area	Failure mode	Control status	Recommended control	Evidence/audit artifact	Owner
Identity and access	Shared or overly broad access to model endpoints	Recommended enterprise control; vendor SSO/RBAC claims need review if used	SSO/RBAC at enterprise entry points, service identities, tenant-aware authorization, least-privilege service accounts	Identity policy, access review, service account inventory, audit logs	Security + platform
Data and prompts	Sensitive data enters prompts, outputs, logs, traces, or eval datasets	Recommended enterprise control; needs security/privacy/legal review	Data classification, logging policy, redaction, retention limits, sampling, security review	Logging policy, retention policy, redaction tests, sampled audit records	Security + app team
Network boundaries	Model endpoints are exposed broadly, or tools/providers have unrestricted egress	Recommended enterprise control	Private ingress, network policy, egress policy, API gateway rules, environment separation	Architecture review, policy manifests, connectivity tests	Platform
Model access	Unapproved model or version is used; tenants cross model endpoints	Recommended enterprise control; kimchi.dev routing is verified only at the proxy/provider level	Registry or approval gate, allowed-model list, version pinning, rollback path	Model card, approval record, deployment manifest, rollback log	MLOps
Retrieval/RAG	User retrieves documents they are not authorized to see	Recommended enterprise control	Document-level ACLs, tenant-scoped indexes, query-time authorization, source attribution, retrieval logging	ACL tests, retrieval traces, index ownership records, incident samples	Data + security
Tools and agents	LLM triggers unsafe action through a tool	Recommended enterprise control	Tool allowlists, scoped credentials, sandboxing, human approval for high-impact actions, tool-call audit logs	Tool permission matrix, sampled traces, approval records	App + security
Runtime and Kubernetes	Container escape, over-privileged pod, GPU contention, unpatched runtime	Recommended enterprise control; CAST AI autoscaling/GPU features are verified infrastructure inputs	Namespace boundaries, pod security, node isolation, requests/limits, image scanning, patch runbooks	Admission reports, image provenance, vulnerability reports, Kubernetes events	Platform
Supply chain	Untrusted model artifact, container, or dependency reaches production	Recommended enterprise control	Trusted sources, checksum/signature checks where available, image scanning, dependency review, promotion gates	SBOM if available, scan report, artifact provenance, approval ticket	MLOps + security
Observability and audit	Team cannot reconstruct who called which model, with what retrieval context, through which tool, at what cost and latency	Recommended enterprise control; vendor audit/log schemas require validation	Trace correlation across app, gateway, runtime, Kubernetes, cost, and audit systems	Traces, logs, metrics, eval reports, cost records, policy decision logs	Platform + SRE
Cost and capacity	Unbounded GPU spend, starvation of critical workloads, capacity shortfall	Recommended enterprise control; CAST AI cost monitoring verified for Kubernetes concepts such as cluster and namespace	Quotas, labels, autoscaling policy, workload rightsizing, anomaly response, capacity runbooks	Namespace/cluster cost report, scaling events, budget alerts, incident tickets	FinOps + platform

CAST AI public documentation includes governance and security surfaces such as Single Sign-On (SSO), Role-Based Access Control (RBAC), Service Accounts, Kubernetes permissions, Audit log, and Security reports. Treat those pages as diligence inputs, not blanket product assurances or compliance claims. Final wording that describes governance or security capabilities should go through CAST AI product/security approval.

Where governance should live: do not make the model your policy engine

The model can follow instructions. That does not make it a reliable enforcement boundary.

Identity and authorization should be enforced before the request reaches the LLM gateway, and again when retrieval or tools are invoked. Model access policies belong in the gateway/workflow and deployment process: allowed model IDs, approved versions, environment-specific endpoints, and tenant routing rules. Retrieval governance belongs near the data plane: document ACLs, tenant-scoped indexes, source attribution, and retrieval-time authorization checks.

Tool governance belongs at the application/tool layer: scoped credentials, tool allowlists, policy checks, human approval for high-impact actions, and traceable tool calls. Runtime controls belong in Kubernetes: pod security, network boundaries, workload identity, secrets, image provenance, resource policy, and node isolation. Cost governance belongs in both engineering workflow and infrastructure: requests/limits, quotas, autoscaling policy, allocation reporting, and anomaly runbooks.

This is the same design instinct behind good platform engineering and FinOps: put deterministic controls where they can be inspected and changed through reviewable workflows. For related operating-model context, see DevOps, platform engineering, and SRE distinctions, Platform Engineering vs DevOps, Kubernetes FinOps: Build Cost Ownership Into Engineering, Kubernetes Cost Allocation: Spend by Team & Namespace, and cost-aware architecture review.

kimchi.dev’s verified role: LLM routing/workflow surface, not an assumed Kubernetes control plane

The verified kimchi.dev role in this review is specific.

The supplied public documentation describes Kimchi Proxy as routing requests to different LLM providers through an OpenAI-style request/response contract. The supported-provider documentation describes the proxy as a gateway integrating with LLM providers. The custom-model documentation describes taking model artifacts from Google Cloud Storage and deploying them to kimchi.dev hosted infrastructure.

That supports evaluating kimchi.dev in the LLM workflow, routing, and model-access part of the design. It does not support treating kimchi.dev as a self-hosted Kubernetes control plane or as the whole enterprise governance control plane unless additional capabilities are validated.

Questions to validate before production:

• Which deployment modes are supported for your environment?
• Where do the control plane and request-processing paths run?
• What data passes through kimchi.dev, including prompts, outputs, metadata, retrieval context, files, and logs?
• How are provider credentials stored, encrypted, rotated, and revoked? Needs security review.
• What SSO/RBAC and tenancy models are available? Needs product/security review.
• What audit events are emitted, and can they be exported?
• Can prompts and outputs be excluded, redacted, sampled, or retained under policy? Needs privacy/legal/security review.
• How are custom model artifacts promoted and rolled back?
• How are model/version allowlists enforced, if supported?
• What network paths exist among enterprise apps, kimchi.dev, hosted models, provider APIs, and artifact storage?

CAST AI’s verified role: Kubernetes optimization, autoscaling, cost visibility, and governance support

CAST AI’s verified role is the Kubernetes operating layer, not the LLM policy brain.

CAST AI documentation describes Kubernetes automation, optimization, autoscaling, security-related reporting, and cost monitoring. For LLM workloads, the relevant control surfaces are infrastructure-heavy: node provisioning, GPU placement, workload requests, HPA behavior, permissions, and operational evidence.

The CAST AI autoscaler documentation describes scaling clusters by adding right-sized nodes. The GPU documentation describes GPU-optimized instances. The workload autoscaling documentation describes automatically scaling workload requests, while the HPA documentation describes management of native Kubernetes HPA objects. The cost monitoring documentation describes cost breakdowns by Kubernetes concepts such as cluster and namespace. CAST AI also publishes partner content on why it positions itself for running AI/LLM workloads in Kubernetes, Kubernetes FinOps, and namespace/team cost allocation.

The architectural implication: pair CAST AI capabilities with explicit platform policies. Define which namespaces may request GPU nodes, which workloads may autoscale, what labels are mandatory, who can change autoscaling policy, and what incident path applies when spend or capacity behaves unexpectedly.

Do not claim guaranteed savings, performance, uptime, or compliance from this architecture. Those outcomes depend on workload behavior, cloud constraints, configuration, operating discipline, and contractual details.

Tenant, environment, model endpoint, secrets, and data-path separation

Start with environment separation. Development, staging, and production should have distinct model endpoints, credentials, artifact promotion paths, network policies, and logging retention rules. This is an architecture recommendation, not a verified kimchi.dev feature.

Then separate tenants by identity, namespace, data index, model endpoint policy, and cost labels. Do not rely on prompt instructions to maintain tenant boundaries.

Separate model endpoints by sensitivity and operational profile. Experimentation models, internal assistants, production customer-impacting models, batch inference jobs, and tool-enabled agents should not all share identical controls.

Separate secrets by scope and lifecycle: provider keys, model artifact storage access, vector database credentials, tool credentials, CI/CD tokens, and CAST AI service accounts require different owners, rotation policies, and review cadences. Any final permission design involving CAST AI governance surfaces needs product/security review.

Finally, separate sensitive data paths. Prompts, completions, RAG chunks, embeddings, source documents, trace payloads, eval datasets, and logs may have different retention and access requirements. Legal, privacy, and security review is required before production handling of sensitive prompts, outputs, retrieval data, credentials, or audit logs.

Observability for LLM platforms must join app traces, model behavior, Kubernetes events, and cost signals

For each LLM request, an enterprise team should be able to reconstruct user/application identity, tenant, model ID and version, endpoint, latency, retrieval sources, tool calls, policy decisions, errors, and deployment version. Where token counts or request-level cost proxies exist, they are useful, but do not assume CAST AI provides LLM token accounting. The verified CAST AI cost evidence supports Kubernetes cost breakdowns, such as cluster and namespace.

For runtime operations, teams need Kubernetes events, pod restarts, requests/limits, GPU node placement, autoscaling events, HPA behavior, saturation indicators, queue depth where available, and model-server health.

For governance, teams need audit events for model promotion, policy changes, access changes, secret rotation, provider/model registration, deployment changes, and autoscaling or cost-policy changes. CAST AI audit-log documentation should be treated as a diligence input for the infrastructure layer and needs product/security review before being used as a published capability claim. kimchi.dev observability, prompt retention, and audit schemas were not verified in the supplied materials; validate them directly.

OpenTelemetry Generative AI semantic conventions are an emerging interoperability area. A practical, caveated implementation choice is to define stable internal field names first: model ID, provider, request ID, tenant, application ID, deployment version, token or cost proxy, retrieval source IDs, tool call IDs, policy decision ID, and trace/span correlation IDs. Map those fields to OpenTelemetry Generative AI conventions where they fit, but do not let an evolving convention decide what your incident review requires. Also be careful with payload capture: logging prompts and outputs creates security and privacy risk. Define redaction, sampling, retention, access, and incident-review policies before collecting sensitive content.

Operating model: production evidence, incident runbooks, and vendor diligence

A governance-first architecture review should end with operating evidence, not a prettier diagram. The following tables group the final sign-offs: what must be true before promotion, what runbooks must exist, and what questions must be answered by vendors and internal owners.

Production evidence: what must be true before a model or LLM app reaches production

Gate	Required evidence	Owner	Blocking questions
Model approval	Source, license review, artifact provenance, version pinning, limitations, rollback target	MLOps	Can we explain why this model/version is allowed?
Security	OWASP-style threat model, RAG authorization tests, tool review, logging review, scoped secrets, approved network paths	Security	Can unauthorized users retrieve or trigger anything they should not?
Runtime	Image provenance, vulnerability scan, resource requests/limits, GPU placement, autoscaling policy, readiness/liveness behavior	Platform	Can we patch, scale, and roll back safely?
Data/RAG	Document ownership, index tenancy, ACL enforcement, embedding retention, deletion workflow	Data + security	Can retrieved context be tied to authorization and source?
Observability	Dashboards, traces, logs, eval reports, audit events, cost reports	SRE/platform	Can we reconstruct an incident?
FinOps	Allocation labels, quotas or guardrails, budget alerts, scaling-policy owner	FinOps	Who pays for this endpoint, and who can change scaling behavior?
Operations	On-call path, severity definitions, rollback authority, vendor escalation path	Engineering leadership	Who acts at 02:00, and what evidence do they need?
Publication/vendor review	Approved kimchi.dev deployment and data-handling wording; approved CAST AI governance/security wording; security, privacy, and legal review for prompts, outputs, logs, credentials, auditability, retention, and incident-response language	Product/partner + legal/security/privacy	Are public claims supported by approved evidence and review?

Model license review and legal obligations vary by model and organization; this table is not legal advice.

Incident runbooks: write them before LLM paths become critical infrastructure

Scenario	First 15 minutes	Evidence to collect	Rollback/escalation path
Model rollback	Identify current and previous approved version; freeze routing changes	Deployment manifests, eval deltas, routing logs	Roll back endpoint rule; record approval
Prompt/data leakage	Freeze affected logs/traces; identify data paths; disable unsafe tool or retrieval path	Prompt/output samples under policy, access logs, retention config	Security incident process; rotate secrets if affected
RAG authorization failure	Disable affected index/connector; identify impacted documents/users	Retrieval traces, ACL tests, index ownership	Rebuild index; validate ACL enforcement
Unexpected spend	Identify namespace, workload, and model endpoint where possible; correlate traffic and scaling events	Cost reports, scaling events, requests/limits, HPA changes	Pause noncritical jobs; adjust quotas/policies
GPU capacity shortfall	Identify pending GPU workloads; prioritize production endpoints	Scheduling events, node scaling history, queue depth	Fallback model, queueing, capacity escalation
Degraded inference quality	Compare evals, model/runtime changes, retrieval changes, prompt-template changes	Eval reports, traces, deployment changes	Roll back model, prompt, retrieval source, or tool
Runtime/security issue	Assess vulnerable images/dependencies; isolate affected workloads	Image scan, Kubernetes events, network exposure review	Patch/redeploy; incident review
Vendor/tooling incident	Determine whether CAST AI, kimchi.dev, cloud provider, or internal platform owns next action	Evidence package, timestamps, affected endpoints	Escalate through defined support path

Do not imply CAST AI or kimchi.dev owns incident response unless the contract says so. Vendor escalation is a runbook input; ownership remains an internal operating decision.

Vendor diligence questions: answer these before relying on the architecture

For kimchi.dev

• What deployment model is supported?
• Where do the control plane and data/request paths run?
• What data passes through kimchi.dev?
• How are provider keys stored, encrypted, rotated, and revoked?
• What tenancy, RBAC, and SSO options exist?
• What audit events are emitted?
• Can prompts and outputs be excluded, redacted, sampled, or retained under policy?
• How are custom model artifacts promoted and rolled back?
• How are model allowlists enforced, if supported?

For CAST AI

• Which Kubernetes permissions are required?
• Who can change autoscaling and workload policies?
• How are GPU workloads discovered and scaled?
• What guardrails exist for GPU node provisioning?
• How are workload requests adjusted?
• How is cost broken down by Kubernetes concepts such as cluster and namespace?
• What audit logs and security reports are available, and what product/security review is needed before relying on them?

For internal teams

• Which namespaces, node pools, service accounts, and network policies define tenant and environment boundaries?
• Who owns model-serving runtime patching?
• What is the golden path for LLM apps?
• How are secrets and artifact stores managed?
• What counts as sensitive prompt data?
• Can prompts and outputs be logged?
• Which evals block production?
• What latency, quality, retrieval, or tool-call regressions trigger incident response?
• How are GPU costs allocated internally, and who approves autoscaling policy changes?

The main lesson: self-hosted or hybrid open source LLM systems are not governed by the model. They are governed by the control planes around the model. kimchi.dev can be evaluated, based on supplied public documentation, as an LLM routing/workflow surface and as a path for custom-model deployment to kimchi.dev hosted infrastructure. CAST AI can be evaluated, based on public documentation, as a Kubernetes optimization, autoscaling, cost visibility, and governance-support layer. The enterprise team still owns the threat model, the evidence, and the operating model unless validated architecture and contracts say otherwise.