AI Forward Deployed Engineer Interview Questions
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
51 small wins to finish your pathNext question →

Q43HardSystem design

The customer requires everything to run inside their own VPC or on-prem, with data in Oracle databases and a mainframe. Design the deployment.

30-second answerSay your answer out loud first, then reveal.
Inside the customer's VPC, users sign in via SSO to an app and orchestration layer on Kubernetes that calls self-hosted model serving, a vector index, read-only data connectors to an Oracle replica and the mainframe, and the customer's SIEM, while vendor releases reach an internal registry only by controlled transfer.

Considerations

  1. Models: pick open-weight models that fit their GPUs and meet the eval thresholds; quantize if needed; plan capacity (Q36 of the LLM guide); update models via the internal registry.
  2. Data access: prefer replicas or change-data-capture over hitting production Oracle; mainframe through existing integration layers (MQ, CICS web services, nightly extracts) owned by their team.
  3. Air-gapped operations: all images, packages and weights mirrored internally; signed releases; no telemetry leaving the network unless approved.
  4. Security: network policies, secrets in their vault, encryption, integration with their IAM and audit tools.
  5. Operations: runbooks, health checks, backups, upgrade procedures performed by or with their ops team; support access via their approved remote access process.
  6. Evals inside the perimeter: the eval harness runs locally; results shared as aggregate reports.

Trade-offs: lower model quality vs frontier APIs (quantify it with evals), slower update cycles, higher operational burden. Make these explicit to the sponsor.

PreviousThe pilot succeeded in one business unit. Create a 90-day plan to scale to five business units.

Slow is fine. Stopping is the only problem.