Backend Engineer - AI Gateway

NJ At First And Then Plano TX

**++▶ Job Details++** **• Job Title:**Backend Engineer - AI Gateway **• Client:**Korean IT Services and IT Consulting **• Working Location:**NJ then Plano TX **• Working Style:**Onsite **• Employment Type:** 1 year Contract / **W-2** **• Salary:**$100,000 - 130,000/year (DOE) **• Benefit:**Insurance and 401K March **• Visa Support:**No **• Working Hours:**9AM to 6PM **• Language:**English **++▶ Position Summary++** We are building an AI Gateway --- a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale. **++▶ The Right Candidate Will Possess++** A proactive, detail-oriented professional with strong communication, ownership, and problem-solving skills who can collaborate effectively across business and technical teams. **++▶ Required Qualifications \& Skills++** • Async Python \& Network Programming (3 yrs) -- Strong hands‑on experience with asyncio, aiohttp/anyio, low‑level TCP/UDP sockets, WebSocket \& SSE servers, TLS handshake, connection‑pooling, keep‑alive, and zero‑copy buffering. • Streaming \& Large‑Payload Handling -- Ability to manage chunked transfer, bidirectional streaming, back‑pressure and high‑throughput data paths for real‑time LLM inference. • Protocol Knowledge -- Working familiarity with HTTP/2 \& HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers. • FastAPI‑Based Backend (3 yrs) -- Design, develop and test production‑grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs. • Containerization \& Kubernetes -- Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service‑mesh (e.g., Istio/Envoy) basics. • Async Messaging \& Event‑Driven Design -- Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back‑off logic. • PostgreSQL \& Vault -- Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management. • Cloud Networking \& Security -- L4/L7 load balancing, TLS termination, reverse‑proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let's Encrypt). • IaC \& CI/CD -- Terraform modules for multi‑env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue‑green deployments. **++Preferred Qualifications++** • Enterprise Security Gateways -- Integration with CASB/DLP/WSS or similar edge‑security platforms; centralized logging \& audit trails. • IAM \& SSO -- Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access \& SCIM provisioning. • AI/LLM Pipelines -- Building prompt‑engineering workflows, PII anonymization, and monitoring Azure OpenAI (or comparable) usage, rate limits \& cost. • API‑Gateway Operations -- Custom webhook/REST integrations, circuit‑breaker patterns, rate‑limiting, auto‑retry and health‑checking. • Observability \& FinOps -- End‑to‑end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost‑optimization and 24/7 incident response (SLA/SLO). **▶ Important Notice** Beware of scams: Cinter Technology will never contact candidates from an email address ending in @cintertechnology.work. Any such email is fraudulent. **No C2C: We do not accept Corp-to-Corp (C2C) arrangements for this position.**