AI Systems Analyst III

• Analyze incoming AI use cases to determine technical feasibility, data availability, and appropriate risk tiering (green/yellow/red).
• Perform detailed data lineage and quality assessments to ensure training/RAG datasets meet governance standards for accuracy and PII/PHI protection.
• Maintain the "AI Agent Registry & Catalog," documenting agent capabilities, API dependencies, and ownership within the Azure APIM and Mesh architecture.
• Draft technical requirements and "Definition of Ready" artifacts for the Platform Engineering team, ensuring a smooth handoff from CoE to Engineering.
• Support the AI Engineering Committee (AIEC) by documenting architectural fit and identifying potential technical debt in proposed solutions.
• Monitor and aggregate telemetry data on token usage, cost, and error rates to support the "Value Analyst" in ROI reporting.
• Assist in the creation of "Data Dictionaries" and "Knowledge Graphs" required to ground RAG (Retrieval Augmented Generation) pipelines.
• Validate that yellow layer automations (UiPath, Databricks) utilize approved MCP connectors and do not bypass API gateways.
• Collaborate with Data Governance to tag and classify datasets specifically approved for LLM training or fine-tuning.
• Track the lifecycle of AI models from "Pilot" to "Production" to "Retirement" in the enterprise inventory system.
• Support the configuration of "Model Routers" by analyzing performance benchmarks across different LLMs (GPT-4 vs. Llama) for specific tasks.
• Create process flow diagrams for "Agentic Workflows" to visualize how multiple agents interact and hand off tasks.
• Perform any other job related duties as requested.

• Bachelor's degree in Information Systems, Computer Science, Data Analytics, or related field required required
• Equivalent years of relevant work experience may be accepted in lieu of required education
• Five (5) years in Systems Analysis, Data Analysis, or Technical Product Ownership required
• Experience documenting technical requirements for data-intensive applications, API integrations, or cloud platforms required
• Background in Healthcare Payer data (Claims, Member, Provider) is preferred

• Proficiency in SQL and data profiling, capable of analyzing datasets for quality, lineage, and PII/PHI exposure
• Understanding of API specifications (REST/JSON) and microservices architecture to document agent dependencies
• Ability to translate business needs into technical user stories and "Definition of Ready" criteria for engineering teams
• Familiarity with Data Governance tools (e.g., Purview, Collibra) and metadata management
• Knowledge of RAG (Retrieval Augmented Generation) concepts to help define knowledge base requirements

• CBAP (Certified Business Analysis Professional) or PMI-PBA preferred
• Azure Data Fundamentals certification preferred

• General office environment; may be required to sit or stand for extended periods of time
• Travel is not typically required

Compensation Range:

CareSource takes into consideration a combination of a candidate’s education, training, and experience as well as the position’s scope and complexity, the discretion and latitude required for the role, and other external and internal data when establishing a salary level. In addition to base compensation, you may qualify for a bonus tied to company and individual performance. We are highly invested in every employee’s total well-being and offer a substantial and comprehensive total rewards package.

Compensation Type (hourly/salary):

Organization Level Competencies

• Fostering a Collaborative Workplace Culture

• Cultivate Partnerships

• Develop Self and Others

• Drive Execution

• Influence Others

• Pursue Personal Excellence

• Understand the Business