IT Business Analyst (AI/ML)

• We are seeking an experienced Business Analyst to support the development and implementation a comprehensive AI platform featuring third-party LLM integration, enterprise data source connectivity, and agentic orchestration capabilities. This role will be critical in translating complex business requirements into technical specifications for AWS Professional Services and ensuring successful delivery of this advanced AI platform.

Key Responsibilities:

Magic Platform Requirements Analysis

• Lead requirements gathering workshops with up to define business objectives

• Analyse and document requirements for the three core components:

o Third-Party LLM Integration Framework (e.g. OpenAI, Gemini, WOG Maestro)

o Enterprise Data Source Integration with RAG pipeline (e.g. Sharepoint, Teams, JIRA, Confluence)

o Agent Orchestration System with autonomous task completion

• Define functional requirements for enterprise data source integrations based on identified use cases

• Specify requirements for up to agentic bots aligned with business scenarios

Enterprise Data Integration Analysis

• Analyse current enterprise systems and define secure connector requirements

• Document requirements for Backend ELT Pipeline processes and user-friendly frontend interfaces

• Define secure data transfer workflows and data security implementation requirements

• Specify RAG pipeline requirements including query processing, retrieval mechanisms, and response generation

• Analyse document processing needs for various formats and vector embedding requirements

Agent Orchestration & Automation

• Define requirements for backend infrastructure supporting agent management

• Analyse workflow orchestration needs and tool integration requirements

• Document multi-step workflow automation requirements and autonomous task completion specifications

• Work with stakeholders to identify connected data source requirements

• Define performance metrics and monitoring requirements for agent systems

Administrative & Monitoring Requirements

• Specify user management requirements including CRUD operations, role assignment, and access control

• Define bot configuration interface requirements and deployment controls

• Document data source management requirements including connection configuration and status monitoring

• Analyse dashboard requirements for bot usage tracking, conversation metrics, and user engagement

• Define custom metric requirements and reporting capabilities

Testing & Quality Assurance

• Define comprehensive test scenarios for bot functionality using Litmus or Moonshot frameworks

• Specify requirements for LLM-as-a-Judge evaluation systems

• Document performance testing requirements including load testing and stress testing parameters

• Define remediation support requirements for UAT and performance testing phases

• Analyse security testing requirements including VAPT and vulnerability assessment needs

Stakeholder Management & Communication

• Facilitate coordination between client’s stakeholders and technical teams

• Support project management activities including sprint planning and milestone tracking

• Participate in bi-weekly status reporting and stakeholder alignment meetings

• Manage expectations regarding deliverables, timelines, and technical constraints

• Support knowledge transfer training sessions for up to 5 participants per session

• 5+ years of experience in business analysis for complex technology projects
• Minimum 2 years of experience with AI/ML or LLM implementation projects
• Experience in Talend is a must have
• Proven experience working with AWS cloud services and enterprise system integrations
• Experience with government or public sector digital transformation projects preferred

AI & Technical Expertise

• Strong understanding of Large Language Models (LLMs) and their business applications
• Knowledge of RAG (Retrieval-Augmented Generation) architectures and vector databases
• Familiarity with AI governance frameworks and ethical AI considerations
• Understanding of enterprise data integration patterns and API management
• Experience with cloud platforms (AWS preferred) and containerisation technologies
• Knowledge of AI safety testing frameworks and guardrail implementations

Platform Specific Skills

• Experience with multi-modal AI systems and agent orchestration concepts
• Understanding of prompt engineering and chain-of-thought implementations
• Knowledge of AI model evaluation metrics and performance optimization
• Familiarity with enterprise security requirements and compliance frameworks
• Experience with agile methodologies and MLOps practices

Core Competencies

• Excellent workshop facilitation and stakeholder management skills
• Strong analytical thinking and complex problem-solving abilities
• Ability to translate technical AI concepts for non-technical stakeholders
• Detail-oriented approach to requirements documentation and traceability
• Experience with requirements management tools and agile project management
• Understanding of government security clearance processes and information handling protocols

Desirable Experience

• Knowledge of Singapore government digital services and WOG systems
• Familiarity with AI testing frameworks (Litmus, Moonshot, Sentinel)
• Experience with vector databases and embedding technologies