Build and evaluate benchmark datasets for LLM systems processing patent documents and deal-sourcing data. Responsibilities include reviewing and refining annotations, running blind-test evaluations, measuring extraction reliability, identifying hallucinations and edge cases, cataloging schemas, and preparing quality-acceptance reports. The role offers direct engineering supervision, stakeholder-facing projects, and deployment experience.
NUSX Labs is NUSX's technical arm, turning NUS research into inventions, patents, and licensed deals. We're a small, hands-on team, and every AI system we ship is graded against a ground-truth dataset built by hand: for six months, you're the one building it, and your judgments become the benchmark the next model is measured against. It's judgment work, not data entry, across an LLM pipeline that reads patent documents into structured data, a natural-language query layer, and early-signal deal-sourcing tools. You'll be supervised directly by an engineer on the team, with real stakeholder-facing projects and deployment experience.
Roles and Responsibilities
- Build benchmark datasets used to evaluate model performance, across approximately 23 document types, using Claude and Antigravity to draft and an annotation tool to review and refine.
- Run the blind-test evaluation matrix on vendor deliverables and measure extraction reliability against the ground truth.
- Identify edge cases, structural mismatches, and hallucinations that surface-level review would miss, and document them.
- Review existing system interfaces to determine what each screen stores, and build the gold field and schema catalog.
- Compile findings into quality-acceptance reports used by the team to sign off on deliveries.
- Scope may extend beyond the responsibilities listed above.
Requirements
- Demonstrated experience building and shipping a data pipeline (personal, academic, or hackathon) to completion.
- Working knowledge of Python for processing structured data (JSON/CSV). • Familiarity with evaluation concepts (precision, recall, accuracy) and LLM evaluation metrics.
- Willingness to learn domain knowledge (e.g. patents).
- Basic familiarity with version control (git).
- Willingness to use AI tools (Claude, Antigravity) daily.
- Nice to have: basic SQL; basic cloud experience (GCP/AWS); familiarity with data engineering concepts; comfortable reading and working in English and Chinese.
NUS Enterprise Singapore, Singapore, SGP Office
Singapore, Singapore
NUS Enterprise Singapore Office
Singapore
NUS Enterprise Singapore Office
Singapore
NUS Enterprise Singapore Office
Singapore
Similar Jobs
Cloud • Security • Software • Cybersecurity • Automation
Source, screen, and engage passive Account Executive and other sales candidates across the APJ region. Build diverse talent pipelines and benches, map competitive talent markets, partner with recruiters and hiring managers, assess candidate alignment, deliver a strong candidate experience, and maintain accurate recruiting data. Improve sourcing workflows and outreach methods while managing multiple searches in GitLab’s fully remote, asynchronous environment.
Top Skills:
GitlabGreenhouseLinkedInLinkedin Recruiter
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Support SSD products from development and qualification through high-volume manufacturing. Drive manufacturing test bring-up, product qualification, firmware and test validation, production readiness, yield improvement, and system-level failure analysis. Analyze test data and manufacturing performance, resolve root causes with cross-functional engineering teams, and develop automation, analytics, and AI-enabled solutions to improve product quality and engineering efficiency.
Top Skills:
Ai/MlAsicFirmwareGenerative AiNand FlashNvmePciePythonSemiconductor DevicesSQLSsd
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Lead Micron’s carbon-free electricity strategy across the U.S. and APAC, including roadmap development, financial modeling, project due diligence, regulatory engagement, supplier and developer relationships, and executive reporting. Monitor carbon-free energy markets and technologies, support procurement and public affairs initiatives, and communicate sustainability progress. Use AI tools to improve project management, analysis, forecasting, reporting, and stakeholder communications. Travel domestically and internationally 10–20% as needed.
Top Skills:
Ai ToolsCdpEnvironmental Attribute CertificatesExcelGreenhouse Gas ProtocolMS OfficePowerPointRe100Word
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
