Applied Data Scientist, Finance AI Evaluation & Datasets

Ironwood Digital (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers. Scope of the Role: Financial services is one of the highest-stakes domains for generative AI. Numerical accuracy, regulatory compliance, model risk management, auditability, and customer harm prevention, among other concerns, are the bar for shipping anything real. Ironwood Digital partners with foundation model labs, banks, asset managers, fintechs, and other enterprise AI teams building LLMs, multimodal systems, and AI agents for financial workflows. As an Applied Data Scientist, Financial AI Evaluation & Datasets, you own the design, measurement quality, and domain validity of the datasets used to train, fine-tune, evaluate, and monitor financial-domain LLMs, vision-language models, multimodal document models, and AI agents. You bring financial-domain fluency and data science rigor: you can read a risk policy, financial statement, or customer transcript, among other financial-services documents; turn it into a measurable dataset and evaluation specification; define what correct, grounded, compliant, and safe mean for the use case; and produce evidence that sophisticated financial-services customers, model-risk teams, and AI governance stakeholders can trust. This role has a special emphasis on unstructured and multimodal financial data — PDFs, scanned documents, spreadsheets, charts, call transcripts, and other mixed-document workflows where text, numbers, visuals, and metadata all matter. You will work in a pod with a Technical Solutions Architect (scopes the engagement), an Applied Research Scientist (shapes evaluation methodology), an AI/ML Research Engineer (builds training and evaluation infrastructure), and Language Data Scientists (run annotation at scale), making sure what the team produces is domain-valid, statistically defensible, compliant, auditable, and useful for evaluation and post-training. What You’ll Own: Translate customer goals — such as improving financial reasoning, building an eval suite for earnings-call summarization, or evaluating an AML/fraud copilot — into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria. Design training and evaluation datasets across the financial AI surface: financial QA, filings and earnings analysis, credit and underwriting, fraud/AML investigation, and compliance, among other financial workflows. Foreground unstructured and multimodal financial data in dataset design — PDFs, scanned statements, tables, charts, and call transcripts — used by analysts, advisors, compliance reviewers, and operations teams. Design datasets and evaluations for retrieval-augmented and source-grounded systems: evidence citation and faithfulness to source documents, data freshness, conflict resolution across sources, and failure modes caused by incomplete or incorrectly parsed context. Evaluate agentic and workflow-integrated financial AI systems: tool use, retrieval, transaction boundaries, escalation behavior, and controls that prevent unsafe or unauthorized actions. Develop evaluation methodology that goes beyond surface accuracy — numerical consistency, hallucination rates on high-risk claims, refusal and escalation appropriateness, robustness under ambiguity, and fairness across protected or sensitive customer segments. Define sampling strategies, label schemas, and adjudication workflows with Language Data Scientists and finance SMEs; write annotation guidelines that make subjective finance-domain judgments explicit, calibratable, and auditable. Build the statistical and ML tooling that makes large financial datasets trustworthy: stratified sampling across products, markets, and modalities; bias analysis; leakage detection; and distribution shift checks, among other reliability checks. Build evaluation and dataset-quality evidence to support financial-services model risk management: assumptions, limitations, validation results, and residual risks, packaged as reproducible evidence. Partner with the AI/ML Research Engineer to instrument datasets into training, evaluation, and monitoring pipelines — rubric-grounded LLM-as-judge prompts, regression suites, and continuous monitoring. Own data quality end-to-end, from intake through delivery: PII handling, provenance tracking, versioning, and modality-specific QA checks. Reason about financial workflow context: where AI outputs enter analyst, advisor, compliance, risk, or customer-facing workflows; what evidence a reviewer ne