Backbase
Principal AI Evaluation Engineer
We opened the doors of our Business Development Centre (BDC) in the heart of Hyderabad in December 2020. Young at heart, the BDC is a wonderful space that strengthens Backbase’s presence in the APAC region. The physical presence of great talent and support also makes way for long-term relationships with customers.
Nestled in the middle of Hyderabad, the state-of-the-art building sits amidst other global brands. The location offers stunning views of hills and lakes, while nearby there is a selection of restaurants, traditional bazaars, and other cultural hotspots.
As a a Principal AI Evaluation Engineeryou will be leading the evaluation efforts in our AI-powered SDLC team. You will own the evaluation strategy for AI assistants and agentic workflows, ensuring they are reliable, observable, and safeguarded with strong guardrails. Beyond hands-on work, you will mentor engineers, lead triage and reporting, and make evaluation a cornerstone of release decisions.
What you'll do
- Define and lead the evaluation strategy and roadmap for AI-powered SDLC core product
- Build and oversee evaluation pipelines and guardrails.
- Build and maintain evaluation datasets (synthetic and real project data) to benchmark AI behavior.
- Analyze evaluation results, identify gaps, and produce clear, actionable reports for engineering and product stakeholders.
- Build a culture of innovation and excellence, encouraging continuous improvement and adoption of best practices in AI evaluation and deployment.
- Collaborate with cross-functional teams to integrate evaluation insights into development.
Who you are
- Strong understanding of software engineering principles and the software development lifecycle (SDLC).
- Hands-on experience with test design, test management, observability, and data analysis.
- Proficiency in Python (or another scripting language) for automating evaluations.
- Familiarity with AI Agent evaluation methods (faithfulness, answer relevancy, contextual accuracy, tool correctness).
- Excellent analytical and problem-solving skills.
- Strong communication and collaboration abilities, able to work with cross-functional teams and stakeholders.
- Demonstrated ability to mentor engineering talent, fostering collaboration and technical excellence.
- (Nice to have) Experience with evaluation frameworks, RAG systems, or agentic workflows.