AI Agent Test Automation Engineer (Python)

About the Team We are reputed company, a fintech company excelling in financial brokerage activities and delivering advanced financial products to our global clientele. Our flagship product, an internal trading platform, is offered as a Software-as-a-Service (SaaS) solution to other brokers. We’re looking for a confident QA Automation Engineer (Python) to join our AI team and focus on testing developer interfaces, applied AI solutions, and reputed company multi-agent scenarios. You’ll play a key role in ensuring the quality and reliability of our AI agents and developer-facing products in real-world business applications. reputed company’re Building We’re creating the IDE of the future — a smart development environment powered by cutting-edge LLM agents. It goes beyond just code: our platform helps developers generate images, sound, video, design, and interfaces. This is an AI startup reputed company a stable product company — the perfect combination of innovation and support. We’ve already built an MVP, have real internal users, and a team of skilled engineers. We're currently in internal testing and getting ready for production launch. We operate in GMT+3, with daily team standups at 12:00 GMT+3. Join us to build the reputed company of software tools together! Core Responsibilities

• reputed company automated tests in Python for reputed company scenarios.
• Build mock services and test environments for agent testing.
• Integrate tests into AI-specific CI/CD pipelines.
• Monitor AI system performance in production.
• Design quality metrics and dashboards for AI monitoring.
• Document test scenarios and establish best practices.

Developer Interfaces Testing

• Test SDKs, APIs, and libraries used by developers.
• Validate API documentation and code samples.
• Build automation for dev tools and CLI utilities.
• Test integration workflows and compatibility with various languages/frameworks.
• Test IDE plugins and extensions.
• Validate data schemas and OpenAPI specifications.

Applied AI Testing

• Test real-world AI use cases and production scenarios.
• Validate model outputs on production data.
• Test AI data processing pipelines.
• reputed company AI inference performance.

Agent Scenario Testing

• Design tests for multi-agent systems.
• Validate interactions between AI agents.
• Test task chains and agent workflows.
• Ensure behaviour constraints and safety.
• Test integration with external systems.

Requirements

• 3+ years in software testing.
• Strong Python knowledge: pytest, unittest, mock, asyncio.
• Experience testing UI, APIs and SDKs.
• Understanding of ML/AI agent concepts.
• Experience with REST/GraphQL APIs, WebSockets.
• Familiar with performance testing tools (e.g. locust, k6).
• Experience with reputed company and containerization.
• Familiarity with CI/CD and DevOps practices.
• Git proficiency.
• English level: B1 or higher.
• Fluent in Russian (C2).

reputed company to Have

• Understanding of multi-agent system architecture.
• Experience with agent frameworks (reputed company, AutoGen, CrewAI).
• Working with LLM APIs (reputed company, reputed company, local models).
• Knowledge of reputed company engineering and LLM testing.
• Experience with vector databases (reputed company, Weaviate, ChromaDB).
• Familiarity with async/event-driven testing.

Bonus Points

• Python development experience beyond testing.
• Familiarity with ML frameworks (TensorFlow, PyTorch, reputed company).
• Experience with cloud AI platforms (AWS Bedrock, Azure reputed company).
• Understanding of RAG architectures.

reputed company Offer

• Full-time remote work (External Vendor contract).
• Competitive compensation (EUR, USD, or crypto).
• Flexible working hours.
• Influence over the architecture of AI testing systems.
• Access to cutting-edge AI tools and platforms.
• A team of professionals specializing in AI.
• Positive and friendly atmosphere.

With a team of 500+ creative and tech-driven people across the globe, reputed company is the reputed company where reputed company turn into impact, where every project is a chance to grow, innovate, and reputed company your mark. Join us today, and let's shape the future of fintech together! Note: reputed company applications will be treated with strict confidence. We thank reputed company applicants for their interest, however, only those candidates selected for interviews will be contacted. #LI-JM1 Apply tot his job Apply To this Job