About the role
We are looking for a Senior Systems Test Engineer to take the lead on quality across the agents we build and the systems they connect to. This is a hands-on job. You will decide how we test, do the harder testing yourself, and your call is the one that decides whether an agent or a release goes live with a client. You will report to the Director and work with our developers, AI engineers and client teams from discovery through to post-deployment review. You will also be the person our less experienced staff come to.
Key responsibilities
Own the test plan for every agent build, release and system change, deciding what gets tested and what does not, working out which environments and test data you need, setting the entry and exit criteria, and giving the project a realistic estimate of how long testing will take
Go through discovery notes, process maps, business requirements and agent design specifications early, before anyone writes code or a prompt, and build the mapping from requirements to test cases
Write the test conditions, test cases and test scripts used across projects, including invalid input, boundary values, negative cases, ambiguous customer requests and error handling
Plan and run functional, business process, system, integration and regression testing across the agents and the platforms they connect to, including booking, point-of-sale, CRM, practice management and document systems
Find, reproduce and diagnose faults in systems and application code, working to testing protocols, guidelines and quality standards
Build and maintain automated tests, knowing which checks are worth automating, keeping scripts current as agents, prompts, APIs and interfaces change
Keep the regression suite current by adding scenarios from defects, updating or dropping cases that no longer match the system
Set up and manage test environments and test data, including sandbox client systems and de-identified data sets
Plan and run non-functional testing, including load, response time, cost per interaction, failover and recovery scenarios
Design and run evaluation suites for AI agents with realistic scenarios drawn from each client's real workflows, with clear pass criteria for task completion, accuracy, tone and correct escalation
Test how agents plan and act across multiple steps, whether they choose the right tool, call it with the right parameters, handle failed or slow API responses, recover from errors and stop when they should
Check agent output for factual accuracy and grounding, confirming that answers, summaries and drafted documents are supported by the client's own source material
Run adversarial and safety testing, including prompt injection, attempts to extract confidential or personal information, off-topic and abusive inputs, and guardrail bypass attempts
Coordinate User Acceptance Testing with clients and support them while they run it, carry out pre-change validation, cutover checks and post-implementation testing
Review and assess the systems architecture behind each agent, looking at how a proposed change will land on the client's interfacing systems
Produce the test summary and defect reporting that management and clients use at each gate
Create and maintain technical documentation for agents and environments, including test procedures, evaluation sets, technical reports and specifications
Look at test results, evaluation scores and defect trends over time to pick up recurring faults and put forward improvements to the way we test and release
About you
Bachelor degree or higher in Information Technology, Computer Science, Software Engineering or something close to it, or at least three years of relevant work experience
At least three years of hands-on systems or software testing, including having owned the testing for a release or a project from start to finish
Real depth across functional, integration, system, regression and user acceptance testing, in a place where several applications talk to each other
Comfortable in test management and defect tracking tools such as Jira, Azure DevOps, TestRail or similar, and keep traceability from requirement through to result
Have built or maintained automated tests, and can tell which ones are worth automating and which are not
Experience testing APIs and integrations, using tools such as Postman or scripted tests, and enough SQL to check data yourself
Exposure to testing AI or machine learning features, such as chatbots, LLM-based applications or AI agents, including evaluation design, prompt injection or output quality checks
Some scripting ability, ideally Python, and some exposure to load, performance and failover testing
Working knowledge of cloud platforms, web applications, networking basics and common business systems
Strong fault finding, methodical, evidence led, and not satisfied until you know the actual cause
Write clearly, with documentation and test reports usable by someone who is not technical, including clients
Full and unrestricted working rights in Australia
About us
Monster Hub is an Australian AI agency based in Spring Hill, Brisbane. We build custom AI agents for hospitality and professional services businesses: restaurants, law firms, accountants and financial advisers. Our agents take customer enquiries and bookings, manage orders, support rostering, onboard clients, review and summarise documents, track deadlines and prepare reports. Every one of them is built around a real client's operation and plugged into the systems that client already runs, so when an agent gets something wrong, a customer or a client notices it.
Benefits
Permanent, full time, 38 hours a week, Monday to Friday
Real ownership of quality across AI agents running in live businesses, with the authority that goes with it
Hands-on experience in agentic AI testing, one of the fastest growing areas of software quality
Support for training and certification
Spring Hill office, close to Brisbane CBD and public transport