JOB DESCRIPTION

Job Summary:

• We are seeking a senior Production Support Analyst professional to join our team on a contract basis to support a multi-year observability and monitoring modernization initiative.
• The ideal candidate has deep, hands-on experience designing, deploying, and operating the Grafana stack (Grafana, Loki, Tempo, Mimir/Prometheus, Alloy) in enterprise environments.
• They will assess our current monitoring tool landscape, identify overlap and gaps, and assist the technical onboarding of the Grafana stack into our ecosystem.

Key Responsibilities:

• Assist in assessing the current monitoring and observability toolset to identify functional overlap, coverage gaps, and consolidation opportunities, and help document findings and recommendations.
• Assist with the design and implementation of the Grafana stack (Grafana, Loki, Tempo, Mimir/Prometheus, Alloy) for metrics, logs, and traces, including architecture, sizing, high availability, and retention strategy.
• Assist with integrating Grafana with existing enterprise tooling and data sources (e.g., Dynatrace, Splunk, xMatters) and with identity/SSO providers.
• Assist in building and standardizing dashboards, alert rules, SLIs/SLOs, and on-call notification routing in partnership with application and infrastructure teams.
• Support the instrumentation of applications and infrastructure using OpenTelemetry, exporters, and agents; help define telemetry standards and onboarding patterns for new teams.
• Support the migration of existing dashboards, alerts, and monitoring use cases from legacy tools to Grafana, and assist with decommissioning of redundant tools.
• Assist with automating deployment, configuration, and maintenance using infrastructure-as-code and CI/CD (e.g., Ansible, Helm, Terraform, Git-based pipelines).
• Assist with deploying and operating observability components on Kubernetes/Red Hat OpenShift and Linux (RHEL) platforms.
• Help apply security best practices and compliance requirements, including RBAC, secrets management, data retention, and auditability.
• Contribute to runbooks, architecture documentation, and training materials, and assist with knowledge transfer to ensure operational ownership by staff by the end of the engagement.

Qualifications:

• Requires minimum 7 years of job-related work experience in excess of degree requirements, including at least 3 years of hands-on experience with the Grafana stack in production.
• Bachelor’s Degree in Computer Science, Information Systems, or a related field (Required).
• Or a combination of education and experience that provides equivalent knowledge to a major in such fields is required.

Required Skills:

• Proven, hands-on experience implementing and administering Grafana, Loki, Tempo, and Prometheus or Mimir at enterprise scale.
• Strong experience with PromQL, LogQL, and Grafana alerting.
• Experience with OpenTelemetry instrumentation and collectors.
• Experience with Kubernetes and Red Hat OpenShift, including deploying observability components via Helm or Operators.
• Scripting and automation skills (e.g., Bash, Python) and experience with infrastructure-as-code tools (e.g., Ansible, Terraform).
• Experience with CI/CD and Git-based workflows.
• Working knowledge of at least one commercial APM/monitoring platform (e.g., Dynatrace, Splunk) and experience conducting tool rationalization or migration.
• Linux administration skills (RHEL preferred).
• Solid understanding of networking, storage, and security concepts as they apply to monitoring and telemetry pipelines.
• Experience integrating with SSO/identity providers (e.g., Okta, OIDC/SAML) (preferred).
• Strong analytical and problem-solving skills.
• Strong written and verbal communication skills, with the ability to document clearly and present recommendations to technical and management audiences.

TekWissen® Group is an equal opportunity employer supporting workforce diversity.