Senior SRE/Support Engineer
POSITION SUMMARY
- We are seeking a highly skilled DevOps/Site Reliability Engineer (SRE) with strong expertise in market data platforms, Python, production support, and DevOps tools. The role focuses on ensuring reliability, performance, and data quality of market and investment data pipelines. The candidate will handle production support, incident management, and automation, while collaborating with data teams, business stakeholders, and vendors to ensure seamless operations and timely delivery of accurate, high-quality data to downstream users.
POSITION RESPONSIBILITIES
- 1. Incident Resolution & Production Support
- Investigate and resolve job failures, delayed or missing data, application issues, infrastructure problems, and vendor feed disruptions. Coordinate with internal teams and external data vendors to restore services quickly and minimize business impact.
- 2. Data Quality & Validation
- Validate, reconcile, and analyze market and investment data using SQL. Identify root causes of data anomalies before downstream consumption.
- 3. Monitoring & Observability
- Design, implement, and maintain dashboards and alerts using Datadog, Splunk, and AWS services. Automate health checks and enable proactive monitoring for data pipelines and applications.
- 4. Operational Documentation & Runbooks
- Create and maintain accurate runbooks, workflows, and operational documentation. Capture post-incident learnings and drive preventive improvements.
- 5. Stakeholder & Vendor Collaboration
- Partner with business users, data engineering teams, and market data vendors to ensure smooth data delivery and timely issue resolution.
- 6. Data Workflow Reliability
- Monitor pipeline health, enforce SLAs, and ensure consistent end-to-end data flow. Support production data pipelines and investment data applications.
- 7. Continuous Improvement
- Identify opportunities to enhance automation, monitoring, and operational efficiency. Reduce manual intervention and improve overall system reliability.
- 8. Market Data Integration Support
- Manage and troubleshoot third-party data feeds such as SFTP, file drops, and APIs. Ensure data accuracy, completeness, and uninterrupted ingestion.
- SQL/Oracle: Advanced ability to write complex queries for data validation, reconciliation, root-cause analysis, and troubleshooting delayed or incorrect datasets.
- Data Flow & Pipeline Fundamentals: Solid understanding of end-to-end data pipelines, dependencies, data lineage, SLAs, critical path analysis, job failure triage, and data reprocessing.
- GitLab: Hands-on experience with version control, repository management, and CI/CD pipelines for deploying automation and workflows.
- Market Data Knowledge: Familiarity with file drops, SFTP, APIs, streaming feeds, and validation of pricing, reference data, and corporate actions.
- Market Data Vendor Platforms: Experience with Bloomberg, Refinitiv, and similar API-based providers.
- AWS Core Services: Working knowledge of S3, Lambda, Step Functions, and IAM for secure, event-driven processing.
EXPERIENCE AND REQUIRED SKILL SETS
- Python: Strong experience in developing production-ready scripts for data orchestration, processing, monitoring automation, log parsing, alerting, and incident remediation tools.
- SQL/Oracle: Advanced ability to write complex queries for data validation, reconciliation, root-cause analysis, and troubleshooting delayed or incorrect datasets.
- Data Flow & Pipeline Fundamentals: Solid understanding of end-to-end data pipelines, dependencies, data lineage, SLAs, critical path analysis, job failure triage, and data reprocessing.
- GitLab: Hands-on experience with version control, repository management, and CI/CD pipelines for deploying automation and workflows.
- Market Data Knowledge: Familiarity with file drops, SFTP, APIs, streaming feeds, and validation of pricing, reference data, and corporate actions.
- Market Data Vendor Platforms: Experience with Bloomberg, Refinitiv, and similar API-based providers.
- AWS Core Services: Working knowledge of S3, Lambda, Step Functions, and IAM for secure, event-driven processing.