Position Overview
We are seeking an experienced Senior Data / Azure Engineer to support a large-scale cloud modernization initiative. This role will focus on transforming a legacy, on-premises data processing environment into a scalable, cloud-native Azure platform.
The ideal candidate will have deep expertise in Azure serverless technologies, distributed processing, data engineering, and SQL Server optimization. This individual will play a key role in designing and implementing modern data ingestion, processing, monitoring, and validation frameworks while ensuring reliability, scalability, and maintainability. The role may also provide technical leadership to offshore engineering resources and leverage modern AI-assisted development tools to improve engineering productivity and delivery quality.
Key Responsibilities
Azure Application & Data Engineering
- Design, develop, and support Azure Durable Functions utilizing orchestrator and activity patterns.
- Build and maintain solutions using .NET 8 (Isolated Worker) and distributed processing architectures.
- Modernize existing automation processes by converting legacy PowerShell-based workloads into scalable, stateless Azure Functions.
- Implement event-driven architectures leveraging Azure Event Hubs and related Azure services.
- Develop high-performance data ingestion pipelines utilizing watermark-based change detection, parallel processing, and scalable ingestion patterns.
Data Platform Modernization
- Support the migration of legacy data processing systems to Azure-based cloud infrastructure.
- Contribute to the design and implementation of modern cloud-native data processing solutions.
- Analyze and optimize existing workflows to improve scalability, reliability, and operational efficiency.
- Collaborate with architects, engineers, and stakeholders to define future-state cloud data platform capabilities.
Data Quality, Validation & Monitoring
- Design and implement data validation, reconciliation, and quality assurance frameworks.
- Develop parallel-run validation strategies to ensure accuracy during modernization efforts.
- Build comprehensive observability solutions including logging, monitoring, telemetry, and alerting.
- Implement robust error handling, retry mechanisms, and operational support processes using Azure Application Insights and related monitoring tools.
Database Engineering
- Design, optimize, and maintain SQL Server solutions including stored procedures, queries, and database performance tuning.
- Troubleshoot complex database performance issues and implement scalable solutions.
- Ensure data integrity, consistency, and reliability across integrated systems.
Cross-Functional Collaboration & Technical Leadership
- Evaluate downstream dependencies including reports, integrations, data feeds, and business applications.
- Partner with technical and business teams to ensure successful solution delivery.
- Participate in architecture discussions, code reviews, and technical design sessions.
- Document solutions, processes, and technical standards.
- Provide technical leadership and mentorship to engineering resources, including offshore development teams when applicable.
- Coordinate development activities across distributed teams and promote engineering best practices, code quality standards, and knowledge sharing.
- Encourage the effective use of AI-assisted development tools such as GitHub Copilot (or similar) to accelerate code generation, documentation, testing, and engineering productivity.
Required Qualifications
- 7+ years of experience in software engineering, data engineering, or cloud engineering roles.
- Strong experience developing Azure Durable Functions and Azure serverless solutions.
- Expertise with .NET 8 and C# development.
- Experience designing distributed processing and event-driven architectures.
- Hands-on experience with Azure Event Hubs and Azure integration services.
- Strong SQL Server development and performance tuning experience.
- Experience building scalable data ingestion and transformation solutions.
- Experience implementing monitoring, observability, and operational support frameworks.
- Strong understanding of cloud architecture principles and modern software engineering practices.
- Excellent communication, collaboration, and stakeholder management skills.
Preferred Qualifications
- Experience leading cloud migration or modernization initiatives.
- Experience leading and coordinating offshore engineering teams in a distributed development environment.
- Experience with Azure Data Factory, Azure Storage, Azure Service Bus, or related Azure services.
- Knowledge of enterprise data architecture and large-scale data platforms.
- Experience with CI/CD pipelines, DevOps practices, and infrastructure automation.
- Familiarity with data governance, data quality, and validation frameworks.
- Hands-on experience using GitHub Copilot or similar AI-assisted development tools to improve development velocity, documentation quality, testing, and overall engineering efficiency.
Project Overview
This role will support a multi-phase modernization initiative focused on transforming a legacy data processing environment into a scalable, cloud-native platform. Initial efforts will focus on migrating processing workloads to Azure, followed by a broader transition to a modern cloud-based data platform