Data Engineer *Eligible for signing bonus of up to $10k!*

*This position is eligible for a signing bonus of up to $10k!*

Overall responsibility:

Responsible for developing, implementing, and operating stable, scalable, low cost solutions to source data from client systems into the data lake, data warehouse and end-user facing BI applications. Responsible for ingestion, transformation and integration of data to provide a platform that supports data analysis, data enrichment and data science as well as making data operationally available for analysis.

Essential Duties:

  • Build and maintain scalable automated data pipelines. Support critical data pipelines with a highly scalable distributed architecture - including data ingestion (streaming, events and batch), data integration, data curation.
  • Deploy, automate, maintain and manage Azure cloud-based production system, to ensure the availability, performance, scalability and security of productions systems.
  • Good architectural understanding to build and ensure customer success when building new solutions and migrating existing data applications on Azure platform.
  • Conduct full technical discovery, identifying pain points, business and technical requirements, “as is” and “to be” scenarios.
  • Design and arrangement of scalable, highly attainable, and fault tolerant systems on Azure platform.
  • Ownership and responsibility for end-to-end design and development, testing, release of key components.
  • Understand and implement best practices in management of data, including master data, reference data, metadata, data quality and lineage.
  • Experience with code versioning tools and a command of configuration management concepts and tools, CI-CD including DevOps.
  • Other duties as assigned.

Minimum Education/Experience:

Bachelor’s degree (BA or BS) from an accredited college or university plus a minimum of four (4) years of experience in the specific or related field. Or High School Diploma or equivalent plus a minimum of eight (8) years of experience in the field.

Other Experience:

  • Expert level SQL knowledge and experience.
  • Expert level experience with Python/Pyspark.
  • Experience with Microsoft SQL Server Integration Services, or other ETL/ELT tools.
  • Experience with streaming integration and cloud-based data processing systems such as Streamsets, Kafka, and Databricks.
  • Experience with Snowflake/SnowSQL preferred.
  • Hands-on knowledge of cloud-based data warehouse solutions.
  • Experience with cloud architecture and solutions.
  • Experience with cloud-based data storage such as Azure data lake and blob storage
  • Experience with Microsoft SQL Server database and Oracle database.
  • Experience with Microsoft Windows and Linux virtual servers.
  • Experience implementing data pipelines.
  • Experience deploying developed code.
  • Moderate skill in Power BI.