Sr. Data Engineer

2 Days Old

Overview
The Senior Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and modern data architectures that support business intelligence, analytics, and data science initiatives. This role plays a critical part in implementing robust ETL processes, optimizing medallion architecture, and ensuring observability and reliability across the data infrastructure. The ideal candidate will have deep expertise in cloud platforms-particularly Azure and Databricks-and a passion for delivering high-quality, secure, and performant data solutions.
Responsibilities
Design, build, and maintain scalable and efficient ETL/ELT pipelines. Implement and optimize medallion architecture (bronze, silver, gold layers) in a cloud data lake environment. Develop and manage data workflows using Databricks (Spark, Delta Lake). Ensure data quality, integrity, and governance across all stages of the data lifecycle. Collaborate with data scientists, analysts, and business stakeholders to identify data needs and deliver effective solutions. Leverage Azure Data Services (e.g., Azure Data Factory, Azure Synapse, Azure Data Lake) for data integration and orchestration. Establish monitoring and observability for data pipelines using tools such as Azure Monitor, Datadog, Prometheus, Grafana, and OpenTelemetry. Proactively detect and resolve data pipeline issues and performance bottlenecks. Work with CI/CD pipelines and version control systems for data engineering workflows. Maintain documentation and contribute to best practices in data engineering. Administer and manage cloud data platform resources including storage accounts, compute clusters, and networking components. Configure and manage access controls, roles, and permissions to ensure secure data access. Monitor and optimize cloud resource usage to ensure cost-efficiency and performance. Ensure platform security, compliance, and adherence to organizational policies and industry standards. Collaborate with cloud infrastructure teams to support platform scalability and reliability. Qualifications
Required Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 5+ years of experience in data engineering or a similar role. Strong proficiency in ETL/ELT development and data pipeline orchestration. Hands-on experience with Databricks and Apache Spark. Solid understanding of medallion architecture and Delta Lake. Proficiency in SQL, Python, or Scala. Experience with Azure cloud services (preferred) or AWS (nice to have). Familiarity with monitoring and observability tools for data infrastructure. Preferred Experience with infrastructure-as-code tools (e.g., Terraform, ARM templates). Knowledge of data governance and security best practices. Exposure to real-time data processing (e.g., Kafka, Azure Event Hubs). Familiarity with DevOps practices and tools (e.g., GitHub Actions, Azure DevOps).
Must be authorized to work in the U.S. without the need for employment-based visa sponsorship now or in the future. The Company will not sponsor applicants for U.S. work visa status for this opportunity (no sponsorship is available for H-1B, L-1, TN, O-1, E-3, H-1B1, F-1, J-1, STEM OPT, OPT, CPT or any other employment-based visa.)
What we offer you: Multiple benefit plans to suit your needs Paid Time Off 401K Opportunities for advancement Positive and respectful work environment where diversity is valued Generous employee discounts on dining, retail, amusements, and hotels Community volunteer opportunities
Disclaimer : The above information is intended as a high-level overview of the responsibilities and qualifications for the position and does not represent the full job description which will include all responsibilities, skills, duties, requirements, and working conditions associated with the job. For more information or to see the full job description, contact the Human Resources department at the location in which you are applying.
#li-EG1
Apply
Location:
Houston, TX, United States
Category:
Computer And Mathematical Occupations

We found some similar jobs based on your search