Senior GCP Data Engineer

New Today

Key Responsibilities: • Design, develop, and deploy cloud-based solutions using GCP services such as DataFlow, DataFusion, Dataproc, Composer, GCS, BigQuery, and more. • Build and maintain robust, scalable, and efficient Python applications and scripts that integrate with GCP. • Collaborate with cross-functional teams, including product owners, architects, scrum masters and other stakeholders, to define project requirements, timelines, and deliverables. • Ensure the development adheres to best practices in coding standards, code reviews, and continuous integration/continuous deployment (CI/CD). • Implement and maintain security best practices across all cloud-based solutions to ensure data integrity and compliance with industry standards. • Stay updated with the latest trends and technologies in cloud computing and Python development, and drive the adoption of new tools and techniques that enhance productivity and efficiency.
Required Skills and Qualifications: • Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. • 8+ years of experience in software development with a focus on ETL, Python and cloud technologies, with at least 2 years in a senior/lead developer role. • Extensive hands-on experience with Google Cloud Platform, including services like DataFlow, DataFusion, Dataproc, Composer, GCS, BigQuery, and Pub/Sub. • Advanced proficiency in Python programming and Messaging platforms like Kafka • Strong understanding and experience in both batch and streaming ETLs pipelines • Strong knowledge of CI/CD pipelines and tools such as Jenkins, GitHub, UCD, Harness or similar. • Proficient in Git and branching strategies. • Excellent communication and collaboration skills, with the ability to articulate technical concepts to non-technical stakeholders. • Strong understanding of data systems architecture and data pipelines • Knowledge and Experience in Unix shell scripting
Preferred Skills: • Exposure to software development methodologies such as Agile or Scrum. • Experience and knowledge on orchestration tools such as Airflow, Autosys, SQL Agent etc., • Relevant GCP certifications such as Associate/Professional Cloud Architect or Associate/Professional Data Engineer are highly desirable. • Understanding of SQL Server and SSIS packages a plus
Location:
Charlotte, NC, United States
Category:
Computer And Mathematical Occupations