Senior Data/AI Platform Engineer

New Today

The Role: • Design and implement batch and streaming data ingestion pipelines from diverse sources (e.g., files, APIs, Kafka, databases). • Develop real-time and near-real-time data workflows using tools like Apache Flink, Kafka Streams. • Optimize performance for high-volume and high-velocity datasets. • Implement data quality checks and automatic monitoring system to ensure consistently accurate and available data. • Design and manage storage solutions such as Microsoft Fabric, Delta Lake, and Databricks. • Design and implement data virtualization layer to centralize visibility and control of data across diverse sources. • Apply best practices for schema design, partitioning, and data lifecycle management. • Support data discovery and cataloguing in coordination with governance tools. • Work with data scientists, analysts, and business users to understand data needs and translate them into technical solutions. • Partner with DevSecOps and platform engineers to automate deployment and orchestration of data pipelines. • Document data flows, transformations, and quality checks in accordance with governance standard. The Requirements: • Bachelor or Master degree in Computer Science, Engineering, or a related field. • Has at least 5 years of experience in data engineering. • Has at least 1 year in a leadership or senior technical role delivering data engineering projects. • Deep expertise in Spark, Databricks, and data processing frameworks. • Strong knowledge of streaming technologies such as Apache Kafka, Apache Flink, or Azure Event Hub. • Experience working with data lake and/or lakehouse architectures such as Hadoop, Delta Lake, Iceberg and Microsoft OneLake. • Familiarity with data virtualization tool and architecture involving diverse source. • Proficient in Python and SQL. • Familiar with workflow orchestration (e.g., Apache Airflow) and CI/CD principles. • Analytical mindset with a focus on data quality, performance, and maintainability. • Able to work independently and collaboratively in a dynamic environment. • Strong communication and documentation skills to support cross-functional collaboration. • Knowledge of data types across network and IT in telco environment. To Apply: Please send your latest CV in word file to jennifer@quinnox-solutions.com Kindly indicate your availability, current and expected remuneration package. We regret that only shortlisted candidates will be notified. Quinnox Solutions Pte. Ltd. (License Number: 06C3126) Registered EA Personnel (Reg. No.:R1111100)
#J-18808-Ljbffr
Apply
Location:
West Islip, NY, United States
Category:
IT & Technology

We found some similar jobs based on your search