Data Engineer

New Yesterday

Responsibilities and Qualifications:
· Build data applications using Python, SQL, and AWS;
· Build ETL pipelines for batch data ingest from APIs, databases and other sources as needed
· Apply knowledge of basic principles, methods and practices to simple and moderately complex assignments;
· Proactively identify and implement opportunities to automate tasks and develop reusable frameworks;
· Adhere to standard methodologies for coding, testing and designing reusable code/component;
· Work as a data engineer within the Information Security team that uses several Data, Search and AWS technologies;
· Implement standardized, automated operational and quality control processes to deliver accurate and timely data and reporting to meet or exceed SLAs;
· Collaborate with the other engineering team members to ensure all services are reliable, maintainable, and well-integrated into existing platforms;
· Review functional and technical designs to identify areas of risk and/or missing requirements;
· Provide technical write-ups and drawings to promote the proposed solutions.
· 4-5 years experience AWS cloud and AWS services such as S3 Buckets, Lambda, API Gateway, SQS queues;
· 4-5 years experience with batch job scheduling and identifying data/job dependencies;
· 4-5 years experience with data engineering using AWS platform and Python;
· Familiar with AWS Services like EC2, S3, Redshift/Spectrum, Glue, Athena, RDS, Lambda, and API gateway;
· Familiar with software DevOps CI/CD tools, such Git, Linux, and Shell Script;
· Ability to learn quickly, be organized and detail oriented;
· Experience in Software Engineering and Development;
· Understanding of database schema design;
· Proficient in Python
Partner with engineers, program managers and management to understand data needs
Design, build and deploy efficient data pipelines to third party and internally developed systems
Craft and build reports / dashboards using visualization tools like Tableau, Python and R.
Experience analyzing data to discover opportunities and address gaps
Experience with Python, Tableau, SQL and Excel
Experience with working in cloud or on-prem big data technologies.
Experience with data quality validation
Looking for individuals with security driven attitude.
4-5+ years of Python & SQL.
4-5+ years of data modeling
4-5+ years of ETL design implementation and maintenance.
Apply
Location:
New York

We found some similar jobs based on your search