Work Authorization: US Citizen, Green Card Holder or Visa SponsorshipPay: $140k+ Location: Remote ***Candidate must reside in the Midwest or East Coast*** ___________________________________________________________________________________ MSI International is seeking to hire a Sr. Data Engineer to specialize in Hadoop development for our client in Jacksonville, FL. A Hadoop developer is responsible for the design, development and operations of systems that store and manage large amounts of data. ____________________________________________________________________________________ Responsibilities Include: o Write code for moderately complex system designs. Write programs that span platforms. o Code and/or create Application Programming Interfaces (APIs). o Write code for enhancing existing programs or developing new programs. o Review code developed by other IT Developers. o Provide input to and drive programming standards. o Write detailed technical specifications for subsystems. Identify integration points. o Report missing elements found in system and functional requirements and explain impacts on subsystem to team members. o Consult with other IT Developers, Business Analysts, Systems Analysts, Project Managers and vendors. o “Scope” time, resources, etc., required to complete programming projects. Seek review from other IT Developers, Business Analysts, Systems o Analysts or Project Managers on estimates. o Perform unit testing and debugging. Set test conditions based upon code specifications. May need assistance from other IT Developers and team members to debug more complex errors. o Supports transition of application throughout the Product Development life cycle. Document what has to be migrated. May require more coordination points for subsystems. o Research vendor products / alternatives. Conducts vendor product gap analysis / comparison. o Accountable for including IT Controls and following standard corporate practices to protect the confidentiality, integrity, as well as availability of the application and data processed or output by the application. o The essential functions listed represent the major duties of this role, additional duties may be assigned. _______________________________________________________________________________________ Required Skills/Experience: o Experience and understanding with unit testing, release procedures, coding design and documentation protocol as well as change management procedures o Proficiency using versioning tools o Thorough knowledge of Information Technology fields and computer systems o Demonstrated organizational, analytical and interpersonal skills o Flexible team player o Ability to manage tasks independently and take ownership of responsibilities o Ability to learn from mistakes and apply constructive feedback to improve performance o Must demonstrate initiative and effective independent decision-making skills o Ability to communicate technical information clearly and articulately o Ability to adapt to a rapidly changing environment o In-depth understanding of the systems development life cycle o Proficiency programming in more than one object-oriented programming language o Proficiency using standard desktop applications such as MS Suite and flowcharting tools such as Visio o Proficiency using debugging tools o High critical thinking skills to evaluate alternatives and present solutions that are consistent with business objectives and strategy _____________________________________________________________________________________________ Specific Tools/Languages Required: o Big Data Technologies and framework : HADOOP Edge nodes, HDFS, HIVE ,HBase, Map reduce, Hue, Oozie, Pig, Zookeeper, Cloudera ETL – Apache Spark , Sqoop o Programing languages: Scala- Required (Python is helpful) o Fast Access – PostgreSQL, Mongo, HBase o Kafka/Spark Streaming/Batch/Real-Time Processing o Relational DB - DB2, Hive, SQL Server o Knowledge integrating reporting in UI developed with React framework and Node service ____________________________________________________________________________________________ Must Have Experience/Education: o 5+ years related work experience (or equivalent combination of transferable experience and education) in IT development/programming/coding professional within a Hadoop environment o Related Bachelor's degree in an IT related field or relevant work experience o Preferred: Java, Java script, Shell scripting, Nifi, Yarn, Node Service o Front end UI development using React is required o Experience working with Agile Methodology o Transformation/Load (TL) skills required o SPARK/Scala/SQL development knowledge required (all three) o Unix/Linux shell scripting experience required