Senior Data Engineer (Cloud)

Description



The Senior Cloud Data Engineer will work directly with product lead on multiple algorithmic data science products. design, code, test, and analyze software programs and applications. This includes researching, designing, documenting, and modifying software specifications throughout the production lifecycle. This role will also create business critical reports, analyze, and amend software errors in a timely and accurate fashion and provide status reports where required. The position responsibilities outlined below are not all encompassing. Other duties, responsibilities, and qualifications may be required and/or assigned as necessary.



Responsibilities:

• Work with Product team to determine requirements and propose approaches to address users' needs

• Analyze requirements to determine approach/proposed solution

• Design and build solutions using relevant programming languages

• Thoroughly test solutions using relevant approaches and tools

• Conduct research into software-related issues and products

• Bring out-of-box thinking and solutions to address challenging issues

• Effectively prioritize and execute tasks in a fast-paced environment

• Work both independently and in a team-oriented, collaborative environment

• Flexible and adaptable to learning and understanding new technologies

• Highly self-motivated and directed

• Demonstrate a commitment to core values



Experience and Skills:

• Experience in designing, developing, and maintaining data pipelines on AWS cloud platform

• Experience in developing solutions using Snowflake database

• Hands-on software troubleshooting experience

• Proven analytical and problem-solving abilities

• Experience with every phase of the software development life cycle including requirements gathering, requirements analysis, design, development, and testing

• Work closely with the product leads to identify and prioritize data needs for the algorithmic data science products

• Collaborate with cross-functional teams to ensure data quality and accuracy

• Continuously monitor and optimize data pipeline performance

• Stay up-to-date with the latest technologies and industry trends in data engineering



Must have skills:

• Hands on experience with AWS cloud architecture and development data pipelines using S3, Redshift, Athena, DynamoDB, Lambda, Glue, EMR, Kinesis, API Gateway, and other AWS technologies

• Hands on experience is building and monitoring CloudWatch alarms

• Hands on experience with AWS CICD suite (Code commit, Code pipeline, Cloud formation)

• Hands on experience is using Python (intermediate/expert level)

• Hands on experience using Spark (strongly preferred)

• Hands on experience building solutions using Snowflake database

• Hands on experience in trouble shooting complex SQL problems

• Experience working with cross-functional teams in a fast-paced, dynamic environment

• Strong problem-solving skills and attention to detail

• Good communication and collaboration skills



Good to have Skills:

• Hands on experience building visualizations/reports

• Good understanding of data modeling



#Tech2022