Job description

Job Title: Sr. Data Engineer

Location: Fully remote

Duration: Full Time

Required and Preferred Skill Sets:

• 1 -2 years of hands-on experience in AWS - EMR [Hive, Pyspark], S3, Athena or any other equivalent cloud; Ability to solve complex problems
• 1-2 years of hands-on experience Spark Batch Processing and some familiarity with Spark Structured Streaming; Ability to solve complex issues
• 1-2 years’ experience working experience with Hadoop stack dealing huge volumes of data in a scalable fashion
• 2-3 years of hands-on experience with SQL, ETL, data transformation and analytics functions; Ability to solve complex problems
• 2-3 years of hands-on Python experience including Batch scripting, data manipulation, distributable packages; Ability to solve complex problems
• 2-3 years’ experience working with batch orchestration tools such as Apache Airflow or equivalent, preferable Airflow
• 2-3 years working with code versioning tools such as GitHub or BitBucket; expert level understanding of repo design and best practices
• 2-3 years working with deployment automation tools such as Jenkins and familiarity with containerization concepts such as Docker and Kubernetes
• 2-3 years of hands-on experience designing and building ETL pipelines; expert with data ingest, change data capture, data quality; hand on experience with API development; some exposure to Nifi or Kafka
• 2-3 years designing and developing relational database objects; knowledgeable on logical and physical data modelling concepts; some experience with Snowflake
• Preferred 1+ years of experience supporting Tableau or Cognos use cases

Familiarity with Agile; working experience preferred

Education:

• Bachelor's degree in IT or related field
• One of the following alternatives may be accepted:
• Associates + 6 yrs;

Skills and Certifications [note: bold skills and certification are required]

• AWS
• Python SQL
• PySpark Scala

Job Type: Full-time

Salary: $110,000.00 - $120,000.00 per year

Benefits:

• 401(k)
• Dental insurance
• Health insurance
• Paid time off
• Vision insurance

Schedule:

• 8 hour shift
• Monday to Friday

Work Location: Remote

arclintfl.com is the go-to platform for job seekers looking for the best job postings from around the web. With a focus on quality, the platform guarantees that all job postings are from reliable sources and are up-to-date. It also offers a variety of tools to help users find the perfect job for them, such as searching by location and filtering by industry. Furthermore, arclintfl.com provides helpful resources like resume tips and career advice to give job seekers an edge in their search. With its commitment to quality and user-friendliness, arclintfl.com is the ideal place to find your next job.