Job description

Job Description

Role: - Data Engineer

Must Have:

Python; Pandas; Pyspark; APIs; Knowledge of distributed computing to optimize Spark data pipeline.

Nice to have Skills :

Typescript; Spark ML; Scikit Learn; Experience with Palantir Foundry platform;

Front end tools: PowerBi, Tableau.

Responsibilities:

Create and maintain optimal data pipeline architecture;

Develop time series data pipelines;

Assemble large, complex data sets that meet functional / non-functional business requirements;

Identify, design, and implement internal process improvements:

automating manual processes, optimizing data delivery;

Build the data pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources using PySpark.;

Build analytics tools that utilize the data pipeline to provide actionable insights (nice to have). ;

Performance tune and optimize data pipeline on Spark/Palantir Foundry; Follow development and implement object maturity standards;

Create and maintain documentation (i.e., Business Requirements, Design Documents) for handover to Operation Support.

Job Type: Contract

Schedule:

• 8 hour shift

Experience:

• Python: 5 years (Required)
• Pyspark: 5 years (Preferred)
• Pandas: 5 years (Preferred)
• APIs: 5 years (Preferred)

Work Location: Remote

arclintfl.com is the go-to platform for job seekers looking for the best job postings from around the web. With a focus on quality, the platform guarantees that all job postings are from reliable sources and are up-to-date. It also offers a variety of tools to help users find the perfect job for them, such as searching by location and filtering by industry. Furthermore, arclintfl.com provides helpful resources like resume tips and career advice to give job seekers an edge in their search. With its commitment to quality and user-friendliness, arclintfl.com is the ideal place to find your next job.