FASTTEK GLOBALpowered by Fast Switch - Great Lakes
Chennai, Tamil Nadu
Data Engineer #1058237Job Description:
- As a GCP Data Engineer, you will integrate data from various sources into novel data products. You will build upon existing analytical data, including merging historical data from legacy platforms with data ingested from new platforms.
- You will also analyze and manipulate large datasets, activating data assets to enable enterprise platforms and analytics within GCP.
- You will design and implement the transformation and modernization on GCP, creating scalable data pipelines that land data from source applications, integrate into subject areas, and build data marts and products for analytics solutions.
- You will also conduct deep-dive analysis of Current State Receivables and Originations data in our data warehouse, performing impact analysis related to North America's modernization and providing implementation solutions.
- Moreover, you will partner closely with our AI, data science, and product teams, developing creative solutions that build the future.
- Experience with large-scale solutions and operationalizing data warehouses, data lakes, and analytics platforms on Google Cloud Platform or other cloud environments is a must.
- We are looking for candidates with a broad set of analytical and technology skills across these areas and who can demonstrate an ability to design the right solutions with the appropriate combination of GCP and 3rd party technologies for deployment on Google Cloud Platform.
Skills Required:
- Big Query,, Dataproc, Google Cloud Platform - Biq Query, Data Flow, Dataproc, Data Fusion, TERRAFORM, Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API
Skills Preferred:
- AI/ML
Experience Required:
- Engineer 3 Exp: 7+ years Data Engineering work experience
Education Required:
- Bachelor's Degree
Education Preferred:
- Bachelor's Degree
Additional Information :
- Design and build production data engineering solutions on Google Cloud Platform (GCP) using services such as BigQuery, Dataflow, Dataform, Astronomer, Data Fusion, DataProc, Cloud Composer/Air Flow, Cloud SQL, Compute Engine, Cloud Functions, Cloud Run, Artifact Registry, GCP APIs, Cloud Build, App Engine, and real-time data streaming platforms like Apache Kafka and GCP Pub/Sub.
- Design new solutions to better serve AI/ML needs.
- Lead teams to expand our AI-enabled services.
- Partner with governance teams to tackle key business needs.
- Collaborate with stakeholders and cross-functional teams to gather and define data requirements and ensure alignment with business objectives.
- Partner with analytics teams to understand how value is created using data.
- Partner with central teams to leverage existing solutions to drive future products.
- Design and implement batch, real-time streaming, scalable, and fault-tolerant solutions for data ingestion, processing, and storage.
- Create insights into existing data to fuel the creation of new data products.
- Perform necessary data mapping, impact analysis for changes, root cause analysis, and data lineage activities, documenting information flows.
- Implement and champion an enterprise data governance model.
- Ensure knowledge transfer and ease of system maintenance.
- Utilize GCP monitoring and logging tools to proactively identify and address performance bottlenecks and system failures.
- Provide production support by addressing production issues as per SLAs.
- Optimize data workflows for performance, reliability, and cost-effectiveness on the GCP infrastructure.
- Work within an agile product team.
- Deliver code frequently using Test-Driven Development (TDD), continuous integration, and continuous deployment (CI/CD).
- Continuously enhance your domain knowledge.
- Stay current on the latest data engineering practices.
- Contribute to the company's technical direction while maintaining a customer-centric approach.