FASTTEK GLOBALpowered by Fast Switch - Great Lakes
Chennai, Tamil Nadu
Data Engineer #1057334Job Description:
- Responsible for gathering business requirements and designing ETL systems to meet application needs
- Work closely with stakeholders to gather requirements and provide technical guidance on the migration process
- Collaborate with the data engineering teams to understand the existing JCL/ETL jobs and their dependency.
- Design and implement migration strategies for moving ETL to GCP native services.
- Develop, test, and deploy data pipelines and workflows, ensuring scalability, reliability, and performance
- Troubleshoot and resolve any production issues or challenges that arise during the migration
- Delivering product features as part of roadmap and business needs
- Lead in Effort estimation for the new developments and enhancements.
- Co-create test plan and lead execution and code deployment
- Lead performance tuning to optimize the job execution.
- Responsible for validating, running, scheduling and monitoring jobs using data stage director
- Responsible for debugging Ticketing issues in the production phase
Skills Required:
- Big Query,, ETL, Google Cloud Platform - Biq Query, Data Flow, Dataproc, Data Fusion, TERRAFORM, Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API
Experience Required:
- Engineer 3 Exp: 7+ years Data Engineering work experience
Experience Preferred:
- Warranty PG Group under FCSD Tech is looking for a dynamic and result oriented candidate for the role of Senior Software Engineer within its Warranty analytics product line.
- The candidate will be responsible for designing the transformation and modernization of big data solutions on the GCP cloud.
- The candidate must have experience leading large-scale solutions and the operationalization of data warehouses, data lakes, and analytics platforms on a cloud platform, GCP preferred.
- We seek individuals with a wide range of technology skills in these domains, capable of designing the right solutions using a mix of GCP and third-party technologies for deployment on the GCP cloud.
- 7+ years of Data Engineering experience
- 5+ years of Cloud experience (GCP preferred) with solutions designed and implemented at production scale - Strong understanding of key GCP services, especially those related to data processing [Batch/Real Time] Big Query, Cloud Scheduler, Airflow, Postgres, Data Flow, Pub/Sub, Cloud Logging and Cloud Monitoring
- Experience with infrastructure as code Terraform/ GitHub
- Experience in design, development and implementation of data pipelines using Data Warehousing applications
- Experience in integrating various data sources like Oracle, Teradata, DB2, Big Query & Flat files
- Hands on experience in performance tuning and debugging ETL jobs - Involving in review meetings and coordinating with the team in job designing and fine-tuning the job performance.
Education Required:
- Bachelor's Degree
Education Preferred:
- Certification Program
Additional Information :
Qualifications:
- BE/BTech in CS/IT or equivalent degree in relevant domain
- Proven experience in data engineering [IBM DataStage or equivalent Cloud ETL preferred via Bigquery]
- Excellent problem-solving skills and ability to work independently as well as team player
- Strong knowledge of ETL processes, data integration and data management
- Proficiency in programming languages such as SQL, Python or Java
- Familiarity with cloud computing platforms and data storage solutions
- Professional Certification in GCP is preferred (e.g., Professional Data Engineer)
- Experience in Power BI considered plus
- Working exposure in Agile Methodology