FASTTEK GLOBALpowered by Fast Switch - Great Lakes
Chennai, Tamil Nadu
Data Engineer #1037949Job Description:
- Employees in this job function are responsible for designing, building, and maintaining data solutions including data infrastructure, pipelines, etc. for collecting, storing, processing and analyzing large volumes of data efficiently and accurately
Key Responsibilities:
- Collaborate with business and technology stakeholders to understand current and future data requirements
- Design, build and maintain reliable, efficient and scalable data infrastructure for data collection, storage, transformation, and analysis
- Plan, design, build and maintain scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow
- Design, implement and maintain existing and future data platforms like data warehouses, data lakes, data lakehouse etc. for structured and unstructured data
- Design and develop analytical tools, algorithms, and programs to support data engineering activities like writing scripts and automating tasks
- Ensure optimum performance and identify improvement opportunities
Skills Required:
- Data Warehousing, Big Query,, Data Governance, Data/Analytics, Data/Analytics dashboards, Data Model, Elasticsearch
Skills Preferred:
- 3dx, AI/ML
Experience Required:
- Engineer 2 Exp: 4+ years Data Engineering work experience
Experience Preferred:
- have expertise on Elastic Search and Data Analytics and BI
Education Required:
- Bachelor's Degree
Education Preferred:
- Certification Program
Additional Information :
- Responsibilities As a Data Engineer (Elasticsearch Specialist), you will design, build, and optimize data pipelines that feed into Elasticsearch clusters, enabling high-performance search and analytics.
- You will be responsible for ingestion, transformation, indexing, and query optimization of large-scale data. This role requires a deep understanding of Elasticsearch architecture, data modeling, and performance tuning, along with strong data engineering and automation skills.
Key Responsibilities:
- Data Ingestion & Pipelines: Design and implement robust ETL/ELT pipelines to ingest structured, semi-structured, and unstructured data into Elasticsearch.
- Elasticsearch: Manage and optimize Elasticsearch clusters for scalability, availability, and performance.
- Configure index settings, mappings, analyzers, and sharding strategies to improve query efficiency.
- Data Modeling & Indexing: Develop efficient index designs to support various use cases (search, analytics, monitoring).
- Ensure proper use of aggregations, filters, and full-text search for optimal performance.
- Proactive Monitoring: design, implement, and manage comprehensive monitoring and alerting systems for our platforms, with a specific focus on Elastic clusters.
- This includes defining key metrics, thresholds, and escalation procedures to proactively identify and address potential issues
- Health Check Management: develop and maintain a suite of automated health checks for critical endpoints, APIs, infrastructure components, and network paths, ensuring rapid detection of anomalies & service degradations
- Collaboration: work closely with development teams to ensure the reliability and scalability of applications that interact with Elastic, providing guidance on best practices and performance optimization