Role Description
A platform software Engineer is a versatile developer with expertise in Java or Python and a strong foundation in cloud platforms to build and manage applications at scale. Generally, platform engineers fall into two categories: backend engineers, who design and implement microservices with robust APIs, and full-stack engineers, who deliver native UI/UX solutions, and ability to develop frameworks and service to enable an enterprise data platform. With a solid understanding of the SDLC and hands-on experience in Git and CI/CD, platform engineers can independently design, code, test, and release features to production efficiently.
Key Responsibilities:
- Design and Build Data Pipelines: Architect, develop, and maintain scalable data pipelines and microservices that support real-time and batch processing on GCP.
- Service-Oriented Architecture (SOA) and Microservices: Design and implement SOA and microservices-based architectures to ensure modular, flexible, and maintainable data solutions.
- Full-Stack Integration: Leverage your full-stack expertise to contribute to the seamless integration of front-end and back-end components, ensuring robust data access and UI-driven data exploration.
- Data Ingestion and Integration: Lead the ingestion and integration of data from various sources into the data platform, ensuring data is standardized and optimized for analytics.
- GCP Data Solutions: Utilize GCP services (BigQuery, Dataflow, Pub/Sub, Cloud Functions, etc.) to build and manage data platforms that meet business needs.
- Data Governance and Security: Implement and manage data governance, access controls, and security best practices while leveraging GCP's native row- and column-level security features.
- Performance Optimization: Continuously monitor and improve the performance, scalability, and efficiency of data pipelines and storage solutions.
- Collaboration and Best Practices: Work closely with data architects, software engineers, and cross-functional teams to define best practices, design patterns, and frameworks for cloud data engineering.
- Automation and Reliability: Automate data platform processes to enhance reliability, reduce manual intervention, and improve operational efficiency.
Key Responsibilities / Additional Info
Design and Build Data Pipelines: Architect, develop, and maintain scalable data pipelines and microservices
that support real-time and batch processing on GCP.
Service-Oriented Architecture (SOA) and Microservices: Design and implement SOA and microservices-
based architectures to ensure modular, flexible, and maintainable data solutions.
Full-Stack Integration: Leverage your full-stack expertise to contribute to the seamless integration of front-
end and back-end components, ensuring robust data access and UI-driven data exploration.
Data Ingestion and Integration: Lead the ingestion and integration of data from various sources into the data
platform, ensuring data is standardized and optimized for analytics.
GCP Data Solutions: Utilize GCP services (BigQuery, Dataflow, Pub/Sub, Cloud Functions, etc.) to build
and manage data platforms that meet business needs.
Data Governance and Security: Implement and manage data governance, access controls, and security best
practices while leveraging GCP's native row- and column-level security features.
Performance Optimization: Continuously monitor and improve the performance, scalability, and efficiency
of data pipelines and storage solutions.
Collaboration and Best Practices: Work closely with data architects, software engineers, and cross-
functional teams to define best practices, design patterns, and frameworks for cloud data engineering.
Automation and Reliability: Automate data platform processes to enhance reliability, reduce manual
intervention, and improve operational efficiency.
Skills Required
Spring Boot, Angular, Google Cloud Platform - Biq Query, Data Flow, Dataproc, Data Fusion, TERRAFORM, Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API