Data Engineer
с 10.2024 - По настоящий момент |Rapid Innovation GmbH
Google Cloud, GKE, SQL, Dbt, Kotlin, Kafka, Pekko, GitHub
Supply chain management consulting and implementation services.
● Support in creation of master data set for a supply chain management application from multiple heterogeneous sources.
● Design and implementation of ETL and data pipelines
● Implementation data quality dashboard for business critical KPIs
● Deployment and monitoring of various data pipelines
Consultant / Data Engineer
09.2023 - 10.2024 |Data warehouse consulting services
AWS Cloud, Spring Boot, Java, Docker, GitLab
Support and implementation services.
● Consulting services data model, deployment strategies, process design.
● Design and implementation of various ETL processes.
● Workshops and knowledge transfer on cloud and data architecture, process design.
Solution Architect / Cloud Architect
07.2021 - 10.2023 |Data warehouse migration
AWS Cloud, LakeFormation, Glue, EMR, Athena, Lambda, SQS, S3, Docker, Spark, Hadoop, Airflow, Terraform, Python, GitLab
Concept and support of an on-premise data warehouse migration towards the cloud.
● Design and implementation of a cloud-based data warehouse infrastructure based on AWS Managed Services.
● Help in migrating various data processing jobs from Cloudera to AWS Managed Services.
● Design and implementation of deployment pipelines for testing and rollout.
● Workshops and knowledge transfer on cloud and data architecture, process design.
Solution Architect / Data Engineer
06.2020 - 12.2022 |Cloud based data warehouse
AWS Cloud, Kubernetes, Docker, Spark, Airflow, Kafka, Terraform, Python, Kustomize, GitLab
Design and implementation of a cloud-based data ware house for processing user related data.
● Implemented workflows to fetch data from various third-party providers.
● Building and enabling a team to create new ETL processes.
● Realtime integration of a market automation software suite.
● Implemented modern ETL processing environment using Airflow, Spark and Kubernetes.
● General advice on data architecture and data management.
Solution Architect
08.2019 - 12.2019 |Sensor data processing
AWS Cloud, Kubernetes, Kafka, Spark, Bamboo, Java, Docker, Terraform
Design of AWS managed infrastructure platform for sensor data processing, extension of an existing data science environment.
● Advice on design and tools for building a Kubernetes based infrastructure platform for sensor data processing.
● Design and implementation of infrastructure components on Kubernetes.
● Design and implementation of the ETL pipeline for data collection.
● Design and implementation of CI/CD pipeline with Bamboo & Kubernetes.
Solution Architect / Data Engineer
01.2019 - 03.2020 |Data analytics on car measurement data
AWS Cloud, Lambda, IAM, Airflow, Kubernetes, Terraform, Python, Jenkins
Design and implementation of a cloud-based data warehouse for evaluation of vehicle data. Design and implementation of a data science environment.
● Extended a prototype and put into operational readiness for a production environment.
● Design and implementation of a CI / CD pipeline.
● Setup of project structure and release management.
● Implemented various ETL pipelines for car measurement data collection, validation and -transformation.
Solution Architect / Data Engineer
10.2017 - 11.2018 |Data science environment
AWS Cloud, Kubernetes, Spark, R, NiFi, Terraform, Docker, Jupyter NB
Design and implementation of a cloud-based data warehouse & data science environment.
● Designing data warehouse architecture & data storage strategies.
● Architecture proposal of a dynamically scalable data warehouse.
● Implementation of infrastructure components in Terraform & Kubernetes.
● Implementation of infrastructure components in Kubernetes.
● Development of ETL pipelines for data collection.
System Architect / Consultant Data Strategy
05.2017 - 12.2018 |Micro service architecture
Micro Services, Java, Docker, Kafka, Liquibase, Jenkins
Support conception and implementation/migration of a monolith into a micro service architecture.
● Alignment and coordination of different teams regarding technology usage.
● Introduction of Kafka as the central message bus for micro service communications.
● Introduction of LiquiBase for database schema management.
● Professional / technical support for a specific micro service.