← В ленту

Портфолио

Sberbank-Technology

Promotion the direction DaaS in Retail Banking Division. Leading a team of 3 engineers (including DevOps and DE). Creating data platform with batch and stream processing. - Migrated old codebase from SSIS to Airflow 2.x.x. - Created full development and delivery processes: * Separate development (local, test, preprod, prod) environments. * CI/CD process on top of Jenkins. * Granulation, planning and task control. - Development Spark and Spark Streaming jobs for creating ETL processes.

Philip Morris International

- Business analyst of a single data warehouse. Data engineering, Designing the DWH structure, organizing data migrationfrom various sources, developing ETL using CloverETL and Matillion, buildingpipelines based on Kappa Architecture. - Extensive experience in writing complex SQL queries: hierarchical queries,queries using analytical window functions, tuning query performance in Amazon Redshift and MS SQL Server (working with query plans, optimizingthe structure of tables and indexes/keys). - Development of the architecture of the NPS (Net Promoter Score) systemintegration project,close work with AWS: Amazon RedShift, S3, AWS Lambda. - Development and support of current ETL processes (Matillion, CloverDX),integration of data from external systems, integration of web services. - Business analysis, system analysis, development. - Implemented data loading via API connection from the Pardot system to theRedshift storage.Working with Python libraries: pandas, boto3, pypardot4, multiprocessing. - Development of a fully satisfactory algorithm for loading data in contexts strongrestrictions on the number of API requests per day and internal limitations ofthe Pardot system. - Business analysis, working with global business partners in a part ofthe coordination of requirements for the integration of external systems,development a data transfer to internal PMI systems (for loading data fromanalytical agencies), system analysis, development: building ETL processesusing data from internal and external sources. - Creating datamarts for legal and financial departments in Tableau. - Downloading data from the Nielsen analytical agency.Collecting business requirements for reports. - Working with VC systems: GitHub, GitLab, Git Bash, TortoiseGit. - Management experience: coordinating the work of the development team ofexternal contractors.

Otto (GmbH & Co KG)

- Development, providing support and maintenance of the entire reportingsystem in a part of collaborating with delivery services: Russian Post, DHL,SPSR, O-Courier, etc. (creating an interface and forms in MS Access, VBA,writing complex server-side SQL queries). - I gained valuable experience in writing complex SQL queries (Oracle syntax):for the purposes of creating operating and analysis report. - I had an appointment as Software Engineer in the development team thatprovided the internal multifunctional system for administering to the businesswants: designing cross-browser modules (client part: HTML, Java Script,ExtJS, CSS; server part - PHP, Oracle, PL/SQL). During this period I've done: - Implementation a project (development of client and server parts) forautomatic weighing of postal items, including calculation of the delivery rateand printing of postal documents (reports), - in MS Access. - Development and maintain an extensive system for calculating salary in unitsfocused on piece-rate job. - Implementation of an integration with the DHL web-service (within theexisting system) for the transmission and dispatch of shipments, providing thedocuments (and printing them, where appropriate), address labels on thermaltransfer printers through data exchange via the SOAP protocol. - Performing analytical tasks: analysis of business processes, search formutually acceptable business solutions, writing technical specifications. - I gain some experience of HR organization development: softwareimplementation, organization and carrying out of tests. I have the skills: PhotoShop, CorelDRAW: during the student years, I wasengaged in design (web design, creation of logos and corporate identity).Participated in the design of the interface part of the system, became theauthor of the project logo. I am able to solve problems effectively on my own in conditions of fuzzyformulation, I am hardworking, independent, organized, responsible, sociable,curious.

Скиллы

Apache Airflow
Apache Spark
Bash
ClickHouse
CloverDX
ETL Tools
Greenplum
HDFS
Informatica PC
Jenkins
Kafka
Matillion
MIT Kerberos
Oracle
PL/SQL
PostgreSQL
Presto
PySpark
Python
Spark Structured Streaming
SQL
SSIS
Teradata

Опыт работы

Software Engineering Manager
с 01.2022 - По настоящий момент |Sberbank-Technology
HDFS, PySpark, Apache Airflow, Presto, Clickhouse,PostgreSQL, Kafka, Spark Structured Streaming
Promotion the direction DaaS in Retail Banking Division. Leading a team of 3 engineers (including DevOps and DE). Creating data platform with batch and stream processing. - Migrated old codebase from SSIS to Airflow 2.x.x. - Created full development and delivery processes: * Separate development (local, test, preprod, prod) environments. * CI/CD process on top of Jenkins. * Granulation, planning and task control. - Development Spark and Spark Streaming jobs for creating ETL processes.
Senior Data Engineer
04.2020 - 12.2021 |Sberbank-Technology
Teradata, Greenplum, Informatica PC, PostgreSQL, Oracle
Data management: - Development of pipelines for data migration using various ETL tools (SSIS, Informatica PC, orchestration of workflows in Apache Airflow). - Creating and implementation complex ETL-processes to loading andmanipulating large amounts of cold data (5 Tb per object) from Teradata toHDFS in efficient way. - Optimization performance and cost at a high level (Teradata platform): responsibility for data accuracy and integrity, output of various datarepresentations (views, tables) for end users. Creation of additional Statisticsand Indexes, assess join strategy, spool file size analizing, redistribution etc. Data platform: - Implementation CI/CD methodology (Bitbucket, Jenkins) in the currentinfrastructure. - Development system integration modules (Hooks, Operators) withimplementation of a LDAP-based access role model to connect to Teradata. - Development a customer connector for transferring data from Greenplum toHDFS based on MIT Kerberos end-to-end authorization system using externallibraries based on JDBC and ODBC drivers. Promoting of the direction Data-as-a-Service in the Retail banking Division: - Creating a web-application (Oracle APEX) for scheduling and running userfunctions and stored procedures (jobs) from various platforms (Teradata, Greenplum, Informatica PC, PostgreSQL, Oracle) in Apache Airflow. - Deploying Trino (PrestoSQL) technology for all key data storage stacks(HDFS, Greenplum, Oracle, Teradata) to implement ad hoc query system fordata analyzing within the framework of the "Heterogeneous Access" pilot. - Development the single platform to collect metadata from various data-platforms including business description of data objects and technicalmetadata with a finite number of users exceeding 2000 people. - Expansion of functionality of ApEx modules via implementation plugins forloading files to the platform.
Data Engineer
06.2019 - 04.2020 |Sberbank-Technology
JSON, CSV, XML, PARQUET, SSIS, Informatica PC, C#, bash-scripts, ClickHouse, Apache Airflow, ELK, Hadoop
- Creation and implementation of batch data exchange with external partners(Yandex, Rambler Group, Delivery Club, Citymobil, etc.) via SFTP in variousformats (JSON, CSV, XML, PARQUET): SSIS, Informatica PC, C#, bash-scripts as a part of the expansion of Sberbank Ecosystem. - Development of a customer tool for scheduling and running users SP onTeradata platform. The solution is assumed to have full automatization and was realised as a self-service for end users (UI based on Oracle ApEx, afterfilling in a dictionary with launch parameters, particular auto-generated DAG isbeing run at the scheduled time). - Design of the DWH structure (EAV, 6NF, Data Vault methodology), settingup of data migration from Kafka, Hadoop to a single DWH (Teradata),development of ETL based on SSIS, Informatica Power Center, ApacheAirflow.- Implementation of the solution for the delivery metadata from different platforms to the unified bank portal via API interaction (pipeline with regulardata deliveries (in JSON-files of the specified format) with authorization via MITKerberos). - Working with version control systems: GitLab, Git Bash, TortoiseGit. - Practical skills of working with the technical stack: ClickHouse, Apache Airflow, ELK, Hadoop, creating web services with REST API based on Flask. - Implementation of the image assembly from the docker file with the start ofFlask-based services, launching docker containers from Jenkins. - Pilot implementation of speed layer loading using Spark Structured Streaming. Writing documentation (Confluence).
Business analyst of a single data warehouse
05.2018 - 05.2019 |Philip Morris International
SQL, ETL, MS SQL Server, AWS, Matillion, CloverDX
- Business analyst of a single data warehouse. Data engineering, Designing the DWH structure, organizing data migrationfrom various sources, developing ETL using CloverETL and Matillion, buildingpipelines based on Kappa Architecture. - Extensive experience in writing complex SQL queries: hierarchical queries,queries using analytical window functions, tuning query performance in Amazon Redshift and MS SQL Server (working with query plans, optimizingthe structure of tables and indexes/keys). - Development of the architecture of the NPS (Net Promoter Score) systemintegration project,close work with AWS: Amazon RedShift, S3, AWS Lambda. - Development and support of current ETL processes (Matillion, CloverDX),integration of data from external systems, integration of web services. - Business analysis, system analysis, development. - Implemented data loading via API connection from the Pardot system to theRedshift storage.Working with Python libraries: pandas, boto3, pypardot4, multiprocessing. - Development of a fully satisfactory algorithm for loading data in contexts strongrestrictions on the number of API requests per day and internal limitations ofthe Pardot system. - Business analysis, working with global business partners in a part ofthe coordination of requirements for the integration of external systems,development a data transfer to internal PMI systems (for loading data fromanalytical agencies), system analysis, development: building ETL processesusing data from internal and external sources. - Creating datamarts for legal and financial departments in Tableau. - Downloading data from the Nielsen analytical agency.Collecting business requirements for reports. - Working with VC systems: GitHub, GitLab, Git Bash, TortoiseGit. - Management experience: coordinating the work of the development team ofexternal contractors.
Software Developer
12.2005 - 12.2015 |Otto (GmbH & Co KG)
PHP, Oracle, PL/SQ, PhotoShop, CorelDRAW, HTML, Java Script,ExtJS, CSS,
- Development, providing support and maintenance of the entire reportingsystem in a part of collaborating with delivery services: Russian Post, DHL,SPSR, O-Courier, etc. (creating an interface and forms in MS Access, VBA,writing complex server-side SQL queries). - I gained valuable experience in writing complex SQL queries (Oracle syntax):for the purposes of creating operating and analysis report. - I had an appointment as Software Engineer in the development team thatprovided the internal multifunctional system for administering to the businesswants: designing cross-browser modules (client part: HTML, Java Script,ExtJS, CSS; server part - PHP, Oracle, PL/SQL). During this period I've done: - Implementation a project (development of client and server parts) forautomatic weighing of postal items, including calculation of the delivery rateand printing of postal documents (reports), - in MS Access. - Development and maintain an extensive system for calculating salary in unitsfocused on piece-rate job. - Implementation of an integration with the DHL web-service (within theexisting system) for the transmission and dispatch of shipments, providing thedocuments (and printing them, where appropriate), address labels on thermaltransfer printers through data exchange via the SOAP protocol. - Performing analytical tasks: analysis of business processes, search formutually acceptable business solutions, writing technical specifications. - I gain some experience of HR organization development: softwareimplementation, organization and carrying out of tests. I have the skills: PhotoShop, CorelDRAW: during the student years, I wasengaged in design (web design, creation of logos and corporate identity).Participated in the design of the interface part of the system, became theauthor of the project logo. I am able to solve problems effectively on my own in conditions of fuzzyformulation, I am hardworking, independent, organized, responsible, sociable,curious.

Образование

Computer scientist-economist
2001 - 2005
Tver State University, Applied Computer Science and Cybernetics

Дополнительное образование

Data Engineer
Certificates/Courses

Языки

АнглийскийПродвинутыйРусскийРодной