Data Engineer
About Candidate
The candidate has over 7 years of experience as a Data Engineer and Data Architect, specializing in designing and deploying data solutions for industries such as FMCG, retail, e-commerce, and airlines. They have expertise in building data mesh architectures using DBT and managing data pipelines with Apache Airflow and other orchestration tools. With proficiency in cloud platforms like Google Cloud Platform (GCP) and Azure, they have successfully transitioned ETL jobs to Airflow for more efficient management.The candidate has extensive experience working with big data technologies, including Hadoop, Spark, and Hive, and is skilled in SQL databases such as SQL Server, MySQL, and PostgreSQL. They have developed and managed CRM data pipelines across platforms like Salesforce Marketing Cloud (SFMC) to GCP and Azure, and SAS to GCP.
Additionally, they have hands-on experience with AWS services like S3, Athena, Glue, and EMR, contributing to the development of scalable, high-performance data solutions.In their previous roles, the candidate has designed and built ETL pipelines using DBT, Python, Airflow, and BigQuery, and has led teams in building models on data mesh architecture. They have also built APIs with cloud functions and API Gateway to update product team records. Their work with orchestration tools like Apache Airflow, Docker, and version control systems such as Git and SVN has been instrumental in streamlining workflows and enhancing operational efficiency.
The candidate is well-versed in the Hadoop ecosystem, including HDFS, Yarn, Hive, HBase, Sqoop, and Impala, and has used Snowflake and GCP-BigQuery for data warehousing. They have also contributed to the development of data engineering components and tools using Java, Spark, and Scala, as well as participated in the creation of data pipelines for retail and e-commerce cases using GCP-BigQuery, Snowflake, and AWS.Throughout their career, the candidate has demonstrated a strong ability to solve complex data engineering challenges and manage cross-functional teams, making them a valuable asset in any data-driven environment.