Description

Data Engineer specialized in distributed data processing using Apache Spark and Scala.

Experience in designing, developing, and optimizing ETL pipelines on Big Data architectures, working with large-scale datasets in critical production environments.

Specialized in:

Spark job optimization and performance tuning
Batch ETL pipeline development
Workflow orchestration with Airflow
Big Data environment migrations
Distributed processing and scalability

I have worked on projects focused on financial data processing, system integrations, and data platform modernization, participating in migrations to cloud architectures and Databricks environments.

Main stack:

Scala, Spark, Airflow, Hive, SQL, Databricks, PostgreSQL, Cloudera, CI/CD, and APIs.

Domaines d’expertise

Langues

Espagnol
Bilingue ou natif
Anglais
Capacité professionnelle limitée

Préférences en matière de lieu de travail

En télétravail uniquement

Travaille majoritairement à distance

BOSONIT S.L.
Data Engineer
BANQUE & ASSURANCES
janvier 2022 - Aujourd'hui (4 ans et 5 mois)
Madrid, Espagne
Desarrollo, mantenimiento y evolución de pipelines ETL para el procesamiento de mensajes de pago (SWIFT, ISO 20022, SEPA, ACH) Procesamiento batch de datos desde capa landing (S3) hasta capa common, aplicando validaciones técnicas y funcionales Normalización de múltiples fuentes de datos en un modelo común para su posterior explotación Optimización de jobs Spark reduciendo tiempos de ejecución de varias horas a minutos mediante mejoras en particionado, configuración y lógica de procesamiento la de procesos
ETL/ELT Apache Spark Data Engineer Scala Databricks
BINAIA
Big Data Engineering Mentor
EDUCATION & E-LEARNING
juillet 2023 - Aujourd'hui (2 ans et 11 mois)
Madrid, Espagne
• Mentoring new Big Data trainees, providing guidance on both theoretical and practical aspects of Big Data technologies.

• Conduct bi-weekly follow-ups to ensure learning progress. The mentorship program begins with foundational knowledge in Hadoop, HDFS, Hive, Apache Spark, Scala/Python, followed by practical ETL simulations and hands-on experience with Apache Airflow for building and orchestrating data pipelines.
Coaching and mentoring Scala Apache Spark ETL/ELT Databricks

Soyez le premier à recommander David

Contribuez à la réussite de ce freelance en partageant votre expérience de collaboration avec lui.

Agatha Frydrych

Backend Java Software Engineer

4.7

(3)

Baptiste Duhen

Fullstack developer

4.6

(4)

Amed Hamou

Senior Lead Developer

(2)

Audrey Champion

Web developer

4.3

(3)

S’inscrire pour les voir

Grado Superior
MEDAC
2022
Grado Superior
Certified Associate Developer for Apache Spark 3.0
Databricks
Certified Associate Developer for Apache Spark 3.0