Data Platform Architect

Project: Development of a Data Platform for clients in the banking sector.

Responsibilities

• Designing and developing the target Data Platform architecture.
• Designing cloud-based and hybrid Data Platforms, including integration with on-premises systems and the selection of managed services and self-managed components.
• Designing integrations and data flows between banking systems and the platform.
• Designing event-driven, streaming, and batch processing solutions.
• Selecting technologies and architectural patterns, taking into account performance, high availability (HA), scalability, security, and costs.
• Incorporating access controls, private connectivity, encryption, data residency, disaster recovery, and the costs of data storage, processing, and transfer into cloud architecture designs.
• Designing data ingestion, ODS, Data Lake / Lakehouse, and analytics layers.
• Conducting technical workshops with clients and justifying architectural decisions.
• Collaborating with Data Engineering, Java, and DevOps teams, as well as analysts and client-side architects.
• Preparing and maintaining architectural documentation and records of technical decisions.

Key Requirements:

• Hands-on experience designing enterprise-grade data platforms.
• Strong knowledge of Apache Kafka or Confluent Platform.
• Experience with event-driven architecture and stream processing.
• Experience with Java.
• Experience with Change Data Capture (CDC) mechanisms, particularly Debezium or comparable solutions.
• Knowledge of Data Lake / Lakehouse architecture and both streaming and batch processing.
• Knowledge of SQL and NoSQL databases, as well as data modelling and data sharing principles.
• Experience with high-availability systems, scalability, and large data volumes.
• Experience with Kubernetes / OpenShift and designing solutions for on-premises, cloud, and hybrid environments.
• Hands-on experience designing or implementing a Data Platform in at least one public cloud; knowledge of data storage, batch and streaming processing, and analytics services, as well as security principles and cost optimisation.
• Ability to lead technical discussions with clients and make architectural decisions.
• Basic knowledge of Feature Store concepts and data flows to ML models.
• Excellent communication skills at the intersection of architecture, engineering, infrastructure, and business.

Nice-to-Have Technologies and Experience:

• Confluent Platform, ksqlDB, Debezium, Apache Spark, and Trino.
• Apache Iceberg / Polaris and S3-compatible storage in on-premises environments, e.g. MinIO, Ceph, or Dell ECS.
• ODS databases.
• OpenShift, Prometheus, Grafana, and OpenTelemetry.
• Data services from at least one cloud provider, e.g. AWS (Amazon S3, Amazon Redshift), Azure (Azure Data Lake Storage, Azure Data Factory), or Google Cloud (BigQuery, Dataflow, Pub/Sub).
• Experience in the banking or financial sector.
• Knowledge of DORA requirements (operational resilience and outsourcing), GDPR in the context of financial data, and the basics of KNF Recommendations D and M.