Role OverviewWe are seeking an experienced Senior Data Architect / Databricks Architect to lead the design and implementation of scalable lakehouse-based data architectures using the Databricks platform. The role focuses on delivering enterprise-grade data solutions, implementing Unity Catalog governance, and enabling end-to-end data lifecycle management across ingestion, processing, storage, and analytics layers.The ideal candidate will have strong expertise in Databricks, Apache Spark, Delta Lake, and cloud data platforms, along with the ability to collaborate with the project teams to design high-performance, secure, and scalable data ecosystems.
Key ResponsibilitiesEnd-to-End Data ArchitectureCollaborate with Databricks Professional Services and project stakeholders to design comprehensive end-to-end data architectures on the Databricks platform.Define scalable data ingestion strategies integrating structured and unstructured data from multiple source systems.Architect scalable lakehouse storage solutions using Delta Lake and modern data platform best practices.Develop robust data processing frameworks leveraging Apache Spark and Databricks workflows.Design data consumption layers that support analytics, reporting, AI/ML, and operational workloads.Ensure seamless data movement and lifecycle management across ingestion, transformation, storage, and consumption layers.
Governance, Security & ComplianceImplement data governance frameworks leveraging Unity Catalog for centralized governance.Configure metastore, catalog and schema structures, and implement access control policies.Design and enforce data security, role-based access control, and data protection strategies.Ensure compliance with regulatory requirements and enterprise data governance standards.Implement data lineage, monitoring, audit logging, and observability for the data platform.Optimize system performance through cluster configuration, workload management, and query tuning.Define and implement data quality frameworks and validation processes.
Data Modelling & DesignDesign business-aligned data models supporting enterprise analytics and operational use cases.Implement dimensional modeling, normalized models, and data vault architectures.Design optimized Delta table structures to improve scalability and query performance.Implement medallion architecture (Bronze, Silver, Gold layers) for structured data refinement.Develop data schemas that support both BI analytics and machine learning workloads.Maintain data dictionaries, metadata documentation, and model specifications.Technical Leadership & CollaborationLead technical workshops with the project team, stakeholders, and cross-functional teams to gather and refine requirements.Provide architectural guidance and best practices for Databricks-based data engineering teams.Collaborate with Infrastructure, Applications, and Cybersecurity teams for integrated enterprise solutions.Mentor data engineers, architects, and platform specialists on modern lakehouse architectures.Present architecture strategies, solution designs, and technical recommendations to leadership and stakeholders.Solution ImplementationLead implementation of Databricks-based solutions from architecture design to production deployment.Oversee proof-of-concept (POC) initiatives and pilot programs to validate technical feasibility.Ensure solutions meet scalability, reliability, security, and performance requirements.Conduct architecture reviews and governance checkpoints aligned with enterprise standards.
Required Technical SkillsDatabricks & Data PlatformStrong hands-on experience with the Databricks platform, including:Workspace administrationCluster configuration and optimizationWorkflow orchestrationUnity CatalogExperience implementing Unity Catalog for unified data governance, including:Metastore configurationCatalog and schema designAccess control and policy managementData Engineering & ArchitectureExpertise in data modeling approaches including:Dimensional modelingData VaultLakehouse architectureDeep knowledge of Delta Lake features, including:ACID transactionsTime travelPerformance optimization techniquesStrong proficiency in Apache Spark (Spark SQL, DataFrames, performance tuning).ProgrammingStrong coding experience in:PythonSQLScalaCloud PlatformsHands-on experience with at least one major cloud platform:Microsoft AzureAmazon Web Services (AWS)Google Cloud Platform (GCP)Additional Technical SkillsData pipeline development and ETL/ELT architectureMetadata management and data governance frameworksCI/CD implementation for data platformsData quality monitoring and validation frameworksPerformance optimization and troubleshootingKnowledge of data security, compliance, and regulatory standards
Professional Experience8–10+ years of experience in data architecture, data engineering, or advanced analytics roles3–5+ years of hands-on Databricks platform experienceProven experience implementing Unity Catalog in enterprise-scale environmentsDemonstrated success designing large-scale enterprise data models and lakehouse architecturesExperience working with Databricks Professional Services or partner ecosystems is highly desirableExperience across multiple industries such as Public Sector, Financial Services, Healthcare, or Retail is advantageous
Preferred CertificationsDatabricks Certified Associate Developer for Apache SparkDatabricks Data Engineer ProfessionalCloud certifications such as:Azure Data Engineer AssociateAWS Data Analytics SpecialtyGoogle Professional Data EngineerOther relevant data management or analytics certifications