Azure Databricks Admin

<p><strong>Core Technical Responsibilities:</strong></p><p>• Architect and optimize big data pipelines using Apache Spark, Delta Lake, and Databricks-native tools.</p><p>• Design scalable data ingestion and transformation workflows, including batch and streaming (e.g., Kafka, Spark Structured Streaming).</p><p>• Create integration guidelines to configure and integrate Databricks with other existing security tools relevant to data access control.</p><p>• Implement data security and governance using Unity Catalog, access controls, and data classification techniques.</p><p>• Support migration of legacy systems to Databricks on cloud platforms like Azure, AWS, or GCP.</p><p>• Manage cloud platform operations with a focus on FinOps support, optimizing resource utilization, cost visibility, and governance across multi-cloud environments.</p><p><br></p><p><strong>Collaboration & Advisory</strong></p><p>• Act as a technical advisor to data engineering and analytics teams, guiding best practices and performance tuning.</p><p>• Partner with architects and business stakeholders to align Databricks solutions with enterprise goals.</p><p>• Lead proof-of-concept (PoC) initiatives to demonstrate Databricks capabilities for specific use cases.</p><p><br></p><p><strong>Strategic & Leadership Contributions</strong></p><p>• Mentor junior engineers and promote knowledge sharing across teams.</p><p>• Contribute to platform adoption strategies, including training, documentation, and internal evangelism.</p><p>• Stay current with Databricks innovations and recommend enhancements to existing architectures.</p><p><br></p><p><strong>Specialized Expertise (Optional but Valuable)</strong></p><p>• Machine Learning & AI integration using MLflow, AutoML, or custom models.</p><p>• Cost optimization and workload sizing for large-scale data processing.</p><p>• Compliance and audit readiness for regulated industries.</p><p><br></p><p><strong>Qualifications:</strong></p><p>• Bachelor’s degree in computer science.</p><p>• At least 6 years of experience in IT cloud infrastructure, architecture and operations, including security, with at least 3 years in a Platform admin role</p><p>• Strong understanding of data security principles and best practices.</p><p>• Expertise in Databricks platform, security features, Unity Catalog, and data access control mechanisms.</p><p>• Experience with data classification and masking techniques.</p><p>• Strong understanding of cloud cost management, with hands-on experience in usage analytics, budgeting, and cost optimization strategies across multi-cloud platforms.</p><p>• Strong knowledge of cloud architecture, design, and deployment principles and practices, including microservices, serverless, containers, and DevOps.</p><p>• Deep expertise in Azure/AWS big data & analytics technologies, including Databricks, real time data ingestion, data warehouses, serverless ETL, No SQL databases, DevOps, Kubernetes, virtual machines, web/function apps, monitoring and security tools.</p><p>• Deep expertise in Azure/AWS networking and security fundamentals, including network endpoints & network security groups, firewalls, external/internal DNS, load balancers, virtual networks and subnets.</p>

Back to blog

Other Jobs To Apply

No other job posts for this day.