Databricks Administrator Remote Job Summary We are seeking a highly skilled and experienced
Databricks Administrator to manage, optimize, and secure our Databricks environment. The ideal candidate will have deep knowledge of the Databricks Lakehouse platform and a strong background in cloud infrastructure, data governance, and system administration. You will be responsible for ensuring the platform's reliability, scalability, and security, and for providing technical support to our data engineering, data science, and business intelligence teams. Your role will be critical in enabling our organization to leverage the full power of data and AI.
Key Responsibilities - Platform Administration & Management : Administer and configure Databricks workspaces, clusters, and jobs. Manage user identities, access controls, and security settings at both the account and workspace levels.
- Infrastructure & Cost Optimization : Monitor cluster performance, resource utilization, and platform costs. Implement and enforce cluster policies to optimize expenses and ensure efficient resource allocation.
- Security & Governance : Implement robust security policies and access controls using Unity Catalog . Manage user provisioning, roles, and permissions. Ensure compliance with data governance standards and industry regulations.
- Automation & DevOps : Develop and maintain automation scripts (using Python , Bash , or PowerShell ) for platform provisioning, configuration, and operational tasks. Collaborate with DevOps teams to integrate Databricks with CI/CD pipelines (Terraform , Azure DevOps , GitHub Actions , etc.).
- Troubleshooting & Support : Provide advanced technical support to Databricks users, diagnosing and resolving issues related to job failures, performance bottlenecks, and platform connectivity.
- Monitoring & Alerting : Set up and maintain monitoring and alerting systems to proactively track platform health, performance, and security events using tools like Datadog or Azure Monitor .
- Collaboration & Documentation : Work closely with data engineers, data scientists, and analysts to understand their requirements and provide technical guidance. Create and maintain detailed documentation of platform configurations, best practices, and standard operating procedures.
Required Skills & Qualifications - Experience : Minimum of 3-5 years of experience in data platform administration, with at least 2 years of hands-on experience as a Databricks Administrator.
- Cloud Expertise : Strong experience with a major cloud provider (AWS , Azure , or GCP ) and a solid understanding of cloud-based data infrastructure.
- Databricks Platform :
- Expertise in administering Databricks workspaces, clusters, jobs, and libraries.
- Deep knowledge of Unity Catalog for data governance and security.
- Familiarity with the Databricks Lakehouse architecture, including Delta Lake and Apache Spark .
- Technical Skills :
- Proficiency in scripting with Python or another scripting language.
- Experience with Infrastructure as Code (IaC ) tools such as Terraform .
- Knowledge of CI/CD pipelines and tools.
- Experience with SQL is required.
- Soft Skills :
- Exceptional problem-solving and analytical skills.
- Excellent communication and interpersonal skills, with the ability to provide clear technical guidance.
- Strong attention to detail and a commitment to maintaining platform reliability and security.
- Certifications (Preferred) :
- Databricks Certified Platform Administrator
- Cloud-specific certifications (e.g., AWS Certified SysOps Administrator, Microsoft Certified: Azure Administrator Associate)
