Acuity Inc. is seeking a highly skilled Senior Databricks Engineer to join our Engineering Team and help drive the design and delivery of cloud-scale data platforms for federal clients. This role requires deep experience with Spark, Delta Lake, and distributed data pipelines on Databricks. The ideal candidate brings both engineering rigor and strategic insight into enterprise data modernization.
Design and implement Delta Lake tables optimized for ACID compliance, partition pruning, schema enforcement, and query performance across large datasets.
Develop robust ETL and ELT workflows that integrate multiple source systems into a centralized, query-optimized data warehouse architecture.
Leverage Spark SQL and DataFrame APIs to implement business rules, dimensional joins, and aggregation logic aligned to warehouse modeling best practices.
Collaborate with data architects and engineers to implement cloud-native data solutions on AWS using S3, Glue, RDS, and IAM for secure, scalable storage and access control.
Optimize pipeline performance through intelligent partitioning, caching, broadcast joins, and adaptive query tuning.
Deploy and version data engineering assets using Git-integrated development workflows and automate deployment with CI/CD tools such as GitLab or Jenkins.
Monitor pipeline health, job execution, and cluster utilization using native Databricks tools and AWS CloudWatch, identifying bottlenecks and optimizing cost-performance tradeoffs.
Conduct technical discovery and mapping of legacy source systems, identifying required transformations and designing end-to-end data flows.
Implement governance practices including metadata tagging, data quality validation, audit logging, and lineage tracking using platform-native features and custom logic.
Support ad hoc data access requests, develop reusable data assets, and maintain shared notebooks that meet operational reporting and analytics needs across teams.
Clearance Requirement:
About Acuity:
Acuity, Inc. is a leading management and technology consulting firm that specializes in serving the federal government. Our innovative, collaborative and rewarding work environment has earned repeat honors from the Washington Business Journal’s Best Places to Work and SmartCEO Corporate Culture awards.
Why Choose Acuity?
Join Acuity, where your talents are valued, your growth is nurtured, and your impact is amplified. Together, let's shape the future of digital strategy and technology consulting.
We are an equal employment opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, national origin, disability status, protected veteran status or any other characteristic protected by law.
#LI-PA1
#LI-Remote
Software Powered by iCIMS
www.icims.com