Databricks Engineer

• Engineer, build and maintain scalable automated data pipelines. Support critical data pipelines with a highly scalable distributed architecture - including data ingestion (streaming, events and batch), data integration, data curation.
• Build cloud data solutions and provide domain perspective on storage, big data platform services, serverless architecture, RDBMS, DW/DM, NoSQL databases and security.
• Deploy, automate, maintain and manage cloud-based production system, to ensure the availability, performance, scalability and security of productions systems.
• Conduct full technical discovery, identifying pain points, business and technical requirements, “as is” and “to be” scenarios.
• Design and arrangement of scalable, highly attainable, and fault tolerant big data eco systems on cloud platform.
• Ownership and responsibility for end-to-end design and development, testing, release of key components data lake solution using DataBricks.
• Understand and implement best practices in management of data, including master data, reference data, metadata, data quality and lineage.
• Support regular ad-hoc data querying and analysis to understand customer behaviors.

• Experience with developing and deploying ETL pipelines using Apache Spark
• Experience with distributed computing, such as Apache Spark and/or Databricks
• Experience in SQL and/or Spark SQL
• Experience with code versioning tools and a command of configuration management concepts and tools, CI/CD including DevOps.
• Experience working with Software Development Life Cycle (SDLC), Agile methodologies (Scrum)
• BA or BS degree in CS, Information Systems, Information Systems

Clearance Requirement:

• Must be US Citizen with an ability to obtain and maintain US Suitability

About Acuity Inc.

Acuity is a leading management and technology consulting firm that specializes in serving the federal government. Our innovative, collaborative and rewarding work environment has earned repeat honors from the Washington Business Journal’s Best Places to Work and SmartCEO Corporate Culture awards. 

We are an equal employment opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, national origin, disability status, protected veteran status or any other characteristic protected by law.

#LI-PA1

#LI-PAremote