Billings Clinic Senior Data Engineer in Billings, Montana
The Data Engineer acts as subject matter experts of the data repositories available within the organization by obtaining an in-depth understanding of each source system. These engineers are primarily focused on the creation, management, and availability of data pipelines to support structured analytics.
Essential Job Functions
• Develops data pipelines using extract, transform, and load (ETL/ELT) processing routines for enterprise level data warehousing initiatives.
• Writes advanced queries using data scripting language including T-SQL, CCL, Python, Scala.
• Create and maintain data structures and data models to support data at all stages of the data pipeline development lifecycle.
• Maintain, tune, and scale big data infrastructure platforms (Snowflake, AWS, Azure) to appropriately handle workloads.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, data acquisition, data enrichment, re-designing infrastructure for greater scalability, etc.
• Perform database performance tuning methods to ensure uptime and data refresh rate SLAs are met.
• Participate in data engineering team code reviews and offer recommendations regarding data architecture.
• Supports departmen t BI Analysts in translating business requirements into Azure DevOps development tasks to properly track and implement solutions that meet the needs of stakeholders by way of agile development lifecycles.
• Align with Billings Clinic data engineering development standards to deliver uniform results providing a consistent and supportable view of data across the organization.
• Duties may require 24 hour, seven days per week on-call responsibility for project support or for escalated support issues
• Supports, monitors, and implements feature enhancements existing data pipeline infrastructure.
• Evaluates and analyzes the current system architecture to improve uptime and responsiveness while providing recommendations to the Principal Engineer and/or Manager of Data Engineering.
• Develops data pipeline solutions as assigned that are automated, scalable, and sustainable while minimizing defects and technical debt.
• Performs re gular data quality checks on source marts and subject area marts to ensure data integrity and proactively identifies areas of opportunity.
• Investigates and troubleshoots complicated analytic application and stability issues.
• Ensures databases are operational and provide valid and relevant data.
• Assists BI Analysts on ad-hoc dataset requests as needed.
• Participates in code reviews and effectively communicate issues and risks to stakeholders
• Incorporate unit testing and regression testing to ensure defect-free builds and releases.
• Supports and models behaviors consistent with Billings Clinic's mission, vision, values, code of business conduct and service expectations. Meets all mandatory organizational and departmental requirements. Maintains competency in all organizational, departmental and outside agency standards as it relates to the environment, employee, patient safety or job performance.
•Performs all ot her duties as assigned or as needed to meet the needs of the department/organization.