Job Description
Designs, builds, and maintains data systems and pipelines that collect, process, store, and deliver reliable data for analysis and business decision making.
Key Responsibilities
- Design and maintain data pipelines and workflows.
- Collect, clean, transform, and validate data.
- Build and manage databases and data warehouses.
- Develop ETL and ELT processes.
- Integrate data from multiple sources and systems.
- Monitor data quality, availability, and performance.
- Optimize queries, databases, and data processing systems.
- Implement data security, backup, and access controls.
- Collaborate with data analysts, scientists, and software engineers.
- Document data architecture, pipelines, and processes.
Required Skills & Competence
- SQL and database management
- Python or similar programming languages
- ETL and ELT development
- Data warehousing
- Data modeling
- Apache Spark or similar technologies
- Cloud data platforms
- Database optimization
- Data quality and validation
- Git and version control
- Data security
- Problem solving and analytical thinking