Senior Data Engineer
Congo Brands, United States
Job Description
Our Company:
Congo Brands is at the epicenter of where consumer demand meets product development. Through brand partnerships with lead influencers, we are afforded an inside look at what consumers truly desire. E-commerce becomes the laboratory for consumer feedback and serves as a data pool for future retail endeavors. Congo Brands is a health and wellness brand that sells supplements, fitness snacks, hydration drinks, energy drinks, and more through ecommerce and retail partnerships. Congo Brands houses some of the World’s most noted brands including Alani Nu, Prime Hydration, Prime Energy, and 3D Energy only to include more in the coming years. Congo Brands is on pace to become one of the World’s most noted consumer product brand companies, with revenue growth increasing exponentially year over year. Congo Brand’s goal is to be the go-to ‘Better for You Brand’ for everything from nutritional supplements, energy drinks, snacks and much more.
The Role:
The Senior Data Engineer is responsible for developing, maintaining, and optimizing data pipelines and databases, ensuring data is accessible, secure, and accurate. Your expertise in coding, database platforms/technologies, and various data tools will be critical to our data-driven decision-making process. Teamwork and collaboration are key traits required to be successful in this role. You must be hungry to learn the industry and our business model to help drive efficiencies and contribute to the longer-term strategic goals.
This role will report to the Manager, Business Intelligence
Key Responsibilities:
- Data Platform Development: Leveraging skill and expertise as a critical contributor to the design and development of data platform development activities, including data warehousing, data analysis, development of various data cubes, and more.
- Data Pipeline Development: Develop, implement, and maintain data pipelines to efficiently extract, transform, and load (ETL) data from various sources into our data platforms.
- Data Modeling: Design and implement data models and schema to support data warehousing and analytics needs.
- Coding: Write and optimize code to ensure the efficiency and reliability of data processing tasks. Proficiency in Python, SQL, and C#.
- Database Management: Manage database platforms, including MS-SQL Server, other SQL databases, SnowFlake, Azure SQL, and Dataverse, ensuring data integrity, performance, and security.
- Reporting and Visualization: Utilize tools like PowerBI to create interactive dashboards and reports, allowing stakeholders to access and analyze data easily.
- Automation: Develop and maintain automation processes using Power Automate and Azure Logic Apps to streamline data workflows and reduce manual efforts.
- Machine Learning Integration: Collaborate with data scientists and machine learning engineers to integrate machine learning models into data pipelines and applications.
- Cloud Services: Leverage Azure services, including Azure ML, Azure Function Apps, and other cloud technologies to optimize data processes and ensure scalability.
- Data Quality Assurance: Implement data quality checks and monitoring to ensure data accuracy, consistency, and completeness.
- Performance Optimization: Continuously identify and implement improvements to enhance data pipeline efficiency, performance, and scalability.
- Documentation: Maintain detailed documentation of data processes, schemas, and code for future reference and knowledge sharing.
Key Skills and Experience Preferred:
- Proficiency in coding, with a strong emphasis on Python and SQL.
- Experience in C#, JavaScript a plus.
- Solid experience with database platforms and technologies, including MS-SQL Server, other SQL databases, PowerBI, Azure SQL, Dataverse, and SnowFlake.
- Familiarity with Power Automate and Azure Logic Apps for workflow automation.
- Knowledge of Azure ML and machine learning concepts.
- Experience working with Azure services, Azure Function Apps, and cloud technologies.
- Strong analytical skills and the ability to work with complex data sets.
- Proficiency in data modeling and database design.
- Other Qualifications:
- Bachelor's degree in computer science, data engineering, or a related field.
- 5+ years of experience in data engineering, data architecture, or a related role.
- A combination of experience and education considered.
- Possess a working knowledge of various forms of business data, such as financial, sales, supply chain, inventory, etc.
- Excellent problem-