Senior Data Engineer (Remote)
Chariot Solutions, United States
Job Description
If you’re interested in working with different technologies and passionate about solving a wide variety of problems and challenges, you’ll feel right at home at Chariot. Our customers bring us in to solve their most complex problems, guide their teams on new initiatives, and provide expert technology advice.
We are proud to be a 4 time winner of Philadelphia Top Workplaces!
Requirements
We are looking for data engineers who can:
- Design, develop, debug, and optimize data pipelines that perform ELT (extract, load, transform) of various data sources (as well as ETL where appropriate)
- Develop and deploy the required infrastructure for reliable pipeline execution, especially on cloud platforms, using IaC (infrastructure as code) tools
- Design and develop data warehouses for large data sets to support business intelligence, understanding end-to-end data needs and modeling accordingly
- Understand where data lakes fit, and how to take advantage of them
- Investigate data quality issues and perform root cause analysis
- Develop supporting utilities and tools
The ability to pick up and run with new technologies is vital. Here are ones we use (deep knowledge of some but not all required):
- Python (or other high level languages)
- SQL: Postgres, MySQL, Microsoft SQL Server, etc.
- Data warehouses: AWS Redshift, Google BigQuery, Snowflake
- NoSQL: Cassandra, AWS DynamoDB, ElasticSearch, etc
- Experience with various file formats, like Parquet and Avro
- Messaging and streaming: Kafka, AWS Kinesis, Apache Flink
- ETL job orchestration, such as Airflow, Dagster, or Luigi
- Big data parallel processing such as Spark and Hadoop
- Terraform, AWS CloudFormation
- Docker and Container Orchestration
- Logging / monitoring (such as ELK, Splunk, CloudWatch, Prometheus, DataDog)
The ability to quickly understand business domains and manage changing priorities is critical. You like getting your hands dirty with data, and are passionate about turning raw, changing production data into something useful for business. You enjoy working directly with business teams and stakeholders, and are eager to be part of an area of growth for Chariot Solutions.
About Chariot Solutions
Chariot Solutions is a consulting firm specializing in software development, systems integration, mobile application development and training. The Chariot team includes many of the top software architects in the Philadelphia area, with deep technical expertise, industry knowledge and a genuine passion for software development.
Here are examples of the types of projects we work on.
Our consultants also love learning, writing, and speaking.
- Chariot’s blog has a wide array of articles across languages, technologies, frameworks, etc.
- We host conferences such as the Philly Emerging Tech Conference. See 2020 talks here.
- We host the Chariot TechCast podcast where we interview tech luminaries and even host our own weekly DevChat shows.
Benefits
We are committed to the well-being of our employees. Here are some of our benefits
- We take care of health, dental, and vision insurance, 100% paid for you and your family
- Paid maternity, paternity and family leave
- Attend an all expenses paid tech conference of your choice every year, in addition to the Emerging Technologies for the Enterprise conference (Philly ETE)
- Opportunities to present at meetups, conferences and run workshops
- A top of the line MacBook Pro or equivalent
- Flexible Spending Account (FSA)
- 401K retirement plan
- Short and Long Term Disability
- Subsidized gym membership
- Company stock
Must be authorized to work in the U.S.
Chariot Solutions provides equal employment opportunities to all employees and applicants without regard to religion, creed, color, national origin, sex, pregnancy, sexual orientation, gender identity, age, ancestry, physical or mental disability, genetic information, marital status or any other classification protected by applicable local, state or federal laws.