The Senior Data Engineer designs scalable data pipelines, collaborates with teams to address business needs, and ensures data infrastructure reliability.
Join a pioneering financial services company that is redefining homeownership through innovative, data-driven solutions. As a Senior Data Engineer, you will be instrumental in designing and building scalable data pipelines, transforming complex datasets, and ensuring the reliability of the data infrastructure. Working closely with data scientists and analysts, you will provide critical insights that shape strategic decisions. If you are passionate about data engineering and thrive in a fast-paced, collaborative environment, this role offers you the opportunity to contribute to cutting-edge projects and help shape the future of data architecture
Requirements
- 5+ years of Python development experience.
- Experience with AWS, including services like DynamoDB, S3, EMR, Glue, Athena, and Redshift.
- Knowledge of distributed computing engines, such as Apache Spark.
- Proficiency in both relational and non-relational databases, along with data modeling expertise.
- Experience setting up code pipelines using CloudFormation or Terraform.
- Familiarity with ETL pipeline development in OLAP and Data Warehouse environments.
- Hands-on experience with orchestration tools like Airflow.
- Understanding of serverless architectures and scalable application design principles.
- Proficiency with version control systems, particularly Git.
- Experience with Node.js, DevOps, and CI/CD practices is a plus.
- Background in cloud technologies (AWS or Azure) and fintech or banking is desirable.
Responsibilities
- Design and implement scalable data pipelines in a horizontally scaled environment.
- Collaborate with cross-functional teams to understand business needs and define transactional scenarios across multiple microservices.
- Write high-quality, maintainable code that meets established standards and best practices.
- Build strong relationships with clients by understanding their business model and delivering solutions that align with their goals.
- Influence and communicate effectively at all levels, including management and engineering teams.
- Utilize distributed computing systems to transform and manage large datasets efficiently.
- Work with cloud-based technologies such as AWS to support the company’s data infrastructure.
- Contribute to continuous integration/continuous delivery (CI/CD) pipelines and DevOps practices.
- Collaborate with data scientists to ensure the availability and accessibility of critical insights for decision-making.
Top Skills
Airflow
Spark
Athena
AWS
Ci/Cd
CloudFormation
DevOps
DynamoDB
Emr
ETL
Git
Glue
Node.js
Olap
Python
Redshift
S3
Terraform
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
As a Senior Data Engineer/Scientist, you'll design and maintain data pipelines, collaborate with analytics teams, and leverage SQL and Python for data manipulation, building ETL pipelines, and integrating datasets for insightful analytics.
Top Skills:
DatabricksPythonSparkSQLTableau
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
As a Senior Data Engineer, you'll design data models, build scalable data pipelines, and improve data quality, collaborating across teams.
Top Skills:
AirflowSparkAthenaAws Data ServicesEmrFlinkHiveJavaKafkaPythonRedshiftSparkSQL
Artificial Intelligence • eCommerce • Food
As a Senior Data Engineer, you'll develop and maintain data pipelines that inform analytics, ensuring data quality for decision-making across teams.
Top Skills:
DatabricksPythonScalaSpark
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
