Full Stack Data Engineer
Caixa Mágica Software · Lisbon, Portugal
Software Development · 51-200 employees
About the role
Develop and maintain end-to-end data analytics features on an AWS cloud-based big data platform. Manage the full development lifecycle including architecture, testing, deployment, and ongoing maintenance.
What they look for
Requirements
Requires strong experience with Python, SQL, and AWS cloud services such as S3, Lambda, and Athena. Candidates should have a background in data processing, Spark, and a proactive approach to problem-solving.
Benefits
Full description
Overview:
The ideal candidate has experience with Python and SQL and is capable of developing, operating, and maintaining data analytics applications on an AWS cloud-based big data platform. They have a strong interest in data science, can work independently and proactively, and integrate well into a small, dynamic team.
What will you do?
- Develop end-to-end features on an AWS cloud-based big data platform, focused on data analysis and processing.
- Actively contribute to discussions on feature architecture and the definition of the required data flows and processes.
- Create proofs of concept.
- Develop production-ready versions, perform testing, deploy to development, quality assurance, and production cloud environments, monitor initial performance, and carry out debugging and ongoing maintenance.
- Document the developed features using Confluence.
- Support data extraction and exploratory data analysis activities.
What are we looking for?
- Experience with Python, particularly in data manipulation and processing;
- Experience with SQL;
- Experience in programming and debugging;
- Interest in data science;
- Experience with AWS cloud services, such as S3, Lambda, Athena, EC2, Glue, AWS CDK/CloudFormation, Step Functions, Batch, and Fargate;
- Experience with Spark/PySpark;
- Experience with Git;
- Experience with Jupyter Notebooks;
- Experience developing applications and/or working in production environments;
- Proactivity and initiative;
- Strong teamwork skills;
- Ability and motivation to independently find solutions to problems;
- Interest in learning new skills.
What can you expect from us?
- A permanent job contract for a long term project;
- Tech equipment + SIM Card + personal smartphone;
- Health and Life Insurance;
- Social events and team buildings;
- The commitment of letting you grow with us, and be rewarded accordingly;
- A dynamic and young team that will be always there to support you;
- Training in the latest technologies;
- Coffee, fruits, snacks and a warm welcoming when you pass by the office.
