Description & Requirements
About AET
Abyres Enterprise Technologies Sdn. Bhd, a subsidiary of Abyres Holdings Sdn. Bhd. is a leading data engineering consultancy & professional services company based in Cyberjaya, Malaysia. At AET, we believe that data is the most valued asset in today’s digital world. We specializes in leveraging open-source software and open standards to unlock the true potential of data.
Our mission is to empower businesses by transforming raw data into actionable insights and tangible business value. With our expertise and dedication, we turn complexity into clarity, making data work for our clients.
Visit http://elixier.co to know more about AET!
Your Roles
Our clients need expert advise and guidance in implementing solutions for managing and leveraging their data for improving their business. Your primary goal will be to help clients implement data solutions that drive business improvement. You will play a crucial role in understanding client requirements, proposing suitable solutions, and delivering them successfully. As the Lead Engineer, you will oversee end-to-end ETL processes, manage a team, and collaborate with various stakeholders to ensure effective data extraction, transformation, and loading into a Kimball data warehouse. You’ll employ technologies such as Python, SQL (MSSQL, MySQL, Hive, PostgreSQL), Apache Spark (PySpark), and orchestration tools like Apache Airflow and Kubeflow. Your role also involves containerization using Kubernetes, Helm, and Kubedirector to maintain data quality and compliance for datadriven decision-making.
Responsibilities
- Take ownership of the proposed data solutions and drive their successful implementation. Work closely with cross-functional teams, including data engineers, developers, and project managers, to ensure timely and high-quality delivery.
- Establish strong relationships with clients and serve as their trusted advisor. Understand their pain points and objectives, and provide strategic guidance on how data solutions can address their business challenges.
- Lead and manage ETL development activities, ensuring adherence to project timelines, budgets, and quality standards. Proactively identify and mitigate risks to ensure smooth project execution.
Requirements
- Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, or a related field.
- 5+ years of hands-on experience in ETL development and data engineering.
- Ability to work independently and self directed.
- Proven experience in data engineering, including ETL pipelines development and implementation and data warehousing.
- Able to lead ETL development team, plan the tasks and work collaboratively with the team.
- Proficiency in programming languages such as Python, and SQL (MSSQL, MySQL, Hive, PostgreSQL)
- Proficiency in data technologies (Apache Spark/PySpark, Apache Airflow, KubeFlow and S3)
- Solid understanding of Kimball Data Warehouse design (dimension table, fact table, SCD tables)
- Strong understanding of data management, data integration, and data governance principles.
- Hands on experience with deployment on major platforms (eg: Kubernetes, Linux, Docker, Helm, Kubedirector)
- Excellent problem-solving skills and the ability to translate business requirements into technical solutions.
- Exceptional communication and interpersonal skills, with the ability to effectively interact with clients and stakeholders.
- Nice To Have
- Experience in data technologies such as Spark, Kubernetes, Jupyter, Neo4j, KNIME, Cloudera, HPE Ezmeral, PostgreSQL, Citus, Presto, Trino, Hive, Airflow, Superset, Pynecone, NodeJS, MinIO, Rancher, Gitlab CE, FalconLLM, or OpenAI.
- Experience theories and practices around software engineering, software architecture, Kimball data warehouse dimensional modeling, knowledge graph, natural language processing, intent parsing, or knowledge management.
Why Abyres?
Why choose Abyres for your next career move? Here are just a few reasons:
- Cutting-edge Technology – Join our team and work with the latest open-source software and open standards, ensuring you stay at the forefront of data engineering advancements. We believe in the power of collaboration and openness, driving innovation and pushing boundaries.
- Impactful Projects – Be a part of exciting and challenging projects that make a real difference for our clients. From data integration and transformation to advanced analytics and machine learning, you’ll have the opportunity to work on diverse projects across various industries, sharpening your skills and expanding your knowledge.
- Collaborative Environment – We foster a collaborative and inclusive work culture where everyone’s ideas are valued. At AET, you’ll work alongside talented professionals who are passionate about data engineering and share a common goal of delivering exceptional results. Our team-oriented approach ensures that you’ll always have the support and guidance you need to excel.
- Continuous Learning – We believe in the continuous development of our team members. Through mentorship programs, training workshops, and access to learning resources, we provide ample opportunities for you to enhance your skills and stay ahead in this rapidly evolving field.
Join Abyres and embark on an exciting journey where you’ll be empowered to make a real impact with data. Apply now and become a valued member of our team!
