Software Engineer (Big Data) - Recommendation Ecosystem Infrastructure

The Data Ecosystem Team has the vital role of crafting and implementing a storage solution for offline data in our recommendation system, which caters to more than a billion users. Their primary objectives are to guarantee system reliability, uninterrupted service, and seamless performance. They aim to create a storage and computing infrastructure that can adapt to various data sources within the recommendation system, accommodating diverse storage needs. Their ultimate goal is to deliver efficient, affordable data storage with easy-to-use data management tools for the recommendation, search, and advertising functions. Responsibilities: - Design and implement real-time (streaming computing) data systems for large-scale recommendation systems. - Create flexible, scalable, stable, and high-performance storage systems and computing models. - Troubleshoot production system failures, design and implement necessary mechanisms and tools to ensure overall stability of the production systems - Construct industry-leading streaming computing frameworks and other distributed systems to provide reliable infrastructure for massive data and large-scale business systems - Research, design, and develop computer and network software or specialised utility programs. - Analyse user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis. - Update software, enhances existing software capabilities, and develops and direct software testing and validation procedures. - Work with computer hardware engineers to integrate hardware and software systems and develop specifications and performance requirements.