Architecting ML-enabled systems: Challenges, best practices, and design decisions

Context: Machine learning is increasingly used in a wide set of applications ranging from recommendation engines to autonomous systems through business intelligence and smart assistants. Designing and developing machine learning systems is a complex process that can be eased by leveraging effective design decisions tackling the most important challenges and by having a good system and software architecture. Goal: The research goal of this work is to identify common challenges, best design practices, and main software architecture design decisions of machine learning enabled systems from the point of view of researchers and practitioners. Method: We performed a mixed method including a systematic literature review and expert interviews. We started with a systematic literature review. From an initial set of 3038 studies, we selected 41 primary studies, which we analysed according to a data extraction, analysis, and synthesis process. In addition, we conducted 12 expert interviews that involved researchers and professionals with machine learning expertise from 9 different countries. Findings: We identify 35 design challenges, 42 best practices and 27 design decisions when architecting machine learning systems. By eliciting main design challenges, we contribute to best practices and design decisions. In addition, we identify correlations among design challenges, decisions and best practices. Conclusions: We believe that practitioners and researchers can benefit from this first and comprehensive analysis of current software architecture design challenges, best practices, and design decisions.