2024 1

CAVALCANTE et al. LOOKING BACK AND FORWARD: A RETROSPECTIVE AND FUTURE
DIRECTIONS ON SOFTWARE ENGINEERING FOR SYSTEMS-OF-SYSTEMS

Corresponding author Everton Cavalcante, Instituto Metrópole Digital,
Universidade Federal do Rio Grande do Norte, Campus Universitário - Lagoa
Nova, 59078-900, Natal, Rio Grande do Norte, Brasil.

# Looking back and forward: A retrospective and future directions on Software
Engineering for systems-of-systems

Everton Cavalcante  Thais Batista  Flavio Oquendo  Federal University of Rio
Grande do Norte, Natal, Brazil  IRISA-UMR CNRS, Université Bretagne Sud,
Vannes, France<EMAIL_ADDRESS>Cavalcante E  Batista T  Oquendo F

###### Abstract

[Abstract]Today’s complex systems are becoming increasingly larger and more
integrated into other existing systems, thus leading to the so-called systems-
of-systems (SoS). An SoS can be understood as a set of independent
heterogeneous systems that interact with each other to offer, through emergent
behaviors arising at runtime, new functionalities and accomplish global
missions. The peculiarities that distinguish SoS from traditional systems
impose research challenges from the Software Engineering point of view,
motivating the need for a paradigm shift and new approaches for designing,
developing, deploying, and evolving these systems. The International Workshop
on Software Engineering for Systems-of-Systems (SESoS) series started in 2013
to fill a gap in scientific forums addressing SoS from the Software
Engineering perspective, becoming the first venue for this purpose. This
article reports a study aimed at understanding the past, present, and future
of Software Engineering for SoS by providing an overview of the research
reported in papers that appeared in the SESoS workshop throughout its eleven
editions (2013-2023). The study combined scoping review and scientometric
analysis methods to categorize and analyze research in Software Engineering
for SoS presented in 57 papers concerning time, geographic location, topics of
interest, types of research conducted, application domains, and research
impact. Based on the panorama presented, this article discusses current and
future directions in Software Engineering for SoS.

, , . Looking back and forward: A retrospective and future directions on
Software Engineering for systems-of-systems. J Softw Evol Proc.
2024;00(00):1–LABEL:LastPage.

###### keywords:

systems-of-systems, Software Engineering, scoping review, scientometric
analysis

††articletype: Special Issue - Review Article††journal: J Softw Evol
Proc††volume: 00††footnotetext: Abbreviations: ACM CCS, ACM Computing
Classification System; CBSoft, Brazilian Conference on Software: Theory and
Practice; ECSA, European Conference on Software Architecture; ICSA, IEEE
International Conference on Software Architecture; ICSE, IEEE/ACM
International Conference on Software Engineering; INCOSE, International
Council on Systems Engineering; SBES, Brazilian Symposium on Software
Engineering; SESoS, International Workshop on Software Engineering for
Systems-of-Systems and Software Ecosystems; SLR, systematic literature review;
SMS, systematic mapping studies; SoIS, systems of information systems; SoS,
systems-of-systems; WDES, Workshop on Distributed Software Development,
Software Ecosystems and Systems-of-Systems.

##  1 Introduction

Current systems have become increasingly complex mainly due to the diverse
hardware and software resources they require, the processes (processing,
monitoring, management, communication, etc.) involved in their life cycle, and
the interaction with other systems, actors, and artifacts. In this context, it
has been possible to observe a growing interest in the research and
development of large-scale complex systems resulting from integrating other
independent systems, the so-called systems of systems (SoS). This class of
systems has been observed in several application domains, such as defense,
environmental monitoring, energy, emergency management and coordination,
transportation, healthcare, and smart cities. In these scenarios, it is common
to have high-level objectives that cannot be addressed separately by the
elements that make up the system but that can eventually be satisfied as a
result of the integration and interaction between them.1

Due to the fact that SoS represent a topic in evident research activity,
several studies in the literature present different definitions and
characterizations for SoS.2 Recently, the international standard ISO/IEC/IEEE
21840:2019 defined SoS as “a set of systems and system elements that interact
to provide a unique capability that none of the constituent systems can
achieve on its own.” 3 This is exactly one of the most notable characteristics
of SoS, in that the interaction between constituent systems makes an SoS able
to offer new functionalities not provided by any of those constituents if they
operate in isolation. Furthermore, the community has achieved convergence in a
set of characteristics inherent to this class of systems, all of them to be
satisfied to consider a system as an SoS.4. These characteristics, which
distinguish an SoS from other complex and large-scale systems, are:

  * •

Operational independence: Every constituent system of an SoS operates
independently of each other to fulfill its own mission;

  * •

Managerial independence: Every constituent system of an SoS is managed
independently and may decide to evolve in ways unforeseen when they were
originally combined;

  * •

Geographical distribution: Constituent systems of an SoS are physically
decoupled in the sense that only information can be transmitted among
constituent systems;

  * •

Evolutionary development: As a consequence of the independence of the
constituent systems, an SoS as a whole may evolve to respond to changes in its
constituent systems and operational environment. Moreover, constituent systems
may be only partially known at design time;

  * •

Emergent behaviors: In an SoS, new behaviors emerge from the local interaction
of its constituent systems, i.e., an emergent behavior that cannot be
performed by a constituent system alone. These emergent behaviors may be
ephemeral because the systems composing the SoS evolve independently, which
may impact their availability.

SoS have been studied for several years within Systems Engineering, but they
still represent an emerging domain in the research community. Following the
trend of software evolution, many SoS have been fundamentally driven by
software.5, 6, 7 Therefore, the existence of software-intensive SoS suggests a
paradigm shift characterized by the emergence of systems whose software part
is becoming increasingly larger and more complex in terms of scale and
dynamics.8

The SoS characteristics that make these systems unique are the same ones that
pose significant Software Engineering challenges.1 Many software engineers are
used to designing and developing centralized or distributed software, but not
decentralized. Even worse, the characteristics of SoS are often misunderstood,
especially the concept of emergent behavior, which arises from the interaction
between constituent systems and that may be desired but unexpected,
undesirable, or ephemeral. This leads to the following oxymoron: how to design
and develop an SoS to exhibit unexpected (i.e., emergent) expected (by design)
behaviors that arise from interactions between its constituent systems?

Unlike traditional systems, SoS exhibit emergent behavior. Even though the
behavior of a system can be understood as the sum of the behavior of its
components, this reductionism fails in SoS. An SoS behaves in ways that cannot
be predicted exclusively from the individual analysis of its constituent
systems. Furthermore, SoS are characterized by evolutionary development that
makes it possible to maintain emergent behavior to sustain SoS missions.
Furthermore, SoS must be designed to operate in open-world environments where
the constituent systems are only partially known at design time.

All these aforementioned issues require theories, methods, techniques, and
tools to properly design, develop, operate, and evolve such complex, dynamic
software-intensive SoS. In contrast to current closed-world systems, such
open-world, mission-critical systems run on pervasive devices and networks
providing services that are dynamically discovered and used to deliver more
complex services. Dealing with these challenges in the context of Software
Engineering for SoS is a major challenge, but it comes with the great promise
of advancing the state of the art and state of practice. However, research in
the area has already recognized that existing Systems Engineering and Software
Engineering methods have limited applicability or do not apply to SoS.9, 10.
This challenge creates a need for forums where researchers and professionals
can exchange ideas and experiences, analyze research and development issues,
and discuss solutions in Software Engineering for SoS.

The Annual System of Systems Engineering Conference (IEEE
SoSE)†††http://www.sosengineering.org/, held for 18 years by the IEEE System,
Man, and Cybernetics Society and the International Council on Systems
Engineering (INCOSE), is an international conference whose interest lies in a
wide range of topics related to theories, methods and applications in Systems
Engineering. Although there certainly are scientific contributions as papers
dealing with Software Engineering for SoS, IEEE SoSE does not have this
background. Due to the lack of a specific forum about this field, the
International Workshop on Software in Software Engineering for Systems-of-
Systems (SESoS) emerged in 2013 with a focus on the theory and practice of how
Software Engineering could be explored to construct software-intensive SoS.

SESoS is a workshop that has been held annually as part of important Software
Engineering international conferences, such as the IEEE/ACM International
Conference on Software Engineering (ICSE), the IEEE International Conference
on Software Architecture (ICSA) and the European Conference on Software
Architecture (ECSA). The workshop has focused on various topics related to the
design, construction, analysis and evolution of software-intensive SoS, in
addition to being interested in reporting experiences and successful cases
related to SoS and discussing research challenges and future perspectives in
Software Engineering for SoS. With these characteristics, SESoS became the
first forum to address SoS from a Software Engineering perspective, annually
attracting researchers and professionals from different locations around the
world. As an evolution of the research field and the workshop, SESoS 2022
revisited and expanded its scope to address systemic thinking and complexity
in SoS and software ecosystems, making it possible to reflect on how the
different dimensions of these systems (technological, organizational and
social) must be addressed in research and practice.11 A consequence of this
change was renaming the workshop to International Workshop on Software
Engineering for Systems-of-Systems and Software Ecosystems since 2022.

As a result of the 11 editions held to date, the volume of scientific material
documented in SESoS papers can be regarded as significant for analyzing the
nature, extent, and impact of research in Software Engineering for SoS. From
this perspective, this article reports a study aimed at understanding the
past, present, and future of Software Engineering for SoS through an overview
of the research reported in papers published at the SESoS workshop between
2013 and 2023. The study combined scoping review12, 13 and scientometric
analysis14, 15 methods to categorize and analyze research in Software
Engineering for SoS presented by a corpus of 57 papers manually collected from
SESoS proceedings available in the ACM Digital Library and IEEE Xplore. The
screening process used to extract data and the subsequent data analysis
allowed for understanding the research concerning temporal distribution,
geographic location, more or less prevailing topics, types of research,
application domains, and research impact. In addition to the presented
panorama, this article discusses current and future directions in Software
Engineering for SoS.

This remainder of this article is structured as follows. Section 2 presents a
brief motivation and history of the SESoS workshop. Section 3 describes the
methodology used in this study, including the research methods, questions, and
adopted procedures adopted. Section 4 contains a detailed analysis of the
SESoS research contributions, discussed from different perspectives. Section 5
highlights current and future directions in Software Engineering for SoS.
Finally, Section 6 contains some conclusions.

##  2 The SESoS workshop

The SESoS workshop series was created ten years ago, aiming at providing a
forum for researchers and practitioners to exchange ideas and experiences,
analyze research and development issues, discuss promising solutions, and
propose visions for the future in the field of Software Engineering for SoS.
At that time, SoS were becoming a hotspot from both the research and the
industrial viewpoints, in particular related to crucial classes of
applications, such as safety-critical or mission-critical SoS. However, there
was a lack of forums addressing SoS from the Software Engineering perspective.
SESoS was the indeed first venue specifically addressing Software Engineering
for SoS.

In 2013, the first edition of SESoS was organized in Montpellier, France, as a
one-day, full-day workshop within ECSA, one of the most relevant international
conferences about software architectures, attracting participants from
different places in the world. Two categories of papers are solicited, namely
(i) regular research papers describing original, substantial contributions in
research and/or practice, and (ii) short/position papers and experience
reports describing emerging results, lessons learned, open problems or novel
ideas and perspectives on research. The workshop program has been typically
organized upon invited keynotes and technical sessions for paper presentations
and discussions among participants. The topics of interest of SESoS cover
different aspects of SoS development from a Software Engineering perspective,
including architectural design, development techniques, extra-functional
concerns, and verification and validation. The workshop also welcomes
contributions reporting experiences and successful case studies, and papers
discussing open issues, challenges, and future perspectives for future
research. Table 1 summarizes the general topics of interest of the SESoS
workshop.

Table 1: General topics of interest of the SESoS workshop. Category |  Topics
---|---
Analysis & Architecture |  Requirements Engineering, architectural description, architectural evaluation, architectural evolution
Model-Based Engineering |  Modeling and simulation, Model-Driven Engineering, models at runtime
Construction & Evolution |  Techniques and technologies, service-orientation, component framework, evolutionary development
Experience |  Industrial perspectives, successful case studies
General Issues |  Characteristics of SoS, quality in SoS, SoS development processes, acquisition and project management, verification and validation of SoS
Challenges & Directions |  Future perspective, challenges, and directions of SoS research and development

Figure 1 illustrates a world map showing the location of all SESoS editions.
Eleven editions of SESoS were held so far, in conjunction with ECSA, ICSE, and
ICSA. The co-location of SESoS with premier venues on Software Engineering has
attracted researchers and practitioners working on or interested in
approaching SoS, besides contributing to a greater international visibility of
the workshop. The two first SESoS editions (2013 and 2014) were organized as
part of ECSA and eight editions have been yearly held with ICSE since 2015. In
2020, the workshop was part of the ICSA program. Table 2 presents some
historical information about the SESoS editions. The upcoming SESoS edition,
the 13th of the series, is already planned to happen again with ICSE in May
2024: http://sesos2024.icmc.usp.br.

Table 2: SESoS editions.

Edition | Date | Venue | URL | Submitted papers | Accepted papers | Acceptance rate
---|---|---|---|---|---|---
SESoS 2013 | ECSA 2013 | July 2, 2013 | http://sesos2013.icmc.usp.br | 16 | 10 | 62.5%
SESoS 2014 | ECSA 2014 | August 29, 2014 | http://sesos2014.icmc.usp.br | N/A | 11 | N/A
SESoS 2015 | ICSE 2015 | May 17, 2015 | http://sesos2015.icmc.usp.br | 13 | 8 | 61.5%
SESoS 2016 | ICSE 2016 | May 16, 2016 | http://sesos2016.icmc.usp.br | 9 | 6 | 66.7%
SESoS/WDES 2017 | ICSE 2017 | May 23, 2017 | http://sesos-wdes-2017.icmc.usp.br | 21 | 15 | 71.4%
SESoS 2018 | ICSE 2018 | May 29, 2018 | http://sesos2018.icmc.usp.br | N/A | 5 | N/A
SESoS/WDES 2019 | ICSE 2019 | May 28, 2019 | http://sesos-wdes-2019.icmc.usp.br | 14 | 10 | 71.4%
SESoS/WDES 2020 | ICSA 2020 | November 2, 2020 | http://sesos-wdes-2020.icmc.usp.br | 18 | 8 | 44.4%
SESoS/WDES 2021 | ICSE 2021 | June 3, 2021 | http://sesos-wdes-2021.icmc.usp.br | 11 | 5 | 45,4%
SESoS 2022 | ICSE 2022 | May 16, 2022 | http://sesos2022.icmc.usp.br | 11 | 6 | 54.5%
SESoS 2023 | ICSE 2023 | May 14, 2023 | http://sesos2023.icmc.usp.br | 21 | 10 | 47.6%
Average | 14.89 | 8.55 | 58.4%

Data about the number of submitted and accepted papers were collected from the
front matter of SESoS proceedings or defined based on the number of published
papers. No information was available about the number of papers submitted to
SESoS 2014 and SESoS 2018 (N/A in the table).

Figure 1: Locations of SESoS editions (2013-2023).

Four editions of SESoS (2017, 2019, 2020, and 2021) were held jointly with the
Workshop on Distributed Software Development, Software Ecosystems, and
Systems-of-Systems (WDES). The WDES workshop series started in 2007 in Brazil
and was co-located with the leading national software engineering conferences,
the Brazilian Symposium on Software Engineering (SBES) and later the Brazilian
Conference on Software: Theory and Practice (CBSoft). WDES initially focused
on distributed software development, an approach based on distributed
resources and personnel to reduce cost, improve productivity and software
quality, and reach new Information Technology markets. The workshop evolved
over the years to include software ecosystems and SoS, which have Software
Engineering-related problems and challenges amplified due to their distributed
nature. A software ecosystem refers to a system formed over a common
technology platform that joins a set of actors and artifacts that are
responsible for software solutions based on contributions from a socio-
technical network16 so that the social and technical issues related to
software development made this topic as of necessary investigation in Software
Engineering. WDES gained international projection since 2017 when it joined
SESoS in ICSE 2017.

The successful four editions of SESoS/WDES as a joint workshop and the synergy
between SoS and software ecosystems resulted in merging SESoS and WDES as a
single workshop in 2022, when SESoS was renamed to International Workshop on
Software Engineering for Systems-of-Systems and Software Ecosystems, now
explicitly incorporating software ecosystems. Putting SoS and software
ecosystems together enables researchers and practitioners to understand how
different intertwined dimensions (technical, social, business, etc.) affect
the quality and evolution of these systems16. Therefore, SESoS took a step
forward to cope with the challenges and opportunities in the design,
development, deployment, and evolution of complex software-intensive systems
such as SoS and software ecosystems, now considering technological,
organizational, and social aspects in this context.11

##  3 Methodology

This section details the research methodology adopted in this work. Section
3.1 briefly introduce the used research methods, namely scoping review and
scientometric analysis. Section 3.2 poses the research questions guiding this
study. Section 3.4 describes the procedures for collecting SESoS papers and
filtering out the ones for further analysis. Section 3.4 details the data
extraction and analysis procedures.

###  3.1 Research methods

Scoping review. A scoping review is a type of secondary study that seeks to
synthesize evidence by identifying and evaluating the literature on a specific
topic, field, concept, or issue, primarily to establish an overview or
categorization of existing work in a given area of research, irrespective of
type (primary studies, secondary studies, non-empirical evidence). Scoping
reviews are used to understand the nature and extent of available research
evidence12, thereby tending to be even more exploratory than other types of
secondary studies, such as systematic literature reviews (SLRs) and systematic
mapping studies (SMS).17, 13 Scoping reviews thus become suitable for
categorizing bodies of knowledge in complex, heterogeneous areas18 like
Software Engineering for SoS.

The scoping review conducted in this study followed the guidelines organized
by Peters et al.18 The goal was to categorize research in Software Engineering
for SoS presented in SESoS papers concerning time, geographic location, topics
of interest, types of research conducted, and application domains or contexts.
It is worth mentioning that scoping review is a term that is sometimes used in
a similar way or even interchangeably with SMS (in Software Engineering as
well19, 20) since both methods have much in common regarding objectives and
procedures.21, 22 However, this study follows Campbell et al.13 in placing
scoping reviews and SLRs/SMS as distinct concepts.

Scientometric analysis. Scientometrics refers to the study of measurement of
aspects such as performance, impact, international collaboration, etc., of
scientific activities based on the academic production of a given field, i.e.,
papers and citations.14 In recent years, scientometrics has come to play a
major role in the measurement and evaluation of research performance and
mapping scientific knowledge.15

Scientometric analysis typically uses four techniques: (i) co-word analysis,
which comprises building networks of co-occurring terms or keywords; (ii) co-
author analysis, which focuses on collaboration networks established among
authors, institutions, and countries; (iii) citation analysis, which concern
analyzing the number of citations of papers and venues as measures of impact;
and (iv) document clustering, which reveals similarities between papers. These
techniques are often aided by software tools such as
CiteSpace‡‡‡http://cluster.cis.drexel.edu/~cchen/citespace/,
VOSViewer§§§https://www.vosviewer.com, and
NVivo¶¶¶https://lumivero.com/products/nvivo/.

The scientometric analysis in this work used co-word, co-author, and citation
analysis. VOSViewer was used to map and visualize research collaborations in
SESoS papers, thus allowing for deductions about people and locations carrying
out research in Software Engineering for SoS. Searches in Google Scholar
obtained the number of citations for each SESoS paper.

###  3.2 Research questions

This study aims to understand the past, present, and future of Software
Engineering for SoS by providing an overview of the research reported in the
papers that appeared in the SESoS workshop. To achieve this goal, six research
questions (RQs) were formulated as follows:

  1. RQ1:

Where research on Software Engineering for SoS has been conducted?
Rationale: To identify the SESoS authors’ origins and collaboration networks
(academic and academia–industry) based on the authors’ affiliations.

  2. RQ2:

What are the most prevalent topics addressed by the papers?
Rationale: The classification of each paper into the topics listed in the
workshop’s Call for Papers allows identifying the most addressed topics and
understanding of how the interest in them has changed over time.

  3. RQ3:

Which Software Engineering fields do the papers intersect with?
Rationale: The classification of each paper into a taxonomy of Software
Engineering topics allows mapping the research on Software Engineering for SoS
to a broader field.

  4. RQ4:

Which types of research do the studies concern?
Rationale: The classification of each paper into a research type allows
identifying the proposed approaches and assessing the degree of maturity of
the contributions.

  5. RQ5:

What application domains or contexts are the studies inserted into?
Rationale: to verify if the research reported in SESoS papers is specific to a
given application domain or context.

  6. RQ6:

How is the impact of SESoS papers? Rationale: to identify the most cited SESoS
papers and understand how the research has been put forward.

###  3.3 Search strategy and eligibility criteria

The collection of all the papers published in the SESoS editions from 2013 to
2023 followed a manual process. It has started by accessing the SESoS 2023
webpage (the last edition) available at
http://sesos2023.icmc.usp.br/Previous.php and following each link to the
workshop proceedings from 2013 to 2022 as the SESoS webpage is updated each
year to list the proceedings of all the past editions. ACM and IEEE have
published SESoS proceedings since the first edition of the workshop, so all
the papers are available in the ACM Digital Library∥∥∥https://dl.acm.org and
IEEE Xplore******https://ieeexplore.ieee.org/. In the case of SESoS 2023, the
workshop proceedings are available in IEEE Xplore, and all the papers were
manually retrieved from there. The collection process on SESoS proceedings
resulted in 94 papers.

Even though the focus of SESoS has always been on Software Engineering for
SoS, not all papers appearing in the workshop strictly concern this since the
topics of interest listed in the Call for Papers are non-exhaustive and admit
similar themes. Therefore, the SESoS proceedings often contain papers
addressing research that is not necessarily related to SoS. As this work is on
Software Engineering for SoS, the second activity was filtering out the
collected papers to exclude those not related to SoS. The presence of papers
not directly focusing on SoS but notably on software ecosystems became evident
mainly in the joint editions of SESoS and WDES (2017, 2019, 2020, and 2021).
Moreover, since its tenth edition (2022), the workshop changed its name and
the topics of interest listed in the current Call for Papers of SESoS now
explicitly reflect addressing software ecosystems.

###  3.4 Data extraction and analysis

All the papers collected from the SESoS proceedings were screened and 37
papers were excluded as not being related to SoS, thereby resulting in 57
papers. Figure 2 depicts the steps for obtaining the corpus of analysis for
this work.

Figure 2: Process for obtaining and analyzing the SESoS papers.

The next step consisted in manually screening each paper to extract relevant
data for answering the defined RQs. Table 3 lists the data items extracted
from SESoS papers, which were organized in an electronic sheet. After filling
out the data extraction sheet, data items were quantitatively analyzed with
the support of the R language and environment. The Goals/Contributions item of
data extraction underwent inductive open and axial coding procedures23, 24,
which allowed for qualitatively classifying each paper into the SESoS topics
of interest and the Software Engineering concepts from the 2012 ACM Computing
Classification System (CCS)25 and identifying the research type it addressed.
Moreover, the citation count of each paper was automatically scraped by using
the SerpAPI Google Scholar API††††††https://serpapi.com/google-scholar-api.

Table 3: Data items to be extracted from SESoS papers. Item |  Description
---|---
Year |  SESoS edition where the paper was published
Title |  Paper title
Paper type |  Regular or short/position paper (regular papers: up to eight pages; short/position papers: up to four or five pages)
Number of citations |  Citation count (automatically retrieved from Google Scholar)
Authors’ affiliation country |  Countries of origin of the authors
SESoS topics |  Topics of interest as listed in the SESoS Call for Papers
Related Software Engineering field |  One or more Level 2 concepts from the ACM CCS within the Software and its engineering category
Goals/Contribution |  Goals of the paper and its contributions
Research type |  One of Validation, evaluation, solution proposal, philosophical, opinion, experience, secondary study
Empirical methods used (if applicable) |  Empirical methods used in the research (notably in the evaluation research type)
Application domain or context |  Application domain or context which the paper refers to, or non-specified

Finally, it is important to mention three actions related to classifying SESoS
papers. First, the classification into the Software Engineering topics
considered only the first two levels of the Software and its engineering
category of ACM CCS (see Table 4) to reduce the probability of bias due to
misclassification. Second, the classification into research types was done
based on the categories presented in Table 5, which were adapted from Wieringa
et al.’s work26 by including the secondary study category due to the presence
of this type of study in the analyzed papers. Third, the classification into
the topics of interest of SESoS considered the ones presented in Table 1,
which were maintained for most workshop editions. Although the division of the
topics of interest into these categories has not been adopted since SESoS
2022, choosing to use them allowed for a uniform classification of the papers.

Table 4: Concepts within the Software and its engineering of the 2012 ACM Computing Classification System25 used to classify SESoS papers. Concept (Level 1) |  Sub-concepts (Level 2)
---|---
Software organization and properties |  Contextual software domains, Software system structures, Software functional properties, Extra-functional properties
Software notations and tools |  General programming languages, Formal language definitions, Compilers, Context specific languages, System description languages, Development frameworks and environments, Software configuration management and version control systems, Software libraries and repositories, Software maintenance tools
Software creation and management |  Designing software, Software development process management, Software development techniques, Software verification and validation, Software post-development issues, Collaboration in software development, Search-based software engineering
Table 5: Taxonomy of research types – adapted from Wieringa et al.26 Research type |  Description
---|---
Validation |  Investigation of the properties of a solution proposal through experiments, prototyping, formal analysis, and similar techniques.
Evaluation |  Investigation of a problem or an implementation of a technique in practice. Causal relations and implications in terms of benefits and drawbacks are discussed upon solid empirical evidences.
Solution proposal |  Proposal of a solution for a problem, either novel or a significant extension of an existing technique. The potential benefits and the applicability of the solution are demonstrated by a small example or proof-of-concept or via a thorough line of argumentation.
Philosophical |  Sketch of a new way of looking at things, a new conceptual framework, etc.
Opinion |  Expression of author’s opinions whether a certain technique is good or bad, or how things should been done
Experience |  Explanation of what and how something has been done in practice through case studies, surveys, and similar empirical research. It also encompasses the author’s experience in a given context (e.g., an industrial case) or personal experience, typically describing insights and lessons learned.
Secondary study |  Report on the conduction and results of a literature review (whether systematic or not) upon studies available in the literature.

##  4 Results

This section presents the results of this study. We provide an overview of the
distribution of the included papers (Section 4.1), followed by answers to the
defined RQs based on the information extracted from them (Sections 4.2 to
4.7).

###  4.1 Distribution of studies

Figure 3 depicts the temporal distribution of papers along the SESoS editions,
with an average of 5.18 papers per year. The chart also shows a significant
decrease in the number of papers in 2021 and 2022, undoubtedly due to the
COVID-19 pandemic, which impacted scientific production worldwide.27, 28
However, it is not possible to state that the pandemic affected the number of
papers in 2020 because the submission deadline to the SESoS edition in that
year was in mid-January, before the COVID-19 outbreak and the first acute
phase of the crisis.

Figure 3: Temporal distribution of papers along the SESoS editions.

Another important facet to analyze is the proportion of papers on SoS to the
number of papers accepted in SESoS editions. The topics of interest of the
first four editions (2013-2016) and the sixth one (2018) significantly focused
on Software Engineering for software-intensive SoS, thus resulting in the
majority of the papers accepted to the workshop being related to this theme,
ranging from 75% (2015) to 100% (2018). In 2017 and from 2019 to 2021, SESoS
was held as a joint workshop with WDES. Consequently, the topics of interest
included other themes not necessarily related to SoS, such as software
ecosystems and distributed software development. Moreover, the last two
editions of SESoS (2022 and 2023) have explicitly incorporated other topics
beyond SoS, primarily those related to software ecosystems. These facts also
explain the reduced number of papers about SoS in the SESoS workshop.

###  4.2 Research loci

RQ1 concerns identifying where the research on Software Engineering for SoS
has been conducted, including collaboration networks and industry involvement.
Table 6 shows the number of papers per affiliation country of the authors. It
is possible to notice that the research has been concentrated in Brazil,
Europe (more specifically, France, Spain, Germany, Sweden, Austria, and
Italy), and South Korea. Minor participation also comes from other places,
e.g., the USA and Australia.

Table 6: Top-five contributing countries to SESoS between 2013 and 2023. Country | Number of papers
---|---
Brazil | 25 (47.83%)
France | 14 (24.56%)
Sweden | 6 (10.52%)
South Korea | 6 (10.52%)
Germany | 6 (10.52%)
Spain | 5 (8.77%)

Figure 4 shows a co-authorship network based on the authors’ affiliation
countries. An analysis of this network reveals four loosely-tied, small
clusters, thereby indicating that research is mainly fragmented with few
international collaborations. Brazil ($n=25$) and France ($n=14$), which have
the two highest number of papers in SESoS, are the countries with the most
robust collaborations with six co-authored papers.

Figure 4: Co-authorship country network for SESoS papers (2013-2023). The
larger the node, the greater the number of papers; the thicker the link, the
greater the number of co-authored papers.

The analysis of the authors’ affiliation also concerned the involvement of
industry and collaborations with academia. Almost all the papers are authored
by researchers from universities and research institutes, and only one has an
author from industry (paper P9). Nonetheless, it is worthwhile mentioning that
some papers report research in collaboration with or applied to industrial
scenarios, such as papers P43, P45, P56, and P57. Some institutions
contributing to SESoS are also well-known for significant applied research in
partnership with industry, such as the Fraunhofer Institute for Experimental
Software Engineering in Kaiserslautern, Germany.

###  4.3 Prevailing topics

RQ2 aims at classifying the papers across the topics presented in the SESoS
Calls for Papers and understanding how the interest in those topics changed
over time. Figure 5 shows the classification of the papers according to the
categories of topics of interest in SESoS (see Table 1). It is possible to
notice that the most addressed category is Analysis & Architecture, which
comprises the design, representation, evaluation, and evolution of SoS
software architectures. In both traditional systems and SoS, software
architectures are key elements for determining the success of these systems,
but novel architectural solutions are required to handle the inherent
complexity, evolutionary development, and emergent behavior of SoS.6
ISO/IEC/IEEE 42010:202229, one of the main international standards related to
software architectures, indeed corroborates the importance of software
architectures for SoS. Nonetheless, architectural activities such as
evaluation, implementation, and evolution are not sufficiently discussed by
the literature.1

Figure 5: Topics addressed by SESoS papers.

The General Issues category comprises several topics of interest. Data
collected from SESoS papers reveal a significant interest in quality issues
for SoS with a particular focus on the interoperability among constituent
systems, which is fundamental for SoS to be built and work (11 papers). In
SESoS 2015, paper P21 reported the results of an SLR on quality attributes in
the SoS context, pointing out interoperability as one of the five most
relevant quality attributes. In the following year, paper P30 presented the
results of another SLR focused on integrating constituent systems in SoS,
highlighting that this is a high-demand concern. In general, the literature on
architecting SoS has been focusing mostly on quality attributes concerning how
constituent systems cooperate with each other and the

Another recurrent topic of interest concerns verification, validation, and
testing of SoS, which appear in 12 papers. These processes are of paramount
importance considering the distinguishing characteristics of SoS and that many
of these systems are designed for mission-critical domains, thus requiring
robust techniques to ensure their proper operation. In SESoS 2018, paper P39
brought interesting insights into how the SoS characteristics impact
verification, validation, and testing processes, which are well-established in
Software Engineering but are limited or cannot be applied to SoS.

A quarter of SESoS papers on SoS has accounted for the Challenges & Directions
category (see Figure 5), which is related to raising and discussing challenges
and future perspectives of research on Software Engineering for SoS. This
characteristic of the papers aligns with the purposes of the SESoS workshop,
which aims to be a forum where researchers and practitioners can discuss ideas
and experiences, analyze research and development issues, and propose
inspiring visions. Secondary studies play a relevant role in achieving these
purposes, and nine papers (P8, P17, P18, P19, P21, P30, P33, P46, and P49)
reported the results of SLRs addressing different topics discussing relevant
gaps in research and development related to SoS.

It is possible to observe in Figure 5 the low number of SESoS papers related
to the Model-Driven Engineering category. In SESoS 2014, paper P18 presented
the state of the art on using model-driven techniques for SoS, advocating that
this approach could fit these systems due to its success in Software
Engineering in taming software complexity through high-abstraction models.
Most of the SESoS papers in this category concern design-time modeling, which
is only one piece of a broader context of using models in the life cycle of
SoS and their constituent systems. Even though using model-based techniques is
an active area in SoS engineering2, the extent to which they can be applied to
SoS still needs to be better understood and assessed.

Figure 6 shows the temporal distribution of the topics addressed by SESoS
papers. It is possible to observe that the topics within the Analysis &
Architecture category stood out in the first editions of the workshop,
possibly due to the fact that (i) they were held with ECSA, one of the main
international conferences on Software Architecture field, and (ii) most of the
authors of those papers are recognized for their research related to software
architectures, so driving the research towards SoS software architectures
would be a natural choice. The noteworthy number of papers in the Challenges &
Directions category is also observed in the first edition of the workshop
(2013), which was significantly characterized by papers discussing different
challenges and research perspectives on exploring Software Engineering for
software-intensive SoS. The Construction & Evolution and General Issues
categories maintained relatively constant along the years with papers focusing
on different concerns related to building effective, high quality SoS.

Figure 6: Distribution of the topics addressed by SESoS papers along the
years.

###  4.4 Software Engineering topics

RQ3 seeks to cross-reference the topics addressed in SESoS papers and the main
topics in the Software Engineering field as organized by the ACM CCS under the
Software and its engineering category (see Table 4). Figure 7 shows the
classification of the papers according to the concepts defined in the ACM CCS
taxonomy. Designing software is the topic with the highest occurrence,
accounting for almost half of SESoS papers on SoS (45.6%). This prevalence
might be due to research reported in those papers seeking to propose
approaches, methods, and techniques to construct SoS.

Figure 7: Classification of SESoS papers according to the Software and its
engineering category of the 2012 ACM Computing Classification System.25

Figure 7 also shows results in line with observations of the topics of
interest of the SESoS workshop (see Section 4.4). Extra-functional properties
represents a topic of significant interest (15/57 papers) with emphasis on
interoperability, followed by Software system structures (14/57 papers) mainly
due to the focus on the design of SoS software architectures. Software
verification and validation is another relevant topic (12/57 papers)
considering the critical role of verification, validation, and testing in many
SoS.

It is also possible to notice that only one paper addressed the Software post-
development issues topic, which comprises software evolution. Evolutionary
development, one of the distinguishing (and complicating) characteristics of
SoS, allows these systems to continuously evolve to respond to changes,
whether related to the environment, their constituent systems, their
functionalities or missions, or the independent evolution of constituent
systems. This low number of papers concerning SoS evolution might indicate
that research on Software Engineering for SoS does not focus on how SoS and
their constituent systems evolve, thereby revealing a gap in this context.

###  4.5 Research types

RQ4 aims to analyze the research types addressed by SESoS papers from 2013 to
2023, also as an indicator of the nature and maturity of the contributions.
Figure 8 shows the classification of SESoS papers according to the type of
research as summarized in Table 5. More than a quarter of the papers ($n=15$)
are philosophical papers, i.e., they discuss current issues and promising
research directions for SoS from a Software Engineering perspective. This type
of research occurred more frequently (five out of nine papers) in the first
edition of the workshop when the aim was to set up an agenda and lay the
foundations for future developments in Software Engineering for SoS.
Furthermore, the last two editions of the workshop brought papers on SoS
focusing on the discussion of challenges related to SoS considering scenarios
that have been drawing attention from academia and industry in recent years,
such as digital twins (paper P55 in SESoS 2022) and cyber-physical systems
(paper P57 in SESoS 2023). In SESoS 2023, paper P56 aimed to discuss the
dynamic nature of SoS, a topic that has yet to be more explored in the
literature even though it is inherent to this class of systems.

Figure 8: Classification of SESoS papers into research types.

The second most addressed research type in SESoS papers from 2013 to 2023 is
solution proposal (22.8%). In general, these papers present the proposal of a
solution to a relevant research problem without going any further in
validating or evaluating it but rather presenting a demonstration of the
feasibility of their proposal through a proof of concept or example. These
data indicate a lower maturity of some research presented in the workshop,
considering the small number of papers with more robust evaluation ($n=6$)
primarily using computational experiments as an empirical method (four
papers). Nevertheless, it is important to highlight that the primary purpose
of any workshop is to provide an opportunity for researchers to exchange and
discuss scientific and engineering ideas at an early stage, before they have
matured to conference or journal publications.

Another research type often appearing in SESoS papers is secondary studies
(nine out of 57 papers), specifically reporting SLR results. This research
type is of significant importance, considering that the typical outcomes of
SLRs are (i) the presentation of a comprehensive overview of the investigated
topic and (ii) the identification and discussion of gaps found in the
literature to commission future research.30, 31 The secondary studies
presented in SESoS from 2013 to 2023 cover diverse topics, mainly designing
and constructing SoS (papers P8, P18, P19, P30, P33). In SESoS 2019, paper P48
shed light on a specific class of SoS, the so-called systems of information
systems (SoIS). SoIS comprise operationally independent information systems
that interoperate with other constituents to provide unique capabilities.32,
33 Besides inheriting the characteristics of SoS, SoIS present specific ones,
such as (i) the existence of information flows among the constituent
information systems, (ii) a business-oriented process nature, and (iii) the
interoperability among the constituent information systems and their
organizations generating information and adding value.34 These additional,
specific characteristics pose new challenges for research and development in
information systems.

Figure 9 depicts some relationships between research types and the categories
of papers solicited by SESoS, i.e., regular or short/position papers. Most
SESoS papers from 2013 to 2023 are regular ones (64.9%), indicating that the
papers generally present substantial content. Papers addressing validation,
evaluation, experience, and secondary study ($n=26$) present the study results
or the reported experiences in more detail, thereby falling into the category
of regular papers. On the other hand, it is possible to notice that SESoS
papers addressing solution proposal ($n=13$) vary in different degrees so that
regular papers addressing this research type tend to present more mature,
detailed solutions than short/position papers.

Figure 9: Classification of SESoS papers into research types and paper
categories.

###  4.6 Application domains or contexts

RQ5 seeks to identify the application domains or contexts SESoS papers address
as a means of assessing how specific the reported contributions are. Figure 10
shows that most papers (82.5%) do not concern a specific domain, i.e., their
contributions are somewhat generic. Nonetheless, some papers demonstrate their
solution proposals, validation, and evaluation in domains such as industry
automation (paper P2), industrial plants (paper P9), military (papers P10,
P12, P37), and disaster prevention and management (papers P23, P26, P39, P42,
P47). On the other hand, some papers report SoS in other contexts, such as the
Internet of Things (papers P6, P34, and P51) and cyber-physical systems (paper
P7), even though the research is not specific to a given application domain
for systems in these contexts.

Figure 10: Application domains and contexts of papers in SESoS 2013-2023.

Figure 10 also shows other application domains with minor occurrences within
the SESoS papers. Smart grids ($n=2$), retail, education, and finance ($n=1$
each) are domains addressed by some papers. Papers P3 and P24 describe the
proposal to design a reference architecture for large-scale smart grid SoS.
Papers P43 and P45 respectively report the experiences of SoS in retail and
finance. These data suggest that research in Software Engineering for SoS
shall be better balanced with practice to allow for a more concrete
realization of the proposed approaches and solutions.

###  4.7 Research impact

RQ6 concerns assessing the impact of SESoS papers by analyzing how much they
have been cited. Citation count is the most straightforward metric for
measuring scientific impact despite its criticism and many factors that may
affect it, such as motivations for citing a paper and the influence of
publication venue.35, 36 Table 7 lists the ten most cited SESoS papers
according to Google Scholar regarding absolute and average number of
citations. Such normalization is important to have a fair assessment of impact
since papers published longer ago are likely to obtain a higher number of
citations. The average number of citations is defined by the ratio between the
absolute number of citations for a paper and the number of years since it has
been published.37

Table 7: Top-ten cited SESoS papers as retrieved from Google Scholar in
October 2023.

Ordered by absolute number of citations
---
ID |  Title | Year | Citation count
P8 |  The state of the art and future perspectives in systems of systems software architectures | 2013 | 89
P41 |  Security assessment of systems of systems | 2019 | 67
P6 |  Towards an IoT ecosystem | 2013 | 64
P19 |  On the characterization of missions of systems-of-systems | 2014 | 49
P4 |  Challenges for SoS architecture description | 2013 | 47
P21 |  Quality attributes of systems-of-systems: A systematic literature review | 2015 | 47
P1 |  On the challenges of self-adaptation in systems-of-systems | 2013 | 43
P18 |  Investigating the model-driven development for systems-of-systems | 2014 | 44
P20 |  Characterizing architecture description languages for software-intensive systems-of-systems | 2015 | 33
P14 |  Live visualization of large software landscapes for ensuring architecture conformance | 2014 | 30

Ordered by average number of citations
---
ID |  Title | Year | Normalized citation count
P41 |  Security assessment of systems of systems | 2019 | 16.8
P55 |  Integration challenges for digital twin systems-of-systems | 2022 | 14.0
P8 |  The state of the art and future perspectives in systems of systems software architectures | 2013 | 8.9
P46 |  The status quo of systems-of-information systems | 2019 | 6.5
P6 |  Towards an IoT ecosystem | 2013 | 6.4
P21 |  Quality attributes of systems-of-systems: A systematic literature review | 2015 | 5.9
P19 |  On the characterization of missions of systems-of-systems | 2014 | 5.4
P38 |  A meta-model for representing system-of-systems ontologies | 2018 | 5.2
P35 |  On the interplay of Business Process Modeling and missions in systems-of-information systems | 2017 | 5.0
P4 |  Challenges for SoS architecture description | 2013 | 4.7

Considering the absolute number of citations, the top-cited SESoS papers
concern issues related to SoS software architectures. Paper P8 reports a
comprehensive SLR addressing the design, representation, evaluation, and
evolution of these architectures, which are essential for contributing to the
success of software-intensive SoS, besides identifying relevant research gaps
in this context. Paper P4 and P20 also concern SoS software architectures with
a specific focus on their architecture description and languages for this
purpose. The ISO/IEC/IEEE 42010 International Standard29 acknowledges the
critical role of architecture descriptions in SoS development, which can be
used for improving communication, analysis, and evolution of these systems.

As mentioned in Section 4.3, the community has demonstrated interest in
quality issues for SoS, mainly due to the critical nature of these systems in
many domains. The SLR reported in paper P21 identified relevant quality
attributes for SoS and limitations of existing quality models, confirming that
addressing extra-functional properties in SoS still needs research efforts38.
Such a criticality in SoS may also explain the significant number of citations
of paper P41, which focuses on SoS security. This work highlights that even if
the constituent systems of an SoS are secure, their operational and managerial
independence and emergent behavior arising from the interactions among them
may raise threats to the overarching SoS.

Another topic of interest refers to missions in SoS, which is the focus of
paper P19. This paper presents a study about how the missions of SoS can be
defined, specified, represented, and implemented. Missions could be the
starting point to adopt when designing an SoS so that mission models can be
used to drive the choice of constituent systems that shall be integrated into
an SoS towards fulfilling its goals. In SESoS 2015, paper P23 (a step further
in that research) pioneered the introduction of a language and tool to specify
SoS missions. Even though several approaches and tools had been proposed for
SoS modeling, they did handle mission modeling. On the other hand, approaches
to handling missions in traditional software systems do not cope with the
characteristics of SoS. The relevant role of missions has made this topic
still an object of active research nowadays.39, 40, 41

Despite its recentness, paper P55 achieved a significant number of citations
in just one year. This paper elicits and discusses some challenges related to
the construction of digital twin SoS. One of the several existing definitions,
a digital twin is a virtual representation of a physical object, process, or
even a real-world system that is continuously synchronized throughout its
lifecycle.42 Data managed by a digital twin about its entity can be analyzed,
aggregated, and extended to support human-oriented or intelligent informed
decisions, generate new information, and provide value-added services on these
data to visualize, monitor, control, or optimize the observed entity. In this
perspective, paper P55 presents a view of composing and integrating digital
twins of smaller-scale systems to become constituents of an SoS. Digital twins
have attracted growing research interest from academia and industry, but they
have not yet been sufficiently investigated from a Software Engineering point
of view43, 44. The integration of heterogeneous digital twins integration
towards forming an SoS hence exacerbates the existing challenges due to the
embodiment of the distinguishing characteristics of SoS.

##  5 Current and future research directions in Software Engineering for
systems-of-systems

The unique characteristics of SoS, such as evolutionary development and
(unexpected) emergent behaviors stemming from the interactions among the
constituent systems of the SoS, raise several research challenges from the
Software Engineering point of view. Due to its inherent complex nature,
designing SoS is a grand research challenge, in particular due to the
combination of the five defining characteristics of these systems.4 Combining
these five defining characteristics amplifies the complexity of developing,
maintaining, managing, and evolving SoS. These features together turn SoS
naturally highly evolvable, frequently changing at runtime in unpredictable
ways. SoS have thereby evolutionary architectures in the sense that they must
dynamically adapt or evolve at runtime subject to the evolutionary development
of the SoS.

A relevant question to pose is what are the main current and future directions
in the field of Software Engineering for taming SoS complexity. To address
this question, it is necessary to analyze how and why SoS are different from
single systems, as the mainstream body of knowledge of Software Engineering
primarily focuses on single systems (and not on SoS).6 The main difference
between an SoS and a single system is undoubtedly the nature of their
constituent systems (specifically their level of independence) and the
exhibition of emergent behavior. It is also worth noting that, in Systems
Engineering, the formalisms and technologies for single systems are well-known
for not being suitable to SoS for a long time. In particular, the limitations
of employing theories, languages, tools, and methods conceived for the design
and implementation of single systems to SoS are well recognized and have
triggered a new research thread in the last decade.

Complexity is thereby innate to SoS as these systems inherently exhibit
emergent behavior. In SoS, missions are achieved through emergent behavior
drawn from the local interaction among constituent systems while attempting to
avoid creating undesirable behaviors as a side-effect, possibly violating
safety. A further complicating factor is that these behaviors may be ephemeral
because the constituent systems of an SoS evolve independently, which may
impact their availability. In addition, the environment in which an SoS
operates is generally known only partially at design time and almost always
too unpredictable to be summarized within a fixed set of specifications.
Consequently, there will inevitably be novel situations, possibly violating
safety, to deal with at runtime.

Overall, major research challenges raised by software-intensive SoS are
fundamentally architectural 1. They are about how to organize the local
interactions among constituent systems to enable the emergence of SoS-wide
behaviors and properties derived from local behaviors and properties (by
acting only on their interactions, without being able to act in the
constituent systems themselves) subject to evolutions that the SoS does not
control due to the independent nature of its constituent systems. Therefore,
enabling to engineer software-reliant SoS is a grand research challenge, and
the mainstream direction for future work is to develop a novel body of
knowledge that integrates the five fundamental SoS since its foundation from
both theoretical and practical viewpoints.

The Software Engineering community has worked on several challenges pertaining
to SoS individually, but not on all of them. Dealing with all the challenges
in the context of Software Engineering for SoS was the main motivation for
creating the SESoS workshop series ten years ago. Up to now, a holistic view
that brings together the development of SoS is missing, starting from mission
models and supporting its design, development, and evolution while assuring
the achievement of quality of services (such as fault tolerance, scalability,
security, and others) and addressing the trade-offs among them. The holistic
view may address the entire lifecycle of an SoS and cover dynamicity, which is
a challenging dimension for the SoS ever-changing context where constituent
systems operate independently and can dynamically join and leave an SoS over
time. A promising research thread to support such a holistic approach is to
explore model-driven techniques to encompass transformation processes among
the different models, automate the mapping process, and provide automated
tools that hide the burden of dealing with the SoS intricacies. Another
important issue to be addressed is how to mainstream the proposed approaches
into industry.

Finally, the importance of SoS is acknowledged by international
standardization organizations working on standards for System-of-Systems
Engineering. Several standards related to SoS have been published, such as the
recent series ISO/IEC/IEEE 21839:201945, ISO/IEC/IEEE 21840:20193, and
ISO/IEC/IEEE 21841:201946. Therefore, works addressing the aforementioned
challenges should be aligned with the guidance of the available standards.

##  6 Conclusion

This article presented a panorama of the research on Software Engineering for
SoS specifically reported in the SESoS workshop, a relevant scientific forum
that had fostered the SoS community since 2013. Besides bringing a historical
perspective of SESoS, this work followed an evidence-based approach by
applying a scoping review and scientometric analysis. The study defined six
research questions and carefully analyzed 57 papers about SoS among the ones
published in SESoS proceedings, covering all workshop editions (2013 to 2023).
The obtained results include the temporal distribution of the papers along the
SESoS editions, the elucidation of top contributing countries to SESoS and co-
authorship networks, the prevailing topics, co-authorship country network, the
categories of topic of interest, research types, application domains, and an
analysis of the research impact of SESoS papers. All this information brings a
consistent, detailed panorama of Software Engineering for SoS through the lens
of the SESoS workshop.

After a decade of research worldwide, it is possible to observe a scenario
with a growing number of high-quality publications addressing several Software
Engineering topics, mainly SoS software architectures, quality issues, and
verification, validation, and testing processes. Furthermore, the community
continues to demonstrate a high interest in shedding light on and discussing
the state-of-the-art related to Software Engineering for SoS as a means of
identifying current research gaps and directions for future research. This
interest is evident, considering that novel contexts such as digital twins and
cyber-physical systems came into play in the most recent editions of the
workshop.

This article also discussed several challenges related to Software Engineering
for SoS. Research in this area is critical to advancing the ability to design,
implement, and maintain SoS. From a Software Engineering standpoint,
addressing these challenges involves designing SoS software architectures to
accommodate changes, ensuring robustness, and developing strategies to detect
and respond to emergent behaviors effectively. Many other challenges are still
ahead, especially concerning the integration of isolated solutions in a
holistic approach, coping with the early to later stages of the SoS
development lifecycle.

*Author contributions E. Cavalcante: Conceptualization, Methodology, Formal analysis, Investigation, Writing - Original Draft, Visualization. T. Batista: Writing - Review & Editing. F. Oquendo: Writing - Review & Editing.

*Financial disclosure None reported.

*Conflict of interest The authors declare no potential conflict of interests.

## References

  * 1 Cadavid H, Andrikopoulos V, Avgeriou P. Architecting systems of systems: A tertiary study. Information and Software Technology. 2020;118.  doi: 10.1016/j.infsof.2019.106202
  * 2 Nielsen CB, Larsen PG, Fitzgerald J, Woodcock J, Peleska J. Systems of Systems Engineering: Basic concepts, model-based techniques, and research directions. ACM Computing Surveys. 2015;48(2):1–41.  doi: 10.1145/2794381
  * 3 ISO/IEC/IEEE 21841:2019 . Systems and software engineering – Taxonomy of systems of systems. ISO; Switzerland: 2019
  * 4 Maier MW. Architecting principles for systems-of-systems. Systems Engineering. 1998;1(4):267–284.  doi: 10.1002/(sici)1520-6858(1998)1:4<267::aid-sys3>3.0.co;2-d
  * 5 Goncalves MB, Cavalcante E, Batista T, Oquendo F, Nakagawa EY. Towards a conceptual model for software-intensive system-of-systems. In: Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE. 2014; USA:1605–1610
  * 6 Oquendo F. Software architecture challenges and emerging research in software-intensive systems-of-systems. In: Tekinerdogan B, Zdun U, Babar A. , eds. Software Architecture, , Lecture Notes in Computer Science, vol. 9839. Switzerland: Springer International Publishing AG, 2016:3–21
  * 7 Dakkak A, Bosch J, Olsson HH, Mattos DI. Continuous deployment in software-intensive system-of-systems. Information and Software Technology. 2023;159.  doi: 10.1016/j.infsof.2023.107200
  * 8 Heinrich J, Balduf F, Becker M, Adler R, Elberzhager F. Industry voices on Software Engineering challenges in dynamic systems of systems. In: Proceedings of the 2023 IEEE/ACM 11th International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS). IEEE. 2023; USA:58–64
  * 9 Azani C. An open systems approach to System of Systems Engineering. In: Jamshidi M. , ed. System of Systems Engineering: Innovations for the 21st Century, , USA: John Wiley & Sons, Inc., 2008:21–43
  * 10 Lana CA, Souza NM, Delamaro ME, Nakagawa EY, Oquendo F, Maldonado JC. Systems-of-systems development: Initiatives, trends, and challenges. In: Proceedings of the 2016 XLII Latin American Computing Conference (CLEI). IEEE. 2016; USA
  * 11 Santos R, Constantinou E, Antonino P, Bosch J. Reporting on a decade of promoting Software Engineering for systems-of-systems and software ecosystems. ACM SIGSOFT Software Engineering Notes. 2023;48(2):24–28.  doi: 10.1145/3587062.3587070
  * 12 Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology. 2018;18.  doi: 10.1186/s12874-018-0611-x
  * 13 Campbell F, Tricco AC, Munn Z, et al. Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): The same but different — the “Big Picture” review family. Systematic Reviews. 2023;12(1).  doi: 10.1186/s13643-023-02178-5
  * 14 Van Raan AFJ. Scientometrics: State-of-the-art. Scientometrics. 1997;38(1):205–218.  doi: 10.1007/BF02461131
  * 15 Mingers J, Leydesdorff L. A review of theory and practice in scientometrics. European Journal of Operational Research. 2015;246(1):1–19.  doi: 10.1016/j.ejor.2015.04.002
  * 16 Santos R, Constantinou E, Antonino P, Bosch J. Software Engineering for systems-of-systems and software ecosystems. Information and Software Technology. 2024;165.  doi: 10.1016/j.infsof.2023.107335
  * 17 Grant MJ, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009;26(2):91–108.  doi: 10.1111/j.1471-1842.2009.00848.x
  * 18 Peters MD, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis. 2020;18(10):2119–2126.  doi: 10.11124/jbies-20-00167
  * 19 Ralph P, Baltes S. Paving the way for mature secondary research: The seven types of literature review. In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM. 2022; USA:1632–1636
  * 20 Kitchenham B, Madeyski L, Budgen D. SEGRESS: Software Engineering Guidelines for REporting Secondary Studies. IEEE Transactions on Software Engineering. 2023;49(3):1273–1298.  doi: 10.1109/tse.2022.3174092
  * 21 Arksey H, O’Malley L. Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19–32.  doi: 10.1080/1364557032000119616
  * 22 Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: Advancing the approach and enhancing the consistency. Research Synthesis Methods. 2014;5(4):371–385.  doi: 10.1002/jrsm.1123
  * 23 Bandara W, Furtmueller E, Gorbacheva E, Miskon S, Beekhuyzen J. Achieving rigor in literature reviews: Insights from qualitative data analysis and tool-support. Communications of the Association for Information Systems. 2015;37.  doi: 10.17705/1cais.03708
  * 24 Stol KJ, Ralph P, Fitzgerald B. Grounded Theory in Software Engineering research: A critical review and guidelines. In: Proceedings of the 38th International Conference on Software Engineering. ACM. 2016; USA:120–131
  * 25 Association for Computer Machinery . ACM Computing Classification System. https://dl.acm.org/ccs; 2012.
  * 26 Wieringa R, Maiden N, Mead N, Rolland C. Requirements Engineering paper classification and evaluation criteria: A proposal and a discussion. Requirements Engineering. 2005;11(1):102–107.  doi: 10.1007/s00766-005-0021-6
  * 27 Raynaud M, Goutaudier V, Louis K, et al. Impact of the COVID-19 pandemic on publication dynamics and non-COVID-19 research production. BMC Medical Research Methodology. 2021;21.  doi: 10.1186/s12874-021-01404-9
  * 28 Heo S, Chan AY, Peralta PD, Jin L, Nunes CRP, Bell ML. Impacts of the COVID-19 pandemic on scientists’ productivity in Science, Technology, Engineering, Mathematics (STEM), and Medicine fields. Humanities and Social Sciences Communications. 2022;9.  doi: 10.1057/s41599-022-01466-0
  * 29 ISO/IEC/IEEE 42010:2022 . Software, systems and enterprise – Architecture description. ISO; Switzerland: 2022
  * 30 Petersen K, Feldt R, Mujtaba S, Mattsson M. Systematic mapping studies in Software Engineering. In: Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering. BCS Learning & Development. 2008; United Kingdom:68–77
  * 31 Petersen K, Vakkalanka S, Kuzniarz L. Guidelines for conducting systematic mapping studies in Software Engineering: An update. Information and Software Technology. 2015;64:1–18.  doi: 10.1016/j.infsof.2015.03.007
  * 32 Saleh M, Abel MH. Information systems: Towards a system of information systems. In: Proceedings of the International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, vol. 3. SciTePress. 2015; Portugal:193–200
  * 33 Fernandes J, Ferreira F, Cordeiro F, Graciano Neto VV, Santos RP. A conceptual model for systems-of-information systems. In: Proceedings of the 2019 IEEE 20th International Conference on Information Reuse and Integration for Data Science (IRI). IEEE. 2019; USA
  * 34 Fernandes J, Cordeiro F, Ferreira F, Graciano Neto VV, Santos RP. A method for identification of potential interoperability links between information systems towards system-of-information systems. iSys - Brazilian Journal of Information Systems. 2022;15(1).  doi: 10.5753/isys.2022.2014
  * 35 Bornmann L, Daniel HD. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation. 2008;64(1):45–80.  doi: 10.1108/00220410810844150
  * 36 Wang M, Yu G, Yu D. Mining typical features for highly cited papers. Scientometrics. 2011;87(3):695–706.  doi: 10.1007/s11192-011-0366-1
  * 37 Garousi V, Fernandes JM. Highly-cited papers in Software Engineering: The top-100. Information and Software Technology. 2016;71:108–128.  doi: 10.1016/j.infsof.2015.11.003
  * 38 Chiprianov V, Falkner K, Gallon L, Munier M. Towards modelling and analysing non-functional properties of systems of systems. In: Proceedings of the 2014 9th International Conference on System of Systems Engineering (SOSE). IEEE. 2014; USA:289–294
  * 39 Axelsson J, Svenson P. On the concepts of capability and constituent system independence in systems-of-systems. In: Proceedings of the 2022 17th Annual System of Systems Engineering Conference (SOSE). IEEE. 2022; USA:247–252
  * 40 Martin J, Axelsson J, Carlson J, Suryadevara J. Towards a core ontology for missions and capabilities in systems of systems. In: Proceedings of the 2023 18th Annual System of Systems Engineering Conference (SOSE). IEEE. 2023
  * 41 Ferreira FHC, Nakagawa EY, Santos RP. Towards an understanding of reliability of software-intensive systems-of-systems. Information and Software Technology. 2023;158.  doi: 10.1016/j.infsof.2023.107186
  * 42 Eramo R, Bordeleau F, Combemale B, van den Brand M, Wimmer M, Wortmann A. Conceptualizing digital twins. IEEE Software. 2022;39(2):39–46.  doi: 10.1109/ms.2021.3130755
  * 43 Rivera LF, Müller HA, Villegas NM, Tamura G, Jiménez M. On the engineering of IoT-intensive digital twin software systems. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. ACM. 2020; USA:631–638
  * 44 Dalibor M, Jansen N, Rumpe B, et al. A cross-domain systematic mapping study on Software Engineering for digital twins. Journal of Systems and Software. 2022;193.  doi: 10.1016/j.jss.2022.111361
  * 45 ISO/IEC/IEEE 21839:2019 . Systems and software engineering – System of systems (SoS) considerations in life cycle stages of a system. ISO; Switzerland: 2019
  * 46 ISO/IEC/IEEE 21840:2019 . Systems and software engineering – Guidelines for the utilization of ISO/IEC/IEEE 15288 in the context of system of systems (SoS). ISO; Switzerland: 2019

Papers on systems-of-systems in SESoS 2013-2023

  1. [P1]

Weyns D, Andersson J. On the challenges of self-adaptation in systems of
systems. In: Proceedings of the First International Workshop on Software
Engineering for Systems-of-Systems. ACM; 2013. doi:10.1145/2489850.2489860

  2. [P2]

Lytra I, Zdun U. Supporting architectural decision making for systems-of-
systems design under uncertainty. In: Proceedings of the First International
Workshop on Software Engineering for Systems-of-Systems. ACM; 2013.
doi:10.1145/2489850.2489859

  3. [P3]

Pérez J, Díaz J, Garbajosa J, Yagüe A, Gonzalez E, Lopez-Perea M. Large-scale
smart grids as system of systems. In: Proceedings of the First International
Workshop on Software Engineering for Systems-of-Systems. ACM; 2013.
doi:10.1145/2489850.2489858

  4. [P4]

Batista T. Challenges for SoS architecture description. In: Proceedings of the
First International Workshop on Software Engineering for Systems-of-Systems.
ACM; 2013. doi:10.1145/2489850.2489857

  5. [P5]

Romay MP, Cuesta CE, Fernández-Sanz L. On self-adaptation in systems-of-
systems. In: Proceedings of the First International Workshop on Software
Engineering for Systems-of-Systems. ACM; 2013. doi:10.1145/2489850.2489856

  6. [P6]

Delicato FC, Pires PF, Batista T, Cavalcante E, Costa B, Barros T. Towards an
IoT ecosystem. In: Proceedings of the First International Workshop on Software
Engineering for Systems-of-Systems. ACM; 2013. doi:10.1145/2489850.2489855

  7. [P7]

Papatheocharous E, Axelsson J, Andrersson J. Issues and challenges in
ecosystems for federated embedded systems. In: Proceedings of the First
International Workshop on Software Engineering for Systems-of-Systems. ACM;
2013. doi:10.1145/2489850.2489854

  8. [P8]

Nakagawa EY, Gonçalves M, Guessi M, Oliveira LBR, Oquendo F. The state of the
art and future perspectives in systems of systems software architectures. In:
Proceedings of the First International Workshop on Software Engineering for
Systems-of-Systems. ACM; 2013. doi:10.1145/2489850.2489853

  9. [P9]

Vierhauser M, Rabiser R, Grünbacher P, Danner C, Wallner S. Evolving systems
of systems. In: Proceedings of the First International Workshop on Software
Engineering for Systems-of-Systems. ACM; 2013. doi:10.1145/2489850.2489851

  10. [P10]

Belloir N, Chiprianov V, Ahmad M, Munier M, Gallon L, Bruel JM. Using RELAX
operators into an MDE security requirement elicitation process for systems of
systems. In: Proceedings of the 2014 European Conference on Software
Architecture Workshops. ACM; 2014. doi:10.1145/2642803.2642835

  11. [P11]

Schneider JP, Teodorov C, Senn E, Champeau J. Towards a dynamic infrastructure
for playing with systems of systems. In: Proceedings of the 2014 European
Conference on Software Architecture Workshops. ACM; 2014.
doi:10.1145/2642803.2642834

  12. [P12]

Khlif I, Kacem MH, Kacem AH, Drira K. A multi-scale modelling perspective for
SoS architectures. In: Proceedings of the 2014 European Conference on Software
Architecture Workshops. ACM; 2014. doi:10.1145/2642803.2642833

  13. [P13]

Amorim SdS, de Almeida ES, McGregor JD, von Flach Chavez C. When ecosystems
collide. In: Proceedings of the 2014 European Conference on Software
Architecture Workshops. ACM; 2014. doi:10.1145/2642803.2642832

  14. [P14]

Fittkau F, Stelzer P, Hasselbring W. Live visualization of large software
landscapes for ensuring architecture conformance. In: Proceedings of the 2014
European Conference on Software Architecture Workshops. ACM; 2014.
doi:10.1145/2642803.2642831

  15. [P15]

Axelsson J, Kobetski A. Architectural concepts for federated embedded systems.
In: Proceedings of the 2014 European Conference on Software Architecture
Workshops. ACM; 2014. doi:10.1145/2642803.2647716

  16. [P16]

Malakuti S. Detecting emergent interference in integration of multiple self-
adaptive systems. In: Proceedings of the 2014 European Conference on Software
Architecture Workshops. ACM; 2014. doi:10.1145/2642803.2642826

  17. [P17]

Maia P, Cavalcante E, Gomes P, Batista T, Delicato FC, Pires PF. On the
development of systems-of-systems based on the Internet of Things. In:
Proceedings of the 2014 European Conference on Software Architecture
Workshops. ACM; 2014. doi:10.1145/2642803.2642828

  18. [P18]

Graciano Neto VV, Guessi M, Oliveira LBR, Oquendo F, Nakagawa EY.
Investigating the model-driven development for systems-of-systems. In:
Proceedings of the 2014 European Conference on Software Architecture
Workshops. ACM; 2014. doi:10.1145/2642803.2642825

  19. [P19]

Silva E, Cavalcante E, Batista T. On the characterization of missions of
systems-of-systems. In: Proceedings of the 2014 European Conference on
Software Architecture Workshops. ACM; 2014. doi:10.1145/2642803.2642829

  20. [P20]

Guessi M, Cavalcante E, Oliveira LBR. Characterizing architecture description
languages for software-Intensive systems-of-systems. In: 2015 IEEE/ACM 3rd
International Workshop on Software Engineering for Systems-of-Systems. IEEE;
2015. doi:10.1109/sesos.2015.10

  21. [P21]

Bianchi T, Santos DS, Felizardo KR. Quality attributes of systems-of-systems:
A systematic literature review. In: 2015 IEEE/ACM 3rd International Workshop
on Software Engineering for Systems-of-Systems. IEEE; 2015.
doi:10.1109/sesos.2015.12

  22. [P22]

Petitdemange F, Borne I, Buisson J. Approach based patterns for system-of-
systems reconfiguration. In: 2015 IEEE/ACM 3rd International Workshop on
Software Engineering for Systems-of-Systems. IEEE; 2015.
doi:10.1109/sesos.2015.11

  23. [P23]

Silva E, Batista T, Cavalcante E. A mission-oriented tool for system-of-
systems modeling. In: 2015 IEEE/ACM 3rd International Workshop on Software
Engineering for Systems-of-Systems. IEEE; 2015. doi:10.1109/sesos.2015.13

  24. [P24]

Perez J, Diaz J, Garbajosa J, Yague A, Gonzalez E, Lopez-Perea M. Towards a
reference architecture for large-scale smart grids system of systems. In: 2015
IEEE/ACM 3rd International Workshop on Software Engineering for Systems-of-
Systems. IEEE; 2015. doi:10.1109/sesos.2015.9

  25. [P25]

Nakagawa EY, Oquendo F, Avgeriou P, et al. Foreword: Towards reference
architectures for systems-of-systems. In: 2015 IEEE/ACM 3rd International
Workshop on Software Engineering for Systems-of-Systems. IEEE; 2015.
doi:10.1109/sesos.2015.8

  26. [P26]

Seo D, Shin D, Baek YM, et al. Modeling and verification for different types
of system of systems using PRISM. In: Proceedings of the 4th International
Workshop on Software Engineering for Systems-of-Systems. ACM; 2016.
doi:10.1145/2897829.2897833

  27. [P27]

Salvaneschi P. Modeling of information systems as systems of systems through
DSM. In: Proceedings of the 4th International Workshop on Software Engineering
for Systems-of-Systems. ACM; 2016. doi:10.1145/2897829.2897832

  28. [P28]

Park S, Park YB. ITE Arbitrator: A reference architecture framework for
sustainable IT ecosystems. In: Proceedings of the 4th International Workshop
on Software Engineering for Systems-of-Systems. ACM; 2016.
doi:10.1145/2897829.2897834

  29. [P29]

Johnson P, Lagerström R, Ekstedt M, Franke U. Modeling and analyzing systems-
of-systems in the multi-attribute prediction language (MAPL). In: Proceedings
of the 4th International Workshop on Software Engineering for Systems-of-
Systems. ACM; 2016. doi:10.1145/2897829.2897830

  30. [P30]

Vargas IG, Gottardi T, Braga RTV. Approaches for integration in system of
systems. In: Proceedings of the 4th International Workshop on Software
Engineering for Systems-of-Systems. ACM; 2016. doi:10.1145/2897829.2897835

  31. [P31]

Silva E, Cavalcante E, Batista T. Refining missions to architectures in
software-intensive systems-of-systems. In: 2017 IEEE/ACM Joint 5th
International Workshop on Software Engineering for Systems-of-Systems and 11th
Workshop on Distributed Software Development, Software Ecosystems and Systems-
of-Systems (JSOS). IEEE; 2017. doi:10.1109/jsos.2017.12

  32. [P32]

Yun W, Shin D, Bae DH. Mutation analysis for system of systems policy testing.
In: 2017 IEEE/ACM Joint 5th International Workshop on Software Engineering for
Systems-of-Systems and 11th Workshop on Distributed Software Development,
Software Ecosystems and Systems-of-Systems (JSOS). IEEE; 2017.
doi:10.1109/jsos.2017.9

  33. [P33]

Motta RC, Oliveira KMD, Travassos GH. Rethinking interoperability in
contemporary software systems. In: 2017 IEEE/ACM Joint 5th International
Workshop on Software Engineering for Systems-of-Systems and 11th Workshop on
Distributed Software Development, Software Ecosystems and Systems-of-Systems
(JSOS). IEEE; 2017. doi:10.1109/jsos.2017.5

  34. [P34]

Alkhabbas F, Spalazzese R, Davidsson P. Emergent configurations in the
Internet of Things as system of systems. In: 2017 IEEE/ACM Joint 5th
International Workshop on Software Engineering for Systems-of-Systems and 11th
Workshop on Distributed Software Development, Software Ecosystems and Systems-
of-Systems (JSOS). IEEE; 2017. doi:10.1109/jsos.2017.6

  35. [P35]

Graciano Neto VV, Cavalcante E, El-Hachem J, Santos DS. On the interplay of
Business Process Modeling and missions in systems-of-information systems. In:
2017 IEEE/ACM Joint 5th International Workshop on Software Engineering for
Systems-of-Systems and 11th Workshop on Distributed Software Development,
Software Ecosystems and Systems-of-Systems (JSOS). IEEE; 2017.
doi:10.1109/jsos.2017.7

  36. [P36]

Mendes A, Loss S, Cavalcante E, Lopes F, Batista T. Mandala: An agent-based
platform to support interoperability in systems-of-systems. In: Proceedings of
the 6th International Workshop on Software Engineering for Systems-of-Systems.
ACM; 2018. doi:10.1145/3194754.3194757

  37. [P37]

Bouziat T, Camps V, Combettes S. A cooperative SoS architecting approach based
on adaptive multi-agent systems. In: Proceedings of the 6th International
Workshop on Software Engineering for Systems-of-Systems. ACM; 2018.
doi:10.1145/3194754.3194756

  38. [P38]

Baek YM, Song J, Shin YJ, Park S, Bae DH. A meta-model for representing
system-of-systems ontologies. In: Proceedings of the 6th International
Workshop on Software Engineering for Systems-of-Systems. ACM; 2018.
doi:10.1145/3194754.3194755

  39. [P39]

Neves VdO, Bertolino A, Angelis GD, Garcés L. Do we need new strategies for
testing systems-of-systems? In: Proceedings of the 6th International Workshop
on Software Engineering for Systems-of-Systems. ACM; 2018.
doi:10.1145/3194754.3194758

  40. [P40]

Bhardwaj N, Liggesmeyer P. A conceptual framework for safe reconfiguration in
open system of systems. In: Proceedings of the 6th International Workshop on
Software Engineering for Systems-of-Systems. ACM; 2018.
doi:10.1145/3194754.3194759

  41. [P41]

Olivero MA, Bertolino A, Dominguez-Mayo FJ, Escalona MJ, Matteucci I. Security
assessment of systems of systems. In: 2019 IEEE/ACM 7th International Workshop
on Software Engineering for Systems-of-Systems (SESoS) and 13th Workshop on
Distributed Software Development, Software Ecosystems and Systems-of-Systems
(WDES). IEEE; 2019. doi:10.1109/sesos/wdes.2019.00017

  42. [P42]

Song J, Torring JO, Hyun S, Jee E, Bae DH. Slicing executable system-of-
systems models for efficient statistical verification. In: 2019 IEEE/ACM 7th
International Workshop on Software Engineering for Systems-of-Systems (SESoS)
and 13th Workshop on Distributed Software Development, Software Ecosystems and
Systems-of-Systems (WDES). IEEE; 2019. doi:10.1109/sesos/wdes.2019.00011

  43. [P43]

Salvaneschi P. Emerging Structures in information systems: A SOS approach. In:
2019 IEEE/ACM 7th International Workshop on Software Engineering for Systems-
of-Systems (SESoS) and 13th Workshop on Distributed Software Development,
Software Ecosystems and Systems-of-Systems (WDES). IEEE; 2019.
doi:10.1109/sesos/wdes.2019.00012

  44. [P44]

de Paula FC, Santos RPd. Systems Thinking as a resource for supporting
accountability in system-of-information-systems: Exploring a Brazilian school
case. In: 2019 IEEE/ACM 7th International Workshop on Software Engineering for
Systems-of-Systems (SESoS) and 13th Workshop on Distributed Software
Development, Software Ecosystems and Systems-of-Systems (WDES). IEEE; 2019.
doi:10.1109/sesos/wdes.2019.00014

  45. [P45]

Carturan S, Goya D. Major challenges of systems-of-systems with cloud and
DevOps - A financial experience report. In: 2019 IEEE/ACM 7th International
Workshop on Software Engineering for Systems-of-Systems (SESoS) and 13th
Workshop on Distributed Software Development, Software Ecosystems and Systems-
of-Systems (WDES). IEEE; 2019. doi:10.1109/sesos/wdes.2019.00010

  46. [P46]

Teixeira PG, Lopes VHL, Santos RPd, Kassab M, Graciano Neto VV. The status quo
of systems-of-information Systems. In: 2019 IEEE/ACM 7th International
Workshop on Software Engineering for Systems-of-Systems (SESoS) and 13th
Workshop on Distributed Software Development, Software Ecosystems and Systems-
of-Systems (WDES). IEEE; 2019. doi:10.1109/sesos/wdes.2019.00013

  47. [P47]

Park S, Belay ZM, Bae DH. A simulation-based behavior analysis for MCI
response system of systems. In: 2019 IEEE/ACM 7th International Workshop on
Software Engineering for Systems-of-Systems (SESoS) and 13th Workshop on
Distributed Software Development, Software Ecosystems and Systems-of-Systems
(WDES). IEEE; 2019. doi:10.1109/sesos/wdes.2019.00009

  48. [P48]

Vargas IG, Nascimento DdL, Braga RTV. Fostering reuse by integration: A
directed system of systems development case. In: 2020 IEEE International
Conference on Software Architecture Companion (ICSA-C). IEEE; 2020.
doi:10.1109/icsa-c50368.2020.00048

  49. [P49]

Lopes SdS, Vargas IG, de Oliveira AL, Braga RTV. Risk management for system of
systems: A systematic mapping study. In: 2020 IEEE International Conference on
Software Architecture Companion (ICSA-C). IEEE; 2020.
doi:10.1109/icsa-c50368.2020.00050

  50. [P50]

Teixeira PG, Lebtag BGA, Santos RPd, et al. Constituent system design: A
software architecture approach. In: 2020 IEEE International Conference on
Software Architecture Companion (ICSA-C). IEEE; 2020.
doi:10.1109/icsa-c50368.2020.00045

  51. [P51]

Fonseca A, Sousa D, Chagas M, et al. Dealing with IoT defiant components. In:
2021 IEEE/ACM Joint 9th International Workshop on Software Engineering for
Systems-of-Systems and 15th Workshop on Distributed Software Development,
Software Ecosystems and Systems-of-Systems (SESoS/WDES). IEEE; 2021.
doi:10.1109/sesos-wdes52566.2021.00009

  52. [P52]

Ailane TM, Abboush M, Knieke C, Lawendy A, Rausch A. Toward formalizing the
emergent behavior in Software Engineering. In: 2021 IEEE/ACM Joint 9th
International Workshop on Software Engineering for Systems-of-Systems and 15th
Workshop on Distributed Software Development, Software Ecosystems and Systems-
of-Systems (SESoS/WDES). IEEE; 2021. doi:10.1109/sesos-wdes52566.2021.00010

  53. [P53]

Pope M, Sillito J. Exploring non-functional coupling in systems of systems.
In: Proceedings of the 10th IEEE/ACM International Workshop on Software
Engineering for Systems-of-Systems and Software Ecosystems. ACM; 2022.
doi:10.1145/3528229.3529383

  54. [P54]

Borges MVdF, Rocha LS, Maia PHM. MicroGraphQL: A unified communication
approach for systems of systems using microservices and GraphQL. In:
Proceedings of the 10th IEEE/ACM International Workshop on Software
Engineering for Systems-of-Systems and Software Ecosystems. ACM; 2022.
doi:10.1145/3528229.3529381

  55. [P55]

Michael J, Pfeiffer J, Rumpe B, Wortmann A. Integration challenges for digital
twin systems-of-systems. In: Proceedings of the 10th IEEE/ACM International
Workshop on Software Engineering for Systems-of-Systems and Software
Ecosystems. ACM; 2022. doi:10.1145/3528229.3529384

  56. [P56]

Heinrich J, Balduf F, Becker M, Adler R, Elberzhager F. Industry voices on
Software Engineering challenges in dynamic systems of systems. In: 2023
IEEE/ACM 11th International Workshop on Software Engineering for Systems-of-
Systems and Software Ecosystems (SESoS). IEEE; 2023.
doi:10.1109/sesos59159.2023.00014

  57. [P57]

Sjöberg P, Mendez D, Gorschek T. Contemporary challenges when developing
cyber-Physical systems of systems - A case study. In: 2023 IEEE/ACM 11th
International Workshop on Software Engineering for Systems-of-Systems and
Software Ecosystems (SESoS). IEEE; 2023. doi:10.1109/sesos59159.2023.00012