Graph Neural Networks for Surfactant Multi-Property Prediction

Christoforos Brozosa,b, Jan G. Rittigb, Sandip Bhattacharyaa, Elie Akannya,
Christina Kohlmanna, Alexander Mitsosd,b,c,∗

  * a

BASF Personal Care and Nutrition GmbH, Henkelstrasse 67, 40589 Duesseldorf,
Germany

  * b

RWTH Aachen University, Process Systems Engineering (AVT.SVT), Aachen, Germany

  * c

Forschungszentrum Jülich GmbH, Institute for Energy and Climate Research
IEK-10: Energy Systems Engineering, Jülich, Germany

  * d

JARA Center for Simulation and Data Science (CSD), Aachen, Germany, Aachen,
Germany

## Abstract

Surfactants are of high importance in different industrial sectors such as
cosmetics, detergents, oil recovery and drug delivery systems. Therefore, many
quantitative structure-property relationship (QSPR) models have been developed
for surfactants. Each predictive model typically focuses on one surfactant
class, mostly nonionics. Graph Neural Networks (GNNs) have exhibited a great
predictive performance for property prediction of ionic liquids, polymers and
drugs in general. Specifically for surfactants, GNNs can successfully predict
critical micelle concentration (CMC), a key surfactant property associated
with micellization. A key factor in the predictive ability of QSPR and GNN
models is the data available for training. Based on extensive literature
search, we create the largest available CMC database with 429 molecules and
the first large data collection for surface excess concentration ($\Gamma$m),
another surfactant property associated with foaming, with 164 molecules. Then,
we develop GNN models to predict the CMC and $\Gamma$m and we explore
different learning approaches, i.e., single- and multi-task learning, as well
as different training strategies, namely ensemble and transfer learning. We
find that a multi-task GNN with ensemble learning trained on all $\Gamma$m and
CMC data performs best. Finally, we test the ability of our CMC model to
generalize on industrial grade pure component surfactants. The GNN yields
highly accurate predictions for CMC, showing great potential for future
industrial applications.

##  1 Introduction

Surfactants are highly relevant molecules used in a wide range of everyday
products, such as food, cosmetics, detergents and drugs (Vieira et al.,, 2021;
Shaban et al.,, 2020; Nitschke and Costa,, 2007; Tadros,, 2005; Adu et al.,,
2020; Szűts and Szabó-Révész,, 2012). Surface-active agents (surfactants) are
amphiphilic molecules with a hydrophobic chain and a polar hydrophilic head.
The surfactant molecule orients itself at the interface between two phases
with the hydrophobic portion oriented towards the hydrophobic phase (e.g.,
air/oil) and the hydrophilic portion oriented towards the hydrophilic phase.
Due to their structure, surfactants are surface/interface active and they are
able to lower the surface/interfacial tension (Shaban et al.,, 2020; Zhang et
al.,, 2019). Owing to their properties, surfactants are widely used in a
variety of applications such as detergents, dispersion stabilizers, foaming
agents, lubricants and as pharmaceuticals among many others (Knop et al.,,
2010; Shaban et al.,, 2020; Schramm et al.,, 2003; Gallou et al.,, 2015).
Surfactants are classified based on their hydrophilic head group, into ionics
and nonionics, with the former further classified into anionics, cationics and
zwitterionics. Important surfactant properties to various applications are the
critical micelle concentration (CMC), the surface tension ($\gamma$), the
surface excess concentration ($\Gamma$m), the Cloud Point (CP) and the Krafft
Temperature (KT) (Yang and Brouillette,, 2016; Al-Sabagh et al.,, 2011). These
properties are used to identify new surfactants with desired performance.

Amphiphilic molecules such as surfactants form micelles, i.e., aggregates.
Micelles ultimately dictate the surface/interface activity and strongly
impacts the solubilization and detergency, or cleaning ability of a surfactant
solution (Patist et al.,, 2001). The minimum surfactant concentration at which
such micelles are formed in a solution is called _critical micelle
concentration_ (CMC) (Rosen and Kunjappu,, 2012; Mukerjee and Mysels,, 1971).
The CMC is an important value in a wide range of applications such as shampoos
(Thompson et al.,, 2023), bio-materials design for drug delivery systems (Su
et al.,, 2020), polymeric micelles (Ghezzi et al.,, 2021; Perumal et al.,,
2022) and oil recovery (Kumar and Mandal,, 2019). In addition, some studies
have reported correlation between CMC and surfactant toxicity (Perinelli et
al.,, 2016) and between CMC and foam stability (Majeed et al.,, 2020). The CMC
is influenced by multiple factors, like temperature, solvent, pH, chemical
structure, pressure conditions and size of the tail and head groups (Katritzky
et al.,, 2008; Thiruvengadam et al.,, 2020; Rosen and Kunjappu,, 2012).
Determination of CMC is time-consuming and expensive, and several methods can
be used like tensiometry (Dahanayake et al.,, 1986), refractive index
(Mukerjee and Mysels,, 1971), calorimetry (Ortona et al.,, 1998), viscosity
and conductivity measurements (Jobe and Reinsborough,, 1984). In most methods,
a break-point in the measured property (e.g., surface tension or conductivity)
vs. surfactant concentration curve is observed, and the CMC is defined to be
at that point (Mukerjee and Mysels,, 1971; Perinelli et al.,, 2020; Moulik et
al.,, 2021).

Since surfactants prefer to exist and adsorb at the interfaces, we can define
their adsorption effectiveness as the surface excess concentration ($\Gamma$m)
(Rosen and Kunjappu,, 2012). $\Gamma$m is an important surfactant property, as
it is a measure of surfactant concentration at air/water and oil/water
interfaces. Additionally, surface excess concentration has been shown to
influence foaming, emulsification and the kinetics of surfactant-induced pore
wetting (Rosen and Kunjappu,, 2012; Wang et al.,, 2019). Like CMC, $\Gamma$m
is influenced by surfactant structure and temperature (Myers,, 2020; Rosen and
Kunjappu,, 2012). The implicit calculation of the surface excess concentration
is possible, as is the CMC, from a surface tension measurement plot using the
Gibbs adsorption equation (Rosen et al., 1982a, ; Rosen and Kunjappu,, 2012).

Due to the tedious and expensive nature of experiments, the prediction of
surfactant properties without experiments has been a focus of research for
many years, mainly, through the use of quantitative structure-property
relationship (QSPR) models. An overview of QSPR models in surfactants was
given by Hu et al., (2010), with most of the QSPR studies aiming to correlate
molecular descriptors with the CMC (Gaudin et al.,, 2016; Katritzky et al.,,
2008; Mattei et al.,, 2013). The developed models showed good predictive
performance (Gaudin et al.,, 2016; Katritzky et al.,, 2008; Mattei et al.,,
2013). Nevertheless, all of them share a common limitation: they are
applicable only on a single surfactant class (nonionics, cations etc.).
Besides the CMC, similar modeling techniques have been applied to other
important surfactant properties like the cloud point of nonionic surfactants
or the minimum surface tension at CMC (Hu et al.,, 2010). Very recently, the
first QSPR model for predicting $\Gamma$m was developed (Seddon et al.,,
2022). The QSPR model was trained on data generated from the Szyszkowski
equation using experimentally measured SFT-log(c) profiles and not on the
experimental data directly (Seddon et al.,, 2022). However, the authors stress
that their QSPR model has limitations in the surfactant categories included,
e.g., the model is not applicable to fluorinated surfactants (Seddon et al.,,
2022).

In the recent years, graph neural networks (GNNs) have been intensively
researched in the field of molecular property prediction, with numerous GNN
models existing in the literature, even regarding the same target property
(Yang et al.,, 2019; Wieder et al.,, 2020). In more detail, GNNs have been
applied to a variety of chemical applications such as ferromagnetic materials
(Pasini et al.,, 2022), the biodegradability of molecules (Rittig et al.,,
2022) and the activity coefficients (Sanchez Medina et al.,, 2022; Rittig et
al., 2023a, ; Felton et al.,, 2022; Rittig et al., 2023b, ). GNNs are a deep
learning technique, where each molecule is represented as a graph, with atoms
corresponding to nodes and bonds to edges. In contrast to classical QSPR
methods, where molecular descriptors are typically selected manually and
therefore require domain knowledge, GNNs can extract, in an automated way, all
the necessary structural-related information which are later used in the
regression task for property prediction. For surfactants, Qin et al., (2021)
used GNNs to predict the CMC of multiple surfactant classes. They showed that
GNNs can be efficiently used as an alternative to classical descriptor-based
QSPR in surfactants, with very promising results, using a database of 200
molecules, which is relatively small for training a machine learning model.

Herein, we create the largest CMC and $\Gamma$m data sets available and we use
them to develop GNN models for their prediction. First, we extend the publicly
available CMC data set of Qin et al., (2021) to 429 molecules through an
extensive literature search. Then, we construct a second data set of 99
molecules with duplicate values with the aim of investigating possible
benefits of transfer learning in CMC prediction. For the $\Gamma$m, no
publicly available database was found during our research. Therefore, we
construct one with 164 surfactant molecules varying from multiple surfactant
class types. Compared to the work of Seddon et al., (2022), we include a wider
range of surfactant categories, e.g., fluorinated components, and we consider
the impact of counterions on $\Gamma$m (Rosen and Kunjappu,, 2012). Note that
the collected data sets include only measurements that can be found in various
sources of publicly available literature.

Furthermore, we establish a GNN model for the prediction of surface excess
concentration ($\Gamma$m) and a GNN model for the prediction of CMC, both
trained on the above mentioned new databases. Additionally, we investigate
multi-task learning to overcome data limitations and ensemble learning to
enhance the predictive performance. Then, we experimentally measure 3
industrial grade surfactants, previously unseen by the GNN model. Finally, we
predict their CMC with our GNN model, which was trained exclusively on
literature data with mainly purified surfactants, and demonstrate the model’s
ability to generalize to unpurified industrial surfactants.

We construct the rest of this work as following: Firstly, we analyze our
databases, data sampling procedure and the industrial surfactants used
(Section 2). Thereafter, we give an overview of how GNN models work, the
methods we applied and a brief overview of the hyperparameter selection
(Section 3). We then present our results, compare them with previous works and
discuss limitations and possible solutions (Section 4). Lastly, we summarize
our work and suggest possible future improvements (Section 5). The test data
used for model evaluation is publicly available in our GitHub repository. The
training data set remains property of BASF and could be made available upon
request.

##  2 Data sets

We now analyze the existing databases and describe our methodology for data
collection (Section 2.1). Following, we discuss the estimation of CMC and how
we handled duplicated values for transfer learning (Section 2.2). Finally, we
analyze the collected data sets (Section 2.3) and we present three industrial
grade surfactants for model testing (Section 2.4).

###  2.1 Existing database analysis and data collection

We started our work by building on the publicly available database of 202
substances from Qin et al., (2021) for CMC prediction. For the surface excess
concentration $\Gamma$m, we had to exclusively rely on tables in books and
publications, such as (Rosen and Kunjappu,, 2012; Dahanayake et al.,, 1986),
because no constructed data set was found in the literature. At first,
literature data (CMC and $\Gamma$m) was extracted from multiple sources (Rosen
and Kunjappu,, 2012; Gaudin et al.,, 2016; Mukerjee and Mysels,, 1971) for all
the molecules at temperatures between 20-28°C. Note that since temperature
massively impacts both properties, we only focus on the temperature range
defined above. We also traced back to the individual articles referenced in
the sources mentioned above (Rosen and Kunjappu,, 2012; Gaudin et al.,, 2016;
Mukerjee and Mysels,, 1971) and extracted additional CMC and $\Gamma$m data.
This procedure resulted in an extended data set of 429 distinct substances for
CMC. In addition, we simultaneously collected 164 different $\Gamma$m values
from multiple sources.

###  2.2 CMC data collection procedure and duplicate values

During data collection, we often found multiple CMC values for the same
surfactant, differing from source to source, due to factors such as purity
levels, measuring method of choice and mathematical evaluation of experimental
data (Moulik et al.,, 2021; Perinelli et al.,, 2020). An example is given in
Table 1, where for the same surfactant 4 different values have been reported.
The CMC variations are discussed in previous works and remain an issue in
surfactant science (Moulik et al.,, 2021; Perinelli et al.,, 2020; Mukerjee
and Mysels,, 1971).

Table 1: CMC values of dodecylpyridinium bromide reported in literature at 25°C. Method | Value (mM) | Source
---|---|---
Tensiometry | 11.5 |  (Rosen et al., 1982b, )
Conductometry | 11.3 |  (Rosen et al., 1982b, )
Conductometry | 10 |  (Škerjanc et al.,, 1999)
Light scattering | 11.6 |  (Ford et al.,, 1966)

To handle duplicate values, we decided for a ranking according to the
measurement method. Here, we prefer CMC values obtained via tensiometry
because one of our targets is to evaluate our model on industrial grade
surfactants using CMC values measured through tensiometry. If tensiometry data
was not available, we favored data from refractometry measurements since it
was found to be reliable by Mukerjee and Mysels (Mukerjee and Mysels,, 1971).
If data only from other methods was available, i.e., neither tensiometry nor
refractometry, we also included it into our main data set. All remaining
values for a surfactant, i.e., duplicates, are not included in the main data
set but rather collected in a separate data set, which we utilize for a
transfer learning approach (cf. Section 3.5). We note that for most
surfactants where duplicate values exist, the values tend to be very similar
to each other and in some cases even equal.

###  2.3 Data sets analysis

The described sampling process in Section 2.2 led us to construct the three
data sets shown in Table 2, together with a detailed surfactant class
distribution. We observe that in the two main databases, nonionic surfactants
is the dominant class followed by anionics. This class distribution matches
with consumption data of surfactants in 2000 (Rosen and Kunjappu,, 2012;
Myers,, 2020), where anionic and nonionic surfactants are the most used in
industrialized areas. In other words, the research focus is matching the
industrial output. Afterwards, a statistical overview of the target
properties, i.e. CMC and $\Gamma$m, is presented in Figure 1. We note that
$\Gamma$m shows a natural normal distribution without applying the logarithm.
Both data sets have similar mean, median, 5th and 95th percentile values
although no comparison between them should be made, as CMC values are scaled.
The smallest and biggest values in both data sets are similar too. Finally, a
correlation plot between $\log$ CMC and $\Gamma$m is given in Figure 2,
containing surfactants for which both CMC and $\Gamma$m values are collected.

Table 2: Number of surfactants per class for each database collected in this work. DV = Duplicate Values. | CMC |  $\Gamma$m | DV-CMC
---|---|---|---
Nonionics | 220 | 86 | 19
Anionics | 130 | 44 | 44
Cationics | 55 | 13 | 27
Zwitterionics | 24 | 21 | 9
Total substances | 429 | 164 | 99

(a)

(b)

Figure 1: Statistical overview of CMC and $\Gamma$m databases, assembled from
literature in this work. Normal distribution of the data set would ensure an
even train-validation-test split without artifact. Figure 2: Correlation plot
between $\log$ CMC and $\Gamma$m for all surfactant classes. In the plot, 141
surfactants are presented, for which both CMC and $\Gamma$m data was collected
from the literature.

###  2.4 Industrial Surfactants

With an estimated market size of around $40 billion in 2020 (Fortune Business
Insights,, 2021), surfactants are also heavily researched in the industry. As
Myers (Myers,, 2020) points out, the majority of academic interest in
surfactants, focuses generally on highly purified compounds while the industry
is either using complex mixtures to obtain the desired performance or
unpurified compounds due to economical reasons. For surface properties like
CMC, many authors have noticed the effect of impurities on CMC through the
years (Mukerjee and Mysels,, 1971; Moulik et al.,, 2021; Patist et al.,,
2000). In this study, we examine to what extent GNN models trained exclusively
on literature data can generalize for unpurified industrial surfactants.

Three industrial-grade pure-component surfactants were used, provided by BASF,
as obtained from production site without further purification. The main
species in these surfactants are (S1) Texapon 842 UP (Sodium Caprylyl
Sulfate), (S2) Texapon EHS (Sodium 2-Ethylhexyl Sulfate) and (S3) Texapon K 12
G (Sodium Dodecyl Sulfate). We exclude those three molecules from the training
set. According to manufacturing process, we expect the presence of unreacted
raw material, alcohols in this case, and reaction by-products.

##  3 Methods

In this section, we first present the fundamentals of a GNN model (Section
3.1), the general training settings of current works (Section 3.2) and the
hyperparameter selection (Section 3.3). Afterwards, we refer to the learning
techniques applied on this paper (Sections 3.4–3.6). The CMC of the three
industrial surfactants was determined by plotting the surface tension as a
function of the logarithm of the surfactant concentration. From this plot, two
linear regions were determined, which correspond to the linear concentration-
dependent and the linear concentration-independent region, respectively. The
CMC value is then obtained from the intersection of the straight lines.
Finally, for the surface tension measurement a Force Tensiometer – K100
(Krüss, Germany), at 23°C was used.

###  3.1 Graph Neural Networks

In GNN models, every molecule is treated as an undirected graph, where atoms
correspond to nodes and bonds to edges. A feature vector, containing chemical
information, is assigned to each atom and each edge. Our node and edge
features of choice are shown in Table 3 and Table 4 respectively, motivated
from our previous works (Schweidtmann et al.,, 2020; Rittig et al., 2023a, )
and past literature (Yang et al.,, 2019). Please note that hydrogen atoms are
not considered as individual nodes but are implicitly represented in the node
feature vector. A surfactant example is given in Figure 3. The molecular graph
then passes through graph convolutions, where neighbor information, i.e.,
neighboring node and edge features, is aggregated for each node in the graph
accordingly. The network depth _L_ , i.e., the number of graph convolutional
layers, defines the neighborhood pool from which structural information will
be aggregated. We herein use edge-conditioned graph convolutional layers
(Simonovsky and Komodakis,, 2017) and a gated recurrent unit (GRU) (Cho et
al.,, 2014), similar to the message passing framework by Gilmer et al., (2017)
and our previous works (Schweidtmann et al.,, 2020; Rittig et al.,, 2022;
Rittig et al., 2023a, ). In contrast to Qin et al., (2021) who used graph
convolutional layers only considering node features, we thus explicitly
include bond type information in learning the molecular structure which
potentially facilitates distinguishing molecules with similar heavy atoms but
different bonds, e.g., alkanes versus alkenes. After the last graph
convolutional layer, the final updated atom feature vectors are pooled into a
final molecular fingerprint vector hFP (Gilmer et al.,, 2017) through a
permutation invariant function, i.e., summation of all node vectors. The hFP
contains all the necessary structure-related information of a specific
molecule required for molecular property prediction, thereby replacing the
selected descriptors in classical QSPR techniques mentioned in Section 1.

Figure 3: A surfactant molecule represented as undericted graph. In every node
(atom) a feature vector (black-white color) of size 30 and in every edge
(bond) a feature vector (green-white) of size 12 is assigned. The feature
vectors encode chemical information about each individual atom and edge
respectively. As can be seen in the atom features vector, different atoms have
different entries, which distinguishes them from the other atoms. Similarly,
bonds are distinguished through their feaure vector too.

Our model is implemented in the Pytorch Geometric (PyG) framework (Fey and
Lenssen,, 2019). For the attributed molecular graph generation, we use the
SMILES string (Weininger,, 1988) of each molecule and RDKit (_version
2022.3.5_), an open-source toolkit for cheminformatics.

###  3.2 General training settings

We use the high-quality data set CMC to define the hyperparameters of our GNN
models. For the target property CMC, the $\log$ CMC is calculated and then
standardized to a zero mean and a standard deviation of one. The train and
test sets are separated randomly in a 85% - 15% ratio respectively. For the
hyperparameter selection an internal validation set is used, which is a subset
of the training set with 20 substances each. The loss function is the mean
squared error (MSE) and the optimizer is Adam (Kingma and Ba,, 2014). In
general, we use the same general training settings as in our previous works
(Schweidtmann et al.,, 2020; Rittig et al., 2023a, ) and the interested reader
can refer to them for more information. For every modeling approach, which
will be introduced in Sections 3.4–3.6, as well as in industrial surfactants
application, 40 models were trained on 40 individual training subsets and the
results are averaged and reported in Section 4.

Table 3: Atom features used in the molecular graph representation. All features are implemented as one-hot-encoding. Feature |  Description | Dimension
---|---|---
atom type |  atom type (C, N, O, S, F, Cl, Br, Na, I, B, K, H, Li) | 13
is aromatic |  if the atom is part of an aromatic system | 1
hybridization |  sp, sp2, sp3, sp3d, or sp3d2 | 5
# bonds |  number of bonds the atom is involved in | 6
# Hs |  number of bonded hydrogen atoms | 5
Total |  | 30
Table 4: Edge features used in the molecular graph representation. All features are implemented as one-hot-encoding. Feature | Description | Dimension
---|---|---
bond type | single, double, triple, or atomatic | 4
is in a ring | whether the bond is part of a ring | 1
conjugated | whether the bond is conjugated | 1
stereo | none, any, E/Z, or cis/trans | 6
Total |  | 12

###  3.3 Hyperparameter selection

With the hyperparameter selection procedure, the aim is to find the suitable
hyperparameters of our GNN model. For a robust hyperparameter selection, we
test each model in 40 different internal validation sets. We perform a grid
search for the following hyperparameters of the GNN model, varying them within
the respective ranges: Graph convolutional type $\in\\{NNConv,GINEConv\\}$,
number of graph convolutional layers $\in\\{1,2,3\\}$, usage of GRU
$\in\\{True,False\\}$, the batch size $\in\\{4,8,16\\}$, the initial learning
rate $\in\\{0.005,0.01,0.05\\}$, dimensions of molecular fingerprint and of
MLP $\in\\{64,128,256\\}$. In other words, the hidden layers of the graph
convolution part and the hidden layers of the MLP are always the same size.
The optimum combination is a GNN architecture with an initial learning rate of
0.005, a hidden state size of 64, a batch size of 16, total graph
convolutional layers of 1, the NNConv graph convolutional type and the usage
of GRU for the message passing scheme. Our edge feature network, similar to
our previous work (Schweidtmann et al.,, 2020), consists three layers with the
following number of neurons: #1 12, #2: 64, and #3: 4096. The architecture
exhibits 306,561 learnable parameters in total.

###  3.4 Single- and Multi-task Learning

A surfactant molecule usually has multiple target properties, as we discussed
above in Section 1. The classical learning approach, _single-task learning_ ,
is to train individual models for every property of interest. In single-task
learning, model parameters are directly optimized based exclusively on a
single target property only, and not transferred to another property
prediction task. In that sense, all available QSPR methods for surfactants
(discussed in Section 1) are single-task learning. On the other hand, in
_multi-task learning_ multiple target properties are simultaneously predicted
(Caruana,, 1997; Zhang and Yang,, 2017). The simultaneous prediction has been
shown to improve the modeling accuracy of GNNs in molecular property
prediction (Schweidtmann et al.,, 2020; Capela et al.,, 2019; Pasini et al.,,
2022). Normally during multi-task learning, the graph convolutional layers are
shared and individual MLPs are constructed for each target property. The
benefits of this approach, are mainly models’ ability to generalize, learn
faster, reduce overfitting (Ruder,, 2017; Crawshaw,, 2020) and data efficiency
(Crawshaw,, 2020).

In the present work, we investigate the prediction of CMC and of $\Gamma$m
with both single- and multi-task learning. Specifically, we develop a single-
task learning model for each property individually and multi-task learning
models for simultaneous prediction of the properties. Since both of the
properties come from the same measurement procedure and therefore are
correlated, we expect an improved model accuracy during multi-task learning.

###  3.5 Transfer Learning

Another technique for improving machine learning models is transfer learning
(Hendrycks et al.,, 2019; Zhuang et al.,, 2020). During transfer learning, a
model is usually pre-trained on a data set, for example a synthetic one, and
then the model parameters are used to initialize the training on a new unseen
data set. This technique is very useful when only small data sets are
available. In the field of GNNs, researchers investigated the benefits and
limitations of transfer learning (Grambow et al.,, 2019; Han et al.,, 2021;
Kooverjee et al.,, 2022).

As we described in Section 2.2, we collect duplicate values to apply transfer
learning to single-task CMC prediction, with the scope of utilizing bigger
portions of experimental data from the literature. We use the data set DV-CMC
(Table 2) to pre-train the model, i.e., learn the graph convolutions and MLP
parameters, and afterwards we initiate our single-task CMC model with them.
All the initialized parameters are optimized based on the CMC data set (Table
2).

###  3.6 Ensemble Learning

Training and using single models can lead to under- or/and over-predictions. A
well-known technique to mitigate this phenomenon in machine learning is
ensemble learning (Breiman,, 1996; Dietterich,, 2000). In ensemble learning,
multiple models are trained on different subsets of training data set and
their final predictions are averaged, resulting in more robust and generalized
predictions (Breiman,, 1996; Dietterich,, 2000; Ganaie et al.,, 2022).

We use ensemble learning both for our single- and multi-task models mentioned
in Section 3.4, by training 40 different models in 40 different subsets of our
training data set, in each case. Afterwards, we use the 40 different models to
perform predictions in our test set, which are averaged to obtain the final
scores.

##  4 Results & Discussion

In this section, we firstly summarize the predictive performance of our models
(Section 4.1). Afterwards we compare our findings with previous similar work
(Section 4.2) and finally we conclude with investigation of model
applicability on the selected industrial surfactants (Section 4.3). An
overview of the performance of the developed models is reported in Table 5.
For every task we report the root mean squared error (RMSE), the mean absolute
error (MAE) and the variances on the validation and test sets.

Table 5: Summary of model accuracy for different predictive tasks over 40 different runs. In each case the standard deviation is also given, except ensemble learning. In the above table we use the following abbreviations: STL = single-task learning, MTL= multi-task learning, TL = transfer learning, EL = ensemble learning, MAE = mean absolute error, RMSE = root mean squared error. | CMC |  $\Gamma$m
---|---|---
| RMSE | MAE | RMSE | MAE
STL (val) | 0.27 ±0.027 | 0.2 ±0.026 | 1.02 ±0.169 | 0.85 ±0.15
STL (test) | 0.33 ±0.033 | 0.25 ±0.026 | 0.8 ±0.143 | 0.57 ±0.146
STL & TL (val) | 0.27 ±0.034 | 0.2 ±0.033 |  |
STL & TL (test) | 0.33 ±0.042 | 0.26 ±0.034 |  |
MTL (val) | 0.26 ±0.075 | 0.2 ±0.053 | 0.3 ±0.121 | 0.26 ±0.116
MTL (test) | 0.36 ±0.041 | 0.27 ±0.031 | 0.59 ±0.051 | 0.43 ±0.044
STL & EL (test) | 0.28 | 0.21 | 0.76 | 0.53
MTL & EL (test) | 0.31 | 0.23 | 0.56 | 0.4

###  4.1 Predictive performance

The single-task GNN model for CMC exhibits an average RMSE of 0.27 on
validation set and 0.33 on test set, while the variance is bigger in the test
set than in the validation set. For $\Gamma$m the average RMSE in test set is
lower than the one in the validation set, with the former equal to 0.85 and
the later equal to 1.02. Using the logarithm of $\Gamma$m did not improve the
performance. Our model exhibits great performance in predicting the $\log$
CMC, but fails to exhibit similar performance in $\Gamma$m prediction. The
reason for the model’s under-performance may be the small size of the data set
used (140 molecules) for the training and the ambiguous measuring procedures.

In multi-task learning, the GNN model for CMC prediction exhibits an average
RMSE of 0.26 on validation set and 0.36 on test set. For $\Gamma$m prediction,
the average RMSE in test set is again lower than the one in the validation
set, with the former equal to 0.59 and the later equal to 0.43. In the CMC
task, the model perform identical with the one in single-task learning, both
for validation and test sets, while in the $\Gamma$m task the multi-task model
exhibits significantly better performance on the validation set, with the RMSE
reducing by 60%, and improved performance on the test set, with the RMSE
reducing by 20%. Therefore, we conclude that the data limitations of the
$\Gamma$m database as single target property can be overcomed by applying
multi-task learning. On the other hand, the CMC model did not benefit from the
additional data and showed identical results with a slightly higher variance
but an overall similar accuracy compared to the single-task learning.

(a)

(b)

Figure 4: Multi-task GNN ensemble models for (a) CMC and (b) $\Gamma$m for all
surfactant classes. The light red dashed lines represent the 10% error and the
dark red dashed lines the 20% error.

The transfer learning approach, i.e., pre-training the model using the 99
collected duplicate values described in Section 2.2, is applied only on the
single-task CMC model. The RMSE on the validation set remains the same as in
single-task learning, equal to 0.27 and on the test set equal to 0.33.
Interestingly, the standard deviation increases in both sets. The increase may
be due to the broader range of target values for the same property, which
leads the model to deviate more from the true value. We observed that transfer
learning slightly reduced the final model training time, i.e., the model
reached its optimum sooner. Besides the slight reduction of final model
training time, using duplicate values for transfer learning led to similar
performance.

Ensemble learning, i.e., averaging the predictions of the 40 trained models,
slightly reduces the RMSE on test set for single-task learning to 0.28 and for
multi-task learning to 0.31 in the CMC case. A similar RMSE reduction is
observed on test set for $\Gamma$m accordingly, to 0.76 for single-task
learning and to 0.56 for multi-task learning. We use the ensembled results in
multi-task CMC and $\Gamma$m learning to draw the parity plots, shown in
Figure 4 on the independent test sets. The parity plots (measured vs predicted
values) for CMC and $\Gamma$m show a high determination coefficient for the
former, R2CMC= 0.94, and moderate one for the latter R2${}_{\Gamma_{m}}$=
0.74. We demonstrate that the GNN approach in the present work, can predict
CMC and $\Gamma$m across all surfactant classes.

In addition, we present the three components with the highest absolute CMC
error in Figure 5. For molecule one, the combination of high CMC value and
lack of similar molecules, i.e. small alkyl chain with high number of ethylene
oxides in the training set, may be the reasons why the model fails to perform
well. For molecules two and three, we suspect the measurement may have an
impact on the result since identical molecules can be found in the training
set.

(a)

(b)

(c)

Figure 5: Outliers in multi-task learning for CMC prediction. Two of them, (a)
and (b) belong to nonionic class and the third, (c) to anionic class.

Similarly, the four components with the highest absolute $\Gamma$m error are
illustrated in Figure 6. Molecule four is similar as in the CMC case, which
supports the measurement impacted hypothesis from before. On the other side
molecules one and three have complex structure, where at a similar chemistry
is lacking in the training set. Therefore, we can assume that the model fails
to capture the property-structure relationship in this case. The same
reasoning can be applied to molecule two, where we also lack similar molecules
in the training set.

(a)

(b)

(c)

(d)

Figure 6: Outliers in multi-task learning for $\Gamma$m prediction. Three of
them, (a), (c) and (d) belong to nonionic class and the second, (b) to
cationic class.

###  4.2 Comparison with previous works

We next compare our results to the work of Qin et al., (2021) and their GNN
model for CMC prediction, which used a subset of our CMC training data but is
also applicable for a wide choice of surfactant classes. Qin et al., (2021)
report a test RMSE of 0.30 which is similar to ours of 0.28. Note that our
test set is almost three times the size of the one used by Qin et al., (2021),
so that we cover a higher variance of molecules. Besides the slight RMSE
improvement in test set, our model reduces also the average RMSE on the
validation set from 0.39 to 0.27. As can be seen in Figure 7, no major
outliers were observed in the 40 models. Finally, most of the models exhibited
a test RMSE in the range of 0.26-0.29 while Qin et al., (2021) reported a
broader test RMSE range of 0.28-0.45.

Overall, the comparison shows that the general performance of our model on the
CMC is similarly high with previous ones, although a direct quantitative
comparison is not possible due to the different data sets used. As there is no
model for predicting surface excess concentration directly from experimental
data (cf. Section 1), we are unable to compare our results for $\Gamma$m with
other works.

Figure 7: Distribution plot of RMSE on internal validation set for all the
learning tasks. The boxplots are the results of 40 runs in different internal
validation sets. The red points represent the outliers.

###  4.3 Industrial Surfactants

We then apply our developed GNN model to predict on the three pure component
industrial surfactants described in Section 2.4. According to the above
discussed results, the best learning approach for CMC is the combination of
single-task with ensemble learning. We use the 40 trained models from Section
4.1 to perform ensembled predictions on the three surfactants. The predicted
log CMC values, as well the experimental measured ones, are given in Table 6
for comparison. For all of the three, the predicted values are very close to
the measured ones. Overall, the data indicates that the developed GNN model
trained on literature data can accurately predict the CMCs for all three
single-molecule industrial unpurified surfactants.

We note that similar molecules are used in the training set and none of the
three exhibits high structural complexity. On the other hand, the impurities
effects on CMC are not learnt from the model and their implementation could
potentially be future area of research. Future work could also focus on
testing the applicability of GNN model in research of bio-surfactants, with
many of them naturally exhibit high structural complexity.

Table 6: Comparison of predicted values from our single-task GNN with ensemble versus experimentally calculated values for three selected industrial grade surfactants. For the predicted values, the standard deviation over 40 individual runs is also given. The above values, are the logarithmic ones. | Predicted CMC ($\mu$M) | Measured CMC ($\mu$M)
---|---|---
S1 | 4.87 ±0.11 | 4.88
S2 | 4.95 ±0.17 | 4.98
S3 | 3.91 ±0.08 | 3.86

##  5 Conclusions and future work

We apply GNNs to pure component surfactants to predict CMC and $\Gamma$m.
Based on extensive literature scanning, we generate a database for CMC with
double the size of existing. We also construct a data set for $\Gamma$m. As
GNNs have been successfully used for CMC prediction (Qin et al.,, 2021), we
extend the GNN architecture to simultaneously predict the surface excess
concentration $\Gamma$m in a multi-task learning, thereby utilizing
correlations between these two properties. Furthermore, we collect additional
CMC values from the literature and investigate if transfer learning can
increase the model accuracy.

All GNN models exhibit high-accuracy CMC predictions, on a comparable level to
a recently developed GNN model by Qin et al., (2021) but for an extended
spectrum of surfactants. For $\Gamma$m on the other hand, the single-task GNNs
fail to capture the property-structure relationship; here, we find that multi-
task GNNs effectively utilizes the CMC data to substantially enhance
prediction accuracy for $\Gamma$m. In all cases, ensemble learning increases
the prediction accuracy. For transfer learning, however, we observe no
improvements in the model accuracy. Finally, we test the best GNN model for
CMC on three unpurified industrial surfactants and find highly accurate
predictions matching our laboratory measurements, thereby indicating strong
potential for further industrial applications.

Furthermore, our GNN models are subject to certain limitations. As is
typically the case for ML models, the applicability of our GNN models is
limited to surfactants with similar structure as the ones contained in the
training data set. Stereochemistry is also not taken into account in this
work, for example in the case of n-dodecyl-D-maltoside we only use the CMC of
the $\alpha$ isomer (Pagliano et al.,, 2012). Another limitation, shortly
mentioned above, is not including information regarding the purity of each
compound. We only considered highly purified compounds reported in literature
and future research should focus in incorporating surface active impurities to
the GNN model.

Future work could extend the relative small database for surface excess
concentration $\Gamma$m, thus yielding higher performance predictive models.
For $\Gamma$m noise is often encountered in reported values due to it is
implicit calculation through various approaches of the Gibbs adsorption
equation (Rosen and Kunjappu,, 2012). This noise prohibits the GNN model to
better capture the structure-property relationship. Finally, prediction of
further surfactant properties based on the structure would be highly
interesting for surfactant formulators.

## Acknowledgments

The BASF authors (C. Brozos, S. Bhattacharya, E. Akanny and C. Kohlmann) were
funded by the BASF Personal Care and Nutrition GmbH. J. G. Rittig and A.
Mitsos acknowledge funding from the Deutsche Forschungsgemeinschaft (DFG,
German Research Foundation) – 466417970 – within the Priority Programme “SPP
2331: Machine Learning in Chemical Engineering”. Additionally, J. G. Rittig
acknowledges the support of the Helmholtz School for Data Science in Life,
Earth and Energy (HDS-LEE).

## Data availability

All python scripts and the test data used in this work are available as open-
source at https://github.com/brozosc/Graph-Neural-Networks-for-Surfactant-
Multi-Property-Prediction.

## Author contribution

Christoforos Brozos: Conceptualization, Methodology, Software, Data curation,
Validation, Formal analysis, Writing - Original Draft, Writing - Review &
Editing, Visualization

Jan G. Rittig: Conceptualization, Methodology, Software, Formal analysis,
Writing - Review & Editing

Sandip Bhattacharya: Conceptualization, Methodology, Formal analysis,
Supervision, Writing - Review & Editing

Elie Akanny: Conceptualization, Methodology, Experimental methodology &
measurements, Writing - Review & Editing

Christina Kohlmann: Writing - Review & Editing, Supervision, Funding
acquisition

Alexander Mitsos: Writing - Review & Editing, Supervision, Funding acquisition

## Bibliography

  * Adu et al., (2020) Adu, S. A., Naughton, P. J., Marchant, R., and Banat, I. M. (2020).  Microbial biosurfactants in cosmetic and personal skincare pharmaceutical formulations.  Pharmaceutics, 12.
  * Al-Sabagh et al., (2011) Al-Sabagh, A., Nasser, N., Migahed, M., and Kandil, N. (2011).  Effect of chemical structure on the cloud point of some new non-ionic surfactants based on bisphenol in relation to their surface active properties.  Egyptian Journal of Petroleum, 20(2):59–66.
  * Breiman, (1996) Breiman, L. (1996).  Bagging predictors.  Machine Learning, 24(2):123–140.
  * Capela et al., (2019) Capela, F., Nouchi, V., Deursen, R. V., Tetko, I. V., and Godin, G. (2019).  Multitask learning on graph neural networks applied to molecular property predictions.  arXiv preprint arXiv:1910.13124.
  * Caruana, (1997) Caruana, R. (1997).  Multitask learning.  Machine Learning, 28(1):41–75.
  * Cho et al., (2014) Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014).  Learning phrase representations using rnn encoder-decoder for statistical machine translation.  arXiv preprint arXiv:1406.1078.
  * Crawshaw, (2020) Crawshaw, M. (2020).  Multi-task learning with deep neural networks: A survey.  arXiv preprint arXiv:2009.09796.
  * Dahanayake et al., (1986) Dahanayake, M., Cohen, A. W., and Rosen, M. J. (1986).  Relationship of structure to properties of surfactants. 13. surface and thermodynamic properties of some oxyethylenated sulfates and sulfonates.  J. Phys. Chem., 90(11):2413–2418.
  * Dietterich, (2000) Dietterich, T. G. (2000).  Ensemble methods in machine learning.  In Proceedings of the First International Workshop on Multiple Classifier Systems, MCS ’00, page 1–15, Berlin, Heidelberg. Springer-Verlag.
  * Felton et al., (2022) Felton, K. C., Ben-Safar, H., and Alexei, A. A. (2022).  Deepgamma: A deep learning model for activity coefficient prediction.  1st Annual AAAI Workshop on AI to Accelerate Science and Engineering (AI2ASE).
  * Fey and Lenssen, (2019) Fey, M. and Lenssen, J. E. (2019).  Fast graph representation learning with pytorch geometric.  arXiv preprint arXiv:1903.02428v3.
  * Ford et al., (1966) Ford, W., Ottewill, R., and Parreira, H. (1966).  Light-scattering studies on dodecylpyridinium halides.  Journal of Colloid and Interface Science, 21(5):522–533.
  * Fortune Business Insights, (2021) Fortune Business Insights (2021).  Surfactants Market Size, Share & COVID-19 Impact Analysis, By Type (Anionic, Nonionic, Cationic, and Amphoteric), By Application (Home Care, Personal Care, Textile, Food & Beverages, Industrial & Institutional Cleaning, Plastics, and Others), and Regional Forecast, 2021-2028.  https://www.fortunebusinessinsights.com/surfactants-market-102385 (accessed 30-12-2023).
  * Gallou et al., (2015) Gallou, F., Isley, N., Ganic, A., Onken, U., and Parmentier, M. (2015).  Surfactant technology applied toward an active pharmaceutical ingredient: more than a simple green chemistry advance.  Green Chem., 18.
  * Ganaie et al., (2022) Ganaie, M., Hu, M., Malik, A., Tanveer, M., and Suganthan, P. (2022).  Ensemble deep learning: A review.  Engineering Applications of Artificial Intelligence, 115:105151.
  * Gaudin et al., (2016) Gaudin, T., Rotureau, P., Pezron, I., and Fayet, G. (2016).  New qspr models to predict the critical micelle concentration of sugar-based surfactants.  Ind. Eng. Chem. Res., 55(45):11716–11726.
  * Ghezzi et al., (2021) Ghezzi, M., Pescina, S., Padula, C., Santi, P., Del Favero, E., Cantù, L., and Nicoli, S. (2021).  Polymeric micelles in drug delivery: An insight of the techniques for their characterization and assessment in biorelevant conditions.  Journal of Controlled Release, 332:312–336.
  * Gilmer et al., (2017) Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. (2017).  Neural message passing for quantum chemistry.  34th International Conference on Machine Learning, ICML 2017, 3:2053–2070.
  * Grambow et al., (2019) Grambow, C., Li, Y.-P., and Green, W. H. (2019).  Accurate thermochemistry with small data sets: A bond additivity correction and transfer learning approach.  J. Phys. Chem. A, 123(27):5826–5835.
  * Han et al., (2021) Han, X., Huang, Z., An, B., and Bai, J. (2021).  Adaptive transfer learning on graph neural networks.  In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 565–574, New York, NY, USA. Association for Computing Machinery.
  * Hendrycks et al., (2019) Hendrycks, D., Lee, K., and Mazeika, M. (2019).  Using pre-training can improve model robustness and uncertainty.  In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2712–2721. PMLR.
  * Hu et al., (2010) Hu, J., Zhang, X., and Wang, Z. (2010).  A review on progress in qspr studies for surfactants.  International journal of molecular sciences, 11:1020–47.
  * Jobe and Reinsborough, (1984) Jobe, D. J. and Reinsborough, V. C. (1984).  Micellar properties of sodium alkyl sulfoacetates and sodium dialkyl sulfosuccinates in water.  Canadian Journal of Chemistry, 62:280–284.
  * Katritzky et al., (2008) Katritzky, A. R., Pacureanu, L. M., Slavov, S. H., Dobchev, D. A., and Karelson, M. (2008).  Qspr study of critical micelle concentrations of nonionic surfactants.  Ind. Eng. Chem. Res., 47(23):9687–9695.
  * Škerjanc et al., (1999) Škerjanc, J., Kogej, K., and Cerar, J. (1999).  Equilibrium and transport properties of alkylpyridinium bromides.  Langmuir, 15(15):5023–5028.
  * Kingma and Ba, (2014) Kingma, D. P. and Ba, J. (2014).  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980.
  * Knop et al., (2010) Knop, K., Hoogenboom, R., Fischer, D., and Schubert, U. (2010).  Poly(ethylene glycol) in drug delivery: Pros and cons as well as potential alternatives.  Angewandte Chemie International Edition, 49(36):6288–6308.
  * Kooverjee et al., (2022) Kooverjee, N., James, S., and van Zyl, T. (2022).  Investigating transfer learning in graph neural networks.  Electronics, 11(8).
  * Kumar and Mandal, (2019) Kumar, A. and Mandal, A. (2019).  Critical investigation of zwitterionic surfactant for enhanced oil recovery from both sandstone and carbonate reservoirs: Adsorption, wettability alteration and imbibition studies.  Chemical Engineering Science, 209:115222.
  * Majeed et al., (2020) Majeed, T., Sølling, T. I., and Kamal, M. S. (2020).  Foamstability: The interplay between salt-, surfactant- and critical micelle concentration.  Journal of Petroleum Science and Engineering, 187:106871.
  * Mattei et al., (2013) Mattei, M., Kontogeorgis, G. M., and Gani, R. (2013).  Modeling of the critical micelle concentration (cmc) of nonionic surfactants with an extended group-contribution method.  Ind. Eng. Chem. Res., 52(34):12236–12246.
  * Moulik et al., (2021) Moulik, S. P., Rakshit, A. K., and Naskar, B. (2021).  Evaluation of non-ambiguous critical micelle concentration of surfactants in relation to solution behaviors of pure and mixed surfactant systems: A physicochemical documentary and analysis.  Journal of Surfactants and Detergents, 24(4):535–549.
  * Mukerjee and Mysels, (1971) Mukerjee, P. and Mysels, K. J. (1971).  Critical micelle concentrations of aqueous surfactant systems.  Technical report, National Standard reference data system.
  * Myers, (2020) Myers, D. (August 2020).  Surfactant Science and Technology, 4th Edition.  John Wiley & Sons, Ltd.
  * Nitschke and Costa, (2007) Nitschke, M. and Costa, S. (2007).  Biosurfactants in food industry.  Trends in Food Science & Technology, 18(5):252–259.
  * Ortona et al., (1998) Ortona, O., Vitagliano, V., Paduano, L., and Costantino, L. (1998).  Microcalorimetric study of some short-chain nonionic surfactants.  Journal of Colloid and Interface Science, 203(2):477–484.
  * Pagliano et al., (2012) Pagliano, C., Barera, S., Chimirri, F., Saracco, G., and Barber, J. (2012).  Comparison of the $\alpha$ and $\beta$ isomeric forms of the detergent n-dodecyl-d-maltoside for solubilizing photosynthetic complexes from pea thylakoid membranes.  Biochimica et Biophysica Acta (BBA) - Bioenergetics, 1817(8):1506–1515.  Photosynthesis Research for Sustainability: From Natural to Artificial.
  * Pasini et al., (2022) Pasini, M. L., Zhang, P., Reeve, S. T., and Choi, J. Y. (2022).  Multi-task graph neural networks for simultaneous prediction of global and atomic properties in ferromagnetic systems*.  Machine Learning: Science and Technology, 3(2):025007.
  * Patist et al., (2000) Patist, A., Bhagwat, S. S., Penfield, K. W., Aikens, P., and Shah, D. O. (2000).  On the measurement of critical micelle concentrations of pure and technical-grade nonionic surfactants.  Journal of Surfactants and Detergents, 3(1):53–58.
  * Patist et al., (2001) Patist, A., Oh, S., Leung, R., and Shah, D. (2001).  Kinetics of micellization: Its significance to technological processes.  Colloids and Surfaces A: Physicochemical and Engineering Aspects, 176:3–16.
  * Perinelli et al., (2016) Perinelli, D. R., Cespi, M., Casettari, L., Vllasaliu, D., Cangiotti, M., Ottaviani, M. F., Giorgioni, G., Bonacucina, G., and Palmieri, G. F. (2016).  Correlation among chemical structure, surface properties and cytotoxicity of n-acyl alanine and serine surfactants.  European Journal of Pharmaceutics and Biopharmaceutics, 109:93–102.
  * Perinelli et al., (2020) Perinelli, D. R., Cespi, M., Lorusso, N., Palmieri, G. F., Bonacucina, G., and Blasi, P. (2020).  Surfactant self-assembling and critical micelle concentration: One approach fits all?  Langmuir : the ACS journal of surfaces and colloids, 36:5745–5753.
  * Perumal et al., (2022) Perumal, S., Atchudan, R., and Lee, W. (2022).  A review of polymeric micelles and their applications.  Polymers, 14.
  * Qin et al., (2021) Qin, S., Jin, T., Van Lehn, R. C., and Zavala, V. M. (2021).  Predicting critical micelle concentrations for surfactants using graph convolutional neural networks.  The Journal of Physical Chemistry. B, 125:10610–10620.
  * (45) Rittig, J. G., Ben Hicham, K., Schweidtmann, A. M., Dahmen, M., and Mitsos, A. (2023a).  Graph neural networks for temperature-dependent activity coefficient prediction of solutes in ionic liquids.  Computers and Chemical Engineering, 171:108153.
  * (46) Rittig, J. G., Felton, K. C., Lapkin, A. A., and Mitsos, A. (2023b).  Gibbs-Duhem-informed neural networks for binary activity coefficient prediction.  Digital Discovery, 2:1752–1767.
  * Rittig et al., (2022) Rittig, J. G., Gao, Q., Dahmen, M., Mitsos, A., and Schweidtmann, A. M. (2022).  Graph neural networks for the prediction of molecular structure-property relationships.  arXiv preprint arXiv:2208.04852.
  * Rosen and Kunjappu, (2012) Rosen, M. and Kunjappu, J. (2012).  Surfactants and Interfacial Phenomena: Rosen/Surfactants 4E.  John Wiley & Sons, Ltd.
  * (49) Rosen, M. J., Cohen, A. W., Dahanayake, M., and Hua, X. Y. (1982a).  Relationship of structure to properties in surfactants. 10. surface and thermodynamic properties of 2-dodecyloxypoly(ethenoxyethanol)s, c12h25(oc2h4)xoh, in aqueous solution.  J. Phys. Chem., 86(4):541–545.
  * (50) Rosen, M. J., Dahanayake, M., and Cohen, A. W. (1982b).  Relationship of structure to properties in surfactants. 11. surface and thermodynamic properties of n-dodecyl-pyridinium bromide and chloride.  Colloids and Surfaces, 5(2):159–172.
  * Ruder, (2017) Ruder, S. (2017).  An overview of multi-task learning in deep neural networks.  arXiv preprint arXiv:1706.05098.
  * Sanchez Medina et al., (2022) Sanchez Medina, E. I., Linke, S., Stoll, M., and Sundmacher, K. (2022).  Graph neural networks for the prediction of infinite dilution activity coefficients.  Digital Discovery, 1:216–225.
  * Schramm et al., (2003) Schramm, L., Stasiuk, E., and Marangoni, G. (2003).  Surfactants and their applications.  Annu. Rep. Prog. Chem., Sect. C: Phys. Chem., 99:3–48.
  * Schweidtmann et al., (2020) Schweidtmann, A. M., Rittig, J. G., König, A., Grohe, M., Mitsos, A., and Dahmen, M. (2020).  Graph neural networks for prediction of fuel ignition quality.  Energy & Fuels, 34(9):11395–11407.
  * Seddon et al., (2022) Seddon, D., Müller, E. A., and Cabral, J. T. (2022).  Machine learning hybrid approach for the prediction of surface tension profiles of hydrocarbon surfactants in aqueous solution.  Journal of Colloid and Interface Science, 625:328–339.
  * Shaban et al., (2020) Shaban, S. M., Kang, J., and Kim, D.-H. (2020).  Surfactants: Recent advances and their applications.  Composites Communications, 22:100537.
  * Simonovsky and Komodakis, (2017) Simonovsky, M. and Komodakis, N. (2017).  Dynamic edge-conditioned filters in convolutional neural networks on graphs.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 29–38.
  * Su et al., (2020) Su, H., Wang, F., Ran, W., Zhang, W., Dai, W., Wang, H., Anderson, C. F., Wang, Z., Zheng, C., Zhang, P., Li, Y., and Cui, H. (2020).  The role of critical micellization concentration in efficacy and toxicity of supramolecular polymers.  Proceedings of the National Academy of Sciences of the United States of America, 117:4518–4526.
  * Szűts and Szabó-Révész, (2012) Szűts, A. and Szabó-Révész, P. (2012).  Sucrose esters as natural surfactants in drug delivery systems—a mini-review.  International Journal of Pharmaceutics, 433(1):1–9.
  * Tadros, (2005) Tadros, T. (2005).  Surfactants in Personal Care and Cosmetics, chapter 12, pages 399–432.  John Wiley & Sons, Ltd.
  * Thiruvengadam et al., (2020) Thiruvengadam, S., Murphy, M., Tan, J. S., and Miller, K. (2020).  A generalized theoretical model for the relationship between critical micelle concentrations, pressure, and temperature for surfactants.  Journal of Surfactants and Detergents, 23(2):273–303.
  * Thompson et al., (2023) Thompson, C. J., Ainger, N., Starck, P., Mykhaylyk, O. O., and Ryan, A. J. (2023).  Shampoo science: A review of the physiochemical processes behind the function of a shampoo.  Macromolecular Chemistry and Physics, 224(3):2200420.
  * Vieira et al., (2021) Vieira, I. M. M., Santos, B. L. P., Ruzene, D. S., and Silva, D. P. (2021).  An overview of current research and developments in biosurfactants.  Journal of Industrial and Engineering Chemistry, 100:1–18.
  * Wang et al., (2019) Wang, Z., Chen, Y., Zhang, F., and Lin, S. (2019).  Significance of surface excess concentration in the kinetics of surfactant-induced pore wetting in membrane distillation.  Desalination, 450:46–53.
  * Weininger, (1988) Weininger, D. (1988).  Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules.  J. Chem. Inf. Comput. Sci., 28(1):31–36.
  * Wieder et al., (2020) Wieder, O., Kohlbacher, S., Kuenemann, M., Garon, A., Ducrot, P., Seidel, T., and Langer, T. (2020).  A compact review of molecular property prediction with graph neural networks.  Drug Discovery Today: Technologies, 37:1–12.
  * Yang et al., (2019) Yang, K., Swanson, K., Jin, W., Coley, C., Eiden, P., Gao, H., Guzman-Perez, A., Hopper, T., Kelley, B., Mathea, M., Palmer, A., Settels, V., Jaakkola, T., Jensen, K., and Barzilay, R. (2019).  Analyzing learned molecular representations for property prediction.  Journal of Chemical Information and Modeling, 59(8):3370–3388.  PMID: 31361484.
  * Yang and Brouillette, (2016) Yang, Z. and Brouillette, C. G. (2016).  Chapter thirteen - a guide to differential scanning calorimetry of membrane and soluble proteins in detergents.  In Feig, A. L., editor, Calorimetry, volume 567 of Methods in Enzymology, pages 319–358. Academic Press.
  * Zhang et al., (2019) Zhang, F., Li, S., Zhang, Q., Liu, J., Zeng, S., Liu, M., and Sun, D. (2019).  Adsorption of different types of surfactants on graphene oxide.  Journal of Molecular Liquids, 276:338–346.
  * Zhang and Yang, (2017) Zhang, Y. and Yang, Q. (2017).  A survey on multi-task learning.  IEEE Transactions on Knowledge and Data Engineering, 34:5586–5609.
  * Zhuang et al., (2020) Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., and He, Q. (2020).  A comprehensive survey on transfer learning.  Proceedings of the IEEE, PP:1–34.