Introduction
The mining of sequence patterns, also called motifs, is one of the most important tasks in protein sequence analysis and continues to be an active topic of research. The large number of proposals found in the literature sustain this claim. Sequence mining is the task of analyzing a set of possible related sequences and detecting subtrings that occur significantly among those sequences. Motif over-representation can be explained by the existence of segments that have been preserved through the natural evolution of the proteins and suggests that the regions described by those substrings play a structural and functional role in the protein's mechanisms 12. Different types of motifs representation have been proposed and two main classes can be distinguished: probabilistic and deterministic. A probabilistic motif consists of a model that simulates the sequences or part of the sequences under consideration. When an input sequence is provided, a probability of being matched by the motif is yielded. Position Weight Matrices (PWM) and Hidden Markov Models (HMMs) are examples of probabilistic motifs. Deterministic motifs are commonly expressed by an enhanced regular expression syntax, either matching or not the input sequences. This paper is devoted to the evaluation of significance measures for deterministic motif discovery in protein databases. A critical aspect of the motif analysis process is that due to the completeness nature of deterministic mining algorithms the number of extracted motifs is often very large. Not all these motifs are particularly interesting and most of them certainly arise by chance. Therefore, it is crucial to propose scoring methods to discriminate the relevant and significant motifs.
By itself, the definition of a significant motif is an interesting problem. One possible solution to assess this significance is to delegate this decision to a biologist. An expert would analyze the target proteins and decide which motifs have biological interest. As this approach is only feasible for small and medium scale experiments, an alternative is to automatically evaluate motifs according to their statistical or informative importance. As pointed by Hart et al. in 3, statistical significance is often correlated with biological significance and provides a meaningful criterion for the analysis of relevant motifs.
In addition to support a better understanding of the protein's structure and function, motifs have also a wide-range of other applications. They can be used to perform clustering 4, family classification 25678910, discovery of sub-families in large protein families 11, gene expression analysis 1213 and the study and discovery of homology relations 5. The selection of the appropriate measures for a specific problem depends on how well they adjust to the problem. In the literature, many measures of interest and significance have been proposed. How to choose the most appropriate significance measure is still an open question.
Similar to this problem is the discovery of significant association rules. In the work of Tan, Kumar and Srivastava 14, a survey and general evaluation of itemset interest measures is presented. Such measures were used to describe the statistical relationship between the items in a itemset 15. This problem is different from the motif evaluation problem, since an item occurs only once per itemset, which is not the case of motifs, where an item (called symbol) may occur repeatedly. Transcription Factor Binding Sites (TFBS) can be described by motifs with very specific characteristics. Typically, they consist of small length contiguous motifs, highly degenerated, i.e., with many ambiguous positions. In Tompa et al. 16, an assessment of 13 popular algorithms for the discovery of TFBS was performed. Later, Li and Tompa 17 have categorized and examined the adequacy of three popular significance functions used by the algorithms described in 16.
Although, these studies were designed for problems other than protein motif analysis, they may bring important improvements to the field. For instance, the results of the unsupervised mining of massive protein datasets, such as the SwissProt 18 comprehensive protein sequence database, are almost impossible to be properly analyzed. This can be mainly due to the inexistence of measures that objectively and automatically evaluate the biological significance of newly discovered motifs and allow the identification of the truly significant motifs among the irrelevant ones.
Different measures evaluate different properties. Thus, the best solution for a particular problem may include the simultaneous use of several measures. Given that some of these measures will show consistent or even very similar results, it is important to identify such relations in order to avoid biased evaluations. We are also interested in studying the impact of different problem characteristics and how certain operations inherent to the mining process affect these measures.
The contributions of this paper can be summarized as follows:
• It surveys and categorizes significance measures presented in the bioinformatics, data mining, statistics and machine learning literature.
• It provides a comprehensive evaluation of the selected measures, in the presence of different motif and dataset characteristics.
• It proposes a methodology that combines the information provided by several measures in order to highlight the most interesting motifs.
The remainder of the paper is organized in two parts. In the first part we describe the characteristics of the evaluated motifs and the sources where the evaluated data is obtained. Significance measures are then introduced according to the considered categorization. The second part is dedicated to the experimental evaluation. We start by describing how motifs are extracted and then go on to the analysis of ranking, consistency and variability of the measures in a wide range of situations. In section "Motif Ranking Visualizer", we propose a methodology for identifying high scoring motifs and demonstrate its application. Finally, we conclude with the main lessons learned.
Evaluating Deterministic Motifs
Deterministic motifs are described in a regular expression based language, which tends to be easily understandable by humans. These motifs can be divided in two types: fixed-length and extensible-length. Fixed-length motifs (a.k.a (l, d)-motifs 1920) consist of a string with a fixed size of l symbols where d possible symbols may have a mismatch with the matched sequences in the database. Extensible-length motifs have an arbitrary length with an arbitrary number of symbols and gaps. Consider the following abstract pattern:
A1 - x(p1, q1) - A2 - x(p2, q2) - ... - An
Ai is a sequence of consecutive amino acids, called component and -x(pi, qi)- represents a gap greater or equal than pi and smaller or equal than qi. A symbol is considered to be concrete if it represents one of the twenty amino acid symbols. Three types of extensible-length motifs can be distinguished:
• Contiguous Motifs contain no gaps, i.e., pi = qi = 0, ∀i, e.g. IPCCPV.
• Rigid Gap Motifs only contain gaps with a fixed length, i.e., pi = qi, ∀i. The symbol '.' is a wild-card symbol used to denote a gap of size one and it matches any symbol of the alphabet, e.g. MN..A.CA
• Flexible Gap Motifs allow a variable number of gaps between events of the sequence, i.e., pi ≤ qi, ∀i, e.g. AN-x(1,3)-C-x(4,6)-D.
Deterministic motifs are typically mined through combinatorial algorithms that perform an exhaustive traversal of the search space and perform filtering using the support metric. The support of a motif is the number of different sequences where it occurs. For a motif to pass the filter, its support has to be equal or greater than a user pre-defined threshold (see 21222324 for a comprehensive overview). Support is an apriori measure of statistical significance. Generally, further assessment of motif significance is done as a post-processing step.
In this scenario, two important facts justify the critical need for the evaluation of significance measures. First it provides means for an early pruning of irrelevant motifs. The combinatorial nature of the deterministic mining process may deliver an exponentially increasing number of motifs. Thus, efficient pruning of irrelevant motifs results in performance improvement of the algorithms. Second, motifs over-representation does not necessarily imply significance.
In this work, three types of extensible-length motifs will be used to perform the evaluation of fourteen significance measures.
The Prosite Database
There is a significant number of motif repositories freely available at the Internet. Examples of well established and reliable databases are: Prosite 25, Prints 26, Blocks 27, InterPro 28 or eMotif 29 (see 30 for an overview). From the listed databases, Prosite deserves a special attention in the context of our work. Prosite 25 is the oldest and best known sequence motif database. It is semi-manually annotated and its motifs are characterized for having a high biological significance. They provide a strong indication of a region in the protein with an important role. A family of protein sequences is then described by one or more motifs. Since this database is considered a standard, new algorithms and methods tend to use it as a benchmark test-bed.
The Dilimot Database
One of the characteristics of the Prosite motifs is that they are strongly conserved in the respective families, covering the majority or the totality of their sequences. In order to perform an evaluation on less conserved motifs, we have used the Dilimot database 31. It provides a service for finding over-represented, short (3 to 8 amino acids), rigid gap motifs in a set of protein sequences. Additionally, it makes available high-confidence pre-computed motif sets from different species. In this work, several motifs from human related proteins will be used.
Significance Measures
As introduced by Brazma et al. 22, a significance measure can be defined as a function of the form: f(M, C) → ℝ, where M represents the motif being evaluated and C is a set of related proteins sequences usually called target family or positive data. This function returns a real value score that expresses how relevant or significant is M with respect to C. These scores may provide hints to biologically or statistically relevant motifs. If additional sequence information is available, for example where motifs are less expected to occur, both positive and negative information can then be considered in the evaluation. The function can be extended to include the negative dataset C¯
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGafm4qamKbaebaaaa@2CFA@: f(M, C, C¯
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGafm4qamKbaebaaaa@2CFA@) → ℝ. The universe of all sequences U corresponds to U = C + C¯
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGafm4qamKbaebaaaa@2CFA@ and the size of each set of sequences is denoted as |C| and |C¯
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGafm4qamKbaebaaaa@2CFA@|, respectively. We now distinguish four possible cases of a motif M matching a sequence of C:
• True Positive (TP): a sequence that belongs to the target family and matches the motif.
• True Negative (TN): a sequence that does not belong to the target family and does not match the motif.
• False Negative (FN): a sequence that belongs to the target family and does not match the motif.
• False Positive (FP): a sequence that does not belong to the target family and matches the motif.
Sagot 32, suggests that motifs can be evaluated according to the following approaches: probability of matching a random sequence, sensitivity/specificity, information content and minimum description length (MDL). Since this categorization does not include all possible measures, nor distinguishes the type of information provided, a different categorization will be considered. Three categories are proposed:
1. Class-based measures, which are calculated based on the information of the motif in relation to positive and negative data.
2. Information-Theoretic measures, which are based solely on Information-theoretic models like probabilistic or entropy models. In this case the calculation is self-contained, i.e., the necessary information is found in the motif itself.
3. Hybrid measures use both Information-theoretic and class information.
Class-based Measures
The ideal motif is one that matches all the sequences of the target family and no other sequence outside this family. It is also known as signature motif. In this context, the measures most widely used to express the quality of the motifs are: sensitivity, specificity and positive predicted value (see Table 1). Sensitivity (Sn), also called recall, measures the proportion of sequences of the target family correctly matched by the motif. Specificity (Sp) measures the proportion of sequences outside the target family that are not matched by the motif. Positive Predicted value (PPV), also called precision, measures the proportion of sequences that are covered by the motif and that belong to the target family. An ideal motif is one with 100% of Sn and PPV. These three measures yield a positive rank of motifs, i.e., their score is proportional to the rank. For comparison purposes, a negative rank measure false positive rate (Fpr) is also considered. This measure returns the proportion of negative instances that were incorrectly reported as being positive. In this case, the greater the score the worst the quality of the motif. Motifs can be ranked according to one or all of these measures. When a unique value is required to score a motif, a combination of these measures can be used. The F-Measure (F) 33 and the Pearson Correlation (Corr) 2234 (also known as Matthews Correlation Coefficient, for its application in secondary structure prediction 35) are examples of such composed measures. As a last example of a class-based measure we refer to the Discrimination power (Dp) 2. This measure is particularly useful as a filter, since Dp is proportionally associated to selectiveness. A characteristic of class-based measures is that they do not rely on the motif structure to be calculated. Hence, they can be applied to any type of deterministic motif. Although a myriad of class-based measures can be found, covering different aspects of a pattern quality, we only review those widely used in a biological context. Please refer to Table 1 and 2 for details on these measures.
<p>Table 1</p>
List of the motif significance measures.
Symbol
Measure
Formula
Range
Type
Sn
Sensitivity
S
n
(
M
)
=
T
P
T
P
+
F
N
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaem4uamLaemOBa4MaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabdcfaqbqabaaabaGaemivaq1aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdAeagnaaBaaabaGaemOta4eabeaaaaaaaa@3B13@
[0,1]
C
Sp
Specificity
S
p
(
M
)
=
T
N
T
N
+
F
P
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaem4uamLaemiCaaNaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabd6eaobqabaaabaGaemivaq1aaSbaaeaacqWGobGtaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaaaaaaaa@3B13@
[0,1]
C
PPV
Positive Predicted Value
P
P
V
(
M
)
=
T
P
T
P
+
F
P
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaemiuaaLaemOvayLaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabdcfaqbqabaaabaGaemivaq1aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaaaaaaaa@3C0A@
[0,1]
C
Fpr
False Positive Rate
F
p
r
(
M
)
=
F
P
F
P
+
T
N
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemOrayKaemiCaaNaemOCaiNaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGgbGrdaWgaaqaaiabdcfaqbqabaaabaGaemOray0aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdsfaunaaBaaabaGaemOta4eabeaaaaaaaa@3C4E@
[-1,1]
C
F
F-Measure
F
(
M
)
=
2
×
S
e
n
s
i
t
i
v
i
t
y
×
P
P
V
S
e
n
s
i
t
i
v
i
t
y
+
P
P
V
=
2
×
T
P
2
×
T
P
+
F
N
+
F
P
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemOrayKaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqaIYaGmcqGHxdaTcqWGtbWucqWGLbqzcqWGUbGBcqWGZbWCcqWGPbqAcqWG0baDcqWGPbqAcqWG2bGDcqWGPbqAcqWG0baDcqWG5bqEcqGHxdaTcqWGqbaucqWGqbaucqWGwbGvaeaacqWGtbWucqWGLbqzcqWGUbGBcqWGZbWCcqWGPbqAcqWG0baDcqWGPbqAcqWG2bGDcqWGPbqAcqWG0baDcqWG5bqEcqGHRaWkcqWGqbaucqWGqbaucqWGwbGvaaGccqGH9aqpjuaGdaWcaaqaaiabikdaYiabgEna0kabdsfaunaaBaaabaGaemiuaafabeaaaeaacqaIYaGmcqGHxdaTcqWGubavdaWgaaqaaiabdcfaqbqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaaaaaaaa@7017@
[0,1]
1
Corr
Correlation
C
(
M
)
=
T
P
×
T
N
−
F
P
×
F
N
(
T
P
+
F
N
)
(
T
P
+
F
P
)
(
T
N
+
F
P
)
(
T
N
+
F
N
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaem4qamKaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabdcfaqbqabaGaey41aqRaemivaq1aaSbaaeaacqWGobGtaeqaaiabgkHiTiabdAeagnaaBaaabaGaemiuaafabeaacqGHxdaTcqWGgbGrdaWgaaqaaiabd6eaobqabaaabaWaaOaaaeaacqGGOaakcqWGubavdaWgaaqaaiabdcfaqbqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaiabcMcaPiabcIcaOiabdsfaunaaBaaabaGaemiuaafabeaacqGHRaWkcqWGgbGrdaWgaaqaaiabdcfaqbqabaGaeiykaKIaeiikaGIaemivaq1aaSbaaeaacqWGobGtaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaacqGGPaqkcqGGOaakcqWGubavdaWgaaqaaiabd6eaobqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaiabcMcaPaqabaaaaaaa@5DDA@
[-1,1]
C
Dp
Discrimination Power
D
p
(
M
)
=
T
P
|
C
|
−
F
P
|
C
¯
|
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiraqKaemiCaaNaeiikaGIaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabdcfaqbqabaaabaWaaqWaaeaacqWGdbWqaiaawEa7caGLiWoaaaGccqGHsisljuaGdaWcaaqaaiabdAeagnaaBaaabaGaemiuaafabeaaaeaadaabdaqaaiqbdoeadzaaraaacaGLhWUaayjcSdaaaaaa@41AF@
[-1,1]
C
IG
Information Gain
IG(
M
)=Info(
M
)×[
Support(
M
)−1
]
where Info(
M
)=−lo
g
|
Σ
|
p(
M
)
MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbiqaaaBafaqaaeGabaaabaGaemysaKKaem4raC0aaeWaaeaacqWGnbqtaiaawIcacaGLPaaacqGH9aqpcqWGjbqscqWGUbGBcqWGMbGzcqWGVbWBdaqadaqaaiabd2eanbGaayjkaiaawMcaaiabgEna0oaadmaabaGaem4uamLaemyDauNaemiCaaNaemiCaaNaem4Ba8MaemOCaiNaemiDaq3aaeWaaeaacqWGnbqtaiaawIcacaGLPaaacqGHsislcqaIXaqmaiaawUfacaGLDbaaaeGabaaPbiaaxMaacqqG3bWDcqqGObaAcqqGLbqzcqqGYbGCcqqGLbqzcqqGGaaicqWGjbqscqWGUbGBcqWGMbGzcqWGVbWBdaqadaqaaiabd2eanbGaayjkaiaawMcaaiabg2da9iabgkHiTiabdYgaSjabd+gaVjabdEgaNnaaBaaaleaadaabdaqaaiabfo6atbGaay5bSlaawIa7aaqabaGccqWGWbaCdaqadaqaaiabd2eanbGaayjkaiaawMcaaaaaaaa@6C7F@
[0, + ∞[
IT
Pratt
Pratt Measure
P
r
a
t
t
(
M
)
=
∑
i
n
I
′
(
A
i
)
−
c
⋅
∑
k
=
1
n
−
1
(
q
k
−
p
k
)
where
I
′
(
A
i
)
=
−
∑
a
i
∈
A
i
(
P
(
a
i
)
×
l
o
g
(
P
(
a
i
)
)
)
+
∑
a
i
∈
A
i
(
P
(
a
i
)
P
(
A
i
)
×
l
o
g
(
P
(
a
i
)
P
(
A
i
)
)
)
and
P
(
A
i
)
=
∑
a
i
∈
A
i
p
(
a
i
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeqabmqaaaqaaGqaciab=bfaqjab=jhaYjabdggaHjabdsha0jabdsha0jabcIcaOiabd2eanjabcMcaPiabg2da9maaqadabaGafmysaKKbauaacqGGOaakcqWGbbqqdaWgaaWcbaGaemyAaKgabeaakiabcMcaPiabgkHiTiabdogaJbWcbaGaemyAaKgabaGaemOBa4ganiabggHiLdGccqGHflY1daaeWaqaaiabcIcaOiabdghaXnaaBaaaleaacqWGRbWAaeqaaOGaeyOeI0IaemiCaa3aaSbaaSqaaiabdUgaRbqabaGccqGGPaqkaSqaaiabdUgaRjabg2da9iabigdaXaqaaiabd6gaUjabgkHiTiabigdaXaqdcqGHris5aaGcbaGaee4DaCNaeeiAaGMaeeyzauMaeeOCaiNaeeyzauMaeeiiaaIafmysaKKbauaacqGGOaakcqWGbbqqdaWgaaWcbaGaemyAaKgabeaakiabcMcaPiabg2da9iabgkHiTmaaqababaGaeiikaGIaemiuaaLaeiikaGIaemyyae2aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkcqGHxdaTcqWFSbaBcqWFVbWBcqWFNbWzcqGGOaakcqWGqbaucqGGOaakcqWGHbqydaWgaaWcbaGaemyAaKgabeaakiabcMcaPiabcMcaPiabcMcaPiabgUcaRmaaqababaGaeiikaGYaaSaaaeaacqWGqbaucqGGOaakcqWGHbqydaWgaaWcbaGaemyAaKgabeaakiabcMcaPaqaaiabdcfaqjabcIcaOiabdgeabnaaBaaaleaacqWGPbqAaeqaaOGaeiykaKcaaiabgEna0kab=XgaSjab=9gaVjab=DgaNjabcIcaOmaalaaabaGaemiuaaLaeiikaGIaemyyae2aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkaeaacqWGqbaucqGGOaakcqWGbbqqdaWgaaWcbaGaemyAaKgabeaakiabcMcaPaaacqGGPaqkcqGGPaqkaSqaaiabdggaHnaaBaaameaacqWGPbqAaeqaaSGaeyicI4Saemyqae0aaSbaaWqaaiabdMgaPbqabaaaleqaniabggHiLdaaleaacqWGHbqydaWgaaadbaGaemyAaKgabeaaliabgIGiolabdgeabnaaBaaameaacqWGPbqAaeqaaaWcbeqdcqGHris5aaGcbaGaeeyyaeMaeeOBa4MaeeizaqMaeeiiaaIaemiuaaLaeiikaGIaemyqae0aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkcqGH9aqpdaaeqaqaaiabdchaWjabcIcaOiabdggaHnaaBaaaleaacqWGPbqAaeqaaOGaeiykaKcaleaacqWGHbqydaWgaaadbaGaemyAaKgabeaaliabgIGiolabdgeabnaaBaaameaacqWGPbqAaeqaaaWcbeqdcqGHris5aaaaaaa@C763@
]- ∞, + ∞[
IT
LogOdd
LogOdd
Logodd
(
M
)
=
l
o
g
(
S
u
p
p
o
r
t
(
M
)
N
u
m
S
e
q
s
P
(
M
)
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaeeitaWKaee4Ba8Maee4zaCMaee4Ba8MaeeizaqMaeeizaqMaeiikaGIaeeyta0KaeiykaKIaeyypa0JaemiBaWMaem4Ba8Maem4zaCMaeiikaGscfa4aaSaaaeaadaWcaaqaaiabdofatjabdwha1jabdchaWjabdchaWjabd+gaVjabdkhaYjabdsha0jabcIcaOiabd2eanjabcMcaPaqaaiabd6eaojabdwha1jabd2gaTjabdofatjabdwgaLjabdghaXjabdohaZbaaaeaacqWGqbaucqGGOaakcqWGnbqtcqGGPaqkaaGccqGGPaqkaaa@57F4@
]- ∞, + ∞[
IT
ZScore
Z-Score
Z
s
c
o
r
e
(
M
)
=
S
u
p
p
o
r
t
(
M
)
−
E
(
M
)
N
(
M
)
where
E
(
M
)
=
N
r
e
s
i
d
×
P
(
M
)
and
N
(
M
)
=
N
r
e
s
i
d
×
P
(
M
)
×
(
1
−
P
(
M
)
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeqabiqaaaqaaiabdQfaAjabdohaZjabdogaJjabd+gaVjabdkhaYjabdwgaLjabcIcaOiabd2eanjabcMcaPiabg2da9KqbaoaalaaabaGaem4uamLaemyDauNaemiCaaNaemiCaaNaem4Ba8MaemOCaiNaemiDaqNaeiikaGIaemyta0KaeiykaKIaeyOeI0IaemyrauKaeiikaGIaemyta0KaeiykaKcabaGaemOta4KaeiikaGIaemyta0KaeiykaKcaaaGcbaGaee4DaCNaeeiAaGMaeeyzauMaeeOCaiNaeeyzauMaeeiiaaIaemyrauKaeiikaGIaemyta0KaeiykaKIaeyypa0JaemOta40aaSbaaSqaaiabdkhaYjabdwgaLjabdohaZjabdMgaPjabdsgaKbqabaGccqGHxdaTcqWGqbaucqGGOaakcqWGnbqtcqGGPaqkcqqGGaaicqqGHbqycqqGUbGBcqqGKbazcqqGGaaicqWGobGtcqGGOaakcqWGnbqtcqGGPaqkcqGH9aqpdaGcaaqaaiabd6eaonaaBaaaleaacqWGYbGCcqWGLbqzcqWGZbWCcqWGPbqAcqWGKbazaeqaaOGaey41aqRaemiuaaLaeiikaGIaemyta0KaeiykaKIaey41aqRaeiikaGIaeGymaeJaeyOeI0IaemiuaaLaeiikaGIaemyta0KaeiykaKIaeiykaKcaleqaaaaaaaa@8B4E@
]- ∞, + ∞[
IT
J
J-Measure
J
(
C
;
M
)
=
P
(
M
)
×
j
(
C
;
M
)
where
j
(
C
;
M
)
=
P
(
C
|
M
)
×
l
o
g
2
P
(
C
|
M
)
P
(
C
)
+
(
1
−
P
(
C
|
M
)
)
×
l
o
g
2
(
1
−
P
(
C
|
M
)
)
(
1
−
P
(
C
)
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeqabiqaaaqaaiabdQeakjabcIcaOiabdoeadjabcUda7iabd2eanjabcMcaPiabg2da9iabdcfaqjabcIcaOiabd2eanjabcMcaPiabgEna0kabdQgaQjabcIcaOiabdoeadjabcUda7iabd2eanjabcMcaPaqaaiabbEha3jabbIgaOjabbwgaLjabbkhaYjabbwgaLjabbccaGiabdQgaQjabcIcaOiabdoeadjabcUda7iabd2eanjabcMcaPiabg2da9iabdcfaqjabcIcaOiabdoeadjabcYha8jabd2eanjabcMcaPiabgEna0kabdYgaSjabd+gaVjabdEgaNnaaBaaaleaacqaIYaGmaeqaaKqbaoaalaaabaGaemiuaaLaeiikaGIaem4qamKaeiiFaWNaemyta0KaeiykaKcabaGaemiuaaLaeiikaGIaem4qamKaeiykaKcaaOGaey4kaSIaeiikaGIaeGymaeJaeyOeI0IaemiuaaLaeiikaGIaem4qamKaeiiFaWNaemyta0KaeiykaKIaeiykaKIaey41aqRaemiBaWMaem4Ba8Maem4zaC2aaSbaaSqaaiabikdaYaqabaqcfa4aaSaaaeaacqGGOaakcqaIXaqmcqGHsislcqWGqbaucqGGOaakcqWGdbWqcqGG8baFcqWGnbqtcqGGPaqkcqGGPaqkaeaacqGGOaakcqaIXaqmcqGHsislcqWGqbaucqGGOaakcqWGdbWqcqGGPaqkcqGGPaqkaaaaaaaa@8BA3@
[0, + ∞[
H
I
Mutual Information
I
(
Q
;
M
)
=
H
(
Q
)
−
H
(
Q
|
M
)
where
H
(
Q
)
=
−
∑
q
∈
{
C
,
C
¯
}
P
(
q
)
×
l
o
g
2
P
(
q
)
and
H
(
Q
|
M
)
=
−
P
(
M
)
×
∑
q
∈
{
C
,
C
¯
}
P
(
q
|
M
)
×
l
o
g
2
P
(
q
|
M
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqbaeqabiqaaaqaaiabdMeajjabcIcaOiabdgfarjabcUda7iabd2eanjabcMcaPiabg2da9iabdIeaijabcIcaOiabdgfarjabcMcaPiabgkHiTiabdIeaijabcIcaOiabdgfarjabcYha8jabd2eanjabcMcaPiabbccaGiabbEha3jabbIgaOjabbwgaLjabbkhaYjabbwgaLjabbccaGiabdIeaijabcIcaOiabdgfarjabcMcaPiabg2da9iabgkHiTmaaqababaGaemiuaaLaeiikaGIaemyCaeNaeiykaKcaleaacqWGXbqCcqGHiiIZcqGG7bWEcqWGdbWqcqGGSaalcuWGdbWqgaqeaiabc2ha9bqab0GaeyyeIuoakiabgEna0kabdYgaSjabd+gaVjabdEgaNnaaBaaaleaacqaIYaGmaeqaaOGaemiuaaLaeiikaGIaemyCaeNaeiykaKcabaGaeeyyaeMaeeOBa4MaeeizaqMaeeiiaaIaemisaGKaeiikaGIaemyuaeLaeiiFaWNaemyta0KaeiykaKIaeyypa0JaeyOeI0IaemiuaaLaeiikaGIaemyta0KaeiykaKIaey41aq7aaabeaeaacqWGqbaucqGGOaakcqWGXbqCcqGG8baFcqWGnbqtcqGGPaqkaSqaaiabdghaXjabgIGiolabcUha7jabdoeadjabcYcaSiqbdoeadzaaraGaeiyFa0habeqdcqGHris5aOGaey41aqRaemiBaWMaem4Ba8Maem4zaC2aaSbaaSqaaiabikdaYaqabaGccqWGqbaucqGGOaakcqWGXbqCcqGG8baFcqWGnbqtcqGGPaqkaaaaaa@9B2C@
[0, 1]
H
S
Surprise Measure
S
(
M
)
=
I
n
f
o
(
M
)
×
P
(
C
|
M
)
=
I
n
f
o
(
M
)
×
S
u
p
p
o
r
t
(
M
∈
C
)
S
u
p
p
o
r
t
(
M
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaem4uamLaeiikaGIaemyta0KaeiykaKIaeyypa0JaemysaKKaemOBa4MaemOzayMaem4Ba8MaeiikaGIaemyta0KaeiykaKIaey41aqRaemiuaaLaeiikaGIaem4qamKaeiiFaWNaemyta0KaeiykaKIaeyypa0JaemysaKKaemOBa4MaemOzayMaem4Ba8MaeiikaGIaemyta0KaeiykaKIaey41aqBcfa4aaSaaaeaacqWGtbWucqWG1bqDcqWGWbaCcqWGWbaCcqWGVbWBcqWGYbGCcqWG0baDcqGGOaakcqWGnbqtcqGHiiIZcqWGdbWqcqGGPaqkaeaacqWGtbWucqWG1bqDcqWGWbaCcqWGWbaCcqWGVbWBcqWGYbGCcqWG0baDcqGGOaakcqWGnbqtcqGGPaqkaaaaaa@690D@
[0, + ∞[
H
Description of the fourteen significance measures according to the respective type (C = Class based; IT = Information-Theoretic based; H = Hybrid). For each measure the abbreviation symbol used throughout the paper, the formula and the respective range.
<p>Table 2</p>
Auxiliary formulas.
Formula
Range
P
(
C
)
=
T
P
+
F
N
T
P
+
F
N
+
F
P
+
T
N
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaeiikaGIaem4qamKaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabdcfaqbqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaaqaaiabdsfaunaaBaaabaGaemiuaafabeaacqGHRaWkcqWGgbGrdaWgaaqaaiabd6eaobqabaGaey4kaSIaemOray0aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdsfaunaaBaaabaGaemOta4eabeaaaaaaaa@436B@
[0,1]
P
(
C
|
M
)
=
T
P
T
P
+
F
P
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaeiikaGIaem4qamKaeiiFaWNaemyta0KaeiykaKIaeyypa0tcfa4aaSaaaeaacqWGubavdaWgaaqaaiabdcfaqbqabaaabaGaemivaq1aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaaaaaaaa@3C3B@
[0,1]
P
(
C
|
M
)
P
(
C
)
=
T
P
×
(
T
P
+
F
N
+
F
P
+
T
N
)
(
T
P
+
F
P
)
×
(
T
P
+
F
N
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqWGqbaucqGGOaakcqWGdbWqcqGG8baFcqWGnbqtcqGGPaqkaeaacqWGqbaucqGGOaakcqWGdbWqcqGGPaqkaaGccqGH9aqpjuaGdaWcaaqaaiabdsfaunaaBaaabaGaemiuaafabeaacqGHxdaTcqGGOaakcqWGubavdaWgaaqaaiabdcfaqbqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaacqGHRaWkcqWGubavdaWgaaqaaiabd6eaobqabaGaeiykaKcabaGaeiikaGIaemivaq1aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaacqGGPaqkcqGHxdaTcqGGOaakcqWGubavdaWgaaqaaiabdcfaqbqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaiabcMcaPaaaaaa@5C1B@
[0,1]
1
−
P
(
C
|
M
)
1
−
P
(
C
)
=
F
P
×
(
T
P
+
F
N
+
F
P
+
T
N
)
(
T
P
+
F
P
)
×
(
T
N
+
F
P
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqaIXaqmcqGHsislcqWGqbaucqGGOaakcqWGdbWqcqGG8baFcqWGnbqtcqGGPaqkaeaacqaIXaqmcqGHsislcqWGqbaucqGGOaakcqWGdbWqcqGGPaqkaaGccqGH9aqpjuaGdaWcaaqaaiabdAeagnaaBaaabaGaemiuaafabeaacqGHxdaTcqGGOaakcqWGubavdaWgaaqaaiabdcfaqbqabaGaey4kaSIaemOray0aaSbaaeaacqWGobGtaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaacqGHRaWkcqWGubavdaWgaaqaaiabd6eaobqabaGaeiykaKcabaGaeiikaGIaemivaq1aaSbaaeaacqWGqbauaeqaaiabgUcaRiabdAeagnaaBaaabaGaemiuaafabeaacqGGPaqkcqGHxdaTcqGGOaakcqWGubavdaWgaaqaaiabd6eaobqabaGaey4kaSIaemOray0aaSbaaeaacqWGqbauaeqaaiabcMcaPaaaaaa@5FB9@
[0,1]
Info(
M
)=−lo
g
|
Σ
|
P(
M
)
MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemysaKKaemOBa4MaemOzayMaem4Ba82aaeWaaeaacqWGnbqtaiaawIcacaGLPaaacqGH9aqpcqGHsislcqWGSbaBcqWGVbWBcqWGNbWzdaWgaaWcbaWaaqWaaeaacqqHJoWuaiaawEa7caGLiWoaaeqaaOGaemiuaa1aaeWaaeaacqWGnbqtaiaawIcacaGLPaaaaaa@427E@
[0, + ∞[
List of auxiliary formulas used for the calculation of measures from Table 1. The respective range is also provided.
Information-Theoretic Measures
When analyzing the probabilistic aspects of genetic sequences, one of two models can be adopted: a Markov or a Bernoulli model. In Markov models, the probability distribution of a given symbol depends on the n previous symbols, where n determines the order of the Markov chain 836.
In Bernoulli models, sequences are generated according to an independent identically distributed (i.i.d.) process. Therefore, the occurrence of a motif M in a given sequence is assumed to be an i.i.d. process 37. This means that both the input sequences and the occurrence of the amino acids are independent. Protein sequences where motifs are sought to be found are often biologically related.
Although the independence of the positions along a sequence and in the motifs is not always verified, it can be considered reasonable to work under the assumption of an i.i.d. model 38. The probability P of a motif M, in the form A1 - x(p1, q1) - A2 - x(p2, q2) - ... - An, can be calculated according to formula 1.
P(M) = P(A1) × P(-x(p1, q1)-) × P(A2) × P(-x(p2, q2)-) × ... × P(An)
Since the probability of matching any symbol from the alphabet (denoted by character '.') is one (P('.') = 1), then P(-x(p, q)-) = 1 and P(Ai)=∏aj∈AiP(aj)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaeiikaGIaemyqae0aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkcqGH9aqpdaqeqaqaaiabdcfaqjabcIcaOiabdggaHnaaBaaaleaacqWGQbGAaeqaaOGaeiykaKcaleaacqWGHbqydaWgaaadbaGaemOAaOgabeaaliabgIGiolabdgeabnaaBaaameaacqWGPbqAaeqaaaWcbeqdcqGHpis1aaaa@40DD@. We consider that the probability of an amino acid aj, P(aj), is given by its frequency in the Swiss-Prot database 18. If ambiguous positions occur in substring Ai, then its probability is given by formula 2.
P
(
A
i
)
=
∏
a
j
∈
A
i
(
∑
k
=
1
|
A
i
|
P
(
a
j
k
)
)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaeiikaGIaemyqae0aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkcqGH9aqpdaqeqbqaaiabcIcaOmaaqahabaGaemiuaaLaeiikaGIaemyyae2aaSbaaSqaaiabdQgaQbqabaGccqWGRbWAcqGGPaqkcqGGPaqkaSqaaiabdUgaRjabg2da9iabigdaXaqaamaaemaabaGaemyqae0aaSbaaWqaaiabdMgaPbqabaaaliaawEa7caGLiWoaa0GaeyyeIuoaaSqaaiabdggaHnaaBaaameaacqWGQbGAaeqaaSGaeyicI4Saemyqae0aaSbaaWqaaiabdMgaPbqabaaaleqaniabg+Givdaaaa@4FD3@
where ajk stands for the k-th amino acid in position j of the substring Ai. For instance, the probability of the substring A - [GC] - · · - V is given by 0.0783 × (0.0693 + 0.0152) × 1 × 1 × 0.0671 = 4.44 × 10-4. Support(M) is the number of times that a motif M occurs in different sequences of the database. Support(M ∈ C) corresponds to the number of sequences in family C where M occurs.
Information-Theoretic measures quantify the degree of information encoded in a motif. We provide examples of five of these measures.
Information Gain (IG) 3940 is used to measure the amount of accumulated information by a motif in relation to an amino acid sequence. In this measure (see Table 1), the self-information content Info(M) (see Table 2) quantifies the information content associated with the motif, i.e., how likely is M to occur. (Support(M) - 1) gives the occurrence of motif M in the positive dataset. The minus one value of this component allows to easily reject motifs that trivially occur once.
The Minimum Description Length (MDL) principle applied in 1138, is also an Information-theoretic measure and can be made equivalent to the IG measure. MDL is used to score the motifs and to measure the fitness of these motifs with respect to the input sequences. Assuming the hypothetical transmission of sequences, the idea is to measure how much can be saved in this transmission, if one knows about the presence of the motif. Neville-Manning et al. 38 demonstrated that K × log2 P(M) is the saving obtained from a motif M over K covered sequences, which is equivalent to the IG formula.
The LogOdd (LogOdd) measure provides the degree of surprise of a pattern. It compares the actual probability of occurrence (relative support value) with the expected probability of occurrence according to the background distribution. The formula presented in Table 1 is a variant of the LogOdd formula introduced in 36, which was first proposed to measure the significance of probabilistic patterns. This measure is particularly useful when comparing motifs with different lengths 1741. Both IG and LogOdd measures can be applied to all types of deterministic patterns.
The Pratt (Pratt) measure was introduced by Jonassen et al. 42 to rank extensible gap motifs obtained from the Pratt algorithm. Its value is calculated in two steps. In the first step, the information encoded by the motif is calculated. The second step corresponds to a penalty that is considered when gaps occur. The last measure used was the Z-Score measure. Although it is essentially a statistical measure, it was included in this group as it can be calculated based on the support, the motif information and the number of amino acids in the database (constant value). This measure can be used to filter out irrelevant motifs by selecting only those whose actual number of occurrences considerably exceeds its expected number. This criteria is based on the following biological motivation: if a motif occurs more than it is expected to occur by chance, then it should have a biological interest 337. Z-Score is one of the most widely used measures for motif evaluation, see for example 3743.
In the Z-Score formula (see Table 1), Support(M) denotes the actual number of occurrences, E(M) the expected number of occurrences of M, and N(M) the square root of the expected variance.
It was generally verified that statistically relevant motifs, discriminated through the Z-Score function, match functionally important regions of the proteins 3743. Another important conclusion obtained from 37 is that for over-represented motifs, the non-maximal motifs (which are contained in other motifs) have a lower degree of surprise than the maximal ones. This result is a good example that significance measures can be used as a clever mechanism to prune motifs not only after, but also before, their significance is computed. The minimum support criterion provides a way to detect those motifs that occur frequently. Significance measures, like Z-Score or IG, allow to detect motifs that although not frequent occur more than expected or that represent a high degree of information. Both criteria are complementary in the task of automatically retrieving significant motifs from a database. Please refer to Table 1 and 2 for details on these measures.
Hybrid Measures
Considering measures that use both Information-theoretic and class-based features to determine the significance of a pattern, we selected two measures that are popular in the machine learning and data mining communities: the J-Measure (J) 44 and the Mutual Information (I), which is derived from the Shannon's entropy theory 344546.
For a class space Q = {C, C¯
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGafm4qamKbaebaaaa@2CFA@}, the component H(Q) of the I measure (see Table 1) provides the degree of information encoded by Q. Given a motif M, component H(Q|M) measures the amount of uncertainty remaining about Q after M is known. The difference H(Q) - H(Q|M) provides the expected information gain about Q upon knowing M.
The J measure is the product of two factors. The first factor, P(M), provides the prior probability of motif occurrence. The second factor, j(C; M), considers a target class C and its complement C¯
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGafm4qamKbaebaaaa@2CFA@ and measures the goodness-of-fit of M with relation to class C. It is also called cross-entropy 47.
In addition, we redefine the IG measure to account for the distribution of motifs among the protein families, leading to the definition of a measure called Surprise-Measure (S). The S measure combines the information content (Info) of the motif M with the conditional probability of M matching a sequence (s) from the target class C. This probability is given by the relative occurrence of M in C, Support(M∈C)Support(M)
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaqcfa4aaSaaaeaacqWGtbWucqWG1bqDcqWGWbaCcqWGWbaCcqWGVbWBcqWGYbGCcqWG0baDcqGGOaakcqWGnbqtcqGHiiIZcqWGdbWqcqGGPaqkaeaacqWGtbWucqWG1bqDcqWGWbaCcqWGWbaCcqWGVbWBcqWGYbGCcqWG0baDcqGGOaakcqWGnbqtcqGGPaqkaaaaaa@4820@, which corresponds to the positive predicted value of M. It expresses the amount of information provided by the motif and its quality as a class descriptor.
These three measures can be easily calculated for all types of deterministic motifs. In general, one can interpret such measures as a way to quantify the uncertainty reduction of a sequence s belonging to the class C, given that s contains the motif M.
In conclusion, the presented measures can be calculated based on two components of motif information: the class match information (TP, TN, FP, FN) component and the motif probability and gap information component. Class-based measures are calculated according to the first component, Information-theoretic measures based on the second and hybrid measures based on both. Table 2 contains formulas to support a better understanding of Table 1.
Evaluation
We start by describing the algorithms applied to mine the three different types of motifs used in the experiments.
To mine contiguous motifs we developed a simple algorithm based on the n-gram methodology. A n-gram is a word of n contiguous symbols. The algorithm takes as input a set of sequences and the target motif, which represents the motif to be primarily spotted. It extracts words with the length of the target motif (n = motif length) through window sliding. Each word is hashed into a table and the respective support count incremented. Finally, the score values for the different measures of all the scanned words are calculated. Due to their popularity within the bioinformatics community, Teiresias 48 and Pratt 49 were used to extract rigid and flexible gap motifs, respectively. Besides the input dataset, Teiresias algorithm accepts as input three parameters: minimum support, L and W, where L defines the minimum number of concrete symbols that a word of length W must contain. Pratt allows specifying the characteristics of the extracted motifs by setting a large number of parameters. It automatically scores the motifs according to the Pratt measure. With the exception of the minimum support value and the number of reported motifs all the remaining Pratt parameters were used assuming the default values recommended by the authors (program available at 50). Additional details for the use of these programs are provided whenever necessary.
The consistency between two measures can be defined as follows:
Definition 1. (Measure Consistency) Given two measures M1 and M2 and the respective score value vectors VM1 and VM2, the respective consistency is determined by the Pearson's Correlation between its vectors, corr(VM1, VM2).
Informally, a motif is considered to be strongly conserved if it occurs in the majority of the input sequences, i.e., its relative support value is approximately 100%. Alternatively, it is considered weakly conserved if its relative support is considerably below 50%.
Ranking Analysis
In this first experiment, the ability of the introduced measures in ranking the three different types of motifs is evaluated. The general evaluation procedure was as follows: select a target motif from Prosite, Dilimot or synthetically generated motif. Gather the set of related protein sequences where false negatives may occur. The parameters of the algorithm are refined until the target motif is included in the reported solution. For motif ranking evaluation only positive information is considered. Since not all the elements of class match information are available, only Information-theoretic measures are used in the ranking evaluation. In order to assess the quality of the measures in ranking the target motifs, a metric called Rm (Formula 3) was used, where Nmotifs is the total number of evaluated motifs and Rankmotifs the sum of the respective rank values. Measures with Rm closer to 1 are the best.
R
m
=
N
m
o
t
i
f
s
R
a
n
k
m
o
t
i
f
s
MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemOuai1aaSbaaSqaaiabd2gaTbqabaGccqGH9aqpjuaGdaWcaaqaaiabd6eaonaaBaaabaGaemyBa0Maem4Ba8MaemiDaqNaemyAaKMaemOzayMaem4CamhabeaaaeaacqWGsbGucqWGHbqycqWGUbGBcqWGRbWAdaWgaaqaaiabd2gaTjabd+gaVjabdsha0jabdMgaPjabdAgaMjabdohaZbqabaaaaaaa@47E2@
Contiguous Motifs
Real protein sequence data was obtained from Prosite. Entries that contain contiguous motifs were selected and the respective sets of sequences retrieved. Additionally, synthetic protein data was generated. Each synthetic dataset consists of 50 sequences of length 300. For each dataset, a motif of a given length was randomly generated and planted once in all its sequences. The generation of sequences and motifs was done according to the Swiss-Prot amino acid frequency. Motifs were then extracted according to the described n-gram methodology.
Table 3 shows the ranking of 11 Prosite motifs and Table 4 the results for a group of 8 synthetic protein datasets. In both cases, the target motifs are highly conserved with a support of around or equal to 100%.
<p>Table 3</p>
Evaluation of contiguous motifs on Prosite data.
PS entry
Motif
NumSeqs
DiffNGrams
Rel. Supp(%)
Supp Rank
ZScore
LogOdd
Pratt
IG
Info
PS00341
IPCCPV
9
702
77.8
9
21
65
166
13
217
PS00415
LRRRLSDS
12
3582
91.6
9
503
1058
2103
11
1784
PS00047
GAKRH
105
653
93.3
21
61
109
216
27
460
PS00984
CFWKYC
19
1256
100
1
1
1
785
1
5
PS00541
SKRKYRK
6
144
100
1
85
110
131
3
134
PS00822
PFDRHDW
9
2251
100
1
1
5
204
1
400
PS00419
CDGPGRGGTC
207
32936
100
1
1
1
3
1
158
PS00349
RKRKYFKKHEKR
18
2929
100
1
38
86
2884
19
310
PS00861
GWTLNSAGYLLGP
32
888
100
1
66
301
179
1
569
PS01024
EFDYLKSLEIEEKIN
60
5527
100
1
620
2427
5266
1
5244
PS00291
AGAAAAGAVVGGLGGY
136
2423
100
1
1033
1770
184
3
1984
R
m
0.2340
4.526E-3
1.854E-3
9.075E-4
0.1358
9.764E-4
Ranking results of eleven Prosite datasets (identified by the Prosite (PS) entry column). For each dataset, the number of protein sequences, the number of different n-grams (Diff NGrams), where n is equal to the motif length and the relative support of the target motifs (Rel. Supp) are presented. Motifs are ranked with Information-theoretic based measures. Ranks obtained by support (Supp Rank) and information gain (Info) are also provided for comparison purposes. Last row gives the Rm values of each measure, where best results are obtained by support and IG.
<p>Table 4</p>
Evaluation of contiguous motifs on protein synthetic data.
Motif
Supp
ZScore
LogOdd
Pratt
IG
SSN
1
3710
1
2130
1
IYKQ
1
1533
2
11817
1
NDFNE
1
1
1
13483
1
PLMPES
1
1
2
4973
1
MRKMVTAG
1
1
6
9818
1
TKYEETGAFK
1
1
43
7350
1
DRTGMHSIFFLP
1
1
3
11721
1
MTENKVGESICPAAPN
1
1
29
9589
1
R
m
1
0.0015
0.0919
1.128E-4
1
Ranking results for eight synthetic protein datasets. Each dataset contains 50 sequences of length 300. Target motifs have a support of 100%. Motifs are ranked with Information-theoretic measures and support. Last row gives the Rm values of each measure, where the best results are obtained by IG and support.
Rigid Gap Motifs
Table 5 shows the ranking of rigid gap motifs from ten datasets of the Dilimot database. This experiment was performed to evaluate weakly conserved motifs. Table 6 presents the results for 8 datasets from Prosite. The evaluation is focused on long and strongly conserved rigid gap motifs. Teiresias algorithm was used to extract the motifs, were L and W parameters were set to conform the characteristics of the target motif and the minimum support set to 80% of its actual support.
<p>Table 5</p>
Evaluation of rigid gap motifs on Dilimot datasets.
Motif
NumSeqs
Abs. Supp
Supp Rank
IG
Pratt
LogOdd
Zscore
LPSN
15
4
1294
520
2429
4
6
WS.WS
34
7
15
22
31
28
28
Q.RLQ..Q
15
4
5259
660
5213
1
1
P.LP.K
24
8
1334
336
<