Meaning of Explicit & Implicit ( in Math, in ML )

I've often come across the terms Implicit Method and Explicit Method while studying machine learning and AI. It's time to revisit these lightly brushed topics and gain a clearer understanding.

Dictionary Definitions

• Explicit:
  : Something explicit is stated clearly and in detail, leaving no room for confusion or doubt.

• Implicit:
  : Something implicit is suggested though not plainly expressed, inherent in the nature of something.

  To summarize, while explicit refers to something that is stated clearly and in full detail, implicit refers to something that is suggested but not directly expressed. This seems to be a bit distant from the context we encounter in academic papers, so let's dig a little deeper.
  
  

Implicit Method & Explicit Method in Mathematics

• Explicit Method
  : The dependent variable is expressed in terms of independent variables. In other words, x and y can be separated.

  $y = x^2 + x + 1$

• Implicit Method
  : An implicit function is defined as a relationship between x and y where the dependent and independent variables cannot be separated.

  $x^2 + 2xy + y^2 = 0$
  
  Things are getting clearer. With the Explicit Method, we obtain a single solution for a given time step directly through computation. In contrast, the Implicit Method might involve solving equations for each time step, potentially yielding multiple solutions.

  However, for most practical problems, using an Implicit Method with appropriate initial guesses and numerical techniques can yield a single physically plausible solution. Mathematical theorems that guarantee the existence and uniqueness of solutions are applied under appropriate conditions. We will not delve into the complex mathematics here.

  In conclusion, Implicit Methods may require more complex computations but have the potential to yield more stable results and can work with larger time steps compared to Explicit Methods.

  ❓ Why are Implicit Methods considered to be more stable?
  
  Why do implicit numeric methods have superior stability properties than explicit ones?
  The discussion in this link provides a simple mathematical example and answers this question. To put it intuitively, Implicit Methods generally include more "feedback" from the numerical solution path, which makes them more stable than Explicit Methods.

  Even from a mathematical standpoint, as well as in ML, these terms seem to be used with slightly different meanings. Let's move on to the explanation in the context of ML.
  
  
  

Implicit Method & Explicit Method in Machine Learning

• Explicit Method
  : Refers to functions that are defined clearly by an equation or algorithm. For example, a linear regression model is explicit because it has a specific equation describing the relationship between input and output variables.
• Implicit Method
  : Defined indirectly by data. For instance, neural networks are implicit since they do not have a specific equation for input-output relationships but learn this through dataset training.

In essence, we can say that an Explicit Method has various parameters based on which it can clearly explain the relationship between inputs and outputs. On the other hand, if the model does not have a clear formula for the input-output relationship, it is considered an Implicit Method.

Examples of Explicit Method Models

Explicit Method models mean that the relationship between inputs and outputs is clearly and directly defined. These models often allow for interpretation and understanding of what is happening inside the model. They are characterized by having explicit parameters. Here are some examples:

1. Linear Regression:

   • One of the most representative explicit models. It predicts continuous output values through a linear combination of input variables.
   • The model is represented by the equation $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + ... + \beta_nx_n + \epsilon$.

2. Logistic Regression:

   • Used for classification problems, it extends linear regression to predict binary or multinomial outcomes.
   • The output is calculated through the logistic function given by $\frac{1}{1 + e^{-(\beta_0 + \beta_1x_1 + ... + \beta_nx_n)}}$.

3. Decision Tree:

   • Represents decision rules in a tree structure to partition the input space and make predictions.
   • Each branch point in the tree represents an explicit decision rule for input variables, and each node signifies a specific output value.

4. Rule-Based Models:

   • Use explicit rules to classify data or make predictions.
   • These models consist of a set of if-then rules, and decisions are made when input data passes through these rules.

Examples of Implicit Method Models

Implicit Method models learn from data without direct formulas or rules. These models internally can learn complex patterns of data, but the process or outcome may not be intuitively clear. Deep learning models are a common example.

1. Artificial Neural Networks (ANNs):

   • Capable of learning complex patterns from data, they learn the mapping from input to output through weights and activation functions.
   • These models are fundamental components of Deep Learning, capable of modeling complex nonlinear relationships.

2. Convolutional Neural Networks (CNNs):

   • Suitable for high-dimensional data like images, they learn features of images using filters (kernels).
   • CNNs have a hierarchical structure that can learn from low-level to high-level features.

3. Recurrent Neural Networks (RNNs):

   • Suitable for sequence data like text or time series, they learn the information in a sequence by using the output of the previous step as part of the input for the next step.
   • LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) are advanced forms of RNNs that learn long-term dependencies better.

4. Autoencoders:

   • Learn how to efficiently compress (encode) input data and use this compressed representation to reconstruct (decode) the input.
   • A form of unsupervised learning, they are used to learn efficient representations of data.

5. Reinforcement Learning Models:

   • An agent learns how to maximize rewards through interactions with the environment.
   • Neural networks like Policy Networks and Value Networks are used to learn optimal strategies.

p.s. This content has been translated using ChatGPT, so there may be some mistranslations

References

Cambridge Dictionary
Why do implicit numeric methods have superior stability properties than explicit ones?
What is the difference between explicit and implicit functions in machine learning?