A KEY TO THE DATA SCIENCE “TOWER OF BABEL”

Terminology—sometimes referred to as “nomenclature”—is important in every field. In data science, New School (NS) and Old School (OS) nomenclature may be identical, sometimes defined in only one school, but often differs between the OS & NS even for the same word. We help mediate this “Tower of Babel” with a glossary of terms so we can understand data science’s different terminologies.

At the bottom of this page, a bibliography appears of other glossaries to which we referred. The referenced glossary appears in parentheses after the entry. Part or all of some entries were initially generated from AI bots and then corrected and edited, and we note that.

These entries are intended to be starting points, and can by no means be considered completely comprehensive. The number of entries will most certainly grow and may change in tone and content as data sciences advance and evolve. We welcome help to continue to improve and refine this Glossary: It may never be completed as it is a work in progress. Please give us your suggestions, comments, and views on any points within it. This should generate highly productive discussions for everyone from hardcore old-schoolers or rookie boot camp freshers. Everyone’s opinion is encouraged, no matter your position, country, industry, or connection to data science.

A

ABLATION:
(NS) A technique for evaluating the importance of a feature or component by temporarily removing it from a model. You then retrain the model without that feature or component, and if the retrained model performs significantly worse, then the removed feature or component was likely important.

For example, suppose you train a classification model on 10 features and achieve 88% precision on the test set. To check the importance of the first feature, you can retrain the model using only the nine other features. If the retrained model performs significantly worse (for instance, 55% precision), then the removed feature was probably important. Conversely, if the retrained model performs equally well, then that feature was probably not that important.

Ablation can also help determine the importance of:

Larger components, such as an entire subsystem of a larger ML system
Processes or techniques, such as a data preprocessing step

In both cases, you would observe how the system’s performance changes (or doesn’t change) after you’ve removed the component. (Google Machine Learning Glossary)

A/B TESTING:
(NS) A statistical way of comparing two (or more) techniques—the A and the B. Typically, the A is an existing technique, and the B is a new technique. A/B testing not only determines which technique performs better but also whether the difference is statistically significant.

A/B testing usually compares a single metric on two techniques; for example, how does model accuracy compare for two techniques? However, A/B testing can also compare any finite number of metrics.
(Google Machine Learning Glossary)

(OS) A/B testing is a family of statistical techniques to evaluate change given 2 different stimuli. It is often used in advertising where 2 (or more) 30-second direct response spots air and we are interested in call and sales volume brought or Internet actions on by each. A/B testing is a much richer term when viewed synergistically because tests can extend to ABCDEF… and beyond that be designed in dynamic blocks and multiple dimensions to quantify a behavior change metric from a hierarchy of marketing, advertising, or other actions. ANOVA and MANOVA are two OS methods that are really A/B testing on steroids as they work on stimuli that differ by time, space, and network.

In marketing, A/B testing can refer to two different service protocols or product features, and customer satisfaction metrics can then be compared side-by-side to see which product or protocol yields higher customer sat scores. This technique is usually implemented as an experiment or quasi-experiment with randomized control and test samples.

ACCURACY:

This term has a general definition and 2 highly specific ones.

In general, accuracy may be thought of how close a model (or network) is to reality for both present and future.

First Specific Definition of Accuracy: Classifier Models

(NS) The number of correctly predicted observations over the sum of all observations. (Used for classifier models for limited (binary or categorical) dependent variables.)
A high accuracy is desirable since it is often chosen as the metric used to select the Champion (best) model from a field of challengers.

However, a more than trivial number of researchers argue that Accuracy is a poor way to characterize the quality of a given model, especially if data are used to model so-called rare events, or the data are otherwise imbalanced.

To assess the quality (relative or absolute) of a classification model, forecasting models, and certain regression models, alternative metrics to accuracy contain more information.

(OS) Accuracy is the only metric many NS practitioners use to test the worth of a classification model. To assess the quality (relative or absolute) of a classification model, forecasting models, and certain regression models, alternative metrics to accuracy contain more information. These alternatives are most useful when several metrics are used rather than a single, flawed accuracy metric. Alternatives to accuracy include precision (Positive Rate, Predicted), recall/sensitivity (TPT-True Positive Rate), specificity (TNR-True Negative Rate), and balanced accuracy (average of TPR and TNR). The nature of the classification problem and business or scientific needs will often dictate specific metrics as success measures. Forecast accuracy is measured by over a dozen metrics that can be absolute, relative, and based on median, mean, or squared values. Sampling may be a 70/30 split of the entire data set or statistical sampling methods employed to lower computing burden and avoid overfitting/underfitting by leveraging true (smaller) random samples for train, test, validation, time series holdout, and feature engineering.

Second Specific Definition of Accuracy: Forecasting and Prediction

(OS) Prediction accuracy refers to how well a model can predict outcomes or values. Often, this is in non-forecast space, e.g. how well does a model do on the data it was estimated with?
It is typically measured using metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), or the coefficient of determination (R-squared).
A higher prediction accuracy indicates that the model’s predictions are closer to the true values of test and train samples. (Data-Scientist-GPT3, Poe.com)

(NS) Forecast accuracy refers to how well a model can forecast outcomes or values in the future. Often, this is in forecast space, e.g. how well does a model do on the data it was NOT estimated with?
It is typically measured using metrics such as those mentioned above. But more typically there are specialized metrics such as (MAPE) Mean Absolute Percentage Error, (MdAPE) Median Absolute Percentage Error (MdAD),
Median Absolute Deviation, and (RMPSE) Root Mean Absolute Percentage Error. (Hans Lenenback & James Cleary, Forecasting: Practice and Process for Demand Management. )
A higher forecast accuracy indicates that the model’s future forecasts are closer to the true values as they unfold through time.

ACF: See autocorrelation function used in the identification stage of time series models, below.

ACTIVATION FUNCTION:
(NS) An activation function is used in artificial neural networks (ANN) that determines whether a neuron should be activated or not by calculating its output to the next hidden layer (or output layer) based on the input from the previous layer (or input layer). The activation function is responsible for the non-linear transformation of a neural network. (Data Camp Glossary of Data Science)

AGGLOMERATIVE CLUSTERING:

(OS & NS) Agglomerative clustering is a hierarchical clustering algorithm used to group similar data points together based on their proximity or similarity. It is a bottom-up approach where each data point initially forms its own cluster and iteratively merges the closest clusters until a stopping criterion is met.

The techniques used in agglomerative clustering include:

Proximity Measures: A proximity measure is required to determine the distance or similarity between data points. Common proximity measures include Euclidean distance, Manhattan distance, cosine similarity, and correlation distance.

Linkage Criteria: Linkage criteria determine the distance between clusters during the merging process. Different linkage criteria can produce different cluster structures. Some commonly used linkage criteria are:

Single Linkage: Measures the distance between the closest pair of points from two different clusters. It tends to form long, string-like clusters.
Complete Linkage: Measures the distance between the farthest pair of points from two different clusters. It tends to form compact, spherical clusters.
Average Linkage: Measures the average distance between all pairs of points from two different clusters.
Ward’s Linkage: Measures the increase in within-cluster variance after merging two clusters. It aims to minimize the variance within each cluster.
Distance Matrix: A distance matrix is computed to store the pairwise distances or similarities between all data points. It is initially empty and is progressively updated as clusters are merged.
Cluster Merging: The algorithm starts with each data point as a separate cluster. At each iteration, it merges the two closest clusters based on the chosen linkage criterion until a stopping condition is met. The process continues until all data points are brought together in a single cluster or until a desired number of clusters is reached.

Dendrogram: (NS) A dendrogram is a tree-like diagram that represents the hierarchical structure of the clusters. It visually displays the order in which clusters are merged and helps determine the number of clusters by analyzing the heights at which clusters are joined. (OS) Scree Digaram. Agglomerative clustering is relatively easy to implement and interpret. However, it may not scale well to large datasets due to its quadratic time complexity, as it requires computing the distance matrix at each iteration. (Poe.com AI assistant)

AGGREGATION (GROUPING):

(OS) The use of group sums or group means (or other grouped data metrics) for modeling purposes rather than individual variables. This can be in an events-trials syntax or a contingency table of grouped values as well as cohort analysis or vintage data structures. Although this leads theoretically to a loss in efficiency because of the loss of information arising from the data aggregation, in applications this is not necessarily so, since aggregation can to a certain extent cancel out errors in measurement or misspecifications of microrelationships. R-squared are higher with grouped data because errors tend to cancel one another when summed. Care must be taken in determining the basis on which grouping is undertaken since different results are usually obtained using different grouping rules. Note that heteroscedasticity results if each group does not contain the same number of observations. (A Guide to Econometrics, Petter Kennedy 6th edition, glossary). Bucketing of continuous variables (also known as binning, discretization, categorization, and interval mapping are synonymous with grouping in that context.

ALGORITHM: (NS & OS):

An algorithm is a set of instructions that is designed to accomplish a task. Algorithms usually take one or more inputs, run them systematically through a series of steps, and provide one or more outputs. Algorithms are typically associated with computing and are an essential element of computer programming. Algorithms can be used to accomplish a variety of computational tasks, such as performing calculations, finding specific data observations, cleansing data, data visualization, logical queries of databases., implementing forecasting models and methods, estimating or describing model results, and evaluating diagnostic tests.

Algorithms can be created and used outside of computer programming as well. They can be executed manually by people or executed automatically by machines: consider performing long division manually on paper versus using a calculator to do the same operation. Users do not need to understand the inner workings of algorithms in order to use them. In fact, many algorithms used by companies are closely guarded secrets, blocking users from seeing exactly how they work. (Network of National Library of Medicine, Data Glossary)

(OS) Note that algorithms are sometimes called “black boxes” because no one in the organization can edit or change key code blocks. The choice of a black box technique could cause a poor audit, be regulatorily unsound, be too complex, and place dependence on a few developers or a data scientist who could leave at any time. Documentation must exist for code, data, result analysis, feature engineering, and back-testing, Sometimes no one at the organization knows how to set hyperparameters away from the default in the background of the algorithm. This and other weaknesses combine to create skepticism among old-school practitioners about weaker algorithms, and complex modeling methods that require a numerous set of hyperparameters to be tuned. or very recently released algorithms with narrow testbeds.

ALGORITHMIC BIAS:

(NS) When something consistently strays from what’s considered normal or standard. For example, bias in statistics can refer to when a sample group might not accurately represent the whole population, or in ethics it can refer to when a group is favored over another. There are many other ways bias can show up.

Algorithmic bias is when bias happens within a computer program or system. This is often talked about in relation to systems that operate on their own, like artificial intelligence.

There are several ways algorithmic bias can happen:

Biases in the data used to train the system. For example, a program that translates text might be biased if the data it was trained on wasn’t good enough in one of the languages it’s translating.
Biases in what information is included or left out of the system. For example, a program that predicts who will miss doctor appointments might unfairly target poor people, racial minorities, or people living in rural areas if it includes data on race, income, or distance from a medical center.
Biases introduced to fix other issues with the system. For example, a programmer might add new biases to try to balance out the original unfairness in the program that predicts doctor appointment no-shows.
Biases caused by using the system in a different context than it was designed for. For example, a program designed for use in the United States might not work well in a different legal, social, or economic context.
Biases in how the system’s results are interpreted. For example, the person using the program might not fully understand what the program’s results mean and might make assumptions or decisions that aren’t reasonable.

When you hear the word “bias,” it’s important to understand what the person using the word means. Also, remember that bias doesn’t always mean someone did something wrong or prejudiced. People using computer systems should always be aware of potential sources of bias and stay involved in making decisions, no matter what area they’re working in. (NNLM Data Glossary)

AKAIKE’S INFORMATION CRITERION (AIC):

(OS) AIC (Akaike’s Information Criterion): The AIC provides a measure of the goodness-of-fit of a model which takes into account the number of terms in the model. It is commonly used with ARIMA models to determine the appropriate model order. The AIC is equal to twice the number of parameters in the model minus twice the log of the likelihood function. The theory behind the AIC was developed by Akaike and is based on entropy concepts. See Order selection criteria. (Forecasting Glossary Terms).

ANALYSIS OF VARIANCE (ANOVA):

(OS) ANOVA performs Analysis OF VAriance (ANOVA) for balanced data from a wide variety of experimental designs. In analysis of variance, a continuous response variable, known as a dependent variable, is measured under experimental conditions identified by classification variables, known as independent variables. The variation in the response is assumed to be due to effects in the classification, with random error accounting for the remaining variation.

The ANOVA procedure is one of several procedures available in SAS/STAT software for analysis of variance. The SAS ANOVA procedure (PROC ANOVA) is designed to handle balanced data (that is, data with equal numbers of observations for every combination of the classification factors), whereas the GLM procedure can analyze both balanced and unbalanced data. Because PROC ANOVA takes into account the special structure of a balanced design, it is faster and uses less storage than PROC GLM for balanced data.

Use ANOVA for the analysis of balanced data only, with the following exceptions: one-way analysis of variance, Latin square designs, certain partially balanced incomplete block designs, completely nested (hierarchical) designs, and designs with cell frequencies that are proportional to each other and are also proportional to the background population. These exceptions have designs in which the factors are all orthogonal to each other.

ANOVA works for designs with block diagonal matrices where the elements of each block all have the same value. The procedure partially tests this requirement by checking for equal cell means. However, this test is imperfect: some designs that cannot be analyzed correctly might pass the test, and designs that can be analyzed correctly might not pass. If your design does not pass the test, PROC ANOVA produces a warning message to tell you that the design is unbalanced and that the ANOVA analyses might not be valid; if your design is not one of the special cases described here, then you should use PROC GLM instead. Complete validation of designs is not performed in PROC ANOVA since this would require the whole matrix; if you are unsure about the validity of PROC ANOVA for your design, you should use PROC GLM.

Caution: If you use PROC ANOVA to analyze unbalanced data, you must assume responsibility for the validity of the results. (SAS Institute, https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.3/statug/statug_anova_overview.htm)

ANOVA Models can normally be written in linear regression form using numerous dummy variables. ANOVA is often used in agriculture, social science experiments with control and class variables, and other problems. After 2010, ANOVA use has become less common except in fields like medicine, controlled human subject studies, psychology, and agricultural experiments.

ANCOVA:

(OS) general linear model which blends ANOVA and regression. ANCOVA evaluates whether the means of a dependent variable (DV) are equal across levels of a categorical independent variable (IV) often called a treatment, while statistically controlling for the effects of other continuous variables that are not of primary interest, known as covariates (CV) or nuisance variables. Mathematically, ANCOVA decomposes the variance in the DV into variance explained by the CV(s), variance explained by the categorical IV, and residual variance. Intuitively, ANCOVA can be thought of as ‘adjusting’ the DV by the group means of the CV(s). (https://en.wikipedia.org/wiki/Analysis_of_covariance)

ANSWER SETS / MODEL SETS:

One of the chief principles of synergistic data science is the answer set. Rather than engage in uni-dimensional thinking using a single method, numerous methods and tunings are used to create a synergistic “set” of answers rather than a singular answer. Thinking in this way bolsters the usefulness of synergistic data science
by providing robust answers that highlight our degree of certainty in the desired outcome. The new-school boot camp answer (so often used without any hyperparameter-tuning) may not match any one of 10 synergistic alternatives.

By observing the average, median, and other metrics the synergistic approaches provide critical insights across “model sets” or “answer sets”.
“Model sets yield predictions, forecasts, or other target (dependent) variables. These variables can be probability-based, dollar-based, numeric-based, or category-based along with
a host of variations and data structures. Metrics that are automatically produced by synergistic data science yield insights that are exponentially more powerful than a single new-school stock ‘answer’ “. We are talking about model-based answer sets for non-stochastic or non-model-based problems. These problems are diverse: anything from hurricane strike probabilities to Who should get a credit card at a bank are founded on practical methods from many sciences and industries.
“Answer sets recognize that much of data science is not a model (black-box or otherwise). Analytical insights about key business metrics, social network architecture, and far more advanced topics depend on non-model-based synergistic data science.
(Dr. Gordy Fairchild, CEO SynergyData.Science)

API: (NS)
is an acronym for Application Programming Interface, a software intermediary that ensures a connection between applications or computers. An example of an API is embedding Google Maps in a Rideshare application. Data scientists often work with APIs to access data (e.g., Twitter API to download tweets), or to
package a solution they made (e.g., an API that calls a machine-learning model in production) [Data Camp Glossary] OS: No such technology.

ARIMA MODELS: (OS) An abbreviation for AutoRegressive Integrated Moving Average. A time series that, when differenced, follows an ARMA model is known as an ARIMA model. It is a very broad class of time series models. See the Autoregressive (AR) model, Differencing, Integrated, and Moving average. (Glossary of Forecasting Terms).

Models are based on three components: autoregression (AR), differencing (I), and moving average (MA).

Autoregression (AR): The autoregressive component of an ARIMA model involves using past observations of the time series to predict future values. It assumes that the value of the series at any given time is linearly dependent on its previous values. The “p” parameter in ARIMA(p, d, q) represents the number of lagged observations used for predicting future values.

Differencing (I): The differencing component of an ARIMA model is used to make the time series stationary. Stationarity refers to a time series having a constant mean, constant variance, and an autocovariance that does not depend on time. If the time series is not stationary, differencing is applied to remove trend and seasonality. The “d” parameter in ARIMA(p, d, q) represents the order of differencing.

Moving Average (MA): The moving average component of an ARIMA model uses the error terms or residuals from the autoregressive component to predict future values. It captures the dependency between the observed value and residual errors from previous predictions. The “q” parameter in ARIMA(p, d, q) represents the number of lagged forecast errors used in the model.

The notation for an ARIMA model is ARIMA(p, d, q), where “p” represents the order of the autoregressive component, “d” represents the order of differencing, and “q” represents the order of the moving average component.

The parameters “p,” “d,” and “q” are determined through statistical techniques such as autocorrelation function (ACF) and partial autocorrelation function (PACF) plots. These plots help identify the appropriate values for the parameters by examining the correlation between the time series and its lagged values.

ARIMA models can be extended to include additional components such as seasonal patterns, resulting in seasonal ARIMA (SARIMA) models. These models incorporate seasonal differencing and seasonal autoregressive and moving average components to capture patterns that repeat over fixed time intervals.

ARIMA models are widely used for short-term forecasting, understanding time series behavior, and identifying trends and patterns in data. (Poe.com AI assitant)

The plot above (from SAS: https://communities.sas.com/t5/SAS-Communities-Library/SAS-Visual-Forecasting-8-4-Interpreting-Results-and-Diagnostic/ta-p/581294 ) shows how the parts of an ARIMA model fit into the whole concept.

ARMA MODELS: (OS) This type of time series forecasting model can be autoregressive (AR) in form, moving average (MA) in form, or a combination of the two (ARMA). In an ARMA model, the series to be forecast is expressed as a function of both previous values of the series (autoregressive terms) and previous error values from forecasting (the moving average terms). (Glossary of Forecasting Terms). No differencing is involved (as with ARIMA) so there can be potential problems with stationarity and stability.

ARTIFICIAL INTELLIGENCE (AI):

(NS) (also cognitive computing, machine intelligence, synthetic intelligence) – Intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by humans or by other animals.

Conversational AI – A type of technology, like a chatbot, that simulates human conversation, making it possible for users to interact with and talk to it.
Curiosity intelligence – Curiosity artificial intelligence (AI) mimics the natural curiosity of humans that allows us to learn things on our own. The goal is to develop curiosity through machine learning (ML) algorithms so AI systems can seek solutions to new problems independently.
Decision intelligence – Decision Intelligence (DI) combines data science with different scientific theories to help people make the best possible decisions. It aims to provide actionable insights by translating raw data into formats that decision-makers can easily understand.
Emotional AI (also affective computing, artificial emotional intelligence) – The study and development of systems and devices that can recognize, interpret, process, and simulate human affects.
Friendly AI (also friendly artificial intelligence, FAI) – A hypothetical artificial general intelligence (AGI) that would have a positive effect on humanity. It is a part of the ethics of artificial intelligence and is closely related to machine ethics. While machine ethics is concerned with how an artificially intelligent agent should behave, friendly artificial intelligence research is focused on how to practically bring about this behavior and ensure it is adequately constrained.
General AI (AGI) (also strong AI, ASI) – AI that could successfully do any intellectual task that can be done by any human being.
Generative AI (also GenAI, evolving AI, self-improving AI)- Generative AI uses machine learning (ML) to produce new content from an extensive training dataset. The format of the result can be text, images, video, code, 3D renderings, or audio. Nowadays, when we interact with a search engine like Google or when we use a traditional question-answer chatbot, we are requesting existing information. In contrast, when using generative AI-based tools, the model is using existing information to generate original content, such as songs, poems, articles, etc. (slang stochastic parrot, word calculator)
Interactive AI – AI systems that can engage in human-like conversations and respond dynamically to user inputs. These systems are designed to understand the nuances of human language, interpret context, and provide appropriate responses.
Nouvelle AI – Nouvelle AI differs from classical AI by aiming to produce robots with intelligence levels similar to insects. Researchers believe that intelligence can emerge organically from simple behaviors as these intelligences interacted with the “real world”, instead of using the constructed worlds which symbolic AIs typically need to have programmed into them.
Pseudo-AI – The phenomenon of so-called pseudo-AI happens when companies promote their ultrasmart AI interfaces and don’t mention the people working behind the scenes as fake chatbots. In their eager quest to gain the attention of wealthy investors, some companies give the impression their platforms or tools are already past the stage of needing thorough [machine learning] training and are fully automated. That’s called The Wizard of Oz design technique, because it reminds people of the famous movie scene where Dorothy’s dog, Toto, pulls back the curtain and reveals a man operating the controls for the Wizard’s giant talking head.
Seed AI – An Artificial General Intelligence (AGI) which improves itself by recursively rewriting its own source code without human intervention. Initially, this program would likely have minimal intelligence, but over the course of many iterations, it would evolve to human-equivalent or even trans-human reasoning. The key to successful AI takeoff would lie in creating adequate starting conditions.
Weak AI (also narrow AI, ANI) – A model that has a set range of skills and focuses on one particular set of tasks. Most AI currently in use is weak AI, unable to learn or perform tasks outside of its specialist skill set. (U of Oregon AI Glossary)

ARTIFICIAL NEURAL NETWORKS:

(NS) Computing systems that are vaguely inspired by the biological neural networks that constitute animal brains. Such systems “learn” to perform tasks by considering examples, generally without being programmed with task-specific rules. For example, in image recognition, they might learn to identify images that contain cats by analyzing example images that have been manually labeled as “cat” or “no cat” and using the results to identify cats in other images. (https://en.wikipedia.org/wiki/Artificial_neural_network)
An Artificial Neural Network is a machine learning model that is loosely inspired by biological neural networks in human brains. Neural networks consist of up to hundreds of layers of interconnected units called neurons. Conceptually, an artificial neural network has the following types of layers: input, output, and hidden layers used to filter the data through, process it with an activation function, and make predictions at the output. ANN are the building blocks of a subset of machine learning called deep learning, which delivers complex outputs such as image or sound recognition, object detection, language translation, and more. (Data Camp Data Science Glossary)

ASSOCIATION RULE MINING: (NS) consists of first finding frequent item sets (sets of items, such as A and B, satisfying a minimum support threshold, or percentage of the task-relevant tuples), from which strong association rules in the form of A=>B are generated. These rules also satisfy a minimum confidence threshold (a pre-specified probability of satisfying B under the condition that A is satisfied). Associations can be further analyzed to uncover correlation rules, which convey statistical correlations between item sets A and B. (http://amzn.to/39flUbX)

ASYMPTOTICALLY UNBIASED ESTIMATOR: (OS) If the bias of an estimator approaches zero as the sample size increases, the estimator is called “asymptotically unbiased.” The formula for mean squared error is a biased estimator of variance, but it is asymptotically unbiased. (Glossary of Forecasting Terms)

AUTOCORRELATION (SERIAL CORRELATION):

Autocorrelation is a term with several meanings.

1) (OS) In linear regression a key assumption of OLS BLUE best Linear Unbiased Estimators) is that the errors (disturbances) are independent among observations. This is not the case with autocorrelation so hypothesis testing, standard errors, and other metrics of inference are biased even asymptotically. Also, although the OLS regression coefficients themselves are unbiased, they no longer have efficiency (minimum variance. Unless models are tested for autocorrelation, the data scientist runs the risk that the model will be of limited use since disturbances are nonspherical. Diagnosis is accomplished via the Durbin-Watson (DW) test, the Durbin-Watson h test for models with lagged dependent variable terms and other alternatives. A Bayesian estimator might be preferable to using an alternative estimator or corrective technique. Most cases occur with time series data but autocorrelation can still occur with non-forecasting models with sequential observations. First-order autocorrelation (AR{1}) is typically most common in OLS estimation corrections although the orders that go back more than 1 period can also occur but those are usually part of a forecasting time series model.

2. (OS) On the forecasting side, According to Poe AI:

Autocorrelation, also known as serial correlation, is a statistical concept that measures the degree of similarity between a time series variable and a lagged version of itself. In other words, it quantifies the linear relationship between observations of a variable at different points in time. Autocorrelation is commonly used in the field of time series analysis to understand the patterns and dependencies present in sequential data. It is particularly relevant when analyzing data that exhibits temporal dependence, such as stock prices, weather data, or economic indicators.

Autocorrelation is often represented by a correlation coefficient, typically denoted as “r” or “ρ” (rho). This coefficient ranges between -1 and 1, where a value of 1 indicates a perfect positive autocorrelation (a high value in the current observation is associated with high values in the lagged observations), a value of -1 indicates a perfect negative autocorrelation (a high value in the current observation is associated with low values in the lagged observations), and a value of 0 indicates no autocorrelation (the current observation is independent of the lagged observations).

Autocorrelation can be visualized using a correlogram, which is a plot that displays the autocorrelation coefficients at different lags. It helps to identify patterns such as seasonality, trends, or random fluctuations in the data.

Understanding autocorrelation is important because it can affect the reliability of statistical analyses and predictions. If autocorrelation is present in a dataset, it violates the assumption of independence, which is often required by statistical techniques. In such cases, specialized methods, such as autoregressive integrated moving average (ARIMA) models or other time series forecasting techniques, can be employed to account for autocorrelation and make accurate predictions.

Autocorrelation can be addressed or accounted for in statistical techniques using various methods. Here are a few commonly employed approaches:

Differencing: Differencing is a technique used to remove autocorrelation by subtracting the previous observation from the current observation. This can be done once or multiple times until the data becomes stationary (i.e., the mean and variance of the series do not change over time). Differencing is a key step in models like autoregressive integrated moving average (ARIMA) and can effectively mitigate autocorrelation.

Autoregressive Models: Autoregressive models, such as ARIMA, explicitly model the autocorrelation in a time series. These models assume that the current value of a series depends linearly on its previous values. By incorporating lagged observations as predictors, autoregressive models can capture and account for the autocorrelation structure in the data.

Moving Average Models: Moving average models, also a part of ARIMA, consider the influence of lagged forecast errors on the current value. These models assume that the current value depends on the past forecast errors rather than the previous observations themselves. By including lagged forecast errors as predictors, moving average models can address autocorrelation.

Autoregressive Integrated Moving Average (ARIMA): ARIMA models combine autoregressive and moving average components, along with differencing, to model and account for autocorrelation. ARIMA models are widely used for time series analysis and forecasting, and they can handle various types of autocorrelation patterns.

Autoregressive Conditional Heteroscedasticity (ARCH) Models: ARCH models are used when the autocorrelation in a time series is not constant but varies with time, particularly in financial data. ARCH models capture the volatility clustering and time-varying variance in the series by incorporating lagged squared residuals as predictors.

State-Space Models: State-space models are a flexible framework that allows for explicitly modeling and estimating the hidden states and their relationships in a time series. By incorporating autoregressive or moving average components within the state space model, autocorrelation can be accounted for.

These are just a few examples of techniques used to address autocorrelation. The choice of method depends on the specific characteristics of the data, the goals of the analysis, and the underlying assumptions of the statistical technique being used.

AUTOCORRELATION FUNCTIONS (ACFS):

The autocorrelation function (ACF) is a statistical tool used to measure the autocorrelation of a time series variable at different lags. It provides information about the correlation between an observation and its lagged values within the same time series. Here’s how you can calculate autocorrelation using the ACF:

Compute the ACF: To calculate the ACF, you need to first calculate the sample autocovariance function (ACOVF) at different lags. The autocovariance at lag k, denoted as γ(k), is calculated as the covariance between the observations at time t and time t-k. The ACF at lag k, denoted as ρ(k), is then calculated by dividing the autocovariance at lag k by the autocovariance at lag 0 (γ(0)), which is the variance of the time series.

ACF at lag k (ρ(k)) = γ(k) / γ(0)

Calculate autocovariance: To calculate the autocovariance at lag k, you can use the following formula:

γ(k) = [Σ((y(t) – mean(y)) * (y(t-k) – mean(y))) / (n – k)]

where y(t) represents the value of the time series at time t, mean(y) is the mean of the time series, and n is the total number of observations.

Calculate autocovariance at lag 0: The autocovariance at lag 0 (γ(0)) is the variance of the time series, which can be calculated as:

γ(0) = [Σ((y(t) – mean(y))^2) / n]

Normalize to obtain autocorrelation: Finally, divide the autocovariance at lag k by the autocovariance at lag 0 to obtain the autocorrelation at lag k:

ρ(k) = γ(k) / γ(0)

By calculating the autocorrelation coefficients at different lags using the ACF, you can observe how the correlation between observations changes as the lag increases or decreases. The ACF is typically represented as a plot, with the lag on the x-axis and the autocorrelation coefficient on the y-axis. This plot helps identify any significant autocorrelation patterns, such as positive or negative correlations at specific lags, which can be useful in selecting appropriate forecasting models and understanding the behavior of the time series. (Poe AI)

ABOVE GRAPH: From Excellent Kaggle article: https://www.kaggle.com/code/iamleonie/time-series-interpreting-acf-and-pacf

AUTOREGRESSIVE(AR) MODELS: Autoregression is a form of regression, but instead of the variable to be forecast being related to other explanatory variables, it is related to past values of itself at varying time lags. Thus an autoregressive model would express the forecast as a function of previous values of that time series. (Glossary of Forecasting Terms). This is different from a model that has lags of the dependent variable and other independent variables.

AXIOMS OF PROBABILITY:
(OS) There are three axioms of probability: (1) Chances are always at least zero. (2) The chance that something happens is 100%. (3) If two events cannot both occur at the same time (if they are disjoint or mutually exclusive), the chance that either one occurs is the sum of the chances that each occurs. For example, consider an experiment that consists of tossing a coin once. The first axiom says that the chance that the coin lands heads, for instance, must be at least zero. The second axiom says that the chance that the coin either lands heads or lands tails or lands on its edge or doesn’t land at all is 100%. The third axiom says that the chance that the coin either lands heads or lands tails is the sum of the chance that the coin lands heads and the chance that the coin lands tails, because both cannot occur in the same coin toss. All other mathematical facts about probability can be derived from these three axioms. For example, it is true that the chance that an event does not occur is (100% − the chance that the event occurs). This is a consequence of the second and third axioms. [Glossary of Statistical Terms, U Cal Berkeley]

B

BACKPROPAGATION:(BP)
(NS) backpropagation is a technique used in training deep learning networks and is based on implementing gradient descent to iteratively tune the weights and biases to enhance the accuracy of a network. The algorithm calculates the error of the output in each training iteration and then propagates it back into the network, thus enabling it to minimize error in future training iterations. (Data Camp Data Science Glossary)

BAGGING:

(NS) Bagging or bootstrap averaging is a technique where multiple models are created on the subset of data, and the final predictions are determined by combining the predictions of all the models. Some of the algorithms that use bagging technique are :
Bagging meta-estimator
Random Forest

BALANCED PANEL:

(OS) a panel (longitudinal) data set in which all coss-sectional units have observations in all time periods. (A Guide to Econometrics, Petter Kennedy 6th edition, glossary)

BATCH:

(NS) The set of examples used in one training iteration. The batch size determines the number of examples in a batch. (Google ML Glossary)

BATCH NORMALIZATION:

(NS) Normalizing the input or output of the activation functions in a hidden layer. Batch normalization can provide the following benefits:
Make neural networks more stable by protecting against outlier weights.
Enable higher learning rates, which can speed training.
Reduce overfitting.

BATCH SIZE:

(NS) The number of examples in a batch. For instance, if the batch size is 100, then the model processes 100 examples per iteration.

The following are popular batch-size strategies:

Stochastic Gradient Descent (SGD), in which the batch size is 1.
full batch, in which the batch size is the number of examples in the entire training set. For instance, if the training set contains a million examples, then the batch size would be a million examples. Full batch is usually an inefficient strategy.
mini-batch in which the batch size is usually between 10 and 1000. Mini-batch is usually the most efficient strategy. (Google ML Glossary)

BAYES’ RULE:

(OS) Bayes’ rule expresses the conditional probability of the event A given the event B in terms of the conditional probability of the event B given the event A and the unconditional probability of A:
P(A|B) = P(B|A) ×P(A)/( P(B|A)×P(A) + P(B|Ac) ×P(Ac) ). (U Cal Berkeley Glossary of Statistical Terms)

In this expression, the unconditional probability of A is also called the prior probability of A, because it is the probability assigned to A prior to observing any data. Similarly, in this context, P(A|B) is called the posterior probability of A given B, because it is the probability of A updated to reflect (i.e., to condition on) the fact that B was observed to occur.

BAYESIAN NEURAL NETWORK:
(NS) A probabilistic neural network that accounts for uncertainty in weights and outputs. A standard neural network regression model typically predicts a scalar value; for example, a standard model predicts a house price of 853,000. In contrast, a Bayesian neural network predicts a distribution of values; for example, a Bayesian model predicts a house price of 853,000 with a standard deviation of 67,200.

A Bayesian neural network relies on Bayes’ Theorem to calculate uncertainties in weights and predictions. A Bayesian neural network can be useful when it is important to quantify uncertainty, such as in models related to pharmaceuticals. Bayesian neural networks can also help prevent overfitting. (Google Machine Learning Glossary)

BAYESIAN NETWORK:

(NS) A Bayesian Network is a probabilistic graph showing the relationship between random variables for an uncertain domain, where the graph nodes represent those variables, and the links between each pair of nodes (the edges) represent the conditional probability for the corresponding variables. An example of Bayesian Networks is in medical diagnoses, where researchers predict health outcomes while taking into account all the factors that may affect an outcome. (Data Camp Glossary)

BAYESIAN OPTIMIZATION:

(NS) A probabilistic regression model technique for optimizing computationally expensive objective functions by instead optimizing a surrogate that quantifies the uncertainty via a Bayesian learning technique. Since Bayesian optimization is itself very expensive, it is usually used to optimize expensive-to-evaluate tasks that have a small number of parameters, such as selecting hyperparameters. (Google Machine Learning Glossary)

BERNOULLI DISTRIBUTION:

(OS) The probability distribution of a random variable that takes on two values 0 and 1. Binary variable distribution. (A Guide to Econometrics, Petter Kennedy 6th edition, glossary)

BERNOULLI THEOREM:
(OS) Sometimes called the law of averages or the law of large numbers, it states that as the number of trials of an event, rises indefinitely, the components of BLUE (Best Linear Unbiased Estimator) is optimal in terms of its statistical properties. It achieves the minimum variance among all linear unbiased estimators.

Linear: A linear estimator is one that can be expressed as a linear combination of the observed data. In other words, the estimate is obtained by multiplying the data by a set of fixed coefficients. Linearity allows for simplicity and ease of computation.

Unbiased: An unbiased estimator is one that, on average, provides estimates that are equal to the true parameter value being estimated. It does not systematically overestimate or underestimate the true value.

Estimator: An estimator is a statistical method or formula used to estimate an unknown parameter based on available data.

The BLUE property ensures that the estimator is not only unbiased but also has the smallest possible variance among all linear unbiased estimators. This makes it an attractive choice in statistical estimation because it provides efficient and precise estimates.

The concept of BLUE is closely related to the Gauss-Markov theorem, which states that in a linear regression model with certain assumptions (such as the absence of multicollinearity and homoscedasticity), the ordinary least squares (OLS) estimator is the best linear unbiased estimator, making it the BLUE in that context.

In summary, the BLUE property is a desirable characteristic for an estimator in statistical estimation, indicating that it is both linear and unbiased while having the minimum possible variance among all linear unbiased estimators.

BIAS:

(OS) Bias is a statistical term that means a systematic deviation from the actual value. It is a sampling procedure that may show some serious problems for the researcher as a mere increase cannot reduce it in sample size. Bias is the difference between the expected value and the real value of the parameter.

In statistics, bias is a term that defines the tendency of the measurement process. It means that it evaluates the over or underestimation of the value of the population parameter. Let us consider an example: to evaluate the mean of the population. Hopefully, you might have found an estimation which is the true reflection of the population. Now, by using the unbiased estimator, it is easy to find the difference between the true value and the statistically expected value of the population parameter.

Types of Bias
The following are the different types of biases, which are listed below-

Selection Bias
Spectrum Bias
Cognitive Bias
Data-Snooping Bias
Omitted-Variable Bias
Exclusion Bias
Analytical Bias
Reporting Bias
Funding Bias
Classification of Bias

The bias is mainly categorized into two different types

Measurement Bias
Measurement bias takes place for the duration of the carrying out the survey, and its consequences chiefly because of three reasons –

(i) The error happens while recording the data

While recording data, errors happen due to the malfunction of instruments that are used for data collection or because of ineffective handling of these tools by the researchers concerned with data collection.

(ii) Leading Questions

The questions prepared for the survey might be put in a manner to lead the responses that are preferred by the researcher. There can be more choices for the preferred retort given than for the conflicting views.

(iii) Respondents gave inadvertent false responses

There can be a situation when many responders may have misunderstood the question and chosen an incorrect option. If the sample groups were composed of numerous senior citizens and if they were asked to give answers by remembering their previous experiences, they might be providing some false inputs because of a deficiency of memory.

Non-Representative Sampling Bias
Non-representative sampling bias is also referred to as selection bias. This inaccuracy occurs because of implementing random methods during the selection process. It results in an excess representation of some of the elements in the population. All the samples collected using convenience sampling are caused by bias. These type of situation are called under coverage bias.
(https://byjus.com/maths/bias/#:~:text=Bias%20is%20a%20statistical%20term,real%20value%20of%20the%20parameter., Accessed 4/21/2024)

C

CANOPY CLUSTERING:

(NS) an unsupervised pre-clustering algorithm that is often used as preprocessing step for the K-means algorithm or the Hierarchical clustering algorithm. It is intended to speed up clustering operations on large data sets, where using another algorithm directly may be impractical due to the size of the data set. (https://en.wikipedia.org/wiki/Canopy_clustering_algorithm).

CATEGORICAL VARIABLE:
(OS) A variable whose value ranges over categories, such as {red, green, blue}, {male, female}, {Arizona, California, Montana, New York}, {short, tall}, {Asian, African-American, Caucasian, Hispanic, Native American, Polynesian}, {straight, curly}, etc. Some categorical variables are ordinal (ranking and order matter). The distinction between categorical variables and qualitative variables is a bit blurry. C.f. quantitative variable

CAUSATION, CAUSAL RELATION:
(OS) Two variables are causally related if changes in the value of one cause the other to change. For example, if one heats a rigid container filled with a gas, that causes the pressure of the gas in the container to increase. Two variables can be associated without having any causal relation, and even if two variables have a causal relation, their correlation can be small or zero. (U Cal Berkeley, Glossary of Statistical Terms)

CENSORED SAMPLE (VARIABLE):

(OS) A type of sample in which some of the observations have incomplete or limited information. In a censored sample, the values of certain observations are not fully observed or are only partially observed due to some censoring mechanism.

Censoring occurs when the exact value of a variable of interest is unknown or unobservable and is replaced or “censored” with a known or limited value. This can happen for various reasons, such as:

Right-censoring: In right-censoring, the values of certain observations are only known to be above a certain threshold or have not yet occurred at the time of data collection. For example, in a study tracking the time to failure of a product, if the product has not failed by the end of the study, the failure time is right-censored. The observed data would only indicate that the failure time is greater than or equal to the length of the study.

Left-censoring: In left-censoring, the values of certain observations are only known to be below a certain threshold. This can occur when the lower limit of detection or measurement of a variable is known, but values below that limit cannot be precisely measured or observed.

Interval-censoring: In interval-censoring, the values of certain observations are known to fall within an interval but the precise value within the interval is unknown. For example, in a study measuring the blood pressure of individuals, if the measurements are only taken at specific time intervals (e.g., every month), the blood pressure values between those intervals are interval-censored.

Censoring poses challenges in statistical analysis because the complete information about the censored observations is not available. Specialized statistical techniques, such as survival analysis or censored regression models, are often used to handle and analyze censored data. These methods take into account the censoring mechanism and make inferences about the underlying distribution or relationship of the variable of interest based on the available information.

In summary, a censored sample refers to a sample in which some of the observations have incomplete or limited information due to censoring, where the exact values of certain observations are unknown or replaced with known or limited values.

CENTRAL LIMIT THEOREM:

(OS) The central limit theorem states that the probability histograms of the sample mean and sample sum of n draws with replacement from a box of labeled tickets converge to a normal curve as the sample size n grows, in the following sense: As n grows, the area of the probability histogram for any range of values approaches the area under the normal curve for the same range of values, converted to standard units. Note from the graphs below that as n, the number of draws rises the histogram takes on the shape of a normal distribution n=4 & n=5 on the left. The bottom left side graph shows that as the number of draws grows until it converges with the black line.

See also the normal approximation. (Glossary of Statistics, U Cal Berkeley. Image Central limit theorem. (2024, March 27). In Wikipedia. https://en.wikipedia.org/wiki/Central_limit_theorem)

CHATBOT: (NS, originally chatterbot) – A software application that aims to mimic human conversation through text or voice interactions, typically online. Types of chatbots:

assistant bot – Assistant bots, like support bots, need to be good at conversations and answering FAQs. They must also be entertaining to maintain user interest.
conversational bot – This type of chatbot must be able to have a conversation with a human in some way or another. Hence, all types of bots are ultimately conversational chatbots.
informational bot – Information gathering bots can act as research assistants by extracting as much information as possible either from a human or from an Internet resource like a website or an ebook.
skills chatbot – It’s a single-turn-type that doesn’t require much of contextual awareness. It can just follow a command to perform an action.
support chatbot – Designed to solve a specific problem, support chatbots require context awareness, a personality, and multi-turn capability. Most of the support chatbots use deep learning and natural language processing to perform actions.
transactional bot – This type of chatbot can be roughly classified within assistant bots as it often acts on behalf of humans to perform various transactions. For example, placing an order, making a reservation, etc. (Google Data Science Glossary)

ChatGPT (CHAT BASED GENERATIVE PRE-TRAINED TRANSFORMER):
A system built with a neural network transformer type of AI model that works well in natural language processing tasks (see definitions for neural networks and Natural Language Processing below). In this case, the model: (1) can generate responses to questions (Generative); (2) was trained in advance on a large amount of the written material available on the web (Pre-trained); (3) and can process sentences differently than other types of models (Transformer). (University of Oregon, AI Glossary)

CHI-SQUARED AUTOMATED INTERACTION DETECTION (CHAID):

(NS & OS) Chi-squared Automated Interaction Detection (CHAID) – a decision tree technique, based on adjusted significance testing (Bonferroni testing); often used in the context of direct marketing to select groups of consumers and predict how their responses to some variables affect other variables (https://en.wikipedia.org/wiki/Chi-square_automatic_interaction_detection)Chi-squared Automated Interaction Detection (CHAID) – a decision tree technique, based on adjusted significance testing (Bonferroni testing); often used in the context of direct marketing to select groups of consumers and predict how their responses to some variables affect other variables (https://en.wikipedia.org/wiki/Chi-square_automatic_interaction_detection)

CITIZEN DATA SCIENTIST:

(NS) Although the term has existed for a couple of years now, you won’t find job listings for “citizen data scientist” on Glassdoor.com. That’s because it’s not a role that an organization is going to hire for, but more like a requirement, they need to fill. People in the industry are using the term citizen, but those doing the hiring are focused on the tasks not currently getting done but require attention.

If it’s not a job title getting posted, what exactly is it? Gartner defines a citizen data scientist as “a person who creates or generates models that leverage predictive or prescriptive analytics, but whose primary job function is outside of the field of statistics and analytics.” They bridge the gap between those doing self-service analytics as business users and those doing advanced analytics as data scientists. InformationWeek states that the “defining trait is that statistics and analytics are secondary in the role.”

As a new concept, it’s not yet clearly defined, although there seems to be a consensus around the skills needed and tasks to be completed. In an article published at Forbes, Mike Guilfoyle of ARC describes the citizen data scientist as having “data science skills such as statistics, but not as advanced as a data scientist.” He describes their job responsibilities as delving into new data, improving existing models, and creating and deploying additional models to gain added insight. (https://www.simplilearn.com/citizen-data-scientists-article)

Often citizen data scientists are not part of a business but are individuals with values that support using data science to help the environment, model crime, or perform other projects based on social need and idealism. They may make contests up on sites like Kaggle.com or volunteer for non-profits, governments, or other entities to solve social, environmental, or scientific problems.

CIVIC ANALYTICS:
(NS) The application of advanced data mining, modeling, and analysis techniques to enable data-informed and evidence-based decision-making in urban and regional operations, policy, and planning.
(AI Glossary, U of Oregon)

CLASS IMBALANCED DATASET:

(OS) A dataset for a classification problem in which the total number of labels of each class differs significantly. For example, consider a binary classification dataset whose two labels are divided as follows:

1,000,000 negative labels
10 positive labels
The ratio of negative to positive labels is 100,000 to 1, so this is a class-imbalanced dataset.

In contrast, the following dataset is not class-imbalanced because the ratio of negative labels to positive labels is relatively close to 1:

517 negative labels
483 positive labels
Multi-class datasets can also be class-imbalanced. For example, the following multi-class classification dataset is also class-imbalanced because one label has far more examples than the other two:

1,000,000 labels with class “green”
200 labels with class “purple”
350 labels with class “orange” (Google Machine Learning Glossary)

NOTE: Imbalanced datasets should not be evaluated for accuracy alone as accuracy will be skewed towards 1 for the majority class.

CLASSICAL STATISTICS:

(OS) The branch of statistics that is not based on Bayesian statistics. (A Guide to Econometrics, Petter Kennedy 6th edition, glossary)

CLASSIFIER/CLASSIFICATION MODEL:

(OS & NS) A model whose prediction is a class. For example, the following are all classification models:

A model that predicts an input sentence’s language (French? Spanish? Italian?).
A model that predicts tree species (Maple? Oak? Baobab?).
A model that predicts the positive or negative class for a particular medical condition.
A model that detects fraud in a universe of transactions.

Two common types of classification models are:

binary classification
multi-class classification (Google ML Glossary)

NOTE: Techniques are numerous for classification models and include:

Logistic Regression: Logistic regression is a popular classification algorithm used when the target variable is binary (two classes) or is categorical (3 or more classes). It estimates the probabilities of the target outcome(s) based on the input features.
Naive Bayes: Naive Bayes is a probabilistic classification algorithm that applies Bayes’ theorem with the assumption of independence between features. Despite its simplicity, it is often effective and performs well on text classification tasks.
Decision Trees: Decision trees are hierarchical models that divide the feature space into partitions based on a set of rules. They are intuitive to interpret and can handle both categorical and numerical data.
Random Forest: Random Forest is an ensemble learning method that combines multiple decision trees. It constructs a large number of decision trees and combines their predictions to make a final classification decision.
Support Vector Machines (SVM): SVM is a powerful classification algorithm that finds an optimal hyperplane to separate different classes. It works well with both linearly separable and non-linearly separable data by using kernel functions.
K-Nearest Neighbors (KNN): KNN is a non-parametric classification algorithm that classifies data points based on the majority class of their k nearest neighbors in the feature space. It does not make any assumptions about the underlying data distribution.
Neural Networks: Neural networks, particularly deep learning models like multilayer perceptions (MLP), can be used for classification tasks. They consist of multiple layers of interconnected nodes (neurons) and are capable of learning complex patterns and representations from data.
Gradient Boosting Models: Gradient boosting models, such as XGBoost and LightGBM, are ensemble methods that create a strong predictive model by combining weak learners (usually decision trees) in a sequential manner.
Probit Regression: Probit regression is used for binary and multi-class dependent variables. Probit models are similar to logistic regression. It models the relationship between the independent variables and the probability of the binary or ordinal outcome(s) using a cumulative distribution function (CDF) of a standard normal distribution, known as the probit function.
Linear Probability Models (LPM) are multiple linear regression estimations with probabilities trimmed at 1 and 0.

These are just a few examples of classification models, and there are many other algorithms and variations available depending on the specific problem and data characteristics. The choice of the model depends on factors such as dataset size, complexity, interpretability requirements, and performance goals. (Poe.com AI Assistant)

CLASSIFICATION PROBABILITY THRESHOLD:

(OS)

In a binary (or multiclass) classifier model, a number between 0 and 1 that converts the raw probability of each case into a prediction of either the positive class or the negative class. (or, multiple classes). Note that the classification threshold is a value that a human chooses typically using a tradeoff curve or threshold metric such as accuracy, precision, recall, or other metric, not a value chosen by model training. Also known as a cutoff, the exact probability can be anywhere from 0 to 1.A classifier model outputs a raw value between 0 and 1. Then:

If this raw value is greater than the classification threshold, then the positive class is predicted.
If this raw value is less than the classification threshold, then the negative class is predicted.
For example, suppose the classification threshold is 0.8. If the raw value is 0.9, then the model predicts the positive class. If the raw value is 0.7, then the model predicts the negative class.
Note that classifiers can be non-binary with more than 2 classes as well.

The choice of classification threshold strongly influences the number of false positives and false negatives and other metrics using the confusion matrix.
As models or datasets evolve, engineers sometimes also change the classification threshold. When the classification threshold changes, positive class predictions can suddenly become negative classes and vice-versa.

For example, consider a binary classification disease prediction model. Suppose that when the system runs in the first year:

The raw value for a particular patient is 0.95.
The classification threshold is 0.94.
Therefore, the system diagnoses the positive class.

A year later, perhaps the values now look as follows:

The raw value for the same patient remains at 0.95.
The classification threshold changes to 0.97.a
Therefore, the system now reclassifies that patient as the negative class. Same patient. Different diagnosis. (google Machine Learning Glossary)

There are several dozen metrics that can be used such as F1 Score, AUC of a ROC curve, False negative rate, False positive rate, sensitivity (True positive rate) vs, specificity (True Negative Rate), Score Separation Graph (% negative vs % positive for X-axis of scores), CZ, J, Euclidian Distance, Phi, Folkes-Mallows Index, cSI thread score, log odds of outcome, accuracy, total accuracy, and dozens of others that vary by field or model type/objective

CLASSIFICATION AND REGRESSION TREE (CART)

(OS) It is a machine-learning algorithm that is used for both classification and regression tasks. CART builds binary trees based on training data and constructs decision rules in the form of a tree structure.

When used for classification, CART creates a decision tree where each internal node represents a feature or attribute, each branch represents a decision based on that attribute, and each leaf node represents a class label. The decision rules in the tree are formed by recursively partitioning the data based on the values of the input features until a stopping criterion is met, such as reaching a maximum tree depth or a minimum number of samples per leaf node. The class label associated with the majority of the samples in a leaf node becomes the predicted class for new instances.

In the case of regression, CART constructs a decision tree where the leaf nodes contain the predicted continuous values rather than class labels. The tree is built in a similar manner, recursively partitioning the data based on feature values to minimize the sum of squared errors between the predicted and actual values in each leaf node.

CART is a popular and widely used algorithm due to its simplicity, interpretability, and ability to handle both categorical and continuous input features. It is often used as a foundation for more advanced ensemble methods such as random forests and gradient boosting. (AI Poe.com Assistant)

CLIPPING :

In the context of data analysis or signal processing, clipping refers to the process of limiting or restricting the values of a variable or signal to a specific range or threshold.

Clipping is often used to handle extreme or outlier values that might negatively affect the analysis or processing of the data. By setting a maximum or minimum threshold, values above or below that threshold are “clipped” or truncated to the threshold value.

For example, if we have a dataset of temperature readings and we know that any values above 100 degrees Celsius are erroneous or beyond the physical limit, we can clip the data by setting a maximum threshold of 100 degrees Celsius. Any temperature readings above 100 degrees will be replaced with the maximum threshold value of 100. (Poe.Com, Data Science GPT3 AI)

Clipping can be useful in various scenarios, such as removing noise or outliers, ensuring data stays within a specified range, or preventing extreme values from affecting subsequent calculations or analysis. However, it’s important to consider the potential impact of clipping on the integrity and representativeness of the data.

CLOUD:

The term “cloud” typically refers to the concept of storing and accessing data and applications over the internet instead of locally on a physical device or a local server. In simple terms, the cloud refers to a network of remote servers hosted on the internet that store and manage data and perform various computing tasks.

Cloud computing allows individuals and businesses to access a wide range of services and resources on-demand, such as storage, processing power, software applications, and databases, without the need for local infrastructure or hardware. These resources are usually provided by cloud service providers, who maintain and manage the underlying infrastructure.

One of the primary benefits of cloud computing is that it offers scalability and flexibility. Users can easily scale up or down their resources based on their needs, paying only for the resources they use. Additionally, the cloud enables collaboration and remote access to data and applications, making it easier to work and share information across different devices and locations.

There are different types of cloud services, including:

Infrastructure as a Service (IaaS): Provides virtualized computing resources such as virtual machines, storage, and networks. Users have more control over the infrastructure and can install and manage their own operating systems and applications.

Platform as a Service (PaaS): Offers a platform with tools and services for developing, testing, and deploying applications. Users can focus on application development without worrying about the underlying infrastructure.

Software as a Service (SaaS): Delivers software applications over the internet on a subscription basis. Users can access and use the software through a web browser without the need for installation or maintenance.

CLOUD COMPUTING:
(NS) Cloud computing has become increasingly popular due to its convenience, cost-effectiveness, and scalability, and it is used in various industries and applications, including data storage, web hosting, software development, artificial intelligence, and more. (Poe.com AI Assistant)

COHORT DATA:
(OS) Cohort data refers to a type of data that is organized and analyzed based on specific groups or cohorts. A cohort is a group of individuals who share a common characteristic or experience during a specific period of time.

In the context of data analysis, cohort data is often used to study and analyze the behavior, characteristics, or outcomes of a group of individuals over time. Cohort analysis allows for the examination of trends, patterns, and comparisons within and between different cohorts.

Cohort data can be collected in various fields and industries. For example, in healthcare, cohort data may be used to study the long-term health outcomes of a group of patients who received a particular treatment. In marketing, cohort data can be used to analyze the purchasing behavior of customers who joined a loyalty program during a specific period.

When working with cohort data, the key aspect is the ability to track and analyze the data over time for each cohort. This enables researchers or analysts to identify trends, make comparisons, and draw conclusions about the impact of specific factors or interventions on the cohort’s behavior or outcomes.

Common analyses performed on cohort data include calculating retention rates, studying conversion rates, assessing customer lifetime value, and conducting survival analysis. Cohort data analysis can provide valuable insights into the dynamics and characteristics of specific groups, enabling businesses, organizations, and researchers to make informed decisions and develop targeted strategies. (Poe.com AI Assistant)

COINTEGRATION:

(OS) Cointegration is a statistical concept that measures the long-term relationship between two or more time series variables. In simple terms, it determines whether a linear combination of these variables is stationary over time. Stationarity refers to a property where the statistical properties of a time series, such as mean and variance, remain constant over time.

Cointegration is particularly relevant when analyzing financial and economic data, as many economic variables are non-stationary and tend to exhibit trends or drifts. By identifying cointegration, we can determine whether two or more non-stationary variables move together in the long run despite exhibiting short-term deviations from their long-term relationship.

When two variables are cointegrated, it implies that there is a stable equilibrium or a long-term relationship between them. This means that any temporary divergence from the long-term relationship is expected to be corrected in the future. Cointegration is often associated with pairs trading or statistical arbitrage strategies, where investors exploit deviations from the long-term relationship between two assets to generate trading opportunities.

Cointegration analysis typically involves using statistical tests, such as the Engle-Granger test or the Johansen test, to determine the presence of cointegration between variables. These tests examine the residuals or the differences between the variables and assess whether they are stationary. If the residuals are stationary, it suggests the presence of cointegration.

Overall, cointegration provides a useful framework for analyzing the long-term relationship between variables, allowing researchers and analysts to better understand the dynamics and interactions among different time series. (Poe AI Assistant)

COMPUTER VISION: Computer vision is an area of computer science concerned with enabling computers to achieve high-level understanding from digital images or videos, near to how humans can see them. Computer vision
became especially popular with the evolution of deep learning and the accumulation of big data. Some of its applications are object and facial recognition, motion analysis, self-driving cars, and optical character recognition. [Data Camp Glossary]

CONDITION INDEX:

(OS) Used to diagnose muticollinearity, the Condition index is a measure of how close the X’X matrix (product moment matric) is to perfect multicollinearity where pearson correlation =1. This would not yield a proper inverse of X’X^-1. As a rule of thumb a condition index over 30 indicates strong collinearity. (Peter Kennedy , A guide to Econometrics 6th ed, p. 199)

CONFIDENCE INTERVAL:

(OS) A confidence interval is a statistical range that provides an estimate of the uncertainty associated with a population parameter. It is commonly used in inferential statistics to quantify the level of confidence we have in our estimate. When we collect a sample from a population, we use the sample data to estimate a population parameter, such as the mean or proportion. However, due to sampling variability, our estimate is unlikely to be exactly equal to the true population parameter. A confidence interval provides a range of values within which we believe the true population parameter is likely to fall. The interval is constructed based on the sample data and a chosen level of confidence, typically expressed as a percentage (e.g., 95% confidence interval). For example, if we calculate a 95% confidence interval for the mean weight of a population, we might find that the interval ranges from 50 kg to 60 kg. This means that we are 95% confident that the true population mean weight falls within this range. The level of confidence chosen reflects the trade-off between precision and certainty. A higher confidence level, such as 99%, will result in a wider confidence interval, providing a more conservative estimate. Conversely, a lower confidence level, such as 90%, will yield a narrower interval but with less certainty. It’s important to note that a confidence interval only provides a range of plausible values for the population parameter. It does not guarantee that the true parameter falls within the interval, nor does it provide information about the distribution of sample data.

Overall, a confidence interval is a statistical tool that helps us estimate the range within which a population parameter is likely to lie, taking into account the uncertainty inherent in sampling. (Poe.com Data-Scientist GPT-3 AI)

CONJOINT ANALYSIS: (OS)

Conjoint analysis is a statistical technique used in market research to understand how customers value different components or features of a product or service. It breaks down a product into a set of attributes and analyzes how these attributes impact users’ perceived value of the item or service.

The findings from a conjoint analysis can be interpreted in several ways:

Importance of attributes: Conjoint analysis provides insights into the importance of different attributes by assigning them importance scores. These scores indicate the relative impact of each attribute on the decision-making process. Higher scores indicate greater importance.
Preference for attribute levels: Conjoint analysis helps identify which levels of each attribute are preferred by individuals. By comparing the utilities or part-worth values assigned to different attribute levels, we can determine which combinations are more desirable.
Trade-offs and preference patterns: Conjoint analysis allows us to understand the trade-offs individuals are willing to make between different attribute levels. By examining the relative utilities of attribute levels, we can identify the preferred trade-offs and uncover preference patterns.
Market segmentation: Conjoint analysis can be used to segment the market based on individual preferences. Cluster analysis or latent class analysis can be applied to group individuals with similar preference patterns, thereby identifying distinct market segments.
Predictive modeling: The results of conjoint analysis can be used to build predictive models that estimate the likelihood of choice for different product configurations. These models can help simulate market scenarios and optimize product designs or pricing strategies.

Overall, the interpretation of conjoint analysis findings involves understanding attribute importance, preference patterns, trade-offs, market segmentation, and utilizing the insights to make informed marketing decisions.

CONFUSION MATRIX: A confusion matrix is a table that is commonly used to evaluate the performance of a classification model. It provides a summary of the predictions made by the model and how they compare to the actual values or labels of the data. The confusion matrix is typically organized into rows and
columns, with each row representing the predicted class or label of the data, and each column representing the observed outcome class or label. The number of instances falling into each combination of actual and predicted classes is recorded in the cells of the matrix. Then, certain metrics are made to describe model accuracy and True/False Positive Rates, True/False Negative Rates, etc. These metrics can be fine-tuned to fit the problem by varying the cutoff threshold (event vs. non-event probability that triggers a predicted event.

NOTE: In the matrix above, four key metrics and their sources in the matrix are shown.

NOTE: In the matrix above, score thresholds are altered until there is a low FPR as FNR is secondary in the problem as each FPR has a cost. However, we would be happier with a higher sensitivity/recall and precision. By changing cut-offs and using other models confusion matrices are regenerated and can be all compared to find optimal models and cut-offs. In practice, this matrix would be compared to other cut-offs of the same model, along with cut-offs from other competing techniques run on the problem. See this blog entry for further details about setting cutoffs to optimize metrics for each unique problem rather than use the default probability of 0.5..

CONSENSUS MODEL: A consensus model combines predictions from multiple individual models to make a final prediction. This approach often leads to more accurate results by leveraging the strengths of different models. It is often used in macroeconomics and stock-portfolio (e.g. Earnings) analysis and in stock picking. To build a consensus model, we first need to train multiple models on the same dataset. These models can be of different types, such as decision trees, random forests, support vector machines, or neural networks. Each model will make predictions on new data. (Poe.com Data Scientist GPT3 AI)

CONTENT ANALYSIS: (OS): a method of evaluating textual (typically semi-structured data) with the objective of making inferences about specific words and meanings.

Content analysis is a research methodology used in various fields, including social sciences, communication studies, and marketing. It involves systematically analyzing and interpreting the content of different types of textual, visual, or audiovisual materials. The goal of content analysis is to identify patterns, themes, and relationships within the content to gain insights or draw conclusions. Content analysis can be both quantitative and qualitative in nature. In quantitative content analysis, researchers apply coding schemes or predefined categories to analyze large amounts of data. They count and measure the occurrence of specific words, phrases, or concepts to identify frequencies, trends, or associations. This approach allows for statistical analysis and generalizations. On the other hand, qualitative content analysis focuses on understanding the underlying meanings, contexts, and interpretations of the content. Researchers use coding schemes that emerge from the data itself rather than predefined categories. They analyze the content in a more interpretive and subjective manner, looking for themes, patterns, or nuanced insights.

Content analysis can be applied to various types of content, such as written documents, social media posts, interviews, news articles, advertisements, or audiovisual materials. It can be used to study public opinion, media representations, social interactions, organizational communication, or the effectiveness of communication campaigns, among many other research areas. To conduct a content analysis, researchers typically follow a systematic process that includes selecting the content to be analyzed, developing coding schemes or categories, training coders, analyzing the content, and interpreting the findings. The process may involve manual coding or the use of software tools for automated coding and analysis.

Content analysis provides a systematic and objective approach to studying textual and visual content, allowing researchers to uncover patterns, trends, and insights that can inform decision-making, policy development, or further research in a particular field. (Poe.com Data-Scientist GPT3 AI)

(NS) Content analysis has been supplanted by techniques such as sentiment analysis, image recognition and analysis, network analysis, topic modeling, and social network analysis (SNA). These alternatives can handle larger corpora (textual bodies or word lists) and make use of NS techniques such as natural language processing (NLP), Latent Dirichlet Allocation (LDA) algorithms, and non-negative Matrix Factorization (NMF).

CONVERGENCE:

OS) A mathematical and statistical term that signifies the process of iteratively reaching a stable or optimal solution. It occurs when a sequence of values or solutions progressively gets closer to a desired or optimal outcome.

In optimization algorithms, convergence is typically measured by monitoring the change in objective function values or the parameters being optimized. The algorithm continues to update and refine the solution until the desired level of convergence is achieved. This level can be defined based on a predefined tolerance or a stopping criterion.

Convergence is important because it indicates that the optimization process is approaching a satisfactory solution. It implies that further iterations are unlikely to significantly improve the solution or lead to substantial changes. Achieving convergence is crucial to ensure the efficiency and effectiveness of optimization algorithms.

Convergence can have different characteristics depending on the specific problem and algorithm being used. For example, in gradient descent optimization, convergence is often observed when the gradient of the objective function approaches zero, indicating that the algorithm is approaching a local minimum. In clustering algorithms, convergence occurs when the assignment of data points to clusters stabilizes and does not change significantly.

It’s worth noting that convergence does not guarantee reaching the global optimum in all cases, particularly for non-convex optimization problems. In such cases, the algorithm may converge to a local optimum instead. Therefore, careful consideration of the problem and algorithm design is necessary to ensure that convergence leads to a desirable solution.

Overall, convergence is a fundamental concept in data science and optimization, indicating the progress toward a stable or optimal solution. It allows data scientists to assess the effectiveness and reliability of their algorithms and make informed decisions based on the achieved level of convergence. (Poe.com Data-Science GPT3 AI).

Hyperparameters or default settings in models and algorithms often control whether convergence occurs. Things like tolerance settings (precision), number of iterations, mathematical method, learning rate, regularization parameters, batch size, and initial value settings can all affect whether a model or algorithm converges at all or results in a global or local solution. Hyperparameter tuning should be used to optimize and check results, especially in unstable or unbalanced solutions.

CONVERSATIONAL USER INTERFACE (CUI):
(NS) (Also conversational UI). A conversational user interface is what allows computers to mimic conversations with real humans. These interfaces use NLP (Natural Language Process) to interpret incoming voice or text and reply with a response. The two primary types of CUIs are voice assistants (like Siri and Alexa) and chatbots. (U of Oregon AI glossary)

CORPUS:
(NS) A large dataset of written or spoken material that can be used to train a machine to perform linguistic tasks.

CORRELATION:

(OS) A measure of linear association between two (ordered) lists. Two variables can be strongly correlated without having any causal relationship, and two variables can have a causal relationship and yet be uncorrelated.(Berkelely Statistics Glossary)

CORRELATION COEFFICIENT (PEARSON):

(OS) A standardized measure of the linear association or mutual dependence between two variables, say X and Y . Commonly designated as r, its values range from -1 to +1, indicating strong negative relationship, through zero, to strong positive association.
The correlation coefficient is the covariance between a pair of standardized variables. (Glossary of Forecasting Terms) To calculate the Pearson correlation, you need to have paired observations of two continuous variables. The Pearson correlation assesses the linear association by computing the covariance of the variables and dividing it by the product of their standard deviations. (Poe.com Assistant AI)

CORRELATION COEFFICIENT (SPEARMAN RANK):

(OS) The Spearman correlation coefficient, denoted as (rho), assesses the monotonic relationship between two variables. It is suitable for both continuous and ordinal variables. Unlike the Pearson correlation, the Spearman correlation does not assume a linear relationship.
Instead, it evaluates whether the variables tend to change together in a consistent manner, regardless of the specific functional form of the relationship. The Spearman correlation ranges from -1 to 1, where -1 indicates a perfect negative monotonic relationship, 1 indicates a perfect positive monotonic relationship, and 0 suggests no monotonic relationship. The Spearman correlation, on the other hand, does not require the data to be continuous. It operates on the ranks of the observations rather than the raw values. It converts the data into ranks and then calculates the Pearson correlation on the ranks. This makes the Spearman correlation suitable for variables with non-linear relationships or variables with outliers. (Poe.com Assistant AI)

CORRELATION MATRIX (PEARSON):

(OS) A matrix of Pearson correlation coefficients designed to showcase the linear dependencies between members of a set of variables.

Variable	Age	BMI	DiabetesPedigreeFunction	Glucose	Insulin	KNOWN_FEMALES	PATIENT_NUMBER	GLUCOSE_INSULIN_RATIO	Outcome
Age	1.000	0.004	0.089	0.342	0.218	0.241	-0.068	-0.095	0.332
BMI	0.004	1.000	0.166	0.187	0.194	-0.312	0.020	-0.016	0.247
DiabetesPedigreeFunction	0.089	0.166	1.000	0.108	0.056	-0.063	-0.042	0.144	0.228
Glucose	0.342	0.187	0.108	1.000	0.541	-0.087	-0.008	-0.107	0.529
Insulin	0.218	0.194	0.056	0.541	1.000	-0.029	-0.020	-0.495	0.257
KNOWN_FEMALES	0.241	-0.312	-0.063	-0.087	-0.029	1.000	-0.022	-0.078	-0.059
PATIENT_NUMBER	-0.068	0.020	-0.042	-0.008	-0.020	-0.022	1.000	-0.060	-0.157
GLUCOSE_INSULIN_RATIO	-0.095	-0.016	0.144	-0.107	-0.495	-0.078	-0.060	1.000	-0.043
Outcome	0.332	0.247	0.228	0.529	0.257	-0.059	-0.157	-0.043	1.000

From Pima Nation Diabetes Data

COVARIANCE:

(OS) This is a measure of the joint variation between variables, say X and Y. The range of covariance values is unrestricted (large negative to large positive). However, if the X and Y variables are first standardized, then covariance is the same as the Pearson correlation and in that case, the range of covariance (correlation) values is from -1 to +1.

CROSS-SECTIONAL DATA / POOLED CROSS-SECTIONAL DATA:

(OS) Cross-sectional data refers to a type of data that is collected at a specific point in time, capturing information from different individuals, entities, or subjects. In cross-sectional data, each observation represents a distinct unit, and the data is collected simultaneously or at the same time point for all units. For example, a survey conducted to collect information about the income, age, and education level of individuals at a specific moment would be considered cross-sectional data.

On the other hand, pooled cross-sectional data refers to a type of data that combines cross-sectional observations from multiple time periods. In pooled cross-sectional data, data is collected at different time points, but each observation still represents a distinct unit. For example, if a survey collects income, age, and education level data from different individuals in different years, and all the data is combined into a single dataset, it would be considered pooled cross-sectional data.

The key difference between cross-sectional data and pooled cross-sectional data lies in the time frame of data collection. Cross-sectional data is collected at a specific moment in time, whereas pooled cross-sectional data combines observations from multiple time periods. Pooled cross-sectional data allows for the examination of changes or trends over time, as it captures information from different points in time for the same units. In contrast, cross-sectional data provides a snapshot of information at a particular time point, without accounting for changes over time. (poe.com Data-Scientist-GPT3)

CROSS-VALIDATION: Cross-validation is a resampling method when training machine learning models that splits labeled data into training and test sets. In each iteration of cross-validation, different parts of the data are used to train and test the model. The training set is used to train a model, and the test set is used to make predictions and compare them with the actual labels for those entries. Afterward, an overall accuracy metric is calculated to estimate the predictive performance of the
resulting model. [Data Camp Glossary]

CUMULATIVE DENSITY FUNCTION / CUMULATIVE PROBABILITY DISTRIBUTION FUNCTION (CDF):

(OS) The cumulative distribution function of a random variable is the chance that the random variable is less than or equal to x, as a function of x. In symbols, if F is the cdf of the random variable X, then F(x) = P( X ≤ x). The cumulative distribution function must tend to zero as x approaches minus infinity, and must tend to unity as x approaches infinity. It is a positive function, and increases monotonically: if y > x, then F(y) ≥ F(x). The cumulative distribution function completely characterizes the probability distribution of a random variable. (U Cal Berkeley Statistics Glossary)

CURSE OF DIMENSIONALITY:

(NS) The curse of dimensionality refers to the challenges and issues that arise when working with high-dimensional data. As the number of features or dimensions in a dataset increases, the volume of the data space grows exponentially. This growth in data space can lead to a range of problems that impact the performance and efficiency of machine learning algorithms. Here are some key points to understand about the curse of dimensionality:

Increased Sparsity: In high-dimensional spaces, data points tend to become more sparse, meaning that the data points are farther apart from each other. This can make it challenging for algorithms to generalize well from the training data to unseen data points.
Computational Complexity: As the dimensionality of the data increases, the computational requirements of algorithms also grow significantly. Many machine learning algorithms struggle to handle high-dimensional data efficiently, leading to longer training times and increased computational resources.
Overfitting: High-dimensional data increases the risk of overfitting, where a model learns noise or irrelevant patterns in the data that do not generalize well to new data. This is especially problematic when the number of features is comparable to or greater than the number of observations.
Curse of Dimensionality in Distance Metrics: Distance-based algorithms, such as k-nearest neighbors, can be severely affected by the curse of dimensionality. In high-dimensional spaces, the notion of distance becomes less meaningful, as all data points tend to be far apart from each other.
Feature Selection and Dimensionality Reduction: To combat the curse of dimensionality, feature selection and dimensionality reduction techniques are often employed to reduce the number of features while retaining relevant information. Techniques like Principal Component Analysis (PCA) and t-SNE can help in reducing dimensionality and improving model performance.

Overall, the curse of dimensionality highlights the challenges that arise when working with high-dimensional data, emphasizing the importance of feature selection, dimensionality reduction, and careful model design to mitigate these challenges and build effective machine learning models. (Poe.com, Data-Science-GPT3)

CYCLIC COMPONENT:

(OS) In a time series, the series is often broken into trend, seasonality, and cycle. The cyclical component of a time series represents fluctuations that are not of fixed frequency like seasonality but occur over a more extended period. Cycles are typically longer-term patterns that do not have a fixed duration, unlike seasonality. Economic cycles, business cycles, and other long-term trends fall under the cyclical component. Identifying cyclical patterns can help in understanding broader trends and making informed decisions.

D

DAMPED TREND EXPONENTIAL SMOOTHING:

(OS) A form of exponential smoothing in which the trend is damped by the volatility of the data; more volatility leads to greater dampening. Glossary, (Hans Lenenback & James Cleary, Forecasting: Practice and Process for Demand Management.)

DASHBOARD:

(NS) A dashboard is a summary of metrics that is used to communicate data trends, business indicators, and KPI (Key Performance Metrics) is an easily understandable manner to aid in executive tracking of models, data, and metrics.

DATA:

(OS) The term “data” does not have one clear definition and can be interpreted differently depending on the context, such as a researcher’s field of study. Examples of data are tables of numbers, transcripts of interviews, survey results, images, video or audio recordings, and genomic data. The NIH Data Management and Sharing Policy defines data as “The recorded factual material commonly accepted in the scientific community as of sufficient quality to validate and replicate research findings, regardless of whether the data are used to support scholarly publications.” While a blood sample or observations in a lab notebook may be considered data, the NIH Policy specifically excludes these, highlighting the caveat that depending on the context, the specific definition of scientific data may differ. (National Library of Medicine Data Glossary)

DATABASE:

(OS) A database is a structured, organized collection of data stored and accessed electronically, typically in a database management system, such as MySQL or Microsoft Access. Databases can contain any type of data, including patient records, scientific observations, transcripts, maps, or historical records. They are structured so they can be easily expanded or updated to include new data, as well as to facilitate searching for and retrieving data.

There are different types of database structures (e.g., object-oriented, hierarchical), but the most common one is a relational database (e.g., MySQL, Oracle). In a relational database, data is stored in different tables (e.g., books, publishers, authors) that are linked to represent the types of relationships between the tables (e.g., a book can have many authors, a book can have only one publisher, an author can have many books). These tables and the relationships between them represent a data model. Retrieval from these databases is typically done using structured query language (SQL) for communicating within the system or an Application Program Interface (API) for communicating from another system. Another increasingly popular type of database is a NoSQL database (e.g., MongoDB) which does not store data in relational tables.

While databases can be created and accessed on individual computers, they are typically stored on network servers so users can access them remotely via the Internet.

DATA CLEANSING:

(OS) The process of preparing data for use in modeling or analytics.

Data cleansing, also known as data cleaning or data scrubbing, is a crucial step in the data preparation process that involves identifying and correcting errors, inconsistencies, and missing values in a dataset to ensure its accuracy and reliability for analysis. The main goal of data cleansing is to improve the quality of the data by making it consistent, complete, and usable for further analysis. Here are the main steps involved in data cleansing:

Identifying Data Quality Issues: The first step in data cleansing is to identify data quality issues such as missing values, duplicate records, incorrect data types, outliers, and inconsistencies in the dataset. This step involves thorough exploration and understanding of the data to pinpoint areas that need to be cleaned.
Handling Missing Values: Missing values are a common issue in datasets and can impact the analysis results. One approach to handling missing values is to impute them with a reasonable value based on the context of the data. This could involve using statistical measures like mean, median, or mode to fill in missing values.
Removing Duplicates: Duplicate records in a dataset can skew the analysis results and lead to inaccurate conclusions. Identifying and removing duplicate records is essential to ensure the integrity of the data. This step involves identifying duplicate entries based on key attributes and keeping only unique records.
Standardizing Data: Standardizing data involves converting data into a consistent format or structure to make it uniform across the dataset. This step may include converting data types, normalizing text values, and ensuring consistent date formats for better analysis.
Handling Outliers: Outliers are data points that significantly deviate from the rest of the data and can distort analysis results. Identifying and handling outliers by either removing them or transforming them can help improve the accuracy of the analysis.
Correcting Inconsistencies: Inconsistencies in data can arise from human error, data entry mistakes, or different data sources. Correcting inconsistencies involves resolving discrepancies in data values, ensuring consistency in naming conventions, and aligning data across different sources.
Validation and Quality Checks: After cleaning the data, it is essential to perform validation and quality checks to ensure that the data is accurate, complete, and consistent. Running validation checks, cross-validating data, and conducting quality assurance tests can help verify the integrity of the cleaned dataset.

By following these main steps in the data cleansing process, data scientists and analysts can ensure that the dataset is accurate, reliable, and ready for further analysis and modeling. Data cleansing is a fundamental step in the data preparation pipeline that lays the foundation for deriving meaningful insights and making informed decisions based on high-quality data.

Data cleaning tasks can include converting dates from one format to another, removing unwanted text, splitting multiple data points in a cell into separate cells, or coding missing or NA values. (National Library of Medicine Data Glossary & poe.com Data-Science-GPT3)

A complete record of pre and post-cleansed data should be made so as to identify the frequency, magnitude, and importance of the data cleansing process on the data itself so that the cleansing’s impact on the business problem is clear. Some kinds of models require less data cleansing as outliers, illegal values, missing data, and other features may not be important in specific models. It may be desirable to run the model with lightly cleansed or uncleansed data first and then with fully cleansed data to determine the impact played by cleansing in a specific problem. The mere act of data cleansing can cause bias and inaccuracies to occur.

(NS) Often, a data engineer cleanses the data rather than the modeler or analyst. This provides specialization to the process but can cause problems when no record is kept of just what was done.

DATA CUBE:

(OS/NS) A data cube is a multidimensional representation of data that allows for the exploration and analysis of information from multiple perspectives. It extends the concept of a two-dimensional table (like a spreadsheet) to higher dimensions, enabling users to visualize and analyze data in multiple dimensions simultaneously.

In a data cube, data is organized along dimensions, which represent different attributes or variables. These dimensions can be categorical (e.g., product category, time period) or numerical (e.g., sales revenue, quantity sold). The intersection of these dimensions forms cells in the data cube, each containing a specific value or measure.

The main advantages of using a data cube in data science include:

Multidimensional Analysis: Data cubes allow for multidimensional analysis of data, enabling users to analyze information along multiple dimensions at once. This provides a more comprehensive view of the data and helps uncover patterns and insights that may not be apparent in traditional two-dimensional data structures.
Aggregation and Summarization: Data cubes facilitate aggregation and summarization of data along different dimensions. Users can easily drill down or roll up data to different levels of granularity, making it easier to analyze trends and patterns in the data.
Efficient Querying: Data cubes can significantly improve query performance by precomputing and storing aggregated data along different dimensions. This speeds up data retrieval and analysis, especially for complex queries involving multiple dimensions.
Interactive Exploration: Data cubes support interactive exploration of data, allowing users to dynamically slice, dice, and pivot data along different dimensions. This interactive approach to data analysis enhances data exploration and visualization.
Decision Support: Data cubes are valuable for decision support and business intelligence applications, as they provide a structured and intuitive way to analyze and interpret data. They are commonly used in areas such as sales analysis, financial reporting, and data mining.

Overall, data cubes play a crucial role in data science by enabling analysts and data scientists to analyze data from multiple dimensions, uncover insights, and make informed decisions based on a comprehensive view of the data. They offer a powerful and flexible framework for exploring and analyzing complex datasets, making them a valuable tool in the data science toolkit. (Poe.com, Data-Sicentist-GPT3)

DATA DICTIONARY:

(NS) A data dictionary is a file that describes each element of your dataset. If your dataset includes tabular (spreadsheet) data, the data dictionary would include a list of the fields in the table and what they mean, including units and precision.

If your data included R or Python code or scripts, the dictionary would provide a brief overview of the purpose of the code (if not already contained in comments); and information about how the code relates to the dataset. [From Smithsonian Data Management Best Practices. Describing Your Data: Data Dictionaries (pdf)].

Data dictionaries have several benefits:

Keeping things consistent across a project. The dictionary can define data names, labels, units, constraints such as acceptable range of values, and other characteristics.
Enabling software to process a data file, by providing details to the software about the file. This information might include the type of data in each column (integer, character, date, etc); the name of the column; the physical units, if relevant; whether nulls are included; etc.
Increasing interoperability and reuse of the data that you want to share and publish.
Providing “human-readable” details to support discovery, interpretation and analysis.

DATA LAKE:

(NS) A data lake is a storage repository that holds vast amounts of raw data in its native format, so this data can be structured, unstructured, or semi-structured. This is in contrast to a data warehouse, in which data is structured in a common data model.‍ While this allows the flexibility of ingesting different types of data into the data lake in different formats, this lack of structure can lead to a “data swamp” if not carefully managed. Since data lakes store raw data, analysis is flexible and can occur at a granular level to ad-hoc queries. At its core, a data lake is a data storage and processing repository in which all of the data in an organization can be placed and data flows into it and insights flow out. Cloud computing can be used for data lakes, or they can be stored locally, with Hadoop being a popular platform for local storage.(Mational Library of Medicine Data Glossary)

DATA MINING:

(NS) Data mining is the process of identifying patterns and relationships in large datasets and extracting this information. This is accomplished with statistics and/or machine learning techniques. Data mining differs from data analysis in that it is approached without a hypothesis. Data mining often involves the automated collection of large quantities of data to “extract” previously unknown or interesting patterns in data.
(OS) Data Mining had a negative connotation since it referred to the case where overfitting had occurred due to excess training of the data to the point where the data was almost all explained and irrelevant or third and second-order small impact variables. (NNLM Data Glossary)

DATA SCIENCE:

(NS)

Data Science as a term dates back to 2004, but as the chart showing “data science” searches and “econometrics” searches shows data science as a term has seen huge search gains since 2016 to where it dominates econometrics, the opposite of 2004-2013.

There are several classes of definitions: AI-dominant, statistics-dominant, Data Science-dominant, and task-based. Here are several examples:

1. Data science is a multifaceted interdisciplinary field of study that uses various scientific methods, advanced analytics techniques, and predictive modeling algorithms to extract meaningful insights from data, to help answer strategic business or scientific questions in many spheres. It combines a wide range of technical and non-technical skills and usually requires solid domain knowledge in the particular industry where it is applied, to be able to correctly interpret the available data and the obtained results.(Data Camp Data Glossary)

2. There are several definitions, typically based on industry, “science paradigm”, or audit, legal, or regulatory requirements. Data science typically includes such asks as data extraction, data taming, data mining, visualization, modeling, forecasting, coding, back-testing, and statistical analysis of analytics solutions to business problems. In addition to statistical and mathematical models, data science includes areas such as machine learning, artificial intelligence, and adaptive learning.

Data science is a relatively new term (2001) that did not gain popular traction for over a decade. Many business data science models are from paradigms (scientific world views) such as engineering or computer science. Those two paradigms typically focus on problems concerning inanimate objects such as computer networks or engineering components. However, many other fields make business-centered contributions and were the major players in the applied data science field before the term was coined.

A new data science can compete with or synergize paradigms (often based on human behavior) such as statistics, predictive analytics, business econometrics, diverse industry practice, and many other social and “hard” sciences. Often, a distinction is made between data scientists accustomed to modeling human behavior vs. the Internet of Things (IoT) or robot “behavior”, adaptive learning, and AI (Artificial Intelligence).

At Synergy Data Science we blend techniques from many industries and paradigms until we have assembled the best mix for a customer’s needs. We obtain superior solutions by exploiting interactions among several client-specific Big Data science approaches. (Synergy Data Science What is Data Science?)

3. Data Science is an interdisciplinary field that uses statistics, computer science, programming, and domain knowledge to collect, process, and analyze data for the purpose of acquiring knowledge or solving a problem. Data science also includes sharing acquired knowledge through storytelling, visualization, and other means of communication. Data science often employs methods such as machine learning, AI, natural language processing, algorithms, and other analytic tools to process and understand data.

Examples
An example of the use of data science would be creating a machine learning model that uses data from large amounts of Electronic Health Records (EHRs) to predict if patients are at a higher risk for readmission after hospital discharge.

Another example would be an AI or neural network that analyzes millions of images of skin lesions and learns to predict which lesions are most likely to turn cancerous.

4. Data Science uses a variety of open-source and proprietary software (e.g, Python and R). Tools include software and programming languages to build and run data science models, tools to extract, organize and clean data, along with tools to visualize and share findings. The tools and processes used vary depending on the institution, data needs, field, and skill set.
(NNLM Data Glossary)

5. Data science is a combination of data analysis, algorithmic development, and technology in order to solve analytical problems. The main goal is the use of data to generate business value. (Analytics Vidhya Glossary of Common Data and ML Terms)

Data Science is an interdisciplinary field that uses statistics, computer science, programming, and domain knowledge to collect, process, and analyze data for the purpose of acquiring knowledge or solving a problem. Data science also includes sharing acquired knowledge through storytelling, visualization, and other means of communication. Data science often employs methods such as machine learning, AI, natural language processing, algorithms, and other analytic tools to process and understand data. See also https://synergy.science/whats-in-a-definition-ai-ml-data-science-statistics-and-analytics-and-econometrics/

DATA REDUCTION:

Data Reduction – obtain a reduced representation of the data while minimizing the loss of information content. These include methods of dimensionality reduction, numerosity reduction, and data compression.

Dimensionality reduction reduces the number of random variables or attributes under consideration. Methods include wavelet transforms, principal components analysis, attribute subset selection, and attribute creation.
Numerosity reduction methods use parametric or non-parametric models to obtain smaller representations of the original data. Parametric models store only the model parameters instead of the actual data. Examples include regression and log-linear models. Non-parametric methods include histograms, clustering, sampling, and data cube aggregation.
Data compression methods apply transformations to obtain a reduced or “compressed” representation of the original data. The data reduction is lossless if the original data can be reconstructed from the compressed data
without any loss of information; otherwise, it is lossy. (http://amzn.to/39flUbX)

DATA TRANSFORMATION:

(OS) Transformations turn lists into other lists, or variables into other variables. For example, to transform a list of temperatures in degrees Celsius into the corresponding list of temperatures in degrees Fahrenheit, you multiply each element by 9/5, and add 32 to each product. This is an example of an affine transformation: multiply by something and add something (y = ax + b is the general affine transformation of x; it’s the familiar equation of a straight line). In a linear transformation, you only multiply by something (y = ax). Affine transformations are used to put variables in standard units. In that case, you subtract the mean and divide the results by the SD. This is equivalent to multiplying by the reciprocal of the SD and adding the negative of the mean, divided by the SD, so it is an affine transformation. Affine transformations with positive multiplicative constants have a simple effect on the mean, median, mode, quartiles, and other percentiles: the new value of any of these is the old one, transformed using exactly the same formula. When the multiplicative constant is negative, the mean, median, and mode, are still transformed by the same rule, but quartiles and percentiles are reversed: the qth quantile of the transformed distribution is the transformed value of the 1−qth quantile of the original distribution (ignoring the effect of data spacing). The effect of an affine transformation on the SD, range, and IQR, is to make the new value the old value times the absolute value of the number you multiplied the first list by: what you added does not affect them. (U Cal Berkeley, Statistics Glossary)

DATA WAREHOUSE:
(OS) A data warehouse is a centralized repository for aggregating data of different types for further analysis. Data from different source databases are loaded into a data warehouse by a process called Extract Transform Load (ETL). Extracting is the process of pulling some or all of the data out of the source databases. Transforming includes various aspects of processing, including data cleaning and structuring the data to adhere to a common format. Loading the data is the process of moving the data into the warehouse. (National Library of Medicine Data Glossary)

DECISION TREES:

(OS & NS):
A decision tree is a type of supervised learning algorithm (having a pre-defined target variable) that is mostly used in classification problems. It works for both categorical and continuous input & output variables. In this technique, we split the population (or sample) into two or more homogeneous sets (or sub-populations) based on the most significant splitter/differentiator in input variables.
Types of tree model machine learning are based on the type of target variable we have. It can be of two types:

Categorical Variable Decision Tree: A decision Tree which has a categorical target (dependent) variable then it is called a categorical variable decision tree.
Continuous Variable Decision Tree: When the decision Tree has a continuous target variable then it is called a Continuous Variable Decision Tree. (Analytics Vidhya)

DEEP LEARNING:

(NS) Deep learning is a subset of machine learning algorithms based on multilayered artificial neural networks (ANN) that are largely inspired by the structure of the brain. ANN are very flexible and can learn from huge amounts of data, to deliver highly accurate outputs. They are often behind some data science and machine learning use cases such as image or sound recognition, language translation, and other advanced problems. (Data Camp, DEata Science Glossary)

DENDROGRAM:

DENSITY-BASED CLUSTER METHOD (DBSCAN):

DEPENDENT VARIABLE:

(OS) The dependent variable (usually expressed in singular form except in ANCOVA or systems of equations) is often expressed as Y, where Y = alpha + beta times X, the equation of a line. Or, in functional terms, Y=f(X). It is the LHS (Left Hand Side) of a relationship between itself and one (univariate case) or more (multivariate case) independent variables. Typically, causality is not assumed in Y=f(X).

(NS) Target variable.

DESCRIPTIVE ANALYTICS:

(OS) Descriptive analytics involves analyzing historical data to understand what has happened in the past. It focuses on summarizing and describing data to gain insights into trends, patterns, and relationships. Descriptive analytics helps in answering questions like “What happened?” and “What is the current state?” (Data-Scientist-GPT3, poe.com AI)

DETERMINISTIC RELATIONSHIP:

(OS) Deterministic Relationship: A deterministic relationship is a precise and predictable relationship between variables. In this type of relationship, the value of the dependent variable can be completely determined by the values of the independent variables. The relationship follows a fixed mathematical function or rule. For example, the relationship between the dependent variable and the independent variables is deterministic because it is defined by the coefficients in the equation. Changes in the independent variables will result in predictable changes in the dependent variable.

DIMENSIONALITY REDUCTION:

(NS) Dimensionality Reduction is the process of reducing the number of random variables under consideration by obtaining a set of principal variables. Dimension Reduction refers to the process of converting a set of data having vast dimensions into data with lesser dimensions ensuring that it conveys similar information concisely. Some of the benefits of dimensionality reduction:

It helps in data compressing and reducing the storage space required
It fastens the time required for performing same computations
It takes care of multicollinearity that improves the model performance.
It removes redundant features
Reducing the dimensions of data to 2D or 3D may allow us to plot and visualize it precisely
It is helpful in noise removal also and as result of that we can improve the performance of models

DISCRIMINANT ANALYSIS:

(OS) Discriminant analysis is a statistical technique used to analyze the differences between two or more groups based on a set of predictor variables. It has the following key characteristics:

Purpose: The main objective of discriminant analysis is to determine the variables that best discriminate between predefined groups and to classify new observations into one of these groups.
Data Requirements: Discriminant analysis requires the dependent variable to be categorical (e.g., group membership) and the independent variables to be continuous (e.g., measurements or ratings).
Types: There are two main types of discriminant analysis – two-group discriminant analysis (when the dependent variable has two categories) and multiple discriminant analysis (when the dependent variable has three or more categories).
Applications: Discriminant analysis is commonly used in various fields, such as:
Marketing research: To classify customers into different segments based on their characteristics.
Psychology: To identify the variables that best distinguish between different personality types or clinical diagnoses.
Biology: To classify species based on their physical characteristics.
Finance: To predict the likelihood of a company defaulting on its debt obligations.
Process: The key steps in conducting discriminant analysis include: (1) formulating the problem, (2) estimating the discriminant function coefficients, (3) determining the significance of the discriminant functions, (4) interpreting the results, and (5) assessing the validity of the analysis.

By analyzing the differences between the classes and maximizing the separation between them, discriminant analysis helps in making informed decisions and predictions based on the available data. [Poe.com Data-Scientist-GPT3]

DISTRIBUTION:

The distribution of a set of numerical data is how their values are distributed over the real numbers. It is completely characterized by the empirical distribution function. Similarly, the probability distribution of a random variable is completely characterized by its probability distribution function. Sometimes the word “distribution” is used as a synonym for the empirical distribution function or the probability distribution function. If two or more random variables are defined for the same experiment, they have a joint probability distribution. [U Cal Berkeley Glossary of Statistics]

The empirical distribution is illustrated below for a particular set of diabetes prediction data, labeled P_1.

In the diagram, we see bars that denote what the diabetes probability for a screening patient, with the tallest bar being the 0 to 0.1 probability scores for over 20% of screened patients.

The normal distribution is but one of many

Other distributions include

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

DISTURBANCE

DOUBLE EXPONENTIAL SMOOTHING

DURBIN-WATSON TEST STATISTIC:

(OS) The Durbin-Watson (DW) statistic, named after its creators, tests the hypothesis that there is no autocorrelation of one time lag present in the errors obtained from forecasting. By comparing the computed value of the Durbin-Watson test with the appropriate values from the table of values of the DW statistic (Table F of Appendix III), the significance can be determined.

DUMMY VARIABLE CONSTRUCTION:
(OS) In old-school data science, dummy variable construction is a technique used to represent categorical variables in regression analysis or other statistical models. It involves creating binary variables, also known as dummy variables, to represent each category or level of a categorical variable.

The process of dummy variable construction involves assigning a value of 1 to the dummy variable if an observation belongs to a specific category and 0 otherwise. This allows us to include categorical variables in statistical models, which typically require numerical inputs.(poe.com Data-Scientist-GPT3)

Typically all categories but 1 are assigned a dummy variable with the remaining variable serving as a “reference level) mean that if all dummy variables are 0, the answer is based on a reference level. In other words, the answer contains the missing dummy within the intercept and the base answer is based on the missing dummy alone. Including all dummies would result in multicollinearity.
But sometimes dummies are constructed for all classes (especially in ANOVA). When fitting a regression model with dummy variables, one common approach is to use the ANOVA (Analysis of Variance) method. ANOVA allows us to test the significance of the categorical variable as a whole and compare the effect of different levels on the response variable.

In ANOVA, the null hypothesis assumes that there is no significant difference in the means of the response variable across the categories of the categorical variable. The alternative hypothesis suggests that at least one category level has a different mean. By calculating the F-statistic and comparing it to the critical value, we can determine whether the categorical variable has a significant impact on the response variable. (Gordy Fairchild and Poe.com (DATA-SCIENTIST-GPT3)

(NS) Hot-level encoding.

E

ECONOMETRICS

EFFICIENT ESTIMATOR

80/20 RULE (PARETO PRINCIPAL):

(OS) States that, for many events, roughly 80% of the effects come from 20% of the causes. In many businesses, 80% of revenue comes from 20% of customers who form an extremely valuable segment of high-spenders. A data-driven loyalty or rewards program can boost spending in this segment substantially. Anti-attrition modeling can ID who may be ready to stop buying and win the customer back before they leave. (Analytics Glossary, Analytics Explained)

ESTIMATOR:

(OS) An estimator is a rule for “guessing” the value of a population parameter based on a random sample from the population. An estimator is a random variable, because its value depends on which particular sample is obtained, which is random. A canonical example of an estimator is the sample mean, which is an estimator of the population mean. There are specialized model-based estimators such as the OLS (Ordinary Least Squares) linear regression coefficients along with estimators in many other model specifications. (U Cal Berkeley Statistics Glossary)

ENSEMBLE MODELING:

ENTERPRISE RESOURCE PLANNING (ERP)

ETL (EXTRACT TRANSFORM AND LOAD):

(NS) ETL (extract, transform, load) is a data pipeline system designed by data engineers. The data is extracted from multiple sources, transformed from its raw form into a proper format to be consistent with the data from other sources, and loaded into the target data warehouse. From here, it can be used for further data analysis and modeling to solve various business problems. (Data Camp Glossary)

EXPECTED VALUE:

EXPERIMENTAL DESIGN

EXPLORATORY DATA ANALYSIS (EDA):

OS) Exploratory Data Analysis (EDA) is an approach/philosophy for data analysis that employs a variety of techniques (mostly graphical) to

maximize insight into a data set
uncover underlying structure
extract important variables
detect outliers and anomalies
test underlying assumptions
develop parsimonious models;
and determine optimal factor settings.

Focus The EDA approach is precisely that–an approach–not a set of techniques, but an attitude/philosophy about how a data analysis should be carried out.

Philosophy EDA is not identical to statistical graphics although the two terms are used almost interchangeably. Statistical graphics is a collection of techniques–all graphically based and all focusing on one data characterization aspect. EDA encompasses a larger venue; EDA is an approach to data analysis that postpones the usual assumptions about what kind of model the data follow with the more direct approach of allowing the data itself to reveal its underlying structure and model. EDA is not a mere collection of techniques; EDA is a philosophy as to how we dissect a data set; what we look for; how we look; and how we interpret. It is true that EDA heavily uses the collection of techniques that we call “statistical graphics”, but it is not identical to statistical graphics per se. (NIST, National Institute of Standards and Technology, Engineering and Statistics Handbook)

(NS) Exploratory data analysis is a broader term that might also include data mining and certain forms of unsupervised machine learning.

Exploratory data analysis (EDA) is the process of analyzing data sets to summarize their main characteristics, often using visual methods. It helps in understanding the data, identifying patterns, and checking assumptions before applying more complex analysis techniques.
Data mining, on the other hand, is the process of discovering patterns, trends, and insights from large datasets using methods such as machine learning, statistics, and database systems. It involves extracting useful information from data to make informed decisions.
Unsupervised machine learning is a type of machine learning where the model is trained on unlabeled data, meaning the algorithm learns to identify patterns in the data without any explicit guidance. Clustering and dimensionality reduction are common techniques used in unsupervised learning to group similar data points or reduce the complexity of the data.

In summary, EDA focuses on exploring and summarizing data, data mining involves extracting valuable insights from data, and unsupervised machine learning aims to find patterns and structures in data without the need for labeled examples. (Poe.com Data-Scientist-GPT3). All 3 could be argued to form new-school EDA.

EXPONENTIAL SMOOTHING:

EXPONENTIAL TREND:

EX-POST FORECASTING:

ETL: (Extract, Transform, Load) is a data pipeline system designed by data engineers. The data is extracted from multiple sources, transformed from its raw form into a proper format to be consistent with the data from other sources, and loaded into the target data warehouse. From here, it can be used for further data analysis and modeling to solve various business problems. (Data Camp Data Science Glossary)

F

FACTOR ANALYSIS:

FAIR DATA:

(NS) FAIR data is Findable, Accessible, Interoperable, Reusable. The principles emphasize machine-actionability (i.e., the capacity of computational systems to find, access, interoperate, and reuse data with no or minimal human intervention) because humans increasingly rely on computational support to deal with data as a result of the increase in volume, complexity, and creation speed of data.

The “F” in FAIR. Metadata and data should be easy to find for both humans and computers. Machine-readable metadata are essential for automatic discovery of datasets and services.
The “A” in FAIR. The ability for a user to know how data can be accessed, possibly including authentication and authorization.
The “I” in FAIR. The capability to communicate, execute programs, or transfer data among various functional units in a useful and meaningful manner that requires the user to have little or no knowledge of the unique characteristics of those units.he “R” in FAIR. Reusability is the ultimate goal of FAIR data. To achieve this, metadata and data should be well-described so that they can be replicated and/or combined in different settings. (U Iowa Data Glossary

FALSE NEGATIVE (FN)

FEATURE:
(NS, from Poe AI): In data science, a feature refers to an individual measurable property or characteristic of a data point (also known as an observation or instance) that is used as input for a machine learning or statistical model. Features are typically represented as variables or columns
in a dataset.

Features can take various forms, such as numerical values, categorical variables, or even more complex structures like images or text. They provide information about the data points and are used to make predictions, classify instances, or uncover patterns and relationships within the data.

Here are a few examples of features in different domains:

Numerical Features: These are features that represent quantitative measurements. Examples include age, income, temperature, or any other continuous or discrete numerical values.

Categorical Features: These are features that represent categories or labels. Examples include gender (e.g., male or female), color (e.g., red, blue, green), or any other non-numeric values that represent distinct categories.

Text Features: In natural language processing (NLP), text features refer to the textual content of documents or sentences. Techniques such as text preprocessing, tokenization, and feature extraction are used to convert text into numerical representations that can be used as features.

Image Features: In computer vision, image features capture the visual properties of images. These features can be derived from techniques like convolutional neural networks (CNNs) or feature extraction algorithms, and they represent aspects of the image such as edges, textures, or shapes.

Features play a crucial role in building predictive models. The selection and engineering of relevant features can significantly impact the performance and interpretability of the models. Data scientists often employ techniques such as feature selection, dimensionality reduction, and feature engineering to
identify the most informative features or create new ones that enhance the model’s ability to learn and generalize patterns from the data.

(OS) Variable, (Database: Column; Marketing; characteristic; Psychology: factors, control variables, independent variables).

FEATURE ENGINEERING: (NS)

Feature engineering is the process of using domain knowledge and subject matter expertise to transform raw features into features that better reflect the underlying problem, and are better suited for machine learning algorithms. It includes extracting new features from the available data, or manipulating the
existing features. For example, if we’re trying to predict a health outcome such as the likelihood of having diabetes, calculating a BMI feature using height and weight features is feature engineering. [Data Camp glossary]

FEATURE REDUCTION:

F-TEST:

FACTOR ANALYSIS:

FIT ERROR:

FIXED EFFECTS MODEL:

(OS)

FORECAST & PREDICTION ACCURACY:

(OS) Prediction accuracy refers to how well a model can predict outcomes or values. Often, this is in non-forecast space, e.g. how well does a model do on the data it was estimated on?
It is typically measured using metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), or the coefficient of determination (R-squared).
A higher prediction accuracy indicates that the model’s predictions are closer to the true values as observed. (Data-Scientist-GPT3, Poe.com) However, the nature of the accuracy is of primary importance as different metrics approach accuracy differently. There are specialized metrics such as (MAPE) Mean Absolute Percentage Error, (MdAPE) Median Absolute Percentage Error (MdAD), Median Absolute Deviation, and (RMPSE) Root Mean Absolute Percentage Error. (Hans Lenenback & James Cleary, Forecasting: Practice and Process for Demand Management.)

A higher forecast accuracy indicates that the model’s future forecasts are closer to the true values as they unfold through time. The size of the confidence interval for model predictions in forecast space helps us understand our degree of certainly that the true future value will be withing a probability-based range.

FORECASTING:

FORECAST ERROR:

FORENSIC ANALYTICS:
Forensic analytics refers to the application of data analysis techniques and technologies to aid in accounting and regulatory audits, law-enforcement investigations, fraud detection, legal proceedings, and dispute resolution.
It involves the systematic examination, interpretation, and presentation of financial, operational, and other types of data to uncover evidence, identify irregularities, decrease risk, and support decision-making.

Some key aspects of forensic analytics include:

Data Collection and Analysis:
Gathering and organizing relevant data from various sources such as financial records, transaction logs, emails, and other digital and physical documents.
Applying advanced analytical methods, including statistical analysis, data mining, and predictive modeling, to identify patterns, trends, and anomalies.
Fraud Detection and Investigation:
Identifying potential fraudulent activities, such as misappropriation of assets, financial statement manipulation, and other types of financial crimes.
Tracing the flow of funds and uncovering complex financial schemes to build a comprehensive understanding of the fraudulent activities.
Compliance and Regulatory Examinations:
Assessing an organization’s adherence to relevant laws, regulations, and industry standards.
Identifying areas of non-compliance and providing recommendations for remediation. (poe.com, AI Assistant)

F-SCORE:

F-TEST

FUZZY LOGIC

G

GAUSS-MARKOV THEOREM

GENERALIZED

GEOMETRIC DATA

GEOMETRIC LAG

GLOBAL OPTIMAL

GOODNESS-OF-FIT MEASURES

GitHub:

GRADIENT DESCENT

H

HADOOP:
Hadoop is an open-source Java-based software framework that enables parallel processing and distributed storage of big data across clusters of many computers. Hadoop allows you to save time and handle much larger amounts of data than could be possible using only one computer. [Data Camp Glossary]

HAZARD FUNCTION:

HEDONIC MODELS:
(OS) Hedonic models, also known as hedonic regression models, are statistical models used to estimate the relationship between the price or value of a product or service and its various attributes or characteristics. These models are commonly employed in real estate, economics, and marketing research to determine the factors that influence the price or value of a particular item.

The term “hedonic” refers to the concept of hedonic pricing, which suggests that the price of a product or service is influenced by its intrinsic attributes. In other words, the value or utility of a good is derived from its individual characteristics, rather than just its overall quality or functionality.

Hedonic models typically involve multiple regression analysis, where the price or value of the item is considered as the dependent variable, and the various attributes or characteristics of the item are treated as independent variables. These attributes can include physical features, location, amenities, brand, and other relevant factors that are believed to impact the price or value.

By estimating the coefficients of the independent variables in the hedonic model, one can quantify the effect of each attribute on the price or value of the item. This information is valuable for understanding consumer preferences, conducting market analysis, pricing strategies, and making informed decisions related to investments, real estate valuation, or product development.

Hedonic models can also be used for prediction and forecasting purposes. By applying the estimated coefficients to new observations with known attribute values, one can predict the price or value of similar items.

Overall, hedonic models provide a framework for analyzing and understanding how specific attributes contribute to the price or value of a product or service, providing valuable insights for various industries and research fields.of information; otherwise, it is lossy. (http://amzn.to/39flUbX)

HETEROSCEDASTICITY: (OS) This condition exists when the errors do not have a constant variance across an entire range of values. For example, if the residuals from a time series have increasing variance with increasing time, they would be said to exhibit heteroscedasticity. (Glossary of Forecasting Terms) Special methods and data adjustments are used in many problems from linear regression to forecasting to prevent heteroscedasticity from biasing results.

HIDDEN MARKOV MODEL

HIERARCHICAL CLUSTERING

HISTOGRAM

HIVE

HOLT-WINTERS FORECASTING

HYPERPARAMETERS

HYPOTHESIS

HYPOTHESIS TESTING

I

IDENTIFICATION

IMPUTATION:

INDEPENDENCE OF IRRELEVANT ALTERNATIVES (IIA)

INFERENTIAL STATISTICS:

(OS) Inferential statistics is a branch of statistics that involves making inferences and generalizations about a population based on sample data. It helps in drawing conclusions and making predictions about a larger group based on a smaller subset of data. Inferential statistics is commonly used in hypothesis testing, confidence intervals, and making predictions about the population parameters. (Data-Scientist-GPT3, poe.com AI)

INFORMATION VALUE:

INSTRUMENTAL VARIABLE ESTIMATION:

ITERATION:

INVERSE AUTOCORRELATION FUNCTION:
(OS) IACF stands for the Inverse Autocorrelation Function. In forecasting, the IACF is a measure that helps identify the order of the Moving Average (MA) component in a time series model.

The IACF is computed by taking the inverse of the autocorrelation function (ACF) of a time series. The ACF measures the correlation between the values of a time series at different lags. By taking the inverse of the ACF, we can determine the potential order of the MA component in an ARMA model. (Poe.com Data-Scientist-GPT3)

J

JACKKNIFE:

JOINT PROBABILITY DENSITY FUNCTION:

JUPYTER NOTEBOOK:

K

KAGGLE:

(NS) Kaggle is an online platform and community for data scientists, machine learning practitioners, and enthusiasts to participate in data science competitions, collaborate on projects, and learn from each other. It was founded in 2010 and has since become one of the leading platforms for data science and machine learning challenges.
On Kaggle, users can find a wide range of datasets across various domains, including finance, healthcare, image recognition, natural language processing, and more. These datasets are often shared by organizations or individuals looking for insights or solutions to specific problems.
One of the main attractions of Kaggle is its data science competitions. These competitions provide participants with real-world datasets and problem statements, challenging them to develop predictive models or solutions that achieve the best performance on an evaluation metric specified by the competition host. Competitions often come with cash prizes, recognition, and opportunities to work on real-world problems with leading companies and organizations.
Kaggle also serves as a collaborative platform where users can form teams, share code and insights, and learn from each other’s approaches. The platform provides features like discussion forums, notebooks (Jupyter notebooks integrated within Kaggle), and public kernels (code notebooks) that allow users to showcase their work and share it with the community.
In addition to competitions, Kaggle hosts datasets, provides access to cloud computing resources for running data analyses and model training, and offers a wide range of learning resources such as tutorials, courses, and community-led discussions. (Poe.com AI Data-Scientist GPT-3)

KERNEL ESTIMATION:

(NS/OS) Kernel estimation, also known as kernel density estimation or kernel smoothing, is a non-parametric technique used to estimate the underlying probability density function (PDF) of a random variable based on observed data. It is commonly used in statistics and data analysis to visualize and estimate the probability distribution of a continuous variable.
The basic idea behind kernel estimation is to approximate the PDF by placing a kernel, which is a smooth and symmetric function, at each data point and then summing these kernels to obtain a smoothed estimate of the PDF. (poe.com, Data-Scientist-3-GPT)

KERNEL SMOOTHING:

(NS) Kernel smoothing, also known as kernel regression or Nadaraya-Watson estimator, is a non-parametric technique used to estimate the relationship between a dependent variable and one or more independent variables in a regression setting. It is commonly used in statistics and data analysis when the relationship between variables is expected to be non-linear or when the underlying functional form is unknown.
The basic idea behind kernel smoothing is to assign weights to the observed data points based on their proximity to a particular point of interest. These weights are determined by a kernel function, which controls the shape and width of the smoothing window. The kernel function can be a Gaussian (normal) kernel, Epanechnikov kernel, or other suitable choices.

K-MEANS:

A popular clustering algorithm that groups examples in unsupervised learning. The k-means algorithm basically does the following:

Iteratively determines the best k center points (known as centroids).
Assigns each example to the closest centroid. Those examples nearest the same centroid belong to the same group.
The k-means algorithm picks centroid locations to minimize the cumulative square of the distances from each example to its closest centroid.

For example, consider the following plot

K-MEANS CLUSTERING (https://pub.towardsai.net/fully-explained-k-means-clustering-with-python-e7caa573176a)

A Cartesian plot with several dozen data points.

If k=4, the k-means algorithm will determine 4 centroids. Each example is assigned to its closest centroid, yielding 4 groups:

The previous data points are clustered into 4 distinct groups, with each group representing the data points closest to a particular centroid.

Imagine that a webstore firm wants to segment customers by customer number and rate of purchase, the 4 centroids identify the customer number and rate in that cluster. So, the manufacturer could probably base a growth model those 4 centroids. Note that the centroid of a cluster is typically not an example in the cluster.

The preceding illustrations shows k-means for examples with only two features. Note that k-means can group examples across many features.

(NS)

KNN:

(OS/NS) :KNN stands for k-nearest neighbors, which is a popular machine learning algorithm used for both classification and regression tasks. It is a non-parametric and instance-based learning method that makes predictions based on the similarity of data points.

In the KNN algorithm, the “k” refers to the number of nearest neighbors to consider when making a prediction. The algorithm works as follows:

Training: During the training phase, the algorithm simply stores the feature vectors and corresponding labels of the training data.
Prediction: When a new data point is provided for prediction, the algorithm calculates the distances between the new point and all the points in the training data. The most common distance metric used is Euclidean distance, but other distance measures can also be employed.
Nearest neighbors selection: The algorithm selects the “k” data points from the training set that are closest to the new data point based on the calculated distances.
Majority voting (classification) or averaging (regression): For classification tasks, the algorithm assigns the class label that is most frequent among the “k” nearest neighbors. For regression tasks, the algorithm calculates the average of the target values of the “k” nearest neighbors.
The choice of the value of “k” is an important consideration in KNN. A smaller value of “k” leads to a more flexible model that can capture local patterns but may be sensitive to noise or outliers. On the other hand, a larger value of “k” provides a smoother decision boundary but may miss out on fine-grained details.

KNN is known for its simplicity and ease of implementation. It does not require explicit training, as the training data is directly used for making predictions. However, it can be computationally intensive, especially with large datasets, as it needs to calculate distances between the new point and all training points during prediction.
KNN is widely used in various domains such as image recognition, recommendation systems, and anomaly detection. It is a versatile algorithm that can handle both numerical and categorical data, making it a popular choice for many machine learning tasks.

KOYCK LAG:

(OS) In econometrics (OS) A Koyck lag, also known as a Koyck distributed lag, is a concept in econometrics that refers to a specific geometric lag structure used in dynamic regression models to capture the lagged effects of a variable on itself. It is named after the Dutch economist Jan Koyck, who introduced this lag structure in the 1950s.
In a Koyck lag structure, the lagged values of the variable of interest are included as explanatory variables in the model, with each lagged value multiplied by a coefficient that represents the decay or attenuation of its effect over time. The lag coefficients follow an exponential decay pattern, where the effect of each lag diminishes exponentially as the lag increases. The Koyck lag structure is often used in time series regression models to capture the gradual adjustment or feedback mechanism between variables. It is particularly suitable for modeling processes with a slow response to changes or where past values have a lasting impact on the current value. (poe.com, Data-Scientist-3 GPT)

KURTOSIS:

(OS) Pearson (1905) introduced kurtosis as a measure of how flat the top of a symmetric distribution is when compared to a normal distribution of the same variance. He referred to more flat-topped distributions (KURTOSIS < 0) as “platykurtic,” less flat-topped distributions (KURTOSIS > 0) as “leptokurtic,” and equally flat-topped distributions as “mesokurtic” (KURTOSIS = 0).

Kurtosis is actually more influenced by scores in the tails of the distribution than scores in the center of a distribution (DeCarlo, 1967). Accordingly, it is often appropriate to describe a leptokurtic distribution as “fat in the tails” and a platykurtic distribution as “thin in the tails.” Values between 0 and 1 suggest a normal distribution.

L

LABEL:

LABELED DATA:

LAG DISTRIBUTION:
(OS) In the context of time series analysis and forecasting, lag distributions refer to the patterns or relationships between a variable and its past values at different time lags. The identification of important lag distributions is crucial for capturing temporal dependencies and building accurate predictive models. The importance of lag distributions can vary depending on the specific dataset and the underlying dynamics being analyzed. However, there are a few commonly considered lag distributions that are often explored in time series analysis:

Autocorrelation (ACF): Autocorrelation measures the correlation between a variable’s current value and its lagged values at different time lags. The autocorrelation function (ACF) plot displays these correlations, with the y-axis representing the correlation coefficient and the x-axis representing the lag. Significant autocorrelation at certain lags indicates important lag distributions that can be leveraged for forecasting.
Partial Autocorrelation (PACF): Partial autocorrelation measures the correlation between a variable’s current value and its lagged values while controlling for the influence of intermediate lags. The partial autocorrelation function (PACF) plot displays these correlations, helping to identify direct relationships between a variable and its lagged values. Significant partial autocorrelation at certain lags indicates important lag distributions that contribute to the variable’s behavior.
Seasonal Patterns: In time series data with seasonal patterns, important lag distributions correspond to the seasonal intervals. For example, in monthly data, the lag distribution of 12 (representing 12 months) may capture annual seasonality. Identifying and incorporating these seasonal lag distributions is crucial for accurate modeling and forecasting.
Trend Patterns: In time series data with trends, important lag distributions correspond to the trend component. For example, in quarterly data with a linear trend, the lag distribution of 4 (representing 4 quarters) captures the trend patterns. Accounting for these trend-related lag distributions is essential for modeling and predicting the underlying trend.
Other Relevant Patterns: Depending on the specific domain and dataset, there may be other important lag distributions to consider. For instance, in financial time series, lag distributions related to market events, economic indicators, or specific calendar events might be of interest.
Identifying the most important lag distributions typically involves analyzing ACF and PACF plots, exploring seasonal and trend patterns, and considering domain-specific knowledge. By understanding and incorporating these lag distributions, analysts can capture the temporal dynamics and improve the accuracy of their time series models and forecasts. (poe.com, Data-Scientist-GPT3)

In econometrics (OS) A Koyck lag, also known as a Koyck distributed lag, is a concept in econometrics that refers to a specific geometric lag structure used in dynamic regression models to capture the lagged effects of a variable on itself. It is named after the Dutch economist Jan Koyck, who introduced this lag structure in the 1950s.
In a Koyck lag structure, the lagged values of the variable of interest are included as explanatory variables in the model, with each lagged value multiplied by a coefficient that represents the decay or attenuation of its effect over time. The lag coefficients follow an exponential decay pattern, where the effect of each lag diminishes exponentially as the lag increases.
The Koyck lag structure is often used in time series regression models to capture the gradual adjustment or feedback mechanism between variables. It is particularly suitable for modeling processes with a slow response to changes or where past values have a lasting impact on the current value. (poe.com, Data-Scientist-GPT3)

LAGGED VARIABLES:

(OS) In pure time series models as well as models with other explanatory variables lagged variables refer to the value of lags t periods back in time, where t is the number of periods that have elapsed for a given data periodicity. The distribution of the lagged variables can be discerned via ACF, PACF, seasonal patterns, trend patters, cycle patterns, and various other forms. Some models use the lagged value of the current period’s independent variable alone instead of the current independent variable value (as in partial adjustment models) to predict future values merely by using the current value as the dependent variable or in combination with other (non-lagged) independent variables.

LAGRANGE MULTIPLIER (LM)

LASSO REGRESSION:
(NS) Lasso regression performs L1 regularization, i.e. it adds a factor of sum of absolute value of coefficients in the optimization objective. Thus, lasso regression optimizes the following:

Objective = RSS + α * (sum of absolute value of coefficients)
Here, α (alpha) works similar to that of a ridge regression and provides a trade-off between balancing RSS and the magnitude of coefficients. Like that of ridge, α can take various values. Let’s iterate it briefly here:

α = 0 : Same coefficients as simple linear regression
α = ∞ : All coefficients zero (same logic as before)
0 < α < ∞ : coefficients between 0 and that of simple linear regression (Analytics Vidhya Glossary of Machine Language Terms)

LAUGH TEST:

(OS) The laugh test of data reduction refers to noticing totals that are ridiculously large or small based on business or statistical knowledge. If data cannot pass the laugh test, it is likely to be seriously flawed in part or in whole.

LEADING INDICATOR:

(OS) Leading indicators usually refer to macroeconomic metrics that lead rather than lag certain kinds of economic activity like a recession or consumption. Leading indicators also include variables that serve to lead a forecast, meaning that some idea of the future is given by the leading indicator. For this reason, leading indicators are highly useful in time series and longitudinal models.

LEAST SQUARES ESTIMATOR:

An estimator that minimizes the sum of squared residuals. (Jeffrey Wooldridge, Introductory Economics)

LEAST SQUARES REGRESSION:

(OS) Also known as OLS (Ordinary Least Squares). A least squares regression is a linear model designed to fit a dependent variable to a set of independent variables. In cross-sectional regression, it is subject to 6 classical linear regression assumptions:

Linear in Parameters. In an equation of the form Y= alpha + beta1 x X1 + beta2 x X2 + epsilon (error). The betas are the parameters of interest. Note that certain non-linear forms such as X1^2 are still linear in parameters.
Random Sampling. There is a random sample of n observations that are used to estimate the betas.
No perfect Collinearity. No one variable is an exactly linear combination of another.
Zero Conditional Mean. The epsilon (error term) has a zero mean given any values of the X’s.
Homoskedasticity. The error has the sample variance as each of the independent variables.
Normality. The error is entirely independent of the X’s and is normally distributed with mean=0 and a standard deviation of 1. This assumption allows the testing of statistical hypotheses about the beta coefficients.

(NS) A linear regression model trained by minimizing L2 Loss. (Google Machine Learning Glossary)

LIFE TABLE METHOD:

(OS) The life table method in survival analysis is a widely used technique to estimate survival probabilities and hazard rates in a cohort of individuals over time. It is a non-parametric method that provides a step-by-step approach to calculate survival metrics such as survival probabilities, hazard rates, and median survival times.

The key components of the life table method include:

Calculation of Survival Probabilities: The life table method calculates the probability of surviving up to a certain time point given that an individual has survived up to the previous time points. This is typically done using the Kaplan-Meier estimator.
Estimation of Hazard Rates: The hazard rate at each time point is estimated as the probability of an event (e.g., death) occurring at that time point given that the individual has survived up to that time point.
Construction of the Life Table: The life table is constructed by organizing the data into intervals of time and calculating the number of individuals at risk, the number of events (deaths), and the survival probabilities at each time interval.
Survival Curve: The survival curve, also known as the Kaplan-Meier curve, is plotted based on the calculated survival probabilities over time. It shows the proportion of individuals surviving at each time point.
Median Survival Time: The median survival time, which represents the time by which 50% of the individuals in the cohort have experienced the event of interest, can be estimated from the survival curve.

Overall, the life table method is a valuable tool in survival analysis to estimate and visualize the survival experience of a cohort of individuals over time. It provides insights into the survival probabilities, hazard rates, and overall survival patterns in a dataset. (Poe.com, Data-Scientist-GPT-3)

While the life table method in survival analysis is a useful and widely used technique, it also has some disadvantages that researchers should be aware of. Some of the drawbacks of the life table method include:

Censoring: The life table method assumes non-informative censoring, meaning that the probability of being censored is unrelated to the individual’s survival time. If censoring is informative (related to the likelihood of the event occurring), the estimates provided by the life table method may be biased.
Interval Width: The life table method requires binning the data into intervals, which can result in loss of information. The choice of interval width can impact the accuracy of the estimated survival probabilities and hazard rates.
Assumptions: The life table method assumes that the survival probabilities are constant within each interval, which may not always hold true, especially in cases where the hazard rate changes over time.
Handling Time-varying Covariates: The life table method does not easily accommodate time-varying covariates, which can be important predictors of survival. Ignoring time-varying covariates can lead to biased estimates.
Limited Flexibility: The life table method is non-parametric and may not be suitable for complex survival data that require more flexible modeling techniques. Parametric survival models or semi-parametric models like Cox proportional hazards model may be more appropriate in such cases.
Difficulty in Handling Missing Data: The life table method may face challenges in handling missing data, especially if the missingness is related to survival outcomes.
Limited in Predictive Ability: The life table method focuses on estimating survival probabilities and does not directly provide predictions for individual survival times or hazard rates beyond the observed data.

While the life table method is a valuable tool for estimating survival metrics, researchers should consider these limitations and choose the appropriate survival analysis technique based on the specific characteristics of their data and research objectives. (Poe.com, Data-Scientist-GPT-3)

LIKELIHOOD RATIO (LR):

(OS) In the realm of data science, the likelihood ratio is a fundamental concept used in statistical inference to compare the likelihood of two competing hypotheses given the observed data. It is a measure that helps assess the strength of evidence in favor of one hypothesis over another.

In essence, the likelihood ratio is defined as the ratio of the likelihood of the data under one hypothesis to the likelihood of the data under another hypothesis. Mathematically, it can be expressed as:

Likelihood Ratio = 𝐿 (Data ∣ Hypothesis 1)/
𝐿(Data ∣ Hypothesis 2)
Likelihood Ratio=L(Data∣Hypothesis 2)/
L(Data∣Hypothesis 1 )

Here,
L(Data∣Hypothesis) represents the likelihood of the observed data given a specific hypothesis.

How is the likelihood ratio used in data science?

Model Comparison: In statistical modeling, data scientists often compare different models to determine which one best fits the data. The likelihood ratio test can be used to compare the fit of nested models (where one model is a special case of the other) and assess whether the more complex model provides a significant improvement in fit.
Hypothesis Testing: Data scientists use the likelihood ratio test as a tool for hypothesis testing. By comparing the likelihood of the data under the null hypothesis (simpler model) to an alternative hypothesis (more complex model), they can evaluate the strength of evidence against the null hypothesis.
Parameter Estimation: Likelihood ratios are also crucial in maximum likelihood estimation, where the goal is to estimate the parameters of a statistical model that maximize the likelihood of the observed data. The likelihood ratio can help compare different parameter values and assess their relative likelihoods.
Variable Selection: In machine learning and regression analysis, likelihood ratios can be used for variable selection by comparing models with and without specific predictors. This helps in identifying the most influential variables in predicting the outcome.

Overall, the likelihood ratio plays a pivotal role in statistical inference and model evaluation in data science, aiding in model comparison, hypothesis testing, parameter estimation, and variable selection. It provides a quantitative measure of the evidence in favor of one hypothesis or model over another based on the observed data. (Poe.com, Data_scientist-GPT-3)

LIMITED DEPENDENT VARIABLE:

(OS) A limited dependent variable can take on only a small number of integer values. There are 4 kinds of limited dependent variables:

Binary Dependent Variables: Binary dependent variables can take on only two possible values, typically coded as 0 and 1. Examples include yes/no outcomes, presence/absence indicators, or success/failure responses. Models that commonly use binary dependent variables include logistic regression, probit regression, Support Vector Machines (SVM), K-Nearest-Neighbor (KNN) models, naive Bayes, Neural Networks, random forest, gradient boosting, Multi-Layered Perceptron (MLP), decision tree models, linear probability models, discriminant analysis, and others. Many of these models can also be used for multiple category limited dependent variables as well.
Multi-Categorical Dependent Variables. Dependent variables are based on multiple classes that are not ordered or ranked and values do not depend on the magnitude. The variable’s value may be something like a color, car model, or genetic expression. Multi-category logit and probit models and several of the binary variable models (using multiple categories) are used to analyze this case.
Ordinal Dependent Variables: Ordinal dependent variables have a specific order or ranking to their categories, but the distance between categories is not well-defined. Examples include Likert scale ratings (e.g., strongly disagree, disagree, neutral, agree, strongly agree) or educational attainment levels (e.g., high school, college, graduate school). Models that can handle ordinal dependent variables include ordered logistic regression, ordered probit regression, continuation ratio models, cumulative logit models, and adjacent category models.
Count-Dependent Variables: Count-dependent variables represent the number of occurrences of an event or outcome within a specific unit of observation. Examples include the number of customer purchases, defects in a manufacturing process, or the count of medical visits. Models that are suitable for count-dependent variables include Poisson regression, negative binomial regression, and zero-inflated models. (Based on poe.com Data-Scientist-GPT3)

LIMITED DEPENDENT VARIABLE PROPERTIES:

Here are some key properties of limited dependent variable models:

Non-Normality: Limited dependent variables often do not follow a normal distribution, which is a common assumption in traditional linear regression models. Models for limited dependent variables are designed to accommodate the non-normality of the data.
Boundaries or Threshold Effects: Limited dependent variables may have boundaries or thresholds that restrict the possible values they can take. For example, binary variables are bounded between 0 and 1. Models obviously need to account for these boundaries in their estimation.
Heteroscedasticity: Limited dependent variables may exhibit heteroscedasticity, meaning that the variance of the error term may not be constant across all levels of the independent variables. Models need to address this issue to ensure statistically meaningful or accurate parameter estimation.
Endogeneity: Limited dependent variable models often need to consider endogeneity, where the independent variables are correlated with the error term. Techniques such as instrumental variables or control functions may be employed to mitigate endogeneity.
Censored or Truncated Data: In some cases, the dependent variable may be censored or truncated. A censored variable is one where the true value is known to fall within a certain range but is not precisely observed. A truncated variable, on the other hand, is a variable where certain values are systematically excluded or not captured in the data set. Models for censored or truncated data need to correct for this truncation in their estimation.
Sample Selection Bias: Limited dependent variable models may also be susceptible to sample selection bias, where the observed sample is not representative of the full population. Techniques like Heckman correction (Heckit model) can be used to address sample selection bias.
Model Interpretability: Limited dependent variable models often provide more interpretable results compared to standard linear regression models, especially when dealing with binary, ordinal, or count data. Interpretability is crucial for understanding the impact of predictors on the outcome. (Based on poe.com, Data-Scientist-GPT3)

LOGARITMIC TRANSFORMATION

LOG-LIKELIHOOD:

LOG LOSS

LOG-ODDS RATIO

LOGIT (LOGISTICAL) MODEL/REGRESSION

LONGITUDINAL DATA/ PANEL DATA:

(OS) Technically, data that has repeated observations on the same cross-section (individuals, households, accounts, etc.) over time. Longitudinal data may also be panel data or vintage data where individuals are followed over a period of time, say to observe shopping habits or TV use. This is not pure time series data, but a mix of random effects (time-varying features/variables) and fixed effects (variables that do not change over time or change infrequently). In many fields, fixed, random, and mixed models are often used with this type of data depending on what the data scientist cares about. A special form of longitudinal data has cohorts of people who join and leave the dataset at different times or so-called vintage data. There are dozens of other ways aside from the fixed-random-mixed models that can be used. A frequent NS mistake is to not take into account specially structured data like these. Many OS practitioners outside of certain academic fields are also untrained in techniques for these data structures. (Econometric Analysis of Cross Section and Panel Data, Jeffrey W. Wooldridge, MIT Press, 2nd Ed, 2010)

LONG SHORT TERM MEMORY (LSTM)

LOSS:
(NS) During the training of a supervised model, a measure of how far a model’s prediction is from its label. (Google Machine Learning Glossary)

LOSS FUNCTION:

(NS) A loss function calculates the loss. (Google Machine Learning Glossary)

LOSS AGGREGATOR:
(NS) A type of machine learning algorithm that improves the performance of a model by combining the predictions of multiple models and using those predictions to make a single prediction. As a result, a loss aggregator can reduce the variance of the predictions and improve the accuracy of the predictions. (Google Machine Learning Glossary)

LOSS CURVE:
(NS) A plot of loss as a function of the number of training iterations. A Cartesian graph of loss versus training iterations, showing a
rapid drop in loss for the initial iterations, followed by a gradual drop, and then a flat slope during the final iterations. Note the loss curve is used to tell if the solution is optimal, whether an algorithm converged, and to set a higher (lower) number of iterations.

Loss curves can help you determine when your model is converging or overfitting.

Loss curves can plot all of the following types of loss:

training loss
validation loss
test loss

(Google Machine Learning Glossary)

M

MARKEDNESS:

(OS) Markedness is a confusion matrix metric that gives an overall measure of the predictive power of a classifier, taking into account both the positive and negative predictive values.Markedness is defined as:

Markedness = Positive Predictive Value (PPV) + Negative Predictive Value (NPV) – 1

Where:

1. Positive Predictive Value (PPV) = True Positives / (True Positives + False Positives)
PPV represents the proportion of true positives among all the positive predictions made by the classifier.
Negative Predictive Value (NPV) = True Negatives / (True Negatives + False Negatives)
2. NPV represents the proportion of true negatives among all the negative predictions made by the classifier.
Markedness ranges from -1 to 1, and it provides a measure of the overall predictive power of the classifier. A higher value of markedness indicates better predictive power.

Specifically:

A markedness value of 1 indicates perfect predictive power, where the classifier correctly predicts all positive and negative instances.
A markedness value of 0 indicates that the classifier has no predictive power, as the PPV and NPV are both 0.5 (equivalent to random guessing).
A negative markedness value indicates that the classifier is worse than random guessing.
Markedness is considered a more comprehensive metric compared to other confusion matrix metrics, such as accuracy, precision, recall, and F1-score, as it takes into account both the positive and negative predictive power of the classifier.

Markedness can be particularly useful when the class distribution in the dataset is imbalanced, as it provides a more balanced assessment of the classifier’s performance on both the positive and negative classes. (Poe.com AI Assistant)

MARKET BASKET ANALYSIS:

(NS) Market Basket Analysis (also called MBA) is a widely used technique among marketers to identify the best possible combination of products or services that are frequently bought by customers. This is also called product association analysis. Association analysis is mostly done based on an algorithm named the “Apriori Algorithm”. The Outcome of this analysis is called association rules. Marketers use these rules to strategize their recommendations. When two or more products are purchased, Market Basket Analysis is done to check whether the purchase of one product increases the likelihood of the purchase of other products. This knowledge is a tool for marketers to bundle products or strategize a product cross-sell to a customer. (Analytics Vidhya Glossary)

MARKOV CHAIN:

(OS) A stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event is a Markov chain. In continuous time, it is known as a Markov process. (https://en.wikipedia.org/wiki/Markov_chain)

MAXIMUM LIKELIHOOD ESTIMATE (MLE):

(OS) The maximum likelihood estimate of a parameter from data is the possible value of the parameter for which the chance of observing the data is largest. That is, suppose that the parameter is p, and that we observe data x. Then the maximum likelihood estimate of p is
estimate p by the value q that makes P(observing x when the value of p is q) as large as possible.

For example, suppose we are trying to estimate the chance that a (possibly biased) coin lands heads when it is tossed. Our data will be the number of times x the coin lands heads in n independent tosses of the coin. The distribution of the number of times the coin lands heads is binomial with parameters n (known) and p (unknown). The chance of observing x heads in n trials if the chance of heads in a given trial is q is nCx qx(1−q)n−x. The maximum likelihood estimate of p would be the value of q that makes that chance largest. We can find that value of q explicitly using calculus; it turns out to be q = x/n, the fraction of times the coin is observed to land heads in the n tosses. Thus the maximum likelihood estimate of the chance of heads from the number of heads in n independent tosses of the coin is the observed fraction of tosses in which the coin lands heads. (U Cal Berkeley Statistics Glossary)

MEAN (ARITHMETIC):

(OS) The sum of a list of numbers, divided by the number of elements in the list. Also called an average but a median may be called an average as well.

MEAN ABSOLUTE DEVIATION (MAD)

MEAN ABSOLUTE ERROR (MAE)

MEAN ABSOLUTE PERCENTAGE ERROR (MAPE):

(OS) The mean absolute percentage error is the mean or average of the sum of all of the percentage errors for a given data set taken without regard to sign. (That is, their absolute values are summed and the average computed.) It is one measure of accuracy commonly used in quantitative methods of forecasting. If the sign of the error does not matter, MAPE is often thought to be the gold standard for judging forecast quality. It can be used easily to compare different models, algorithms, and hyperparameter settings. MAPE=ABS(((FORECAST-ACTUAL)/ACTUAL)*100). It is less sensitive to outliers and distributional problems than mean-based methods. By taking on an absolute value, this method ensures that under-forecasts and over-forecasts do not cancel each other out. This is a measure of precision not a measure of bias. (Hans Levenback & James Cleary, Forecasting: Practice and Process for Demand Management, 2006)

MEAN ERROR (ME):

(OS) the sum of the forecast errors divided by the number of periods in the forecast horizon for which forecasts were made. It is a measure of bias. It gives the mean of forecast errors expressed in units of the data. (Hans Levenback & James Cleary, Forecasting: Practice and Process for Demand Management, 2006)

MEAN PERCENTAGE ERROR (MPE):

(OS) MPE gives the mean of forecast errors in absolute-value percentage form and is unit-free. It is also a measure of bias. (Hans Levenback & James Cleary, Forecasting: Practice and Process for Demand Management, 2006)

MEAN SQUARED ERROR (MSE):

(OS) The mean squared error of an estimator of a parameter is the expected value of the square of the difference between the estimator and the parameter. In symbols, if X is an estimator of the parameter t, then
MSE(X) = E( (X−t)^2 ).

The MSE measures how far the estimator is off from what it is trying to estimate, on average in repeated experiments. It is a summary measure of the accuracy of the estimator. It combines any tendency of the estimator to overshoot or undershoot the truth (bias), and the variability of the estimator (SE). The MSE can be written in terms of the bias and SE of the estimator:
MSE(X) = (bias(X))^2 + (SE(X))^2.

In forecasting, a distinction should be drawn between the MSE of the training data (fit period error) and the MSE of forecasted data (forecast error). The forecast error is composed of fit error, and widening confidence intervals around the forecast line as the researcher forecast gets further and further in the future.

MEDIAN: (OS) If variables are ranked, the middle observation is the median. The relationship between the mean and the median tells us something about the skewness of the data. For some variables such as income or net worth the mean is typically skewed upwards due to very high valid values. The mean is more sensitive to outliers while the median can help ID problems with kurtosis.

MEDIAN ABSOLUTE DEVIATION (MdAD): (OS) MdAD is a viable alternative (measures of spread) to sample standard deviation and MAD when outliers are present and mean-based measures are sensitive to outliers. By taking on an absolute value, this method ensures that under-forecasts and over-forecasts do not cancel each other out. (Hans Levenback & James Cleary, Forecasting: Practice and Process for Demand Management, 2006)

MEDIAN ABSOLUTE PERCENTAGE ERROR (MdAPE): (OS) Instead of the mean being used (as with MAPE) the median is used to construct this precision metric. It is less sensitive to outliers and skewness than the MAPE. By taking on an absolute value, this method ensures that under-forecasts and over-forecasts do not cancel each other out. (Hans Levenback & James Cleary, Forecasting: Practice and Process for Demand Management, 2006)

MEDIAN SMOOTHING:

(OS) Median smoothing is a technique used in forecasting to reduce the impact of outliers and noise in a time series data set. It involves replacing each data point with the median value of a subset of neighboring data points within a specified window size.

Here’s how median smoothing works:

Define a window size: The window size determines the number of neighboring data points considered for smoothing. It can be an odd number to ensure symmetry around each data point.
Slide the window through the time series: Start at the beginning of the time series and slide the window through each data point.
Calculate the median: For each window position, sort the data points within the window and select the median value. The median is the middle value if the window size is odd or the average of the two middle values if the window size is even.
Replace the data point: Replace the original data point with the calculated median value.
By replacing each data point with the median of its neighboring values, median smoothing effectively reduces the impact of extreme values and outliers. It provides a smoother representation of the underlying trend in the data, making it easier to identify patterns and make forecasts.

It’s worth noting that median smoothing can effectively remove outliers but may also smooth out genuine changes in the data. Therefore, the window size should be chosen carefully to balance noise reduction and preserving important features in the time series.

Overall, median smoothing is a simple and robust technique commonly used in forecasting to mitigate the effects of noise and outliers in time series data. (Poe.com AI Assistant)

MEASUREMENT ERROR:

(OS) Measurement error refers to the discrepancy between the true value of a variable and the value that is actually measured or observed. It is a common occurrence in data science and can arise due to various factors such as instrument limitations, human error, sampling issues, or natural variability in the phenomenon being measured. (Poe.com AI Assistant)

MIXED AUTO-REGRESSIVE MOVING AVERAGE (see ARMA):

MIXED-EFFECTS MODELS:

(OS) A mixed-effects model, also known as a mixed-effects regression model or multilevel model, is a statistical model that combines both fixed effects and random effects. It is commonly used in situations where the data has a hierarchical
or nested structure, such as in longitudinal or clustered data. (Poe.com, Data-Scientist-GPT3)

MODEL:

NS & OS Fundamentals
In general, any mathematical construct that processes input data and returns output. Phrased differently, a model is the set of parameters and structure needed for a system to make predictions. In supervised machine learning, a model takes an example as input and infers a prediction as output. Within supervised machine learning, models differ somewhat. For example:

A linear regression model consists of a set of weights and a bias.
A neural network model consists of:
A set of hidden layers, each containing one or more neurons.
The weights and bias associated with each neuron.
A decision tree model consists of:
The shape of the tree; that is, the pattern in which the conditions and leaves are connected.
The conditions and leaves.
You can save, restore, or make copies of a model.

Unsupervised machine learning also generates models, typically a function that can map an input example to the most appropriate cluster. (Google Machine Learning Glossary)

MODEL SETS / ANSWER SETS:

MONTE CARLO SIMULATION:

(OS) Monte Carlo simulation is used to model the probability of different outcomes in a process that cannot easily be predicted due to the intervention of random variables. It is a technique used to understand the impact of risk and uncertainty in prediction and forecasting models. (https://www.investopedia.com/terms/m/montecarlosimulation.asp)

Analytics Vidhya offers a more understandable answer: “The idea behind Monte Carlo Simulation is to use random samples of parameters or inputs to explore the behavior of a complex process. Monte Carlo simulations sample from a probability distribution for each variable to produce hundreds or thousands of possible outcomes. The results are analyzed to get probabilities of different outcomes occurring.”

MOVING AVERAGE:

Moving Average (MA) in a forecasting and prediction sense is different from the moving average of a series.

MOVING AVERAGE PERCENTAGE ERRORS (MAPE):

MOVING AVERAGE PROCESS (MA)

MOVING MEDIAN SMOOTHING:

MULTINOMIAL LOGISTIC MODEL MULTINOMIAL PROBIT MODEL. (OS) logit and probit variables are usually trying to predict a binary outcome (dependent variable). If the outcome has more than 2 levels, the multinomial form is used. IIA (Independence of Irrelevant XXXXXiia——XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

N

NAIVE FORECAST1 (NF1):

NAIVE FORECAST3 (NF3):

NATURAL LANGUAGE PROCESSING (NLP)

NEURAL NETWORKS:

(NS) A model containing at least one hidden layer. A deep neural network is a type of neural network containing more than one hidden layer. For example, the following diagram shows a deep neural network containing two hidden layers.

A neural network with an input layer, two hidden layers, and an
output layer.

Graph from Google Machine Learning Glossary

Each neuron in a neural network connects to all of the nodes in the next layer. For example, in the preceding diagram, notice that each of the three neurons in the first hidden layer separately connect to both of the two neurons in the second hidden layer.

Neural networks implemented on computers are sometimes called artificial neural networks to differentiate them from neural networks found in brains and other nervous systems.

Some neural networks can mimic extremely complex nonlinear relationships between different features and the label. (Google Machine Language Glossaryy

There are several neural network methods and models used in various domains of data science and machine learning. Here are a few examples:

Feedforward Neural Networks (FNN): FNNs are the most basic type of neural network, consisting of an input layer, one or more hidden layers, and an output layer. They are commonly used for tasks like classification and regression.
Convolutional Neural Networks (CNN): CNNs are primarily used for image and video analysis. They utilize convolutional layers to extract hierarchical representations from input data, making them effective for tasks like object detection, image classification, and image segmentation.
Recurrent Neural Networks (RNN): RNNs are designed to handle sequential data by incorporating feedback connections. They have a “memory” that allows them to process sequences of variable lengths, making them suitable for tasks like natural language processing, speech recognition, and time series analysis.
Long Short-Term Memory (LSTM): LSTM is a variant of RNN that addresses the vanishing gradient problem and can retain information over longer sequences. LSTMs are widely used for tasks involving long-range dependencies, such as language modeling and sentiment analysis.
Generative Adversarial Networks (GAN): GANs consist of two neural networks, a generator and a discriminator, competing against each other. GANs are used for generating synthetic data that resembles the training data distribution, making them useful for tasks like image generation, data augmentation, and style transfer.
Autoencoders: Autoencoders are unsupervised learning models that aim to reconstruct their input data. They consist of an encoder that compresses the input data into a latent representation and a decoder that reconstructs the data from the latent space. Autoencoders find applications in dimensionality reduction, anomaly detection, and image denoising.
These are just a few examples of neural network methods and models. Each model has its strengths and is suited for specific tasks and data types. Researchers and practitioners continue to explore and develop new architectures and variations to address various challenges in different domains. (Poe.com (Data-Scientist-GPT-3)

NOMINAL VARIABLE:

(OS) A nominal variable is a type of categorical variable that represents data in distinct categories or groups. It is also known as a qualitative or discrete variable. In the context of statistics and data analysis, nominal variables
are used to classify data into different categories based on their characteristics, but they do not have any inherent order or numerical value associated with them.

For example, consider a dataset of students’ favorite colors, where the colors are represented as categories like “red,” “blue,” “green,” and “yellow.” In this case, the variable “favorite color” is a nominal variable
because the categories are distinct and there is no inherent order or numerical value assigned to them.

Nominal variables can be represented using labels, symbols, or numbers, but these representations do not carry any inherent meaning in terms of magnitude or order. The categories are purely qualitative and can be unordered
or have an arbitrary order.

When analyzing data with nominal variables, common statistical measures such as counts, frequencies, mode, and chi-square tests are often used. Nominal variables are widely used in various fields, including market research, s
ocial sciences, and categorical data analysis, to understand and categorize qualitative characteristics or attributes.

NON-LINEAR LEAST SQUARES

NON-STATIONARITY:

NON-STOCHASTIC RELATIONSHIP:

NORMAL DISTRIBUTION

NORMAL PROBABILITY PLOTS:

NORMALIZATION

NoSQL

O

OBJECTIVE FUNCTION:

(NS)

The mathematical formula or metric that a model aims to optimize. For example, the objective function for linear regression is usually MSE or Mean Squared Loss. Therefore, when training a linear regression model, training aims to minimize Mean Squared Loss.

In some cases, the goal is to maximize the objective function. For example, if the objective function is accuracy, the goal is to maximize accuracy.

ONE-HOT ENCODING:
(NS) one-hot encoding
#fundamentals
Representing categorical data as a vector in which:

One element is set to 1.
All other elements are set to 0.
One-hot encoding is commonly used to represent strings or identifiers that have a finite set of possible values. For example, suppose a certain categorical feature named Scandinavia has five possible values:

“Denmark”
“Sweden”
“Norway”
“Finland”
“Iceland”
One-hot encoding could represent each of the five values as follows:

country Vector
“Denmark” 1 0 0 0 0
“Sweden” 0 1 0 0 0
“Norway” 0 0 1 0 0
“Finland” 0 0 0 1 0
“Iceland” 0 0 0 0 1
Thanks to one-hot encoding, a model can learn different connections based on each of the five countries. However, one country would have to be left out of regression (and other) models or multicollinearity would result. In such cases, the left out dummy variable would be part of the reference level (included in a base scenario) and coefficient interpretation would be affected.

Representing a feature as numerical data is an alternative to one-hot encoding. Unfortunately, representing the Scandinavian countries numerically is not a good choice. For example, consider the following numeric representation:

“Denmark” is 0
“Sweden” is 1
“Norway” is 2
“Finland” is 3
“Iceland” is 4
With numeric encoding, a model would interpret the raw numbers mathematically and would try to train on those numbers. However, Iceland isn’t actually twice as much (or half as much) of something as Norway, so the model would come to some strange conclusions.

(OS) Dummy variable construction.

OOB:

OPEN SOURCE:

(NS)

ORDERED LOGIT MODEL:

ORDERED PROBIT MODEL:

ORDINAL DATA:

OUTLIERS:

(OS) An outlier is an observation that is many Standard Deviations from the mean or possibly above the 99th percentile or below the first percentile. It is sometimes tempting to discard outliers, but this is imprudent unless the cause of the outlier can be identified, and the outlier is determined to be spurious. Also, if the problem at hand is to examine in the top 1% of income, wealth, or net worth for studies of affluent consumers for example. Otherwise, discarding outliers can cause one to underestimate the true variability of the measurement process. Sometimes, listing the top 50 or bottom 50 observations will allow the data scientist to see a natural break above (or below) which signifies extreme outliers or influential observations. In old-school data science, several methods are used to detect outliers:

Visual Inspection: In old-school data science, one of the simplest methods to detect outliers is through visual inspection of data using scatter plots, box plots, or histograms. Outliers may appear as data points that are significantly distant from the majority of the data.
Z-Score: The Z-score method calculates the number of standard deviations an observation is away from the mean. Observations with a Z-score greater than a certain threshold, usually set as 2 or 3, are considered potential outliers.
Modified Z-Score: The modified Z-score is a variation of the Z-score method that uses the median and median absolute deviation (MAD) instead of the mean and standard deviation. It is more robust to outliers.
Range Rule: The range rule is a simple rule of thumb where observations beyond a certain range from the mean are considered outliers. This range is often defined as mean ± a certain number of standard deviations.

Outlier measures for specific models can also be used. In the context of regression models, there are several statistics and graphs that can be used to evaluate the power and influence of outliers. These techniques help assess how influential outliers are in terms of their impact on the regression model’s coefficients, goodness-of-fit measures, and overall model performance. Here are a few commonly used methods:

Cook’s Distance: Cook’s distance measures the influence of each observation on the regression model’s coefficients. It quantifies how much the model’s predictions change when a particular observation is removed. Larger values of Cook’s distance indicate greater influence of the corresponding data point on the model.
DFFITS: DFFITS measures the influence of each observation on the fitted values or predicted response. It calculates the standardized difference between the predicted response values with and without each observation. DFFITS values larger than a threshold, typically ±2√(p/n), where p is the number of predictors and n is the sample size, are considered influential.
DFBETAS: DFBETAS assess the influence of each observation on the regression coefficients. It calculates the standardized difference between the estimated coefficient values with and without each observation. DFBETAS values larger than a threshold, typically ±2/√n, where n is the sample size, are considered influential.
Residual Plots: Residual plots provide a visual assessment of outliers’ impact by examining the patterns of the model’s residuals. Outliers may appear as data points with large positive or negative residuals, indicating substantial deviations from the predicted values. Common residual plots include scatter plots of standardized residuals against predicted values or against the independent variables.
Influence Plots: Influence plots combine both graphical and statistical measures to assess the influence of individual observations. They often display measures such as Cook’s distance or standardized residuals against the observation index. Points outside a certain threshold are considered influential outliers.

(NS) An outlier is an abnormal value in a dataset that deviates considerably from the rest of the observations. Outliers can be evidence of a measurement error or extraordinary event. Often NS automatic cleansing algorithms will automatically remove the top or bottom percentile or data above (or below) a set number of standard deviations from the mean (clipping). At first glance, this is unwise based on OS practice, but outliers can significantly skew results, especially for certain kinds of models (like regressions) so that coefficient values are undesirably skewed. It can depend on the nature of the problem as when you are interested in the middle class or skewed distributions. It can be too time-consuming to do a physical inspection of outliers in cases where speed-to-result is important or data quality is known to be error-prone. Or, techniques such as using Median-based accuracy measures in forecasting are less sensitive to outliers. (Data Camp Glossary and Gordy Fairchild)

In general, robust statistical models are less sensitive to outliers compared to traditional models. They are designed to handle data with outliers or influential observations without significantly affecting the model estimates. Here are a few models that are known for their robustness to outliers:

Robust Regression: Robust regression methods, such as RANSAC (RANdom SAmple Consensus) and Huber regression, are less influenced by outliers compared to ordinary least squares (OLS) regression. These methods downweight or ignore outliers, resulting in more robust parameter estimates.
Decision Trees: Decision trees are less sensitive to outliers because they partition the data based on feature values and thresholds rather than relying on the overall distribution of the data. Outliers may not strongly influence the tree-building process as long as they are not the majority within a specific split.
Random Forests: Random forests, being an ensemble of decision trees, inherit the robustness of decision trees. They aggregate the predictions of multiple trees, reducing the impact of outliers on the overall model prediction.
Support Vector Machines (SVM): SVMs are generally robust to outliers due to the use of support vectors, which are the data points closest to the decision boundary. Outliers that are far away from the support vectors have less influence on the model.
Quantile Regression: Unlike ordinary least squares regression, quantile regression estimates the conditional quantiles of the response variable. It is less sensitive to outliers because it focuses on estimating different parts of the distribution rather than the mean.

It’s important to note that while these models are considered robust to outliers, extreme outliers can still have some influence on the model. Additionally, the definition of an outlier may vary depending on the context and the specific problem at hand. Preprocessing techniques such as outlier detection or data transformation may be necessary to handle outliers effectively in any model.

Also in new-school data science, certain methods can be used to detect outliers:

Box Plot and Interquartile Range (IQR): Box plots provide a visual representation of the distribution of the data, including the median, quartiles, and potential outliers. Observations outside a certain range, typically defined as 1.5 times the IQR, are considered outliers.
Local Outlier Factor (LOF): LOF is an algorithm that measures the local density deviation of a data point compared to its neighbors. Points with a significantly lower density compared to their neighbors are considered outliers.
Isolation Forest: The isolation forest algorithm isolates outliers by randomly selecting a feature and then randomly selecting a split value between the maximum and minimum values of that feature. Outliers can be identified by the number of splits required to isolate them.
Robust Mahalanobis Distance: Mahalanobis distance measures the distance of an observation from the mean of a multivariate distribution, accounting for the covariance structure. Robust Mahalanobis distance is less sensitive to outliers and can be used to detect them.

In new-school models, such as machine learning algorithms or advanced statistical techniques, there are several outlier detection statistics and graphs that can be used to evaluate the power of outliers on model results. These methods help assess the influence of outliers on model performance, accuracy, and generalization. Here are a few commonly used techniques:

Residual Analysis: Residual analysis is a powerful method to evaluate outliers in new-school models. It involves examining the difference between the predicted and actual values. Outliers may be identified as data points with large residuals, indicating significant deviations from the model’s predictions.
Mahalanobis Distance: Mahalanobis distance measures the distance of each observation from the mean of a multivariate distribution, taking into account the covariance structure. Outliers are identified as data points with large Mahalanobis distances, which indicate they are far away from the central distribution of the data.
Local Outlier Factor (LOF): LOF is a popular outlier detection algorithm that measures the local density deviation of a data point compared to its neighbors. Points with significantly lower density compared to their neighbors are considered outliers. LOF provides a measure of the outlier score for each observation.
Isolation Forest: Isolation Forest is another algorithm used for outlier detection. It isolates outliers by randomly selecting a feature and then randomly selecting a split value within the range of that feature. Outliers are identified as data points that require fewer splits to isolate them. Influence Plots: Influence plots combine graphical and statistical measures to evaluate the influence of individual observations on the model. They often display measures such as Cook’s distance, leverage, or standardized residuals against the observation index. Points outside certain thresholds are considered influential outliers.
Receiver Operating Characteristic (ROC) Curve: In classification problems, ROC curves can be used to evaluate the power of outliers. By plotting the true positive rate against the false positive rate, the ROC curve provides a measure of the model’s performance in detecting outliers. A higher area under the curve (AUC) indicates better outlier detection capability.

These statistics and graphs help assess the power of outliers in new-school models and guide further analysis or model improvement. It’s important to note that outlier detection is a complex task and requires consideration of the specific problem, data characteristics, and modeling assumptions. (U Cal Berkely Statistics Glossat, Gordy Fairchild, poe.com (Data-Scientist-GPT3)

OVERFITTING:
(NS) Overfitting refers to when a model learns too much information from the training set including potential noise and outliers. As a result, it becomes too complex, too conditioned on the particular training set, and fails to adequately perform on unseen data. Overfitting leads to high variance on the bias-variance tradeoff. [Data Camp Glossary]

(OS) Cases where a model fits the data “too well” such as a linear regression for example where we obtain an adjusted R-squared of 0.999. The regression is a stochastic relationship (measured with error) as opposed to a deterministic relationship (no error, R-squared would be 1). Very high R-squareds can be symptoms of multicollinearity or examples of where a model has added too many independent variables.

P

PANEL DATA / LONGITUDINAL DATA:

PARAMETERS:

PARETO PRINCIPAL (80/20 RULE):

PARTIAL AUTOCORRELATION:

(OS) This measure of correlation is used to identify the extent of the relationship between current values of a variable with earlier values of that same variable (values for various time lags) while holding the effects of all other time lags constant. Thus, it is completely analogous to partial correlation but refers to a single variable. (Glossary of Forecast Terms)

PARTIAL AUTOCORRELATION FUNCTIONS (PACFs):

ABOVE GRAPH: From Excellent Kaggle article: https://www.kaggle.com/code/iamleonie/time-series-interpreting-acf-and-pacf

PARTIAL CORRELATION:
(OS) This statistic provides a measure of the association between a forecast variable and one or more explanatory variables when the effect of the relationship with other explanatory variables is held constant. (Glossary of Forecast Terms)

PATTERN RECOGNITION

PEARSON CORRELATION:

PEARSON CORRELATION COEFFICIENTS:

Correlation
Variable	Age	BMI	DiabetesPedigreeFunction	Glucose	Insulin	KNOWN_FEMALES	PATIENT_NUMBER	GLUCOSE_INSULIN_RATIO	Outcome
Age	1	0.0039	0.0886	0.3417	0.218	0.2406	-0.0679	-0.095	0.3315
BMI	0.0039	1	0.1663	0.187	0.194	-0.3124	0.0201	-0.0158	0.2465
DiabetesPedigreeFunction	0.0886	0.1663	1	0.1082	0.0556	-0.0628	-0.042	0.1439	0.2278
Glucose	0.3417	0.187	0.1082	1	0.5414	-0.087	-0.0078	-0.1073	0.5292
Insulin	0.218	0.194	0.0556	0.5414	1	-0.0294	-0.0204	-0.495	0.2573
KNOWN_FEMALES	0.2406	-0.3124	-0.0628	-0.087	-0.0294	1	-0.0215	-0.0775	-0.0591
PATIENT_NUMBER	-0.0679	0.0201	-0.042	-0.0078	-0.0204	-0.0215	1	-0.0595	-0.1565
GLUCOSE_INSULIN_RATIO	-0.095	-0.0158	0.1439	-0.1073	-0.495	-0.0775	-0.0595	1	-0.0432
Outcome	0.3315	0.2465	0.2278	0.5292	0.2573	-0.0591	-0.1565	-0.0432	1

PERIODICITY:

(OS/NS) Periodicity refers to the measurement period of a dataset. It can be monthly (one observation per month), weekly(one per week), daily, hourly, by minute, and real time (by second). The data could be a stock (snapshot in time) or a flow (flow of data during the period).

Often different results may appear for a given model by periodicity. the coefficients for price elasticity or demand functions are known to differ by periodicity since consumers have more ability to change behavior the longer the period is. Daily demand functions will differ according to whether the data are hourly or monthly. Often, special variables such as day-of-week effect, monthly seasonality, weekly events, time-of-day effect, etc. are required to estimate time series or fixed effects models. Periodicity is not just important for time series models, as it may just be a way of aggregating/disaggregating data, or the time value is just ignored. The periodicity cn also affect the trend and cycle components in time series and lagged models. In general, one must be careful when aggregating across periodicities .

POISSON MODEL:

POOLED CROSS-SECTIONAL DATA / CROSS-SECTIONAL DATA:

(OS)

Pooled cross-sectional data refers to a type of data that combines cross-sectional observations from multiple time periods. In pooled cross-sectional data, data is collected at different time points, but each observation still represents a distinct unit. For example, if a survey collects income, age, and education level data from different individuals in different years, and all the data is combined into a single dataset, it would be considered pooled cross-sectional data. Political polls that rely on a different sample of respondents over time is an example.

Cross-sectional data refers to a type of data that is collected at a specific point in time, capturing information from different individuals, entities, or subjects. In cross-sectional data, each observation represents a distinct unit, and the data is collected simultaneously or at the same time point for all units. For example, a survey conducted to collect information about the income, age, and education level of individuals at a specific moment would be considered cross-sectional data.

POSTERIOR DISTRIBUTION

PRECISION:

PRECISION-RECALL CURVE:

(OS)

In an optimal precision vs. recall graph, we aim to find a balance between precision and recall based on the threshold settings of a classifier. The graph is typically plotted with precision on the y-axis and recall on the x-axis.

Ideally, we want a high precision value, indicating a low number of false positives, and a high recall value, indicating a low number of false negatives. This balance ensures that our classifier is both accurate and comprehensive in its predictions.

In the graph, the optimal point lies at the top-right corner, where both precision and recall are maximized. This point represents the best trade-off between precision and recall. However, it’s important to note that the optimal point may vary depending on the specific problem and the associated costs of false positives and false negatives.

By analyzing the precision vs. recall graph, we can determine the threshold setting that achieves the desired balance based on the specific requirements and constraints of the problem at hand.

PREDICTION AND FORECAST ACCURACY:

PREDICTIVE ANALYTICS:

(OS) Predictive analytics involves using statistical algorithms and machine learning techniques to predict future outcomes based on historical data. It aims to forecast what might happen in the future by identifying patterns and relationships in the data. Predictive analytics helps in answering questions like “What is likely to happen?” and “What are the potential outcomes?” (Data-Scientist-GPT3, poe.com AI)

PRESCRIPTIVE ANALYTICS:

(OS) Prescriptive analytics goes a step further than predictive analytics by providing recommendations on what actions to take to achieve a desired outcome. It uses optimization and simulation techniques to suggest the best course of action based on predictive models and business objectives. Prescriptive analytics helps in answering questions like “What should we do?” and “How can we achieve the best possible outcome?” (Data-Scientist-GPT3, poe.com AI)

PRINCIPAL COMPONENT:

PRIOR DISTRIBUTION:

PROBABILITY:

PROBABILITY DENSITY FUNCTION

PROBABILITY LIMIT:

PROBIT MODEL

PROPENSITY SCORES:

PROPORTIONAL HAZARDS MODEL

PROXY VARIABLE

PYTHON:

Open-source and free: Python is an open-source programming language, which
means it is freely available and can be used, modified, and distributed without
any cost in contrast with often expensive proprietary software. Python’s
open-source nature fosters a large and active community, resulting in
continuous improvement, frequent updates, and a vast ecosystem of libraries and
tools.

2. Versatility and flexibility: Python is a general-purpose programming language. Python
can be used for a wide range of tasks beyond data analysis, such as web
development, machine learning, artificial intelligence, and automation.

3. Rich ecosystem: Python has a rich ecosystem of libraries and frameworks
specifically tailored for data science and analytics. Popular libraries like
NumPy, Pandas, and Matplotlib provide powerful tools for data manipulation,
analysis, and visualization. Additionally, Python has robust machine learning
libraries like scikit-learn, TensorFlow, and PyTorch, enabling the development
and deployment of advanced models.

4. Community support: Python has a vibrant and supportive community. Online
forums, communities, and platforms like Stack Overflow, GitHub, and Kaggle are
filled with resources, tutorials, and code examples. The Python community
actively contributes to the development of libraries, shares best practices,
and helps resolve issues. This collaborative atmosphere fosters learning,
innovation, and problem-solving.

5. Integration capabilities: Python integrates smoothly with other technologies
and systems. It can interact with databases, web APIs, cloud services, and
other programming languages. This integration capability allows data scientists
to leverage existing infrastructure, access diverse data sources, and integrate
data science workflows into larger applications or systems.

6. Learning curve: Python has a relatively gentle learning curve compared to many
proprietary software applications. Its syntax is intuitive and readable, making
it easier for beginners to grasp. Additionally, Python’s popularity means that
there are abundant learning resources, tutorials, and documentation available
for individuals looking to get started with data science.

7. Reproducibility and scalability: Python’s code can be easily shared,
version-controlled, and reproduced. By using Python for data science, you can
create scripts and notebooks that document and automate your analysis pipeline.
This enhances collaboration, ensures reproducibility, and facilitates the
sharing of research findings. Furthermore, Python’s scalability allows you to
handle large datasets and perform distributed computing when necessary.

OS: Python does have disadvantages.

1. Performance: Python is an interpreted language, which means
it can be slower compared to compiled languages like C or Fortran. For
computationally intensive tasks or large-scale data analysis, Python may not be
as efficient as other languages.

2. Learning Curve: If you’re new to programming or come from a
non-technical background, Python’s learning curve may be steeper compared to
other statistical software like Stata or SPSS. Understanding the syntax and
concepts of Python, as well as learning to work with libraries such as NumPy,
Pandas, or SciPy, can require some initial effort and time investment.

3. Limited GUI Support: This can be a disadvantage if you prefer a
more user-friendly and interactive interface for your statistical analyses.

4. Availability of Specialized Packages: Python has a rich ecosystem
of libraries for statistical and econometric analysis. However, the
availability and validation testing of certain specialized packages may vary
compared to software specifically designed for econometrics, like EViews or
Gretl. It’s important to evaluate whether the required functionality exists in
the Python ecosystem or if alternative software would be more suitable for your
specific needs.

5. Reproducibility: Python’s flexibility and extensive library
support can sometimes lead to challenges in ensuring reproducibility. Managing
dependencies, version control, and documenting code and analysis steps can be
more involved compared to using dedicated statistical software that often
provides built-in solutions for these issues.

Q

Q-Q PLOTS:

QUALITATIVE DEPENDENT VARIABLE

QUANTITATIVE VARIABLES

R

RANDOM EFFECTS MODEL:
(OS) Random effects, on the other hand, refer to the effects that vary across different levels of the data hierarchy. These effects are assumed to be drawn from a population distribution and are used to capture the variability
between different groups or clusters within the data. Random effects allow for the modeling of individual differences or group-level variability.

RECALL:

(OS) Also known as sensitivity and TPR (True Positive Rate)

RECEIVER OPERATING CHARACTERISTICS CURVE (ROC)

(OS/NS)

RECOMMENDATION ENGINE

REGRESSION:

Regression analysis is a statistical technique used in data science to model the relationship between a dependent variable and one or more independent variables. It is commonly
employed to solve business problems by predicting numerical values or estimating continuous outcomes based on given inputs. There are several types of regression methods, each suited for different scenarios. Here are some common
types of regressions and their roles in addressing business problems:

1.
Linear Regression: Linear regression models the relationship
between a dependent variable and one independent variables by fitting a
linear equation to the data. It is widely used in business settings to
understand the linear association between variables and make predictions or
estimate values. Linear regression can be useful for tasks such as sales
forecasting, price estimation, or demand analysis.

2.
Multiple Regression: Multiple regression extends linear regression
by incorporating multiple independent variables to predict a dependent
variable. It allows businesses to analyze the impact of multiple factors on an
outcome. For example, in marketing, multiple regression can be used to predict
sales based on advertising expenditure, pricing, and other relevant variables.

3.
Polynomial Regression: Polynomial regression fits a curved line to
data by including polynomial terms (e.g., quadratic, cubic) in addition to the
linear terms. It can capture nonlinear relationships between variables and
provide more flexible predictions. Polynomial regression can be useful in
scenarios where the relationship between variables is not strictly linear, such
as in engineering or physical sciences.

4.
Ridge Regression and Lasso Regression: Ridge and Lasso regression
are techniques used for regularization in linear regression models when
multicollinearity (high correlation between independent variables) is present.
They help prevent overfitting by adding a penalty term to the regression
equation. These methods are beneficial when dealing with datasets with high
dimensionality and correlated variables, and they help improve model
performance and interpretability.

5.
Logistic Regression: Logistic regression is used for binary
classification problems, where the dependent variable represents two classes or
categories. It models the probability of an instance belonging to a particular
class. Logistic regression is valuable in business applications such as
customer churn prediction, fraud detection, or determining the likelihood of an
event occurring.

6.
Time Series Regression: Time series regression is employed when
the independent variable(s) are associated with time. It considers the temporal
aspect of the data and allows for forecasting future values based on historical
patterns. Time series regression is useful in various business domains,
including finance, sales forecasting, inventory management, and resource
planning.

These regression techniques, among others, provide a range of tools to analyze relationships, make predictions, and derive insights from data to solve business problems. The
choice of regression method depends on the nature of the problem, the data available, and the specific objectives of the analysis.

REGRESSION SPLINE

REGULARIZATION

REINFORCEMENT LEARNING (RL)
(NS) RL is a stand-alone branch of machine learning (neither supervised nor unsupervised) where an algorithm gradually learns by interacting with an environment. RL makes decisions based on its past experience about which actions can bring it closer to a stated goal. By receiving rewards for correct actions and penalties for wrong ones, the algorithm finds out the optimal strategy to maximize its performance. Examples of RL algorithms include game-playing machine learning systems such as chess engines and video game agents. (Data Camp Data Science Glossary)

RELIABILITY:

RESIDUAL:

RESIDUAL ANALYSIS:

RIDGE REGRESSION

ROC-AUC

ROOT MEAN SQUARED ERROR (RMSE):

(OS) The square root of MSE.

ROOT MEAN SQUARED PERCENTAGE ERROR (RMPSE):

S

SAMPLING:

(NS): In new-school problems, practitioners typically use all the available data to perform many kinds of analyses. A random training sample (often about 70%) of the data is distinct from a random testing sample (often 30%). The selection of the sample is often SRS (Simple Random Sample) although other methods can be used. There is sometimes a third sample called a validation sample. Models are estimated (trained) using training sample data only. The resulting model is then applied to the testing sample to see if there is “overfitting” or how well the model predicts with different data. Overfitting means that the model is so dependent on the training sample that it does not fit the test sample very well. Every single datapoint post-cleanse is “sampled” and used, often at great time and resource cost.

(OS): Old-school practitioners often object to calling 70-30 train & test splits as “random samples” since using all the data is hardly sampling and those proportions are necessarily large for many kinds of problems. In reality, if the objective is to minimize overfitting, statistically-based samples of 5-10% may be superior because certain models yield more robust results when large enough samples are needed for statistical sampling. Meanwhile not using all the data avoids the inclusion of spurious or extremely minor variables that could be flukes and catching these minor issues can make the model not overfit the training data. Additionally, the 70-30 random sample split for train and test is unlikely to significantly differ between the two groups on descriptive statistics. With smaller random samples 5/10% the two samples are more likely to differ statistically so the overfitting criteria are more robust. The statistical approach leverages facts about specialized random samples to save time and cost in training, testing, validating, and evaluating models. Ultimately, the model can be used in production to score all new data in real-time or short turnaround types. As an aid to validation, the model can be analyzed using all the data if needed.

SAMPLING DISTRIBUTION:

(OS)

SAMPLING METHOD:

SAS (STATISTICAL ANALYSIS SYSTEM SOFTWARE SUITE)

SCIENTIFIC METHOD:

(OS) an empirical method for acquiring knowledge that has characterized the development of science since at least the 17th century. The scientific method involves careful observation coupled with rigorous skepticism, because cognitive assumptions can distort the interpretation of the observation. Scientific inquiry includes creating a hypothesis through inductive reasoning, testing it through surveys, transactions, experiments and statistical analysis, and adjusting or discarding the hypothesis based on the results.

Although procedures vary from one field of inquiry to another, the underlying process is often similar. The process in the scientific method involves making conjectures (hypothetical explanations), deriving predictions from the hypotheses as logical consequences, and then carrying out experiments or empirical observations based on those predictions. A hypothesis is a conjecture based on knowledge obtained while seeking answers to the question. The hypothesis might be very specific or it might be broad. Scientists then test hypotheses by conducting experiments or studies. A scientific hypothesis must be falsifiable, implying that it is possible to identify a possible outcome of an experiment or observation that conflicts with predictions deduced from the hypothesis; otherwise, the hypothesis cannot be meaningfully tested.

Though the scientific method is often presented as a fixed sequence of steps, it represents rather a set of general principles. Not all steps take place in every scientific inquiry (nor to the same degree), and they are not always in the same order.

SCREE DIGRAM: (OS) See AGGLOMERATIVE CLUSTERING. Diagram used to find the optimal number of clusters. (NS) Similar to Dendogram.

SEASONAL ADJUSTMENT:

SELECTION BIAS:

SELF-SUPERVISED LEARNING:
(NS) A family of techniques for converting an unsupervised machine learning problem into a supervised machine learning problem by creating surrogate labels from unlabeled examples.

Some Transformer-based models such as BERT use self-supervised learning.

Self-supervised training is a semi-supervised learning approach. (Google Machine Learning Glossary)

SELF-SELECTION

SEMI-PARAMETRIC ESTIMATION:

SEMI-SUPERVISED LEARNING

SENSITIVITY:

[OS]
In classification problems, sensitivity (also known as recall, hit rate, or true positive rate) is a performance metric that measures the ability of a model to correctly identify positive instances from all the actual positive instances in the data.

Sensitivity is calculated as the ratio of true positives (TP) to the sum of true positives and false negatives (FN): Sensitivity = TP / (TP + FN)

In other words, sensitivity quantifies the proportion of actual positive instances that the model correctly identifies as positive. It focuses on the model’s ability to avoid false negatives, which are instances that are actually positive but incorrectly predicted as negative.

A high sensitivity value indicates that the model has a low rate of missing positive instances and is good at capturing the positive class. On the other hand, a low sensitivity value indicates that the model is prone to missing positive instances, and there is a higher rate of false negatives.

Sensitivity is particularly relevant in situations where correctly identifying positive instances is critical and missing positive cases can have significant consequences. For example:

Medical Diagnosis: In medical diagnostic testing, sensitivity is important to detect disease. A high sensitivity means the model is good at correctly identifying patients with a particular illness, minimizing false negatives, and ensuring that patients who need further medical attention are not missed.

Fraud Detection: In fraud detection applications, sensitivity is crucial for identifying fraudulent transactions. A high sensitivity ensures that the model captures a high proportion of actual fraudulent cases, reducing the chances of false negatives and preventing fraudulent activities from going undetected.

Risk Assessment: Sensitivity plays a role in risk assessment scenarios, such as identifying high-risk individuals for insurance or credit applications. A high sensitivity ensures that individuals with a higher risk of default or adverse events are correctly flagged, minimizing the chances of false negatives and
potential losses.

It’s important to note that sensitivity is just one metric for evaluating the performance of a classification model. It should be considered in conjunction with other metrics such as specificity, accuracy, precision, and F1 score to gain a comprehensive understanding of a model’s performance and assess its
suitability for a specific problem domain. [AI: Poe.com]

NS:
Sensitivity is sometimes considered a top metric to use when reviewing competing classification model results instead of the sub-optimal term of accuracy. Certain imputed, dynamic, and exploratory work is performed to create new variables based on ratios, rates of change, and rules of thumb. Because the coefficients of these imputations each have a different numerical range, the meaning of each is different and can be hard to communicate, we often skip covering a third aspect of the situation. Accuracy should generally NOT be the only measure that is used to infer the desirability of one classification model over another. See accuracy and confusion matrix.

SENSITIVITY ANALYSIS:

SENTIMENT ANALYSIS: (NS) Using statistical or machine learning algorithms to determine a group’s overall attitude—positive or negative—toward a service, product, organization, or topic. For example, using natural language understanding, an algorithm could perform sentiment analysis on the textual feedback from a university course to determine the degree to which students generally liked or disliked the course. (Google Machine Learning Glossary)

SERIAL CORRELATION:

SIGNIFICANCE TESTS:

SIMULTANEOUS EQUATIONS:

SKEWNESS:

(OS) if the value is greater than + 1.0, the distribution is right skewed. If the value is less than -1.0, the distribution is left skewed. Values between +1 and -1 suggest a normal curve as long as kurtosis is

SPEARMAN (RANK) CORRELATION:

(OS) the Spearman correlation, also known as Spearman’s rank correlation coefficient, is a measure of the strength and direction of the monotonic relationship between two variables. Unlike the Pearson correlation coefficient, which measures the linear relationship between variables, the Spearman correlation assesses the monotonic association, which includes both linear and nonlinear relationships.

To calculate the Spearman correlation, the ranks of the observations for each variable are used instead of the actual values. The correlation coefficient ranges from -1 to 1, where:

A value of 1 indicates a perfect monotonic increasing relationship, meaning that as one variable increases, the other variable also increases.
A value of -1 indicates a perfect monotonic decreasing relationship, meaning that as one variable increases, the other variable decreases.
A value of 0 indicates no monotonic relationship between the variables.
The Spearman correlation is less sensitive to outliers and can capture relationships that may not be linear. It is particularly useful when the relationship between variables is not well approximated by a straight line.

SPECIFICATION ERRORS:

(OS) Specification errors can occur in several ways:

Model misspecification refers to cases where the wrong model for the problem at hand is used:

Such as using the wrong regression model for panel data such as linear regression when one should use a non-linear model if the independent variables are intrinsically non-linear.
Use of over-identified 2SLS when endogeneity is present. The model will be incorrect.
Use of non-stationary models when stationarity is present in a time series model.
Use of models to test hypotheses or theories that do not use known theoretical relationships such as a demand or supply model with no price.
Omitted variable problem. Variables that should be in the model that are not included resulting in sub-optimal results.
Use of linear or other models that fail to correct for heteroscedasticity or autocorrelation.

These are just a few (OS) examples. (Gordy Fairchild)

(NS) Some newer models in the field of data science and machine learning can still be susceptible to specification errors. Here are a few examples of “new-school” models that may be prone to such errors:

Deep learning models: Deep learning models, such as deep neural networks (DNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs), have gained popularity for their ability to learn complex patterns and relationships in data. However, these models can still suffer from specification errors if the network architecture is not appropriately designed or if important features or interactions are not adequately captured. For instance, if the depth or width of the network is not sufficient to capture the complexity of the underlying data, the model may exhibit poor performance or fail to generalize well.
Generative adversarial networks (GANs): GANs are a class of deep learning models used for generating synthetic data that closely resembles a given training dataset. Specification errors can occur in GANs if the generator or discriminator networks are not properly specified, leading to unrealistic or biased synthetic data generation. For example, if the generator network is too simplistic or if the discriminator network is not able to effectively distinguish between real and synthetic data, the GAN may produce inaccurate or low-quality synthetic samples.
Reinforcement learning (RL) models: RL models learn optimal decision-making policies through interaction with an environment. Specification errors can arise in RL models if the environment dynamics or reward functions are mis-specified. For instance, if the reward function does not accurately reflect the true objective or if important state variables are omitted from the model, the RL agent may learn suboptimal policies or exhibit unexpected behavior.
Bayesian models: Bayesian models, such as Bayesian networks or hierarchical Bayesian models, incorporate prior knowledge and uncertainty into the modeling process. Specification errors can occur in these models if the prior distributions or conditional dependencies among variables are incorrectly specified. For example, if the prior distributions do not align with the true beliefs about the variables or if the conditional dependencies do not capture the true causal relationships, the Bayesian model may produce biased or unreliable results.
AutoML models: AutoML frameworks, such as TPOT or AutoSklearn, automate the model selection and hyperparameter tuning process. While these tools aim to find the best model and configuration automatically, specification errors can still occur if the available models or search space do not align with the characteristics of the data or if the evaluation metrics are not appropriate for the task at hand. It is important to carefully validate and interpret the results generated by AutoML tools to avoid specification errors. (Poe.com Data-Science GPT 3)

SPECTRAL ANALYSIS

SPLINE THEORY:

STANDARD DEVIATION

STANDARD ERROR

STATIONARITY

STEEPEST ASCENT:

STEPWISE REGRESSION:

STOCHASTIC GRADIENT DESCENT

STOCHASTIC RELATIONSHIP:

(OS/NS) A stochastic relationship, on the other hand, is a relationship that is not completely predictable or deterministic. It involves random or probabilistic elements that introduce uncertainty into the relationship. In this type of relationship, the value of the dependent variable is influenced by both the values of the independent variables and random factors. The relationship is described in terms of probabilities or statistical distributions. For example, in a time series analysis, the relationship between variables may exhibit a stochastic component due to random variations or noise that cannot be explained by the independent variables alone. (poe.com, Data-Scientist-GPT3)

STRUCTURED LANGUAGE

STRUCTURED QUERY LANGUAGE (SQL):

STRUCTURED DATA

SUPERVISED LEARNING

SVM (SUPPORT VECTOR MACHINE)

T

Tensor:

The primary data structure in TensorFlow programs. Tensors are N-dimensional (where N could be very large) data structures, most commonly scalars, vectors, or matrices. The elements of a Tensor can hold integer, floating-point, or string values.

(Google Machine Learning Glossary)

T-TEST:

TAYLOR SERIES EXPANSION:

TIME SERIES:

TIME SERIES REGRESSION:

TOBIT MODEL:

TRANSFORMATION:

TREND: (OS)

(NS)

TREND STATIONARITY

TRUE NEGATIVE

TRUE POSITIVE

TRUNCATED SAMPLE (VARIABLE):

(OS) Not the same as censored sample (variable). A truncated sample and a censored sample are both types of incomplete data, but they differ in how the data is handled and the implications for statistical analysis.

A truncated sample occurs when data points are missing because they fall outside a specific range or are systematically excluded. In other words, the sample is “truncated” at certain values. For example, if we are studying the heights of individuals and only include individuals taller than 170 cm in our sample, we have a truncated sample. Truncation affects both the observed values and the underlying distribution, as we only have information for a subset of the population.

On the other hand, a censored sample occurs when data points are recorded but with some values either partially or entirely unknown. This typically happens when the measurements are subject to a detection limit or a certain threshold beyond which values cannot be accurately measured. For example, in a study on response times, if any response time exceeding a certain maximum value is recorded as “> 10 seconds,” we have a censored sample. Censoring affects the observed values but does not impact the underlying distribution beyond the censoring threshold.

The main difference between a truncated sample and a censored sample lies in the treatment of the missing data. In a truncated sample, the missing data is completely excluded, and the analysis is based only on the available data within the specified range. In contrast, in a censored sample, the missing data is retained but with some values replaced or adjusted to reflect the censoring.

Statistical analysis methods for truncated and censored samples differ as well. Truncated data requires specialized techniques that account for the truncation point, such as maximum likelihood estimation with truncated distributions. Censored data, on the other hand, often involves survival analysis or methods that handle right-censored or left-censored data, such as Kaplan-Meier estimation or parametric survival models.

In summary, a truncated sample involves missing data points that are excluded because they fall outside a specific range, while a censored sample involves recorded data points with values beyond a certain threshold replaced or adjusted. The treatment and analysis of these types of incomplete data differ, and appropriate statistical methods are applied accordingly. (Data Scientist GPT 3 AI, Poe.com)

T-TEST

TWO-STAGE LEAST SQUARES

TYPE 1 ERROR:

(OS) Type I Error – the rejection of a true null hypothesis (https://www.investopedia.com/terms/t/type-i-error.asp)

TYPE 2 ERROR:

(OS) Type II Error – refers to the non-rejection of a false null hypothesis. It is used within the context of hypothesis testing. (https://www.investopedia.com/terms/t/type-ii-error.asp)

TYPE 1 SUMS OF SQUARES (sequential or incremental SS):
(OS) In linear regression, ANOVA, and GLM Type I sums of squares are determined by considering each source (factor) sequentially, in the order they are listed in the model. The Type I SS may not be particularly useful for analyses of unbalanced, multi-way structures but may be useful for balanced data and nested models. Type I SS are also useful for parsimonious polynomial models (i.e. regressions), allowing the simpler components (e.g. linear) to explain as much variation as possible before resorting to models of higher complexity (e.g. quadratic, cubic, etc.). Also, comparing Type I and other types of sums of squares provides some information regarding the magnitude of imbalance in the data. Types II and III SS are also known as partial sums of squares, in which each effect is adjusted for other effects.

TYPE 3 SUMS OF SQUARES:

Type III is also a partial SS approach, but it’s a little easier to explain than Type II; so we’ll start here. In this model, every effect is adjusted for all other effects. The Type III SS will produce the same SS as a Type I SS for a data set in which the missing data are replaced by the least-squares estimates of the values. The Type III SS corresponds to Yates’ weighted squares of means analysis. One use of this SS is in situations that require a comparison of main effects even in the presence of interactions (something the Type II SS does not do and something, incidentally, that many statisticians say should not be done anyway!). In particular, the main effects A and B are adjusted for the interaction A*B, as long as all these terms are in the model. If the model contains only main effects, then you will find that the Type II and Type III analyses are the same.

TYPE 2 SUMS OF SQUARES:
Type II partial SS is a little more difficult to understand. Generally, the Type II SS for an effect U, which may be a main effect or interaction, is adjusted for an effect V if and only if V does not contain U. Specifically, for a two-factor structure with interaction, the main effects A and B are not adjusted for the AB interaction because the interaction contains both A and B. Factor A is adjusted for B because the symbol B does not contain A. Similarly, B is adjusted for A. Finally, the AB interaction is adjusted for each of the two main effects because neither main effect contains both A and B. Put another way, the Type II SS are adjusted for all factors that do not contain the complete set of letters in the effect. In some ways, you could think of it as a sequential, partial SS; in that it allows lower-order terms to explain as much variation as possible, adjusting for one another, before letting higher-order terms take a crack at it.

TYPE 4 SUMS OF SQUARES:
The Type IV functions were designed primarily for situations where there are empty cells, also known as “radical” data loss. The principles underlying the Type IV sums of squares are quite involved and can be discussed only in a framework using the general construction of estimable functions. It should be noted that the Type IV functions are not necessarily unique when there are empty cells but are identical to those provided by Type III when there are no empty cells.

U

UNBIASED MEDIAN ABSOLUTE DEVIATION (UMdAD):

UNDERFITTING

UNIT ROOT TEST

UNLABELED DATA:

UNSTRUCTURED DATA:

(NS) Data that is not part of a standard file of numeric and short text. Unstructured data includes video, images, audio, long text fields, and network architecture.

UNSUPERVISED LEARNING:

(NS) Unsupervised learning is a class of machine learning algorithms that learn the underlying structure of a dataset without being provided a target variable. Unsupervised learning is used to discover common patterns in data, group the values based on their attributes, and then
later make predictions on unseen data. The most common unsupervised learning algorithm is k-means. Examples of common tasks are anomaly detection and customer segmentation based on common characteristics. (Data Camp Glossary)

V

VARIABLE:

(OS): Variables are the columns in a traditional structured dataset or relational database. Each row is an observation or a case. They are sometimes described by their use in statistical techniques as independent variables (for variables on the right-hand side of an equation of interest) or the dependent variable(s) on the left-hand side of an equation or a series of equations. They can be numeric or character. Typically OS uses a shorter width for character variables than NS since data are structured.

NS: features, characteristics. These may include unstructured, semi-structured, and structured data elements.

VARIABLE IMPORTANCES:

(NS) A set of scores that indicates the relative importance of each feature to the model. For example, consider a decision tree that estimates house prices. Suppose this decision tree uses three features: size, age, and style. If a set of variable importances for the three features are calculated to be {size=5.8, age=2.5, style=4.7}, then size is more important to the decision tree than age or style. Different variable importance metrics exist, which can inform ML experts about different aspects of models.
(OS) In regressions, the variable importances can be seen directly in linear coefficients, standardized regression coefficients, odds ratios, etc. However, some new-school modeling techniques use algorithms to generate metric magnitudes that allow ranking of independent variables by importance and other types of approximations in a broad body of other models. Importantly, data scientists of the new school variety may not be interested in why, they merely want a good forecast or are using signal-based models that don’t require importances.

VARIANCE:

(OS) Variance is used to measure the spread of a given set of numbers and is calculated by the average of squared distances from the mean. (Analytics Vidhya Glossary)

VARIANCE-COVARIANCE MATRIX

VARIANCE INFLATION FACTORS (VIF):

VECTOR AUTOREGRESSION (VAR):

VENN DIAGRAM:

(OS) A Venn Diagram stems from set theory and is used to depict how sets of objects, people, models, and other relationships fit together. The Venn diagram below shows raw data (oil) emerging through the data science process into combustion (data-driven analytics, inference, and prediction). Note that it assumes data science does not include the entire sets of domain knowledge, math/physics, or Computer Science. Data Science is shown to include some (but not all) math, domain knowledge, and computer science. Data science is defined by the diagram as needing to include portions of all 3. There are overlaps (intersections) between domain knowledge and math, between math and computer science and between domain knowledge and computer science that DO NOT include data science. There are also parts of math, computer science, and domain knowledge that are independent of any other concept (set) shown. Venn diagrams are often used to show and quantify how separate datasets are merged (joined) as well as what portion of the data exists alone (disjoint) or with 1 other dataset. Venn diagrams can be greater than 3D but are harder to illustrate.

W

WEB SCRAPING:

(NS) Web scraping is the process of extracting specific data from websites for further usage. Web scraping can be done automatically by writing a program to capture the necessary information from a website. [Data CAMP Glossary]

WHITE NOISE:

(OS/NS) When there is no pattern whatsoever in a data series, it is said to represent white noise or randomness. The concept is often used to relate a model’s prediction to reality. For a deterministic relationship, there is no white noise as Y is identically equal to a+b+c. For a stochastic relationship, models do not precisely get all predictions right, but a desirable model is Y=a+b+c+error. The error term should be white noise.

White noise can be diagnosed graphically using a few different methods. One common approach is to plot the data and examine its characteristics. Here are a few graphical techniques for diagnosing white noise:

Time Series Plot: Plotting the time series data itself can provide insights into whether it exhibits characteristics of white noise. A white noise series will appear as a random, erratic pattern with no discernible trends or patterns.
Histogram: Creating a histogram of the data can help assess whether it follows a normal distribution. In white noise, the histogram should show a relatively even distribution with no prominent peaks or clusters.
Autocorrelation Function (ACF) Plot: The ACF plot shows the correlation between a time series and its lagged values. In white noise, there should be no significant correlation at any lag. The ACF plot should exhibit a pattern of values close to zero for all lags.
Partial Autocorrelation Function (PACF) Plot: The PACF plot shows the correlation between a time series and its lagged values, while removing the effects of intermediate lags. In white noise, the PACF plot should also show no significant correlation at any lag, with values close to zero.
QQ Plot: A Quantile-Quantile (QQ) plot compares the quantiles of the observed data against the quantiles of a theoretical distribution (e.g., normal distribution). In white noise, the QQ plot should show the data points closely following the diagonal line.

These graphical techniques can provide visual evidence of whether the data exhibits white noise characteristics. However, it’s important to note that these methods are not definitive and should be used in conjunction with statistical tests for a more comprehensive analysis.

X

X-11/X-12 ARIMA / X-13 ARIMA SEATS SEASONAL ADJUSTMENT PROGRAMS:

(OS)Methods developed by the US Census to deseasonalize data. According to the Census Bureau:

(NS) X-13ARIMA-SEATS is a seasonal adjustment software program developed and maintained at the U.S. Census Bureau. The program is an expansion of the Census Bureau’s earlier X-12-ARIMA program, which itself was an expansion of the initial X-11 software from the Census Bureau and X-11-ARIMA from Statistics Canada under Estela Dagum.

The X-13ARIMA-SEATS software allows for the same adjustment methods as X-12-ARIMA and offers improvements in diagnostics as well as an enhanced version of the Bank of Spain’s SEATS software. The SEATS routines are the result of collaboration with the developers of the software (Agustin Maravall, Former Chief Economist of the Bank of Spain, now retired, and Gianluca Caporello).

Improvements in X-13ARIMA-SEATS as compared to X-12-ARIMA include

additional regressors for modeling calendar effects in stock (inventory) time series
built-in regressors for new outlier types, including seasonal outliers, quadratic ramps, and temporary level shifts
the ability to designate groups of user-defined holiday regressors and generate model diagnostics for the different groups
regression model-based F tests for stable seasonal and trading day regressors
accessible HTML output generated directly by the software rather than by a separate utility.

Y

Z

ZERO INTERCEPT MODEL:

(OS) A regression model that assumes the relationship between the dependent variable and the independent variables passes through the origin (0,0 in a graph). They are used sometimes because in physics or chemistry, nothing exists if nothing is used to create a reaction, or to simplify the interpretation of coefficients, or because the model predicts better without an intercept.

Z-SCORE NORMALIZATION:

A scaling technique that replaces a raw feature value with a floating-point value representing the number of standard deviations from that feature’s mean. For example, consider a feature whose mean is 800 and whose standard deviation is 100. The following table shows how Z-score normalization would map the raw value to its Z-score:

Raw value	Z-score
800	0
950	+1.5
575	-2.25

The machine learning model then trains on the Z-scores for that feature instead of on the raw values. (Google Machine Learning Glossary)

Z_TEST:

(OS) An hypothesis test based on approximating the probability histogram of the Z statistic under the null hypothesis by the normal curve. (U Cal Berkeley Statistics Glossary)

GLOSSARY LINKS:

Google Machine Learning Glossary

https://developers.google.com/machine-learning/glossary

DataCamp Glossary

https://www.datacamp.com/blog/data-science-glossary

University of Iowa Data Glossary

https://www.lib.uiowa.edu/data/glossary/

National Library of Medicine Network Data Glossary

https://www.nnlm.gov/guides/data-glossary

University of Oregon AI glossary
https://blogs.uoregon.edu/artificialintelligence/ai-glossary/

Glossary of Statistics, U Cal Berkeley
https://www.stat.berkeley.edu/~stark/SticiGui/Text/gloss.htm

Glossary of Forecasting Terms, Rob J. Hyndman, Monash University