Nonlinear Regression

A mathematical model that fits an equation to certain data using a generated line that is not straight

Nonlinear regression, also known as a non-linear model, is an approach to statistical modeling that uses a non-straight line to explain data instead of a straight line.

Nonlinear Regression

This is in opposition to linear regression analysis, which uses a straight-line equation (such as Ỵ= ax + b). Both of these models attempt to measure the relationship of a variable (x) that is dependent on the value of another variable (y).

Nonlinear regression is useful because it is much more flexible than linear regression. Often, the curve can produce more accurate results for your data and display association between variables more effectively.

The goal of non-linear regression analysis is to find the model that minimizes the sum of the squares. The sum of the squares measures the accuracy of the association between the data presented and the regression equation.

How to Model Nonlinear Regression

Nonlinear regressions may use several different equations to fit a dataset. This includes logarithmic functions, exponential functions, square root functions, trigonometric functions, and power functions.

Spreadsheet

Estimating the correct equation is done through a series of iterations that work to get slowly closer to the ideal model for the data, which is measured by the smallest sum of squares. A simple equation for modeling a non-linear regression curve is as follows:

Y = f(X, 𝜷) + 𝝐

where,

  • X = A vector of P predictors
  • 𝜷 = A vector of K parameters
  • f = A known regression function
  • 𝝐 = Error term

Non-linear regression models may be described as a curve that models a vector of parameters multiplied by an independent variable where the graph is nonlinear in the parameters. A parameter is a statistical term referring to the distribution of a particular characteristic of an entire population.

True parameters are often difficult to obtain. Therefore, they are commonly estimated by using a smaller population and making an inference about the whole population, such as the mean, median, or mode.

Limitations of Non-linear Regression

Nonlinear regression has some limitations. While it is beneficial for predicting the relationship between variables, it is not a perfect predictor. The non-linear regression model is fairly versatile considering it accommodates non-linear relationships in data, but it still may produce predictions that aren't reliable.

Limitations

Of course, this is exceptionally prominent if the relationship between the two variables changes, or it may not take into account other variables that can influence the relationship between the two measured variables. While non-linear regression is exceptionally useful for creating forecasts and interpreting correlations, it is very common that past relationships cannot perfectly predict the future.

For example, when the trend in the data is not linear, then the non-linear regression will not work as well. In particular, the curve generated by nonlinear regression may become too steep for certain data sets and thus give unrealistic predictions.

How to Calculate the Sum of Squares

The sum of squares is found by finding the difference between the Y-value of each data point and the Y-value of the mean, squaring the differences, and taking the sum of all the squares of the differences. Hence the name, sum of squares.

Calculator

The sum of squares is a value that is meant to demonstrate the accuracy of a fit curve to a set of data. The differences are squared to get the positive values for each difference. The smaller the differences between data points and the fitted model, the smaller the sum of squares will be. This means that the lower the sum of squares, the more the model fits, or the better it is at predicting the relationship between two variables.

Nonlinear models can be a double-edged sword. While this type of regression is often better suited for modeling relationships between variables because it is more flexible, it can also make it challenging to find the model that best fits the data. 

Often nonlinear regressions are found through a series of iterations made through trial and error. Two commonly used methods are the Levenberg-Marquardt and Gauss-Newton methods.

Applications of Nonlinear Regression

Nonlinear regression is used to analyze data where the relationship between variables is not linear. It is most often used when a line does not accurately represent the data, for example, when the relationship between two variables is concave or convex. Nonlinear regression has many applications in various fields, including economics, engineering, science, and many other fields.

The economic theory of nonlinear regression has been extensively studied by economists such as Tinbergen and Samuelson, and it has been applied to a wide variety of problems both in macro and microanalysis. In finance, nonlinear regression may be used in a number of ways. 

Often it is used to forecast future prices. One form this takes is time-series models, which are commonly used to track the price of an asset or basket of assets over time, usually through certain economic conditions. However, these models typically do not take on a linear trajectory and are therefore graphed using nonlinear regression.

Stock data

Non-linear regression is important in engineering applications that involve dynamic systems with time-varying parameters such as fluid dynamics and control engineering, both of which are domains where the nonlinearity can come from viscosity or other sources of dissipation in the system.

This type of regression has a range of uses, but the most common is predicting or forecasting future events from past data. For example, the equation can help predict a stock price in two weeks, how many people will buy this pair of shoes next month, and more. It also helps determine the dependence between variables that may be related in non-linear ways.

For example, it can help find how much money a business needs to spend on marketing campaigns instead of how many new customers they need to generate.

Example of Non-linear Regression

It may be easier to understand nonlinear regression curves with an example. So let's look at a simple example of data that is best modeled with a curve rather than a line: the height of a plant over time.

Regression Graph

Plants do not grow at a constant rate. In this graph, the dependent variable is the height of the plant, and the independent variable is time or the plant's age. As one can see, time progresses at a constant rate.

On the other hand, the height of the plant is not constant. The plant grows a different amount every week and tends to grow more over time. This shows that this graph is nonlinear in the parameter, creating a non-linear curve.

If we accurately understand the relationship in such an instance, it would be unwise to model it using linear regression. As a result, a simple rule of thumb ensures that the model used for predicting the values is based on the underlying relationship.

Machine Learning Package Course

Everything You Need To Master Algo Trading using Python

To Help you Thrive in One of the Most Future Proof Careers on Wall Street.

Learn More

Free Resources

To continue learning and advancing your career, check out these additional helpful WSO resources: