# Introduction

Prediction, modeling and trading is becoming a more complicated challenge because in all these steps a large number of factors intervene that directly influence the behavior of the market. Therefore, there is a desire to continue developing new computational techniques, that help investors to enter the market with more guarantees. Thus, technical indicators are chart analysis tools that can help trades market operators to understand and act on the movements of the future evolution of the prices of a financial asset. So in this article we will present the Moving Average Indicator and its importance for trend analysis [1].

The goal when using indicators is to identify trading opportunities. Applying the moving average strategy to a price chart allows traders to identify areas where the trend changes the direction creating a potential trading opportunity [2].

# Technical Indicators

A technical indicator is a mathematical calculation based on a series of data of the historical price or volume of the market (previous quotes) with the aim of producing another series of data, that will be analyzed to establish trading rules or simply to observe trends. There are four types of technical indicators used in the market and they are the following [1]:

- Oscillators: compare prices over time. Normally, the closing prices of a session are compared with the closing prices of previous sessions. Based on this concept, a multitude of trading rules can be established. Oscillators are often used when there is a lateral trend in the market, that is, there is no upward or downward direction. For this, the indicator is presented between an upper and a lower level, in such a way that when any of them is exceeded, it can be indicative of a buy or sell signal. Examples of oscillators are the Momentum, Relative Strength Index (RSI or Relative Strength Index) and Stochastic indicators, among many others [3,6].
- Volatility indicators: as their name suggests, they are a measure of the amount and speed at which prices change, regardless of direction. These indicators provide the information to know how long it may take for a security to change direction in order to establish the right moment in which to carry out an operation. Some indicators of this type are Bollinger Bands, Keltner Bands, among others [1,3].
- Volume indicators: show for each moment the amount of money moved for a specific value, that is, the sum of money from purchases and sales. These indicators show the health of a security, since the more volume there is, it means that the greater the confidence to operate with that security. Although this does not always mean that it is a good time to trade, there will lie the ability of the trader together with the use of other indicators to determine whether to enter or exit the market. Some examples of indicators of this type are Volume on Balance or Accumulation / Distribution [3].
- Trend indicators: try to identify trends and determine their strength. These indicators analyze prices over time to determine averages. They allow you to obtain graphs that smooth the price peaks when dealing with averages for different periods, in such a way that locating trends is easier. Indicators of this type are: Moving Average (MA or Moving Averages), Moving Average Convergence Divergence (MACD or Convergent Divergent Moving Average) and Average Directional Index (ADX or Directional Average Index) that serves to measure the strength of a trend [3].

# Moving Average Trend Indicator

A moving average, also called as rolling average or running average is a used to analyze the time-series data by calculating a series of averages of the different subsets of full dataset. So, a moving average (MA) is a technical tool that provides an average of the price over a specific period of time. They are used to filter out market noise and make sense of the trend by eliminating minor movements that would mask what the market is actually doing. Its use is for the detection of trends but they are also useful for the analysis of the evolution of the price in different periods of time. This time-lagged indicator follows price action, eliminates market noise, and smoothes prices to more easily determine the direction of the asset under study.

Thus, they are nothing more than the moving average, and to calculate it, the last periods that we have parameterized for its calculation are taken, for example a moving average of 20 periods, the last 20 data from the closing of the price are taken and its half. Its representation is a line on the chart (See Figure 1), the noise is more filtered, the movements of the (MA) are smoother than the next price movements. The (MA) is always delayed, which is the result of performing a calculation on the prices of the past, they do not predict only a more clear summary of what has happened, the usefulness is that they help us detect trends and when there are trends it is more likely that this I continued, the shorter the (MA) calculation period, the less delay it will have but it will contain much more noise than the (MA) that take a longer period, of course but take longer to react to recent market changes. There are several moving averages, but the most popular are simple and exponential.

<img>
**Figure 1: Moving Average of 20 and 50 Period**

# Simple Moving Average:

All data for the period are weighted equally, have the same weight and are given the same importance as the first day as the last, of which an average is simply taken. Act more slowly to adapt to the most recent price changes.

The formula for calculating the SMA is straightforward [2]:

The simple moving average = (sum of the an asset price over the past n periods) / (number of periods)

Where:

So, in this chart (See figure 2) uses two moving averages, a faster moving average(short-term)) and a slower (long-term) moving average. The faster moving average 20 day period while the slower moving average can be 50 day period.

A short term moving average is faster because it only considers prices over short period of time and is thus more reactive to daily price changes. On the other hand, a long term) moving average is deemed slower as it encapsulates prices over a longer period and is more lethargic [2]. A moving average, as a line by itself, is often overlaid in price charts to indicate price trends. A crossover occurs when a faster moving average (i.e. a shorter period moving average crosses) a slower moving average (i.e. a longer period moving average). In stock trading, this meeting point can be used as a potential indicator to buy or sell an asset.

When the short term moving average crosses above the long term) moving average(media), this indicates a buy signal [2].

Contrary, when the short term moving average crosses) below) the long term moving average, it may be a good moment to sell [2].

<img>

**Figure 2: Simple Moving Average of 20 and 50 Periods**

# Exponential Moving Average:

An additional value is taken to the selected period, that is, for an (EMA) of 20 periods, the last 21 are considered, the new price is weighted in% much more while the others weight everything equally (See figure two).

So, (EMA) give more weight to the most recent periods. This makes them more reliable than (SMAs) as they are comparatively better representation of the recent performance of the asset [2].

The EMA is calculated as [2]:

Where:

So, in this chart (See figure 2) uses two moving averages, a faster moving average (short-term) and a slower (long-term) moving average. The faster moving average 20 day period while the slower moving average can be 50 day period.

**Figure 3: Exponential Moving Average of 20 and 50 Periods**

So, must choose the one that best fits on the chart depending on which best fits what you want to see, it is worth knowing that a simple moving average is weaker than an exponential average, the only thing that matters in the adjustment of a moving average is that you like how it looks on your particular chart. To incorporate (MA) into trading strategies, it is important to indicate that they act as Dynamic Support and Resistances, that is, when the price is in a trend, it behaves as a trend line marking the guideline, it also has the logical quality of attracting the placed price. They are still an average of the price and by the statistical principle that everything returns to its average in a moment the price always tends to approach any moving average that has been plotted on the chart. These two properties can be used in our trading strategy. Another utility of the (MA) is that they use them to detect the extreme of the price, since all the data of a distribution tend to cluster around its mean, if a strong impulse moves the price away from the (MA) at some point this the price will return to its mean This will help us to detect the end of the market. That is, when the price is far from its average and the trend may be ready to make a correction. To adjust a (MA) and parameterize its timing in trading, you need to understand is that every (MA) is still a line chart of price in a higher timing and this is where its usefulness lies. However, a 5 period is equivalent to the line graph of multiplying that temporality * 5, that is, if we are in one minute it is equivalent to the 5 minute line graph, if it were 60 periods it would be the one hour line graph. So to say that (MA) you need you must say in what time frame your operations will be or rather what time context you have your operations in.

# Conclusions

The Moving Averages are the simplest indicators to use but it is important to know their limitations and their utilities, it will help us to maintain the context of the trend of the different time frames, in addition to detecting the extremes of the market and when it is ready to correct for be overbought or oversold. Let us remember that no (MA) or parameter is better than another and no matter how many (MA) you add to the graph, you will never be able to attenuate or completely eliminate its limitations, it is best to choose a simple (MA) with some simple parameters, in such a way as to incorporate them into our own systems of trading.

**References**

- CobuildLab 2021. Technical Indicators for Trading.
- Generating Trade Signals using Moving Average (MA) Crossover Strategy. Towardsdatascience 2020.
- A.-M. Baiynd, «The Trading Book: A Complete Solution to Mastering Technical Systems and Trading Psychology,» McGraw-Hill, p. 272.
- I. Staff. Available: HYPERLINK ["http://www.investopedia.com/articles/technical/070301.asp"](http://www.investopedia.com/articles/technical/070301.asp)

Technical indicators are chart analysis tools that can help currency market operators to understand and act on price movements, that is a stock analysis tool that serves to study the future evolution of the prices of an asset financial. Therefore, they help to interpret whether the prices of an asset are in a situation of overbought or oversold in order to make the right investment decision.

Technical Indicators Specifically Moving Average and Its Use for Financial Market Analysis

# Introduction

The stochastic oscillator was developed in the late 1950s by George Lane. As designed by Lane,
the stochastic oscillator presents the location of the closing price of a stock in relation to the
high and low range of the price of a stock over a period of time, typically a 14-day period. Lane,
over the course of numerous interviews, has said that the stochastic oscillator does not follow
price or volume or anything similar. He indicates that the oscillator follows the speed or
momentum of Price.

# The Stochastic Oscillator

A stochastic oscillator is a momentum indicator comparing a particular closing price of a security
to a range of its prices over a certain period of time. The sensitivity of the oscillator to market
movements is reducible by adjusting that time period or by taking a moving average of the
result. It is used to generate overbought and oversold trading signals, utilizing a 0–100 bounded
range of values [1].

Stochastics have a different approach than RSI [2] and other indicators to calculate price swings.
This indicator reports how far the current price is from the low of the last X periods. The line
created in the stochastic representation (% K), is then used to create a moving average (% D),
which is located on the same chart as the first line. The stochastic is charted like all oscillators,
just below the price chart [3] (See Figure 1).

The Formula for the Stochastic Oscillator is:


Where:

Notably, %K is referred to sometimes as the fast stochastic indicator. The "slow" stochastic
indicator is taken as %D = 3-period moving average of %K. The general theory serving as the
foundation for this indicator is that in a market trending upward, prices will close near the high,
and in a market trending downward, prices close near the low. Transaction signals are created
when the %K crosses through a three-period moving average, which is called the %D [3].

The difference between the slow and fast Stochastic Oscillator is the Slow %K incorporates a %K
slowing period of 3 that controls the internal smoothing of %K. Setting the smoothing period to
1 is equivalent to plotting the Fast Stochastic Oscillator.1 The use of slow stochastic has the
advantage that it smoothes it out and that is, as the formula is constructed, the stochastic can
offer a quite serrated appearance with somewhat unexpected sudden movements if something
is smoothed with a small moving average [3].

Figure 2: Stochastic Oscillator Slow and Fast

Like all normalized oscillators, the stochastic is also interpreted taking into account its
overbought and oversold levels in this case it is generally considered as overbought if the
stochastic is above the 80 level and it is considered oversold if it is below the 20 level, it is
important to consider these levels as approximate not as absolute if you remember the video of
the oscillators that all suffer a displacement effect therefore the upward trends perhaps the
level 80 as overbought is too low and it is convenient that it is higher perhaps the level 85 or
even higher, the same happens with downtrends, the 20th may be too high a level so you
should investigate and test different levels and not interpret them in isolation but in conjunction
with the Price [4].

Like many oscillators and indicators, the generic parameter stochastic is 14 periods, as a general
rule, it will be valid for any time period, but if the type of analysis performed on the charts
requires analysis in several time periods, it is from the highest to the lowest, you may be
interested in playing. Remember there is no better parameter than another, only one more
adapted to your style of trading and how to interpret what is happening, test different
parameters and if you have doubts, just stick with 14 [4].

# Conclusions

The Stochastic measures the closing level of the last period compared to a previous period,
generally the 14 periods 100% is the maximum of the previous period and 0% the minimum
reminds that this oscillator is used looking for divergences, displacements, crossing of the line
mean, etc [4].

It is important to indicate that the primary limitation of the stochastic oscillator is that it has
been known to produce false signals. This is when a trading signal is generated by the indicator,
yet the price does not actually follow through, which can end up as a losing trade. During
volatile market conditions, this can happen quite regularly. One way to help with this is to take
the price trend as a filter, where signals are only taken if they are in the same direction as the
trend [1].

![test](/static/8249acd153af68a9812a9940c5aa9b12/4aAFYHcgSLSirIDVXzcM.png)


**References**

- Investopedia 2020. Stochastic Oscillator.
- CobuildLab 2021. Technical Specifically RSI and Its Use for Financial Market Analysis.
- https://www.fidelity.com/learning-center/trading-investing/technical-analysis/technical-indicator-guide/slow-stochastic.
- FD Life&trading Portal. El Indicador Estocástico 2020.
- George Pruitt. "The Ultimate Algorithmic Trading System Toolbox + Website: Using Today's
Technology To Help You Become A Better Trader. John Wiley & Sons, 2016.

The stochastic oscillator was developed in the late 1950s by George Lane. As designed by Lane, the stochastic oscillator presents the location of the closing price of a stock in relation to the high and low range of the price of a stock over a period of time, typically a 14-day period. Lane, over the course of numerous interviews, has said that the stochastic oscillator does not follow price or volume or anything similar. He indicates that the oscillator follows the speed or momentum of Price.

Technical Indicators Stochastic and Its Use for Financial Market Analysis

# Introduction

The Average True Range Indicator (ATR Indicator) was developed by Welles Wilder as an
instrument for determining market volatility. The indicator tries to appreciate the interest of the
market in the price movement, since sudden movements and changes in the trend are usually
accompanied by wide ranges [1]. So, ATR can help day traders confirm when they might want to
initiate a trade, and it can be used to determine the placement of a stop-loss order.

In the world of “trading”, Volatility generates opportunities that is why it is interesting to
measure it and one of the best ways to do it is using the ATR Indicator. So, Volatility is the
amplitude or range of movement that quotes make in a certain time, in trading where we seek
to carry out short-term operations, as a general rule, we do not feel more comfortable when
there is volatility, because more volatility also means greater travels in the same time and
therefore greater profit potential, but must be accompanied by proper risk management. Many
trading systems perform at their best, when Volatility is highest [2]. Volatility is Cyclical, which
means that periods of low volatility will be followed by periods of high volatility or the opposite.
So, being aware of this is useful to learn to wait for the best periods to operate our trading
systems but also to adapt the management of the risk to current volatility in the market [2].

# The Average Real Range (ATR)

The ATR is plotted in a similar way to an oscillator below the price and it looks like this (See
Figure 1). The Volatility should be analyzed as something cyclical but not static, the ATR is useful
for any market and timeframe. Its calculation method is very simple, the true range is calculated
taking the highest of these three values, the last maximum minus the last minimum, the last
maximum minus the previous close and the last minimum minus the previous close, then a
Moving Average is calculated generally of 14 periods and its value is reflected in a graph below
the price graph as it happens with the MM, the higher the calculation period, the smoother the
behavior of the ATR will be and vice versa (See Figure 1). As a general rule, 14 works well
enough in any temporality and it is interesting to modify it depending on the temporal
proportions that you work with. If you don't know which parameter to put 14 will be enough.

Wilder originally developed the ATR for commodities, although the indicator can also be used
for stocks and índices [1]. Simply put, a stock experiencing a high level of volatility has a higher
ATR, and a low volatility stock has a lower ATR.

The first step in calculating ATR is to find a series of true range values for a security. The price
range of an asset for a given trading day is simply its high minus its low. Meanwhile, the true
range is more encompassing and is defined as:

Where:

The ATR is commonly used as an exit method that can be applied no matter how the entry
decision is made. One popular technique is known as the "chandelier exit" and was developed
by Chuck LeBeau. The chandelier exit places a trailing stop under the highest high the stock
reached since you entered the trade. The distance between the highest high and the stop level
is defined as some multiple times the ATR [3].

The data that the ATR gives us can therefore interpret as the average path that the price has
made in each average period or what is the same as how much we should expect on average
that the price travels in a single candle or bar. We know what route each candle has been doing
on average, we have a much clearer dimension of how close or far our stop loss or profit target
can be, we will also have an objective and measurable data of whether the market is moving a
lot or Little in more advanced analysis will help us to measure the state of volatility cycles, since
most cycles occur with some regularity, so defining the regularity of these volatility cycles helps
us to predict when we can expect it to increase or decrease. Look at this example (See Figure 2)
an analysis of volatility cycles and the regularity with which the cycles are repeated [2].

The ATR is also useful to decide the placement of stop losses in certain trading systems, if we
know on average how much the price has been traveling per period we could take a multiple of
the value of the ATR as a minimum reference for our stop losses based on the typical duration of
our operations, the same could be done with the profit taking zones, whether partial or total, if
we reference it at a multiple of the ATR we will have a clearer perception of how likely it is that
it has been reached in a certain time.

# Conclusions

The ATR measures the current Volatility of the market, a data that can be very useful as
confirmation in the breaking of levels in risk management and monetary management, but also
in the analysis of volatility cycles, it is an extremely intuitive data of interpreting that makes it
easy to apply, although it is graphical as an oscillator, its interpretation is not the same, although
it also helps us measure the market momentum through its volatility and therefore what
average route we can expect per unit of time.

There are two main limitations to using the ATR indicator. The first is that ATR is a subjective
measure, meaning that it is open to interpretation. There is no single ATR value that will tell you
with any certainty that a trend is about to reverse or not. Instead, ATR readings should always
be compared against earlier readings to get a feel of a trend's strength or weakness. Second,
ATR only measures volatility and not the direction of an asset's price. This can sometimes result
in mixed signals, particularly when markets are experiencing pivots or when trends are at
turning points. For instance, a sudden increase in the ATR following a large move counter to the
prevailing trend may lead some traders to think the ATR is confirming the old trend; however,
this may not actually be the case [5].

**References**

- Chart-formations. "J. Welles Wilder, Jr." Accessed Aug. 7, 2020.
- FD Life&trading Portal. El Indicador ATR 2020.
- Corporate Finance Institute. "Chandelier Exit." Accessed Aug. 8, 2020.
- CobuildLab 2021. Technical Specifically Moving Average and Its Use for Financial Market
Analysis.
- Investopedia 2020. Average True Range

The Average True Range Indicator (ATR Indicator) was developed by Welles Wilder as an instrument for determining market volatility. The indicator tries to appreciate the interest of the market in the price movement, since sudden movements and changes in the trend are usually accompanied by wide ranges [1]. So, ATR can help day traders confirm when they might want to initiate a trade, and it can be used to determine the placement of a stop-loss order.

Technical Indicators ATR and Its Use for Financial Market Analysis

# Introduction

MACD (Moving Average Convergence / Divergence) is a technical indicator that measures the
strength of the price movement. It is made up of two MACD and Signal Lines and a Histogram.
The author did it thinking about the daily charts, so he decided that they were 12 and 26 which
is approximately the Moving Average of half a month and a month of stock trading days. As it is
of the Oscillator type, it allows us to measure the momentum of trends, in addition to detecting
overbought or oversold states, that is, when the trend has gone too far and is ready for a short-
term consolidation [1].

# The Moving Average of Convergence / Divergence (MACD)

MACD is commonly used by analyzing crossovers, divergences, and periods of steep slope
(positive or negative). It is calculated through the Difference between two exponential moving
average lines of 12 and 26 periods [2]. This difference can be seen on a separate chart (called
the MACD line). It also has a 9-period moving average [1,3] line called the signal line (See Figure
1).

<img>

Figure 1: MACD and Signal Lines

So to understand how the MACD oscillator is built we can see that when the 2 (MA) [2] above
approach the main line of the MACD approaches its zero line, the central line measures the
difference between the two Moving Averages (MA) when they touch, the main line MACD will
show zero, there is no distance between them (See Figure 2). On the contrary, when the MAs
separate, the main line moves away from 0 either above or below depending on which (MA) is
above the other, thus indicating a possible upward or downward trend [4].

<img>

Figure 2: MA in the MACD Behavior

The histogram, which are these bars that are seen on the MACD chart, is nothing more than the
difference between the main and signal lines (See Figure 3) [4].

<img>

Figure 3: Histogram in MACD

With these three elements main line, signal line and histogram we can obtain information that
can be incorporated into our trading system. The MACD is very versatile and serves both to
detect trends and states of overbought or oversold. The most common signals that the MACD
throws are the following: when the main line crosses the 0 either up or down it could be
indicating a possible change in trend and that is when the fast (MA) the 12 is above the slow at
26, the main MACD line will be above 0 indicating that the (MA) are aligned for a possible
uptrend (See Figure 4). Otherwise, if the fast is below the slow, the main MACD line will be
below zero [4].

<img>

Figure 4: MACD in Overbought and Oversold Behavior

Traders will then look for a long position when the MACD line crosses the signal line from the
bottom up, and will take a short position when the MACD line crosses the signal line from the
top down. On the other hand, it is also possible to find valuable information on the divergence
between the MACD line and the current price.

Divergences (See Figure 5) are another type of signals that the MACD offers us, a divergence
occurs when the price chart offers new highs or new lows during a trend and the main line of
the MACD does not accompany, that is, they do not make the new ones. Highs new lows and we
have two types of divergence, the bearish and bullish divergence, the bearish divergence occurs
when the main MACD line does not accompany making new maximums of the trend and in this
case it is warning us that the uptrend is losing momentum or momentum and could be
preparing to turn around, the bullish divergence which is just the opposite when the price chart
is making new lows in a downtrend and the main MACD line does not accompany indicating loss
of bearish momentum and possible exhaustion of the pure downtrend common sense because
then because if a divergence occurs it means that the two (MA) [2] with which it is co It builds
the MACD and generally those of 12 and 26 are getting progressively closer although new highs
or lows are being produced which is a clear indication that the trend is losing strength I have
used this signal a lot as confirmation of exhaustion patterns if you have a Exhaustion pattern
with divergence in the MACD and breakout of the confirmation line is quite likely to produce
some anti-trend movement.

<img>

<img>

Figure 5: MACD Divergence Behavior

# Python implementation MACD

To create a program that uses MACD, we must first understand the ATR indicator. It is an instrument for
determining the strength of the price movement. MACD is commonly used by analyzing crossovers,
divergences, and periods of steep slope (positive or negative). It is made up of two MACD and Signal
Lines.

These are the prerequisites for the program. Yahoo Finance API is used to download stock data, MACD
is to calculate the indicator values. Matplotlib of course is to plot the data as a graph. Extracting closing
price data of Yahoo Finance stock for the aforementioned time-period and Observe general price
variation of the closing price for the give period. For this example, I have taken the 4 years of historical
data of APY Yahoo Finance stock from 1st March 2017 to 1st Feb 2021.

We start by installing API Yahoo Finance pip install yahoo finance after importing libraries
#importing variables

import pandas as pd

import numpy as np

import datetime as dt

import pandas_datareader as pdr
import seaborn as sns

import matplotlib.pyplot as plt

#extracting data from Yahoo Finance API with two índices AAPN and NFLX from 1990 to date today.
tickers = ['AAPL','NFLX']

all_data = pd.DataFrame()

test_data = pd.DataFrame()

no_data = []

for i in tickers:
try:

test_data = pdr.get_data_yahoo(i, start = dt.datetime(1990,1,1), end = dt.date.today())
test_data['symbol'] = i

all_data = all_data.append(test_data)

except:
no_data.append(i)

Now we will calculate MACD and Signal Lines. It’s calculated through the Difference between
two exponential moving average lines of 12 and 26 periods and It also has a 9-period moving
average line called the signal line.

all_data['12Ewm'] = all_data.groupby('symbol')['Close'].transform(lambda x: x.ewm(span=12,
adjust=False).mean())

all_data['26Ewm'] = all_data.groupby('symbol')['Close'].transform(lambda x: x.ewm(span=26,
adjust=False).mean())

all_data['MACD'] = all_data['26Ewm'] - all_data['12Ewm']
all_data['9Ewm']=all_data['MACD'].transform(lambda x: x.ewm(span=9, adjust=False).mean())

Now Plotting (MACD, MAs, Signal Line)

start = dt.datetime.strptime('2019-01-01', '%Y-%m-%d')
end = dt.datetime.strptime('2019-12-31', '%Y-%m-%d')
sns.set()

fig = plt.figure(facecolor = 'white', figsize = (20,10))

ax0 = plt.subplot2grid((6,4), (1,0), rowspan=4, colspan=4)
ax0.plot(all_data[all_data.symbol=='AAPL'].loc[start:end,['Close','12Ewm','26Ewm']])
ax0.set_facecolor('ghostwhite')

ax0.legend(['Close','12Ewm','26Ewm'],ncol=3, loc = 'upper left', fontsize = 15)
plt.title("Apple Price with MACD difference between slow and long indicator", fontsize = 20)

ax1 = plt.subplot2grid((6,4), (5,0), rowspan=1, colspan=4, sharex = ax0)
ax1.plot(all_data[all_data.symbol=='AAPL'].loc[start:end,['MACD','9Ewm']])

ax1.legend(['MACD','9Ewm'],ncol=3, loc = 'upper left', fontsize = 12)
ax1.set_facecolor('silver')

plt.subplots_adjust(left=.09, bottom=.09, right=1, top=.95, wspace=.20, hspace=0)
plt.show()

<img>

Other stocks closing price data of NASDAQ (National Association of Securities Dealers Automated
Quotation) and Dow Jones (Dow Jones Industrial Average).

<img>

<img>

<img>

# Conclusions

We have seen the amount of profits that an indicator as versatile as the MACD, it is widely used
because it is simple and easy, since it provides information about the strength of the trend (up
or down), but also about the strength of the trends (buy and sell) signals. Its disadvantage is
that it is a Short-Term Indicator, since its longest average is 26 days, which could not be useful
for long-term investments. As it is also a trend-following indicator, this means that the indicator
communicates a signal when the trend takes place, not before it begins.

**References**

[1] CobuildLab 2021. Technical Indicators for Trading.

[2] CobuildLab 2021. Technical Specifically Moving Average and Its Use for Financial Market
Analysis with python.

[3] Implementing MACD in Python https://towardsdatascience.com/implementing-macd-in-
python-cc9b2280126a. 2020

[4] FD Life&trading Portal. El Indicador MACD. 2019.

MACD (Moving Average Convergence / Divergence) is a technical indicator that measures the
strength of the price movement. It is made up of two MACD and Signal Lines and a Histogram.
The author did it thinking about the daily charts, so he decided that they were 12 and 26 which
is approximately the Moving Average of half a month and a month of stock trading days. As it is
of the Oscillator type, it allows us to measure the momentum of trends, in addition to detecting
overbought or oversold states, that is, when the trend has gone too far and is ready for a short-
term consolidation [1].

Technical Indicators MACD and Its Use for Financial Market Analysis with Python

# Introduction

Momentum trading is a technique where traders buy and sell financial assets after being influenced by recent price trends. They look to take advantage of upward or downward trends within the financial markets until the trend starts to fade.

Momentum trading strategies focus on price action and price movements rather than fundamental factors, such as company growth or economics. This is a form of technical analysis​ that is very popular with short-term traders. However, traders calculate momentum price projections based on historical price trends and data, and given the volatility of financial markets, prices can change and the market can move in an unforeseen direction at any time. The markets are also affected by news releases and other macroeconomic events​, which need to be taken into account when building a momentum trading strategy and risk management​ plan.

Momentum refers to the speed of price changes within a financial security. It is determined by many factors, the most significant being asset volume and volatility of the market. Momentum traders look to enter and exit positions quickly, therefore they often prefer markets with high liquidity, such as the https://www.cmcmarkets.com/en-gb/forex-trading forex market or https://www.cmcmarkets.com/en-gb/markets-shares share market. Liquid markets are also particularly volatile and this means that momentum traders can take advantage of short trades through price swings, which is similar to other short-term trading strategies, such as day trading.

Trend following is another trading strategy that is used to identify https://www.cmcmarkets.com/en-gb/trading-guides/forex-trend trendlines ​within price charts when trading assets. Trend following and momentum strategies share similarities in that they both focus on price action, buy high on uptrends and sell low on downtrends. However, whereas momentum based trading relies on aspects of fundamental analysis to make predictions, trend followers tend to focus solely on price and the size of their trades. This is partially to reduce time and make quicker trades, but also to help reduce losses. Trend followers measure their position size with the trend rather than placing a large amount of capital in one position.

Whereas momentum strategies focus on following the current trends of an asset, https://www.cmcmarkets.com/en-gb/trading-guides/swing-trading swing trading​ takes a different approach to this. Instead, swing traders trade within ranges and tend to focus on buying and selling at support and resistance levels. As they trade on short-term price changes, most swing trades usually only last for a short duration, while momentum trades can last either in the short-term or long term, depending on the strength of a trend. Swing traders also do not take into account fundamental and economic factors that may have an effect on price trends.

Momentum in trading is often influenced by timeframe. Although some momentum traders prefer to take positions in the long-term, one of the most appropriate strategies for trading on momentum is the short-term approach of day trading. The aim of https://www.cmcmarkets.com/en-gb/trading-guides/day-trading day trading​ is to enter and exit multiple positions as quickly as possible throughout the day, with the aim of making a profit from small price movements. Therefore, momentum traders look for markets and securities with a high volume, so that they can buy and sell stocks quickly without interruption.

If each security has a range of at least $5, this is considered profitable enough for momentum intraday trading. Smaller price movements are more suited to scalping strategies, which are very common within the forex market, where currencies fluctuate by a number of ‘pips’.

As we have already discussed, momentum strategies are arguably the most useful for volatile markets, where there is a constant stream of traders. With this in mind, momentum in share trading is one of the most monitored markets. Share trading is a financial market where stocks are constantly changing in price, due to external factors. These include aspects of fundamental analysis, such as company earnings reports, updated P/E ratios and takeover bids.   https://www.cmcmarkets.com/en-gb/trading-guides/company-fundamentals Company analysis​ is an important process of stock screening when a trader is interested in momentum in share trading.

The share market also applies to https://www.cmcmarkets.com/en-gb/trading-guides/what-is-an-exchange-traded-fund exchange-traded funds​ (ETFs), which are investment funds that grant a trader access to a collection of underlying assets. Various ETFs seek to track the performance of momentum-based indices within the stock market, most often large cap or blue-chip stocks, where is a constant stream of price action. Developing an ETF momentum trading strategy follows the same protocol as the share market, although you will be trading on multiple securities at once, rather than a single share.

Below are some of the most popular and efficient technical indicators for a momentum trader to succeed in their strategy. The best momentum indicators tend to focus on price action rather than a stock’s long-term economic situation. (See Articles Indicator Techniques where we to explain your uses with python RSI, Moving Average, Average directional index and Moving average convergence divergence (MACD).

# Momentum Trading

Momentum is a phase in which an asset appears to be moving based on past changes in prices rather than due to any stock specific fundamental or news. When prices move higher in reaction to higher prices is known as a Bull phase, and when prices move lower just because they'd been going lower it's known as a Bear phase.

A momentum run is clearly seen with the candlestick charts where green represent up days and red represents down days. On the chart you can see that up appears to be in a bowl momentum phase for a few months in 2014 and is hugging what is known as a trend line.

A trend line is a straight line that connects two or more points which can be temporary highs or lows, since stock prices tend to be trendy. Trend lines that connect the highest or lows in a stock's price history, can help identify the current trend and predict what the stock price might do in the future. Volume (red box in the chart) is critical to confirming momentum trends. Technical analysts use trend lines above and below a series to identify potential buy or sell signals.

Figure 2: Momentum Trends

Quantitative traders need a measure that is more objective but still versatile. They use moving averages to clearly see stock price trends that may not be apparent in a forest of price bars. The slope of a moving average's line can show you whether a stock is an upward or downward trend and can be used to generate trading signals.

Momentum traders use moving or rolling averages to identify trends and specified trade entry and exit parameters. You saw earlier that a trend line is a good way to look at whether a stock is in a sustain momentum phase, either a Bull or Bear phase. The key word here is sustained(sostenido). Trends don't tell us whether the trend will continue or will break abruptly. This is where moving averages and crossovers come in.

Figure 3: Moving Averages Short and Long

Let's add some moving averages and look at the 2014-15 period when the stock was in a Bull momentum phase. We'll demonstrate this on the same chart you saw earlier, but with some added lines. The slope of the moving average line shows whether the stock is in an upward or downward trend. A simple moving average is just the average of the close price over a specified period. It is sometimes hard to spot a trend in a forest of price pars. The slope of rolling averages or moving averages can tell you whether a stock is in an uptrend or downtrend.

Figure 4: Moving Averages Short and Long

A 50-day simple moving average is the sum of all the closing prices in the 50-day period divided by 50. You can see this on the plot where the green line begins 50 days after the first price data point red and exponentially weighted moving average works the same as a simple moving average except the most recent prices are given more weight in the average than the older prices. Here you see the closing price plotted along with a 10-day and 50-day simple moving average. Each closing price in the window is equally weighted as opposed to an exponential average where more recent prices are given a heavier weight.

Figure 5: Simple Moving Averages 50 Day

This gives you an indicator that more closely reflects current market conditions while still including earlier data.

A very common way to obtain a buy or sell signal is to look for moving average crossovers in stock charts. This means computing two moving averages of different lengths and waiting for one to cross over the other. The direction of the cross will indicate the direction of the momentum. At least that's what technical traders believe. Volume is once again a critical component in determining whether the crossover is a real change in trend or temporary.

A Bear cross is when the shorter moving average moves below the longer moving average, and a Bull cross is when the shorter moving average moves above a longer moving average. Traders take a long position after a Bull cross and a short position in a stock after a Bear cross. They hold that position while the cross is in place. They switch positions when the bowl cross moves into a Bear cross or vice versa.

We can see here that there are four points where the averages cross each other. Assume you are currently long the stock, the first two crossings come close to each other and indicate a quick sell and then buy back phase, but the second two crossings predict a longer up face. This can be one way to make profits using moving average crossovers. However, since most traders are watching the same signals, there is no guarantee these will work in the real world. For that you will need to develop more complex trading strategies based on more unconventional trading signals.

Figure 6: Simple Moving Bear and Bell Cross

A very common way to obtain a momentum signal is to look for moving average crossovers. This mean computing two moving averages of different lengths, and waiting for one to cross the other. The direction of the cross will indicate the direction of the momentum.

Next let's look at how you choose the size of your moving average window or length. Choosing moving average length is very complicated. There are many choices of lengths such as 20, 30, 50, 65, 200 etcetera. In fact, any integer is a possible choice.

So the tendency is to look at the chart and find the two moving averages that seem to fit the given chart nicely, and then use that as a trading strategy for all stocks or even for the same stock in a future time period. This is known as over-fitting and you need to be aware of the danger of over-fitting. What works in one time period may not work in another time period, even for the same stock.

Choosing Moving Average Lengths is very complicated. There are many choices of lengths such as 20, 30, 50, 65, 200, etc. in fact any integer is a possible choice. So you need to be aware of the dangers of Overfitting.

The reason is the choice of lengths will strongly affect the signal you receive from your moving average crossover strategy. There may be better windows and you can attempt to find them with robust optimization techniques. However, it's incredibly easy to over-fit your moving window lengths. So, Dangers of overfitting the choice of lengths will strongly affect the signal that you receive from your moving average crossover strategy. There may be better windows and attempts to find them can be made with robust optimization techniques. Hovewer, it is incredibly easy overfit your moving window lengths.

Figure 7: Simple Moving Averages with Different Lengths

One approach is to use moving average crossover ribbons, draw many moving averages at a time, and attempt to extract statistics from the shape of the ribbon rather than any two moving averages. You can see how ribbons look here. You can combine quantitative measures of ribbon shape and so generate a trading signal.

Figure 8: Moving Average Crossover

Last we will look at two methods for evaluating or scoring the strength of a combination of moving average signals with different window lengths. The first is a distance metric and the second is correlation. One way of scoring is to use a distance metric. You can use a distance metric to see how far away from some given ranking our ribbon is. Here we combine ribbons to come up with a one to 10 ranking which we normalized to a score of zero to one. A perfectly increasing order of ribbons results in a score of zero. This can be a signal to buy or go long. A perfectly decreasing order of ribbons results in a score of one, this can be a signal to sell or go short.

Figure 9: Scoring Moving Averages Distance Metric

After ranking the rolling means and normalizing the scores, we plot them on the price chart. We see a number of strong cell signals followed by a weak buy signal another strong sell signal and then a weak buy signal. Scoring Moving Average Distance Metric. We see a number of strong sell signals and two buy signals towards the end of the data.

Next we will look at what happens after the final buy signal. When we look at what happens after the buy signal from the previous chart. We see that the stock price rises for bit and then drops by about 20-30 percent over the next few months, not good. So you have to constantly be on the lookout for better signals.

Another slightly better way maybe to use correlation to rank ribbons. You can use a correlation metric, a perfectly increasing order of ribbons results in a score of one, this can be a signal to buy or go long. A perfectly decreasing order of ribbons results in a score of negative one, this can be a signal to sell or go short. First, we rank the ribbons using a Spearman's correlation and normalize the score once again. A score close to one means a buy signal and a score close to negative one is a sell signal. This is somewhat better since it triggered three sell signals pretty close to the short-term top of the stock. This may be worth exploring some more.

A Score close to 1 means a buy signal and a score close to -1 is a sell signal. This is somewhat better since it triggered 3 sell signal pretty close to the short term top of the stock. This may be worth exploring some more.

Figure 10: Correlation to Rank Ribbons

Momentum Trading with Python

**Moving Average Crossover**

A very common way to obtain a momentum signal is to look for moving average crossovers. This means computing two moving averages of different lengths, and waiting for one to cross the other. The direction of the cross will indicate the direction of the momentum.

We'll demonstrate this on some real asset data.

```python
#importing variables

import pandas as pd

import numpy as np

import datetime as dt

import pandas_datareader as pdr

import seaborn as sns

import matplotlib.pyplot as plt

data = "dow_jones.csv"

df = pd.read_csv(data)

all_data = pd.DataFrame()

all_data =df

\# create 20 days simple moving average column

all_data['30_SMA'] = all_data['Close'].rolling(window = 30, min_periods = 1).mean()

\# create 50 days simple moving average column

all_data['200_SMA'] = all_data['Close'].rolling(window = 200, min_periods = 1).mean()

\# display first few rows

print(all_data.head())

all_data['Signal'] = 0.0

all_data['Signal'] = np.where(all_data['30_SMA'] > all_data['200_SMA'], 1.0, 0.0)

all_data['Position'] = all_data['Signal'].diff() #diff() out[i]=a[i+1]-a[i]

\# display first few rows

print(all_data.head())

fig = plt.figure(facecolor = 'white', figsize = (20,10))

ax0 = plt.subplot2grid((6,4), (1,0), rowspan=4, colspan=8)

ax0.plot(all_data['Close'],color = 'r',label = 'Close')

#all_data['CloseBar']=all_data['12Ewm']

#all_data['CloseBar'].plot.bar(label='e', width=0.8)

ax0.plot(all_data['30_SMA'], color = 'b',label = '30-day SMA')

ax0.plot(all_data['200_SMA'], color = 'g', label = '200-day SMA')

ax0.set_facecolor('ghostwhite')

plt.plot(all_data[all_data['Position'] == 1].index, all_data['30_SMA'][all_data['Position'] == 1], #index() method only returns the first occurrence of the value.

 '^', markersize = 15, color = 'g', label = 'buy')

\# plot ‘sell’ signals

plt.plot(all_data[all_data['Position'] == -1].index, 

 all_data['30_SMA'][all_data['Position'] == -1], 

 'v', markersize = 15, color = 'r', label = 'sell')

plt.legend()

#ax0.legend(['Close','20_SMA','50_SMA','Position'],ncol=5, loc = 'upper left', fontsize = 15)

plt.title("Moving Average Crossover", fontsize = 20)

```

We can see here that there are five crossing points once both averages are fully populated. The first seems to be indicative of a following upturn, second of a downturn and so on.

Choosing Moving Average Lengths

WARNING: Overfitting

The choice of lengths will strongly affect the signal that you receive from your moving average crossover strategy. There may be better windows, and attempts to find them can be made with robust optimization techniques. However, it is incredibly easy to overfit your moving window lengths. For an exmaple of this see the Dangers of Overfitting.

An extended version of the moving average crossover system is the Moving Average Ribbon. This moving average strategy is created by placing a large number of moving averages onto the same chart (the chart shown below uses 8 simple moving averages). One must factor the time horizons and investment objectives while selecting the lengths and type of moving averages. When all the moving averages are moving in the same direction, the trend is said to be strong. Trading signals are generated in a similar manner to the triple moving average crossover system, the trader must decide the number of crossovers to trigger a buy or sell signal. Traders look to buy when the faster moving averages cross above the slower moving averages and look to sell when the faster moving averages cross below the slower moving averages.

**References**

- M. O. Anthony Brabazon, An introduction to evolutionary computation in finance. , University College Dublin, IRELAND.: Pipeline Financial Group, Inc., USA.
- A.-M. Baiynd, «The Trading Book: A Complete Solution to Mastering Technical Systems and Trading Psychology,» McGraw-Hill, p. 272.
- CobuildLab 2021. Technical Specifically Moving Average and Its Use for Financial Market Analysis with python.
- CobuildLab 2021. Technical Specifically MACD and Its Use for Financial Market Analysis with python.
- «Trader Kingdom.
- I. Staff. [En línea]. Available:  http://www.investopedia.com/articles/technical/070301.asp.
- «IFC Markets,» [En línea]. Available: https://www.ifcmarkets.mx/forex-trading-strategies/tecanalysis-strategy.
- «Investopedia,» [En línea]. Available: http://www.investopedia.com/terms/g/goldencross.asp
- http://www.algotrades.net/is-black-box-trading-investing-the-way-of-the-future/
- Coursera Using Machine Learning Trading and Finance. 2020.

Momentum trading is a technique where traders buy and sell financial assets after being influenced by recent price trends. They look to take advantage of upward or downward trends within the financial markets until the trend starts to fade.

Momentum Trading with Python

# Introduction

MACD (Moving Average Convergence / Divergence) is a technical indicator that measures the
strength of the price movement. It is made up of two MACD and Signal Lines and a Histogram.
The author did it thinking about the daily charts, so he decided that they were 12 and 26 which
is approximately the Moving Average of half a month and a month of stock trading days. As it is
of the Oscillator type, it allows us to measure the momentum of trends, in addition to detecting
overbought or oversold states, that is, when the trend has gone too far and is ready for a short-
term consolidation [1].

# The Moving Average of Convergence / Divergence (MACD)

MACD is commonly used by analyzing crossovers, divergences, and periods of steep slope
(positive or negative). It is calculated through the Difference between two exponential moving
average lines of 12 and 26 periods [2]. This difference can be seen on a separate chart (called
the MACD line). It also has a 9-period moving average [1,3] line called the signal line (See Figure
1).
<img>

So to understand how the MACD oscillator is built we can see that when the 2 (MA) [2] above
approach the main line of the MACD approaches its zero line, the central line measures the
difference between the two Moving Averages (MA) when they touch, the main line MACD will
show zero, there is no distance between them (See Figure 2). On the contrary, when the MAs
separate, the main line moves away from 0 either above or below depending on which (MA) is
above the other, thus indicating a possible upward or downward trend [4].
 <img>

<img>

The histogram, which are these bars that are seen on the MACD chart, is nothing more than the
difference between the main and signal lines (See Figure 3) [4].
<img>

**Figure 3: Histogram in MACD**


With these three elements main line, signal line and histogram we can obtain information that
can be incorporated into our trading system. The MACD is very versatile and serves both to
detect trends and states of overbought or oversold. The most common signals that the MACD
throws are the following: when the main line crosses the 0 either up or down it could be
indicating a possible change in trend and that is when the fast (MA) the 12 is above the slow at
26, the main MACD line will be above 0 indicating that the (MA) are aligned for a possible
uptrend (See Figure 4). Otherwise, if the fast is below the slow, the main MACD line will be
below zero [4].
 <img>

**Figure 4: MACD in Overbought and Oversold Behavior**

 

 

 

Traders will then look for a long position when the MACD line crosses the signal line from the
bottom up, and will take a short position when the MACD line crosses the signal line from the
top down. On the other hand, it is also possible to find valuable information on the divergence
between the MACD line and the current price.
 

Divergences (See Figure 5) are another type of signals that the MACD offers us, a divergence
occurs when the price chart offers new highs or new lows during a trend and the main line of
the MACD does not accompany, that is, they do not make the new ones. highs new lows and we
have two types of divergence, the bearish and bullish divergence, the bearish divergence occurs
when the main MACD line does not accompany making new maximums of the trend and in this
case it is warning us that the uptrend is losing momentum or momentum and could be
preparing to turn around, the bullish divergence which is just the opposite when the price chart
is making new lows in a downtrend and the main MACD line does not accompany indicating loss
of bearish momentum and possible exhaustion of the pure downtrend common sense because
then because if a divergence occurs it means that the two (MA) [2] with which it is co It builds
the MACD and generally those of 12 and 26 are getting progressively closer although new highs
or lows are being produced which is a clear indication that the trend is losing strength I have
used this signal a lot as confirmation of exhaustion patterns if you have a Exhaustion pattern with divergence in the MACD and breakout of the confirmation line is quite likely to produce
some anti-trend movement.
<img>

<img>

**Figure 5: MACD Divergence Behavior**
 

# Conclusions

We have seen the amount of profits that an indicator as versatile as the MACD, it is widely used
because it is simple and easy, since it provides information about the strength of the trend (up
or down), but also about the strength of the trends (buy and sell) signals. Its disadvantage is
that it is a Short-Term Indicator, since its longest average is 26 days, which could not be useful
for long-term investments. As it is also a trend-following indicator, this means that the indicator
communicates a signal when the trend takes place, not before it begins.
 

**References**

[1] CobuildLab 2021. Technical Indicators for Trading.

[2] CobuildLab 2021. Technical Specifically Moving Average and Its Use for Financial Market

Analysis with python.

[3] Implementing MACD in Python https://towardsdatascience.com/implementing-macd-in-

python-cc9b2280126a. 2020

[4] FD Life&trading Portal. El Indicador MACD. 2019.

MACD (Moving Average Convergence / Divergence) is a technical indicator that measures the strength of the price movement. It is made up of two MACD and Signal Lines and a Histogram. The author did it thinking about the daily charts, so he decided that they were 12 and 26 which is approximately the Moving Average of half a month and a month of stock trading days.

Technical Indicators MACD and Its Use for Financial Market Analysis

# **Introduction**

One of the major goals of the modern enterprise of data science and analytics is to HYPERLINK "https://www.quora.com/Is-optimization-related-to-data-science-And-how" solve complex optimization problems for business and technology companies to maximize their profit. So, Optimization is the science / process behind finding the best solution for a problem with given constraints. We come across optimization problems on a daily basis. These can be for finding the shortest path between your work place and office; maximizing revenues / customer happiness or minimizing costs / debts etc. We basically take a real world problem, model it mathematically and then solve it using mathematical techniques with in the constraints. Optimization is useful in Marketing, Manufacturing, Finance, Online advertising, Machine Learning and all fields you can imagine.
Profit maximization is a main business requirement of corporate organizations, hence a data science application should focus on the profitability of a given business setting. The adaptation of analytics to a specific business application (e.g., churn) is crucial, since it leads to better decision making. Insights from the analytical model support business decisions that allow attaining the maximum net profit, return, payoff, or value. We refer to this value-centric approach as profit-driven business analytics.
Profits are the lifeline of small businesses. As your profits increase and become more predictable, your small business has a greater chance of surviving—and most businesses don't. In fact, only around half of businesses tend to make it past the five-year mark, HYPERLINK "https://www.bls.gov/bdm/entrepreneurship/entrepreneurship.htm" according to data from the Bureau of Labor Statistics.

Because of this survival rate, every day counts when it comes to turning and increasing your small business profits. Most of the time, small business owners can't afford to wait for the results of long-term, large-scale changes to their business model. As much as possible, if you want to turn a bigger profit as a small business owner, the quicker you can do it, the better. The following simple changes can help you get started maximizing your profitability right away.
The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of HYPERLINK "https://360digitmg.com/data-science" data science helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

So, Predictive analytics is about predicting future phenomena using historical data sets. Therefore, for accurate forecasting, predictive analytics is highly helpful as it shows various mathematical models of predictive forecasting. Also, the predictive analytics tools help compare two roadmaps and decide which is more profitable and efficient. It takes into consideration various factors, both internal and external. This process is known as the optimization of supply chain processes. Therefore, it can be said that the predictive analytics of HYPERLINK "https://360digitmg.com/data-science"data science are highly helpful in forecasting, optimization, and visualization of various processes in a supply chain of businesses.
There is a HYPERLINK "https://www.math.uni-bielefeld.de/documenta/vol-ismp/25_bixby-robert.pdf" long and rich history of the theoretical development of robust and efficient solvers for optimization problems. However, focusing on practical applications, we will skip that history and move straight to the part of learning how to use programmatic tools to formulate and solve such optimization problems. There are many excellent optimization packages in Python.

Knowledge of such optimization techniques is extremely useful for data scientists and machine learning (ML) practitioners as discrete and continuous optimization lie at the heart of modern ML and AI systems as well as HYPERLINK "https://www.edx.org/course/optimization-methods-business-analytics-mitx-15-053x" data-driven business analytics processes. These tools and concepts can also be used to maximize the profit of your Supply Chain by focusing your production on high margin products. So, Optimization to help your local bakery to improve its business profitability by selecting the right items to sell.
In this article, we will specifically talk about PuLP. But before going to the Python library, let us get a sense of the kind of problem we can solve with it.

# _Problem Statement_

You want to help your local bakery to maximize its profit. They sell several types of product including cakes, pastries and sandwiches. Example of business case taken from article “Maximize your Business Profitability with Python”.

**Available resources:**
4 bakers working 6 hours per day (Total: 24 hours/day)
1 assistant for sandwiches 4 hours per day
2 ovens that can be used 24 hours per day (Total: 48 hours/day)
100 slots available for stock and display

**Objective**
What do you need to produce to maximize your daily profit?

Business Case – Code in Python
""" Step 1 - Import the required modules"""

```python

from pulp import \*
# Parameters 
items = ["A", "B", "C", "D", "E", "F"]
profit = {"A":6, "B":4.4, "C":7.5, "D":0.9, "E":1.2, "F":2.2}
baker = {"A":50, "B":0, "C":45, "D":35, "E":25, "F":0}
oven = {"A":45, "B":0, "C":90, "D":20, "E":45, "F":0}
display = {"A":4, "B":1.5, "C":3, "D":1, "E":1, "F":1}
assist = {"A":0, "B":12, "C":0, "D":0, "E":0, "F":8}

```

## _Create a LP Minimization problem_

_**Step 1: Define the problem by giving a suitable name to your problem, here I have given the name ‘Problem’.**_

_**Also, specify your aim for the objective function of whether to Maximize or Minimize.**_

```python
model = LpProblem('Maximize Bakery Profits', LpMaximize)
```

## _Create problem Variables_

_**Step 2: Define LpVariable to hold the variables of the objective functions.**_

```python
A = LpVariable('A', lowBound = 0, cat = 'Integer')
B = LpVariable('B', lowBound = 0, cat = 'Integer')
C = LpVariable('C', lowBound = 0, cat = 'Integer')
D = LpVariable('D', lowBound = 0, cat = 'Integer')
E = LpVariable('E', lowBound = 0, cat = 'Integer')
F = LpVariable('F', lowBound = 0, cat = 'Integer')
var_dict = {"A":A, "B":B, "C":C, "D":D, "E":E, "F":F}
```

## _Objective Function_

_**Step 3: Denotes the objective function in terms of defined variables.**_

```python
model += lpSum([profit[i] \* var_dict[i] for i in items])
```

## _Constraints:_

_**Step 4: These are the constraint the variables.**_

```python
# Bakers
model += lpSum([var_dict[i]*baker[i] for i in items]) <= 24*60
# Oven
model += lpSum([var_dict[i]*oven[i] for i in items]) <= 48*60
# Assistant
model += lpSum([var_dict[i]*assist[i] for i in items]) <= 4*60
# Display
model += lpSum([var_dict[i]*display[i] for i in items]) <= 100
```

_**Step 5: This will show you the problem in the output screen.**_

```python
dict_name = dict(zip(["A", "B", "C", "D", "E", "F"], 
 ["Lemon Cake", "Sandwich", "Chocolate Cake", "Croissant", "Chocolate Eclair", "Panini"]))
```

## _Solve Model_

```python
model.solve()
for v in model.variables():
 print(dict_name[v.name], "=", int(v.varValue))
print("Profit reached: {:,} euros".format(sum(var_dict[i].varValue * profit[i] for i in items)))
```

Lemon Cake = 0
Sandwich = 20
Chocolate Cake = 23
Croissant = 0
Chocolate Eclair = 1
Panini = 0
Profit reached: 261.7 euros

_**References:**_

- Maximize your Business Profitability with Python
https://towardsdatascience.com/maximize-your-business-profitability-with-python-fbefebbdf802.
- PuLP: A Linear Programming Toolkit for Python Stuart Mitchell. The University of Auckland, Auckland, New Zealand September 5, 2011.
- Optimization Techniques for Data Scientist and Machine Learning
CobuildLab. 2021. HYPERLINK "https://www.cobuildlab.com/blog/ai" https://www.cobuildlab.com/blog/ai.
- ProfLogit: a Proﬁt Maximizing Logistic Regression Model for Customer Churn Prediction
HYPERLINK "https://www.dataminingapps.com/2018/02/proflogit-a-pro%EF%AC%81t-maximizing-logistic-regression-model-for-customer-churn-prediction/" https://www.dataminingapps.com/2018/02/proflogit-a-pro%EF%AC%81t-maximizing-logistic-regression-model-for-customer-churn-prediction/

[5] Open Source Iniatitive. MIT license.
http://www.opensource.org/ licenses/mit-license.php.

[6] M. O’Sullivan. Dippy.
https://projects.coin-or.org/ CoinBazaar/wiki/Projects/Dippy.

One of the major goals of the modern enterprise of data science and analytics is to solve complex optimization problems for business and technology companies to maximize their profit. So, Optimization is the science / process behind finding the best solution for a problem with given constraints. We come across optimization problems on a daily basis. 

Profit Optimization for Businesses: Small Bakery shop

Artificial Intelligence (AI) and Machine Learning (ML) solutions offer large possibilities to optimize and automate processes, save costs and make less human error possible for many industries. Food and Beverage is not an exception, where it can be beneficially applied in restaurants, bar and cafe businesses as well as in food manufacturing. 

Artificial Intelligent Apply Food Manufacturing and Vehicle Routing Problem 

# Introduction

The vehicle routing problem (VRP) is a combinatorial optimization and integer programming problem seeking to service a number of customers with a fleet of vehicles. Proposed by Dantzig and Ramser in 1959, VRP is an important problem in the fields of transportation, distribution, and logistics.

The vehicle routing problem (VRP) is a https://en.wikipedia.org/wiki/Combinatorial_optimization \o "Combinatorial optimization" combinatorial optimization and "https://en.wikipedia.org/wiki/Integer_programming" \o "Integer programming" integer programming problem which asks "What is the optimal set of routes for a fleet of vehicles to traverse in order to deliver to a given set of customers?". It generalizes the well-known https://en.wikipedia.org/wiki/Travelling_salesman_problem \o "Travelling salesman problem" travelling salesman problem (TSP). It first appeared in a paper by https://en.wikipedia.org/wiki/George_Dantzig\o "George Dantzig" George Dantzig and John Ramser in 1959, https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-DantzigRamser1959-1" in which the first algorithmic approach was written and was applied to petrol deliveries. Often, the context is that of delivering goods located at a central depot to customers who have placed orders for such goods. The objective of the VRP is to minimize the total route cost. In 1964, Clarke and Wright improved on Dantzig and Ramser's approach using an effective greedy approach called the savings algorithm.

Determining the optimal solution to VRP is https://en.wikipedia.org/wiki/NP-hard \o "NP-hard" NP-hard, https://en.wikipedia.org/wiki/Vehicle_routing_problem so the size of problems that can be solved, optimally, using https://en.wikipedia.org/wiki/Mathematical_programming \o "Mathematical programming" mathematical programming or https://en.wikipedia.org/wiki/Combinatorial_optimization \o "Combinatorial optimization" combinatorial optimization may be limited. Therefore, commercial solvers tend to use heuristics due to the size and frequency of real world VRPs they need to solve.

The VRP has many direct applications in industry. In fact, the use of computer optimization programs can give savings of 5% to a company https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-Springer_Verlag-3" [3] as transportation is usually a significant component of the cost of a product (10%) https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-4" - indeed, the transportation sector makes up 10% of the https://en.wikipedia.org/wiki/European_Union \o "European Union" EU's HYPERLINK "https://en.wikipedia.org/wiki/Gross_domestic_product" \o "Gross domestic product" GDP. Consequently, any savings created by the VRP, even less than 5%, are significant https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-Springer_Verlag-3".

The Vehicle Routing Problem (VRP) is a generic name given to a whole class of problems in which a set of routes for a fleet of vehicles based at one or several depots must be determined for a number of geographically dispersed cities or customers. The objective of the VRP is to deliver a set of customers with known demands on minimum-cost vehicle routes originating and terminating at a depot. In the two figures below we can see a picture of a typical input for a VRP problem and one of its possible outputs:

INCLUDEPICTURE http://neo.lcc.uma.es/vrp/wp-content/uploads/vrp.png * MERGEFORMATINET

## Setting up the problem

The VRP concerns the service of a delivery company. How things are delivered from one or more depots which has a given set of home vehicles and operated by a set of drivers who can move on a given road network to a set of customers. It asks for a determination of a set of routes, S, (one route for each vehicle that must start and finish at its own depot) such that all customers' requirements and operational constraints are satisfied and the global transportation cost is minimized. This cost may be monetary, distance or otherwise. HYPERLINK "https://en.wikipedia.org/wiki/Vehicle_routing_problem" \l "cite_note-toth-2”.

The road network can be described using a https://en.wikipedia.org/wiki/Graph_(discrete_mathematics) \o "Graph (discrete mathematics)" graph where the https://en.wikipedia.org/wiki/Directed_edge \o "Directed edge" arcs are roads and vertices are junctions between them. The arcs may be directed or undirected due to the possible presence of one way streets or different costs in each direction. Each arc has an associated cost which is generally its length or travel time which may be dependent on vehicle type. https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-toth-2".

To know the global cost of each route, the travel cost and the travel time between each customer and the depot must be known. To do this our original graph is transformed into one where the vertices are the customers and depot, and the arcs are the roads between them. The cost on each arc is the lowest cost between the two points on the original road network. This is easy to do as HYPERLINK "https://en.wikipedia.org/wiki/Shortest_path_problems" \o "Shortest path problems" shortest path problems are relatively easy to solve. This transforms the sparse original graph into a https://en.wikipedia.org/wiki/Complete_graph \o "Complete graph" complete graph. For each pair of vertices i and j, there exists an arc (i,j) of the complete graph whose cost is written as Cij and is defined to be the cost of shortest path from i to j. The travel time tij is the sum of the travel times of the arcs on the shortest path from i to j on the original road graph.

Sometimes it is impossible to satisfy all of a customer's demands and in such cases solvers may reduce some customers' demands or leave some customers unserved. To deal with these situations a priority variable for each customer can be introduced or associated penalties for the partial or lack of service for each customer given https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-toth-2" [2].

The objective function of a VRP can be very different depending on the particular application of the result but a few of the more common objectives are: https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-toth-2”.

Minimize the global transportation cost based on the global distance travelled as well as the fixed costs associated with the used vehicles and drivers.

Minimize the number of vehicles needed to serve all customers.

Least variation in travel time and vehicle load.

Minimize penalties for low quality service.

Maximize a collected profit/score.

## VRP variants

Several variations and specializations of the vehicle routing problem exist:

Vehicle Routing Problem with Profits (VRPP): A maximization problem where it is not mandatory to visit all customers. The aim is to visit once customers maximizing the sum of collected profits while respecting a vehicle time limit. Vehicles are required to start and end at the depot. Among the most known and studied VRPP, we cite:

The Team Orienteering Problem (TOP) which is the most studied variant of the VRPP, https://en.wikipedia.org/wiki/Vehicle_routing_problem\lcite_note-5 https://en.wikipedia.org/wiki/Vehicle_routing_problem\lcite_note-6 https://en.wikipedia.org/wiki/Vehicle_routing_problem\lcite_note-7.

The Capacitated Team Orienteering Problem (CTOP),

The TOP with Time Windows (TOPTW).

Vehicle Routing Problem with Pickup and Delivery (VRPPD): A number of goods need to be moved from certain pickup locations to other delivery locations. The goal is to find optimal routes for a fleet of vehicles to visit the pickup and drop-off locations.

Vehicle Routing Problem with HYPERLINK "https://en.wikipedia.org/wiki/LIFO_(computing)" \o "LIFO (computing)" LIFO: Similar to the VRPPD, except an additional restriction is placed on the loading of the vehicles: at any delivery location, the item being delivered must be the item most recently picked up. This scheme reduces the loading and unloading times at delivery locations because there is no need to temporarily unload items other than the ones that should be dropped off.

Vehicle Routing Problem with Time Windows (VRPTW): The delivery locations have time windows within which the deliveries (or visits) must be made.

Capacitated Vehicle Routing Problem: CVRP or CVRPTW. The vehicles have a limited carrying capacity of the goods that must be delivered.

Vehicle Routing Problem with Multiple Trips (VRPMT): The vehicles can do more than one route.

Open Vehicle Routing Problem (OVRP): Vehicles are not required to return to the depot.

Inventory Routing Problem (IRP): Vehicles are responsible for satisfying the demands in each delivery point https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-8”.

Multi-Depot Vehicle Routing Problem (MDVRP): Multiple depots exist from which vehicles can start and end. https://en.wikipedia.org/wiki/Vehicle_routing_problem \l "cite_note-9”.

### Case Study VRP - Open Vehicle Routing Problem (OVRP)

This is a project that demonstrates vehicle routing problem. We solved it with Linear Programming using pulp package, which yields the optimal solution.

K vehicles or sales people

Variables:

Goal:

Constraints:

Code in python

```python

from pulp import *

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

%matplotlib inline

import seaborn as sn

#Data Load

position = pd.read_csv('./data/position.csv', index_col="City")

flighttime = pd.read_csv('./data/flight_time.csv', index_col="City")

distance = pd.read_csv('./data/distance.csv', index_col="City")

position.head(5)

#Sites and Latitude and Longitude 

sites=['Barcelona','Belgrade','Berlin','Brussels','Bucharest','Budapest','Copenhagen','Dublin','Hamburg','Istanbul','Kiev','London','Madrid','Milan','Moscow','Munich','Paris','Prague','Rome','Saint Petersburg','Sofia','Stockholm','Vienna','Warsaw']

latlng = ['latitude', 'longitude']

#make some positions (so we can plot this)

positions = dict( ( city, (position.loc[city, 'longitude'], position.loc[city, 'latitude']) ) for city in sites)

for s in positions:

 p = positions[s]

 plt.plot(p[0],p[1],'o')

 plt.text(p[0]+.01,p[1],s,horizontalalignment='left',verticalalignment='center')

plt.gca().axis('off');

#Positions and Distance between cites

positions = dict( ( city, (position.loc[city, 'longitude'], position.loc[city, 'latitude']) ) for city in sites)

for s in positions:

 p = positions[s]

 plt.plot(p[0],p[1],'o')

 plt.text(p[0]+.01,p[1],s,horizontalalignment='left',verticalalignment='center')

plt.gca().axis('off'); 

distances = dict( ((s1,s2), distance.loc[s1, s2] ) for s1 in positions for s2 in positions if s1!=s2)

#For this example let’s try 4

“””#Step 1: Define the problem by giving a suitable name to your problem; here I have given the name ‘Problem’. #also, specifies your aim for the objective function of whether to Maximize or Minimize.”””

\# Create a LP Minimization problem

Lp_prob = p.LpProblem('vehicle', p.LpMinimize) 

#Step 2: Define LpVariable to hold the variables of the objective functions. 

\# Create problem Variables 

x = LpVariable.dicts('x',distances, 0,1,LpBinary)

u = LpVariable.dicts('u', sites, 0, len(sites)-1, LpInteger)

#Step 3: Denotes the objective function in terms of defined variables.

\# Objective Function

cost = lpSum([x[(i,j)]*distances[(i,j)] for (i,j) in distances])

prob+=cost
#Step 4: These are the constraint the variables.

\# Constraints:

for k in sites:

 cap = 1 if k != 'Berlin' else K

 #inbound connection

 prob+= lpSum([ x[(i,k)] for i in sites if (i,k) in x]) ==cap

 #outbound connection

 prob+=lpSum([ x[(k,i)] for i in sites if (k,i) in x]) ==cap

#subtour elimination

N=len(sites)/K

for i in sites:

 for j in sites:

 if i != j and (i != 'Berlin' and j!= 'Berlin') and (i,j) in x:

 prob += u[i] - u[j] <= (N)*(1-x[(i,j)]) – 1

#Step 5: This will show you the problem in the output screen.

#Solver

status = Lp_prob.solve() # Solver

print(p.LpStatus[status]) # The solution status.

Wall time: 52min 38s

Optimal


The optimal route:

non_zero_edges = [ e for e in x if value(x[e]) != 0 ]

def get_next_site(parent):

 '''helper function to get the next edge'''

 edges = [e for e in non_zero_edges if e[0]==parent]

 for e in edges:

 non_zero_edges.remove(e)

 return edges

tours = get_next_site('Berlin')

tours = [ [e] for e in tours ]

for t in tours:

 while t[-1][1] !='Berlin':

 t.append(get_next_site(t[-1][1])[-1])

for t in tours:

 print(' -> '.join([ a for a,b in t]+['Berlin']))

Berlin -> Copenhagen -> Stockholm -> Saint Petersburg -> Moscow -> Kiev -> Warsaw -> Berlin

Berlin -> Hamburg -> Dublin -> London -> Paris -> Brussels -> Berlin

Berlin -> Munich -> Milan -> Madrid -> Barcelona -> Rome -> Prague -> Berlin

Berlin -> Vienna -> Belgrade -> Sofia -> Istanbul -> Bucharest -> Budapest -> Berlin

Calculate total time:

totalTime = 0;

for t in tours:

 time = 0

 for i in range(0, len(t)):

 time += flighttime.loc[t[i][0], t[i][1]]

\# print(flighttime.loc[t[i][0], t[i][1]])

\# print(time)

 if time > totalTime:

 totalTime = time

print(totalTime)

#draw the tours

colors = [np.random.rand(3) for i in range(len(tours))]

for t,c in zip(tours,colors):

 for a,b in t:

 p1,p2 = positions[a], positions[b]

 plt.plot([p1[0],p2[0]],[p1[1],p2[1]], color=c)


#draw the map again

for s in positions:

 p = positions[s]

 plt.plot(p[0],p[1],'o')

 plt.text(p[0]+.01,p[1],s,horizontalalignment='left',verticalalignment='center')

plt.title('%d '%K + 'people' if K > 1 else 'person')

plt.xlabel('latitude')

plt.ylabel('longitude')

\# plt.gca().axis('off')

plt.show()

print('Longest time spent:', totalTime, '(min)')

print('Total distance:', value(prob.objective), '(km)')

Longest time spent: 1988 (min)

Total distance: 15425.02 (km)
```

## 
References

- Dantzig, George Bernard; Ramser, John Hubert (October 1959). HYPERLINK "http://andresjaquep.files.wordpress.com/2008/10/2627477-clasico-dantzig.pdf" "The Truck Dispatching Problem" (PDF). Management Science. 6 (1): 80–91. HYPERLINK "https://en.wikipedia.org/wiki/Doi_(identifier)" \o "Doi (identifier)" doi: HYPERLINK "https://doi.org/10.1287%2Fmnsc.6.1.80" 10.1287/mnsc.6.1.80.
- Toth, P.; Vigo, D., eds. (2002). The Vehicle Routing Problem. Monographs on Discrete Mathematics and Applications. 9. Philadelphia: Society for Industrial and Applied Mathematics. HYPERLINK "https://en.wikipedia.org/wiki/ISBN_(identifier)" \o "ISBN (identifier)" ISBN HYPERLINK "https://en.wikipedia.org/wiki/Special:BookSources/0-89871-579-2" \o "Special:BookSources/0-89871-579-2" 0-89871-579-2.
- Knut-Andreas Lie; Ewald Quak, eds. (2007). Geometric Modelling, Numerical Simulation, and Optimization:: Applied Mathematics at SINTEF. Berlin: Springer Verlag. HYPERLINK "https://en.wikipedia.org/wiki/ISBN_(identifier)" \o "ISBN (identifier)" ISBN HYPERLINK "https://en.wikipedia.org/wiki/Special:BookSources/978-3-540-68783-2" \o "Special:BookSources/978-3-540-68783-2" 978-3-540-68783-2.
- Comtois, Claude; Slack, Brian; Rodrigue, Jean-Paul (2013). The geography of transport systems (3rd ed.). London: Routledge, Taylor & Francis Group. HYPERLINK "https://en.wikipedia.org/wiki/ISBN_(identifier)" \o "ISBN (identifier)" ISBN HYPERLINK "https://en.wikipedia.org/wiki/Special:BookSources/978-0-415-82254-1" \o "Special:BookSources/978-0-415-82254-1" 978-0-415-82254-1.
- Chao, I-Ming; Golden, Bruce L; Wasil, Edward A (1996). "The Team Orienteering Problem". European Journal of Operational Research. 88(3): 464–474. HYPERLINK "https://en.wikipedia.org/wiki/Doi_(identifier)" \o "Doi (identifier)" doi: HYPERLINK "https://doi.org/10.1016%2F0377-2217%2894%2900289-4" 10.1016/0377-2217(94)00289-4.
- Archetti, C.; Sperenza, G.; Vigo, D. (2014). "Vehicle routing problems with profits". In Toth, P.; Vigo, D. (eds.). Vehicle Routing: Problems, Methods, and Applications (Second ed.). pp. 273–297. HYPERLINK "https://en.wikipedia.org/wiki/Doi_(identifier)" \o "Doi (identifier)" doi: HYPERLINK "https://doi.org/10.1137%2F1.9781611973594.ch10" 10.1137/1.9781611973594.ch10.
- Hammami, Farouk; Rekik, Monia; Coelho, Leandro C. (2020). "A hybrid adaptive large neighborhood search heuristic for the team orienteering problem". Computers & Operations Research. 123: 105034. HYPERLINK "https://en.wikipedia.org/wiki/Doi_(identifier)" \o "Doi (identifier)" doi: HYPERLINK "https://doi.org/10.1016%2Fj.cor.2020.105034" 10.1016/j.cor.2020.105034.
- Ekici, Ali; Özener, Okan Örsan; Kuyzu, Gültekin (November 2015). "Cyclic Delivery Schedules for an Inventory Routing Problem". Transportation Science. 49 (4): 817–829. HYPERLINK "https://en.wikipedia.org/wiki/Doi_(identifier)" \o "Doi (identifier)" doi: HYPERLINK "https://doi.org/10.1287%2Ftrsc.2014.0538" 10.1287/trsc.2014.0538.
- https://github.com/jwang0306/vehicle-routing-problem

The vehicle routing problem (VRP) is a combinatorial optimization and integer programming problem seeking to service a number of customers with a fleet of vehicles. Proposed by Dantzig and Ramser in 1959, VRP is an important problem in the fields of transportation, distribution, and logistics.

Machine Learning  - Open Vehicle Routing Problem

A supply chain encompasses all steps in transferring products from the manufacturer to the end customer. Optimizing this process allows it to function at peak efficiency. To better understand supply chain optimization, it’s important to delve deeper into a supply chain’s three main phases: design, planning, and execution/operation.

Supply chain optimization refers to the tools and processes by which manufacturing and distribution supply chain performance and efficiency are improved, taking into account all constraints. Supply chain network optimization technologies use sophisticated algorithms and analytics to balance supply and demand in such a manner that sufficient raw materials are procured for manufacturing and distribution to meet customer expectations at the highest cost efficiency. Supply chains are complex. To best serve you, we go beyond just managing supply chains to offer complete optimization. It’s imperative you don’t waste money or time on valuable shipments.

<img>

There are a number of definitions for supply chain optimization. Our perspective is a broad definition, where the goal of supply chain optimization is to maximize revenue and/or product margins by building the right thing at the right time for the right people. To achieve this goal requires a sophisticated level of understanding and action. So, there are three internal corporate objectives for supply chain optimization. At a basic level these are the goals for: finance (inventory minimization), sales (flexibility and availability), and production (consistent production line loading).

However, the goal is to optimize revenue and/or product margin. This requires viewing the full end-to-end cycle of the value chain. The value chain adds the processes that happen between the shipment of your product and it reaching the hands of the end customer.

### The three phases of a supply chain:

1. Supply chain design – the strategic decisions taken by upper management, usually long-term decisions that put future predictions into consideration. Studying the market is vital at this phase. A good example of such a strategic decision could be choosing a warehouse’s location.
2. Supply chain planning – this phase is more concerned with demand and supply in the market; therefore, proper market research is required. That being said, it’s also as important to consider competitors’ strategies to respond to demand.
3. Supply chain execution/operations – the final phase in supply chain operations, entailing the execution of the flow of products throughout the entire supply chain. To improve performance and effectiveness, both internal and external operations should be considered as well. This phase is concerned with everything starting with retailers sourcing products from the manufacturer or manufacturing the products themselves as a brand (which also includes steps such as assembling and product testing). It also focuses on logistics such as receiving goods for distribution to regional warehouses before sales, as well as [last mile logistics](https://www.bringg.com/blog/logistics-management/5-steps-last-mile-logistics/).

When it comes to supply chain optimization, the focus should be on optimizing it to be customer-centric. Supply chains that are customer-centric keep the final customer in mind throughout every part of the process – especially when choosing a supply chain approach, and a technology solution for better business processes.

## Best practices for supply chain optimization

Now that the elements and phases of a supply chain are clear, here are some best practices for how to optimize your supply chain that are applicable to every industry:

## Ensure you have the proper team in place for the entire supply chain structure

When you have a particularly large supply chain network, with multiple third party teams and partners, data can too easily become siloed. Products can accumulate different I.D.s as they hop from link to link in the supply chain, which becomes impossible to track. Integrating the systems and managing the data from one central location using a centralized data management system helps ensure a smooth transfer of data and alignment between different parts of the supply chain.

## Identify areas where technology can digitize and add transparency

According to a recent survey 15% of companies reported having either automated reporting or conveyance processes, and another 46% plan on automating reporting in the near future. There is clear recognition that manually-driven supply chain processes have led to an ongoing lack of supply chain visibility. Digitizing and automating supply chain management using a dedicated technology solution can solve this problem by connecting data from processes and resources across the supply chain network in one centralized place.

## Align with your customer expectations

Improving customer expectations should be one of the outcomes of optimizing your supply chain, especially when it comes to speed, price, and availability. Visibility on both ends plays a large part in this. Digitized inventory management can provide visibility into inventory levels at each location. Machine learning and artificial intelligence solutions even allow a business to forecast demand. So, before a particular product runs out of stock, it would be resupplied to meet demand and ensure short lead times from order to delivery.

## Use data to improve demand planning and forecasting

To evaluate a supply chain optimization process, most supply chain managers will look at the bottom line: will it reduce supply chain costs? Will it lower operational expenses? This, as well as increasing revenue, will always be the primary company goals, and a good way to achieve them is to look at inventory holding costs. The cost of storing inventory makes up a major percentage of a product’s total cost. That’s why it’s crucial to properly forecast demand and plan accordingly in order to optimize inventory quantities.

Machine learning would also be very beneficial for proper forecasting. Machine learning automates data analysis as well as analytical model building. In other words, the ability to learn from data and identifying patterns can essentially help improve decision making and performance.

A good example would be eliminating time buffers. If machine learning could predict that the time required for drivers to reach a certain warehouse is one and a half hours rather than two, for instance, then that estimation could be incorporated into [dispatching](https://www.bringg.com/blog/dispatching/dispatch-software/) and route planning, reducing lead times across the last mile of the supply chain. That, of course, means the same resources to perform more deliveries, reducing the related transportation cost. More business, less waste.

## Goals of supply chain optimization

No two supply chains are the same. That’s why the best approach for supply chain optimization is to choose an optimization process based on an organization’s business goals and operational objectives. Here are some examples:

## Inventory optimization

As previously mentioned, inventory holding costs are a significant element in the equation, but that’s not the only end goal behind improved inventory control. Having an inventory management system helps a company identify best-selling and slow-selling items. It also helps with deciding where inventory should be stored, which items require quick access, and what could be stored at warehouses in further locations. Generally, inventory control provides information that is invaluable for achieving business goals and improving multiple business processes.

## Faster shipping & order fulfillment

For any process to flow smoothly, all parties involved (whether suppliers, manufacturers, distributors, or retailers) must be on the same page when it comes to logistics, in particular. If proper coordination and supply chain optimization is put into action, this would automatically work in the favor of other goals as well. That includes reducing shipping costs, boosting revenues and profits, and, of course, improving customer service.

## Operational efficiency

Efficiency is one of the key aspects of supply chain management and optimization across all industries. For instance, streamlined shipping that reduces lead times across the final mile of the supply chain leads to improved customer satisfaction. It’s a domino effect: optimizing one point in your organization or supply chain network will positively affect lead times and performance across the rest.

## Revenue and profit growth

One way to increase revenue is by reducing ‘waste’ along the supply chain (e.g. products that never go to market), and reducing the cost to deliver, reducing operating costs (e.g. getting the same number of drivers to perform more deliveries).

One example of this is transportation costs. You can reduce this across the last mile by using technology such as smart delivery quotes that only show the times when shipping resources are available; real-time tracking for end customers so they know exactly when their delivery will arrive and communication whereby customers can communicate if they will miss a delivery. You can also reduce overhead costs by reducing the number of truck rolls performed (via intelligent dispatch and routing). Every ‘truck roll’ (i.e. a time a truck goes out to perform a run) costs hundreds of dollars. Improving operational efficiency so that you eliminate even one half-empty truck a day per region will turn into tens of thousands of dollars a month.

## Reduced costs

For a business to increase its supply chain’s profitability, it must work on reducing costs and expenses throughout the whole process; be it overheads or variable costs. Overheads, or fixed costs, that don’t change according to the production volume. Good examples of overheads would be the rent of warehouses, managers’ salaries, or depreciation of machinery. On the other hand, variable costs are any costs that increase or decrease according to the number of goods produced: the cost of raw materials, distribution costs and shipping costs, for example.

## Improved customer service

We live in a customer-centric, service-oriented world. No one does once-off sales anymore; everything is (or strives to be) on a subscription model. But with a subscription model, a company has to consistently and continuously prove the worth of the product or service to customers. For a shipper, that would be a consumer. For a logistics provider, it would be both the shipper and the shipper’s customer as well. Improving their experience is what will guarantee customer satisfaction and a larger market share.

So, the end goal of supply chain optimization should relate to improving the overall customer experience. For example, minimizing related costs caused by “where is my order” calls (also known as WISMO calls) will also improve the customer experience. Dealing with WISMO calls requires vast service centers; it’s expensive and can be avoided when you have the technology to provide visibility into orders.

## Case Study - Supply Chain Optimization Techniques

A supply chain is a network of organizations, individuals, activities, information, and resources involved in the production and sale of a service or product to consumers. The chain begins with the transformation of natural, raw resources and components into a finished product or service, and concludes with the delivery of that product or service to the end customer.

Begin your analysis by identifying the current state of the supply chain, including facilities, transportation costs and relationships, inventory levels and locations, and customer distribution facilities and practices. This is the baseline for the rest of the analysis, so it's essential that it be as detailed and accurate as possible.

So, Computer models run alternatives to compare costs and capabilities for various options. Modern supply chain optimization technologies typically provide a platform on which supply chain managers and supply chain optimization analysts gain real-time end-to-end visibility of supply chain operations; standardize, synchronize, and automate business processes and execution; and leverage machine learning capabilities and AI for supply chain optimization.

## Case Supply Chain Network - Code in python

```python
import pandas as pd

from pulp import *

#DataLoad

\# Import Manufacturing Costs

manvar_costs = pd.read_excel('variable_costs.xls', index_col = 0)

\# Import Freight Costs

freight_costs = pd.read_excel('freight_costs.xls', index_col = 0)

\# Variable Cost

var_cost = freight_costs/1000 + manvar_costs 

\# Import Plant Fixed Costs

fixed_costs = pd.read_excel('fixed_cost.xls', index_col = 0)

\# Import Low Capacity and High Capacity Plant

cap = pd.read_excel('capacity.xls', index_col = 0)

\# Import Demand

demand = pd.read_excel('demand.xls', index_col = 0)

# Define Decision Variables

loc = ['USA', 'Germany', 'Japan', 'Brazil', 'India']

size = ['Low', 'High']


\# Create a LP Minimization problem 

model = LpProblem("Supply Chain Optimization", LpMinimize)


\# Create Decision Variables

x = LpVariable.dicts("production_", [(i,j) for i in loc for j in loc], lowBound=0, upBound=None, cat='continuous')

y = LpVariable.dicts("plant_", [(i,s) for s in size for i in loc], cat='Binary')

\# Objective Function

\# Define Objective Function

model += (lpSum([fixed_costs.loc[i,s] * y[(i,s)] * 1000 for s in size for i in loc]) + lpSum([var_cost.loc[i,j] * x[(i,j)] for i in loc for j in loc]))

\# Constraints

for j in loc:

 model += lpSum([x[(i, j)] for i in loc]) == demand.loc[j,'Demand']

for i in loc:

 model += lpSum([x[(i, j)] for j in loc]) <= lpSum([cap.loc[i,s]*y[(i,s)] * 1000 for s in size])

#Solver

model.solve()

print("Total Costs = {:,} ($/Month)".format(int(value(model.objective))))

print('\n' + "Status: {}".format(LpStatus[model.status])) 

\# Dictionnary

dict_plant = {}

dict_prod = {}

for v in model.variables():

 if 'plant' in v.name:

 name = v.name.replace('plant__', '').replace('_', '')

 dict_plant[name] = int(v.varValue)

 p_name = name

 else:

 name = v.name.replace('production__', '').replace('_', '')

 dict_prod[name] = v.varValue

 print(name, "=", v.varValue)

Total Costs = 62,038,000 ($/Month)

Status: Optimal

('Brazil','High') = 1.0

('Brazil','Low') = 0.0

('Germany','High') = 0.0

('Germany','Low') = 0.0

('India','High') = 1.0

('India','Low') = 1.0

('Japan','High') = 1.0

('Japan','Low') = 0.0

('USA','High') = 0.0

('USA','Low') = 0.0

('Brazil','Brazil') = 145000.0

('Brazil','Germany') = 0.0

('Brazil','India') = 0.0

('Brazil','Japan') = 0.0

('Brazil','USA') = 1250000.0

('Germany','Brazil') = 0.0

('Germany','Germany') = 0.0

('Germany','India') = 0.0

('Germany','Japan') = 0.0

('Germany','USA') = 0.0

('India','Brazil') = 0.0

('India','Germany') = 90000.0

('India','India') = 160000.0

('India','Japan') = 200000.0

('India','USA') = 1550000.0

('Japan','Brazil') = 0.0

('Japan','Germany') = 0.0

('Japan','India') = 0.0

('Japan','Japan') = 1500000.0

('Japan','USA') = 0.0

('USA','Brazil') = 0.0

('USA','Germany') = 0.0

('USA','India') = 0.0

('USA','Japan') = 0.0

('USA','USA') = 0.0

```

### Reference

- Supply Chain Optimization – What Is It? [https://tensoft.com/supply-chain-optimization-what-is-it/](https://tensoft.com/supply-chain-optimization-what-is-it/)
- Supply Chaing Optimization: Everything You need To Know [https://www.bringg.com/blog/logistics/supply-chain-optimization/](https://www.bringg.com/blog/logistics/supply-chain-optimization/)
- Everything You Need to Know About Supply Chain Optimization. February 20, 2019. [https://www.warehouseanywhere.com/resources/supply-chain-optimization/](https://www.warehouseanywhere.com/resources/supply-chain-optimization/)
- Supply Chain Optimization [https://www.omnisci.com/technical-glossary/supply-chain-optimization](https://www.omnisci.com/technical-glossary/supply-chain-optimization)
- PuLP: A Linear Programming Toolkit for Python Stuart Mitchell. The University of Auckland, Auckland, New Zealand September 5, 2011.
- Supply Chain Optimization with Python https://towardsdatascience.com/supply-chain-optimization-with-python-23ae9b28fd0b

A supply chain encompasses all steps in transferring products from the manufacturer to the end customer. Optimizing this process allows it to function at peak efficiency. To better understand supply chain optimization, it’s important to delve deeper into a supply chain’s three main phases: design, planning, and execution/operation. 


Supply Chain Optimization Applying Artificial Intelligence

The current dynamic and increasingly serious market has expanded the complexities that constrain organizations around the world from starting to take advantage of Big Data regulations to support competition around the world, as it turns out that they are fetching more data determined in dealing with supply chains. Subjective strategies are applied in this review to better understand the wonder of Supply Chain analytics. An inductive methodology of meeting type was applied to obtain new observational data. Fifteen semiorganized meetings were held with administrative professionals in the study of the supply chain, management and big data analytics. 

The obtained observational data were then analyzed using the topical examination technique by the help of the previous literature and the case studies .Key discoveries in this premise are partially rejected by past reviews and existing articles in terms of the implications, definitions, and uses of the three types of Analytics. Moreover, the discoveries offer new methodologies and perspectives that the latest analytics apply at both basic and operational management levels that shape supply chain management plans.

Supply chain is one of the trickiest parts of a business and as the chances of error are quite high, the need for proper synchronization and analytics is very important. Something simple getting out of hand can make the business lose a lot of money as it can create a break or kink in the supply chain. When it comes to strategic and operational parts of the supply chain management, the need for resolving issues on-the-go to keep the chain continuous is very important or every step in the continuation can become a painful point which will block the resources in the wrong places.

This is the reason why data analytics is now taken very seriously in the management of the supply chain. The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of _[data science](https://360digitmg.com/data-science) in the [supply chain](https://360digitmg.com/india/certification-program-in-supply-chain-analytics)_ is helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

On the expected development in the use of Internet of Things technologies, there are numerous conversation and review discussions on our current strategic approaches along with distributed computing, Analytics of the Data, Artificial intelligence and Machine learnings (Tuptuk & Hailes, 2018) In fact, the importance of analytics has been generally perceived and played a critical role in Supply Chain Management until the transfer of military activities during and after the next World War. In everyday terms, there are several types of assets an association is based on, such as innovative assets, specialized, administrative capabilities, and IT-based assets. To competently deal with these assets, in a best-case scenario, organizations should have the option to adequately deal with their supply chains, keeping in mind the unique competitive environment in current world-class business environments. This underscores the need for effective reconciliation and joint effort among supply chain accomplices; this can be implemented with the latest improvements in innovation and results. These frameworks are, in general, with Supply Chain Management and attached data and financial flows. 

SCM is the management of the progress of labor and products and includes all the cycles that transform raw materials into final results. It includes dynamic flattening from supply-side exercises of a business to increase customer value and gain an edge in the commercial center. Generally, Supply Chain Management tries to half-control or link the creation, shipping, and allocation of an item. By dealing with the supply chain, organizations can reduce excess redundancy costs and deliver products to the customer faster. This is complemented by tighter control of internal inventories, inside creation, circulation, deals, and inventories of organization vendors. Supply Chain Management depends on the probability that almost every item that comes to display is due to the efforts of different associations that make up a supply chain. Although supply chains have been around for a very long time, most organizations have focused on them recently to add value to their tasks. 

The supply chain is the organization of different type, firms, entities, exercises and technologies dealing with the assembly and offering of an effective management. A supply chain begins with raw materials moving from a supplier to a manufacturer and closes with the finished product or administration moving to the end customer. Supply Chain Management oversees every touchpoint of an organization's element or management, from the start of creation to the final deal. With so many points along the supply chain that can gain reputation through efficiency or lose credibility due to extended costs, proper Supply Chain Management can increase revenues, lower expenses and affect an organization's core concern.

Supply chain management is often depicted as having five key components: arrangement, supply of raw materials, assembly, transportation, and returns. While the regulation phase implies promoting a general system for the supply chain, the other four components have some expertise in critical needs to achieve this regulation. Organizations must develop talent in each of the five components to have a competent supply chain and avoid costly bottlenecks. Acknowledging the expanding metrics of data, two experts and researchers highlight the importance of Big Data around the world and the possibility of adding value and improving companies. Big Data is defined as massive or complex data arrangements, whose scope is an exabyte, from which the sky is the limit. 

The capacity framework transcends the specialized capacity domain, such as drafting, overseeing, decoding and depicting a traditional framework. As the Business Analytics (BA) team, where the business follows insightful procedures, strategies and data-based logical systems, the term Big Data Analytics data credits depicted by capacity, diversity, and rapidity are measured and regulated by applying thoughts to choose better decisions. These links practice a wide range of intelligent thoughts. Supply Chain Analytics is clearly here to improve data planning limits and supply chain business. This strengthens working with the unique cycle in the union's supply chain.

**Some of the main steps that are covered in the supply chain of the business are:**

Procurement of raw materials

Inbound logistics

Inventory of parts

Manufacturing of goods

Inventory of finished goods

Fulfillment order of the goods (customer goods)

Outbound logistics

All these processes are very different on their own and require separate attention. However, there is no denying that they are entangled with each other and share dependability. For each and every step to be completed successfully, there is a need for the preceding one to be completed on time. Failing to complete any one step means the entire supply chain will collapse. To handle every process of the supply chain, there is the need for separate operational channels, management, visibility, and data collection.

## Supply Chain and Predictive Analysis with Machine Learning

As mentioned above, supply chain management is all about accurate forecasting where further decisions regarding logistics and production can be taken. This is the reason why predictive analytics can be very useful for the business to make their supply chain more productive. Supply chain forecasting is a tricky situation as there are many variables to take into consideration. Long-term forecasting is still something that can be managed, but short-term forecasting can become quite difficult. This is because the number of variables that come to the scene in short-term forecasting are numerous and hard to track.

Predictive analytics is about predicting future phenomena using historical data sets. Therefore, for accurate forecasting, predictive analytics is highly helpful as it shows various mathematical models of predictive forecasting. Also, the predictive analytics tools help compare two roadmaps and decide which is more profitable and efficient. It takes into consideration various factors, both internal and external. This process is known as the optimization of supply chain processes. Therefore, it can be said that the _predictive analytics of [data science](https://360digitmg.com/data-science)_ are highly helpful in forecasting, optimization, and visualization of various processes in a supply chain of businesses.

## Business Case

A vendor supplies different materials with wide unit prices and total purchase order values. Sometimes a few parts from suppliers fail to pass the quality checks, but the defect percentage doesn’t seem to be following a trend. So, The business would like to predict the defective piece percentage in delivery from the supplier. If the defective piece percentage predicted is below the threshold level, then a quality check will not be performed. Let us consider that if the defect piece percentage in the delivery is more than 0.4%, then explicit incoming inspection is needed to ensure the quality of the end product. This will help to focus on quality checks on only particular purchase orders delivery, to control the endproduct quality, and optimize inspection costs. It will also enable us to uncover the variables/parameters which are influencing the defective deliveries of few materials at times and work collaboratively with the supplier to address it.

Example of business case taken from article “Leveraging Machine Learning for Efficiency in Supply Chain Management”, with new contributions at the code in python.

```python

""" Step 1 - Import the required modules"""

import pandas as pd

import numpy as np 

from sklearn.tree import DecisionTreeRegressor # Import Decision Tree Algorithm

from sklearn.preprocessing import StandardScaler

from sklearn.svm import SVR #import for support vector regressor

import matplotlib.pyplot as plt

import pyEX as p

""" Step 2 - Read the data source"""

SourceData=pd.read_excel("Supplier Past Performance.xls") # Load the training data into Pandas 

Testdata=pd.read_excel("Defect Predict.xls") # Load the test data

""" Step 3 - Declare the independent and dependent train data from the sample"""

SourceData_train_independent= SourceData.drop(["Defect Percent"], axis=1) # Drop depedent variable from training dataset

SourceData_train_dependent=SourceData["Defect Percent"].copy() # New dataframe with only independent variable value for training dataset

SourceData_train_independent.head(5)

""" Step 4 - Scale the independent test and train data"""

sc_X = StandardScaler()

X_train=sc_X.fit_transform(SourceData_train_independent.values) # scale the independent variables

y_train=SourceData_train_dependent # scaling is not required for dependent variable

X_test=sc_X.transform(Testdata.values)

""" Step 5 - Fit the test data in maching learning model - Support Vector Regressor"""

svm_reg = SVR(kernel="linear", C=1)

svm_reg.fit(X_train, y_train) # fit and train the modelpredictions = svm_reg.predict(X_test)

“””

SVR(C=1, cache_size=200, coef0=0.0, degree=3, epsilon=0.1, gamma='scale', kernel='linear', max_iter=-1, shrinking=True, tol=0.001, verbose=False)

“”"

""" Step 6 - Predictions - Support Vector Regressor"""

y_pred_svm = svm_reg.predict(X_test)

print(y_pred_svm)

""" Step 7 - Fit the test data in machine learning model - Decision Tree Model"""

tree_reg = DecisionTreeRegressor()

tree_reg.fit(X_train, y_train)

“””

DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=None,

 max_features=None, max_leaf_nodes=None,

 min_impurity_decrease=0.0, min_impurity_split=None,

 min_samples_leaf=1, min_samples_split=2,

 min_weight_fraction_leaf=0.0, presort='deprecated',

 random_state=None, splitter='best')

“””

""" Step 8 - Predictions - Decision Tree"""

y_pred_tree = tree_reg.predict(X_test)

print(y_pred_tree)

""" Step 9 - Graphs Predictions - SVM - Decision Tree"""

plt.plot(SourceData_train_independent['ID'], y_pred_svm, label='SVM',color = 'black')

plt.plot(SourceData_train_independent['ID'], y_pred_tree, label='Decision Tree',color = 'blue')

plt.legend(loc='upper right')

plt.title('Predictions SVR - Decision Tree')

plt.xlabel('Suppliers')

plt.ylabel('Defect Predict')

plt.show()

```

<img>

## References

- Leveraging Machine Learning for Efficiency in Supply Chain Management. 2020. https://www.analyticsvidhya.com/blog/2020/07/leveraging-machine-learning-for-efficiency-in-supply-chain-management/
- [Enterprises put AI in supply chain to streamline processes](https://pythoniseasytolearn.blogspot.com/2019/08/enterprises-put-ai-in-supply-chain-to.html). [Robotics and Motion Control](https://www.googleadservices.com/pagead/aclk?sa=L&ai=CvfRwOer0YJLvFIHN1fAPwaW92Aiuxv3cY5_a9tHXBcCNtwEQASDVp_5wYN_2-Y3cJaABvZ7yzAPIAQGpAlTO-HYzzrM-qAMByAPDBKoEiQJP0Mx8rJYSYF0Ex7k2At_aFYzjtdxa3oxUqfDqyLWQfxRMYd21hlK8gNqit6yuKpp0o-1myp6CmQwit3-88PAKHmLr6pFlbi2nyEbbqVG9xn6jN23diYIjVhU0ba5Vb3R7GJ5FrnhTVBC-RkDwPYLXONVqKqPJXowGVhAsTmXvPRe380UUMYbk9maTyidkqDaiXmEI5-hoNLAQPaH8OOWOnZaHJ2U5qGbDuLHRAV-wLDsMg1_H-kYeBXgP9QOnMfDZ55YnTjJqj9MDSpOmgaXx4l7btW2CRnZgq-6ChqhRDeRwrDGAGo9a9Kxf_w4qXZe2guT-R3xzTjuSTW83ATRGDinGkObO6sOGwASgntrGtgGgBlGAB6vhjTOoB9XJG6gH8NkbqAfy2RuoB47OG6gHk9gbqAe6BqgH7paxAqgHpr4bqAfs1RuoB_PRG6gH7NUbqAeW2BvYBwHSCAcIgGEQARgfsQnD90GGLUl0C4AKAZgLAcgLAbgMAdgTDdAVAYAXAQ&ae=1&num=1&cid=CAASEuRo78jhhfpBdnEudkxIxt14_g&sig=AOD64_2jtiVWfVtxEvjhDd3ai6np9Fg3Uw&client=ca-pub-4427033987154771&nb=0&adurl=https://www.anertkom.de/epages/78014188.sf/en_GB/%3FViewObjectPath%3D%252FShops%252F78014188%252FCategories%252FIndustrieelektronik%252F%2522B%2526R%2520Automation%2522%26gclid%3DEAIaIQobChMI0u_l15Hu8QIVgWYVCB3BUg-LEAEYASAAEgJmPfD_BwE). 2019. https://pythoniseasytolearn.blogspot.com/2019/08/enterprises-put-ai-in-supply-chain-to.html.
- Applications of Data Science in Supply Chain Analytics. 360 DigiTMG. 2021. https://360digitmg.com/applications-of-data-science-in-supply-chain-analytics.
- Machine learning in supply chain. N-ix. 2021. [https://www.n-ix.com/machine-learning-supply-chain-use-cases/](https://www.n-ix.com/machine-learning-supply-chain-use-cases/)
- Advances in Supply Chain Management using Big Data Business Analytics. International Journal of Innovation, Creativity and Change. www.ijicc.net Volume 15, Issue 8, 2021.

The current dynamic and increasingly serious market has expanded the complexities that constrain organizations around the world from starting to take advantage of Big Data regulations to support competition around the world, as it turns out that they are fetching more data determined in dealing with supply chains. Subjective strategies are applied in this review to better understand the wonder of Supply Chain analytics. An inductive methodology of meeting type was applied to obtain new observational data. Fifteen semiorganized meetings were held with administrative professionals in the study of the supply chain, management and big data analytics.

Supply Chain Analytics with Machine Learning

# Optimization Intelligent in Food Manufacturing

## Introduction

If you’re hungry for a mathematical optimization challenge, then try this food manufacturing problem. You’ll learn how to create an optimal multi-period production plan for a product that requires a number of ingredients – each of which has different costs, restrictions, and features.

So, Food processing is the transformation of either raw materials such as natural animals and plants into food, or of one form of food into other forms which are more suitable for the dietary habits of modern people. Therefore, food processing is closely related to the quality of modern people’s life and the economic development of the whole society. Food processing includes diverse aspects (e.g., the manufacturing, modification and manufacturing of food), which require a series of physical and chemical changes of the raw materials. However, due to an increase in environmental pollution, people are concerned about the safety of both the food sources and the food processing procedures. Throughout the process, it is necessary to ensure that the nutritional properties of the raw materials are retained, and that toxic and harmful substances are not introduced into the food. Therefore, food processing is of high value to food scientists, the food industry, and consumers.

Our perspective is a broad definition, where the goal of supply chain optimization is to maximize revenue and/or product margins by building the right thing at the right time for the right people. To achieve this goal requires a sophisticated level of understanding and action. So, there are three internal corporate objectives for supply chain optimization. At a basic level these are the goals for: finance (inventory minimization), sales (flexibility and availability), and production (consistent production line loading). However, the goal is to optimize revenue and/or product margin. This requires viewing the full end-to-end cycle of the value chain. The value chain adds the processes that happen between the shipment of your product and it reaching the hands of the end customer.

So, According to a recent survey 15% of companies reported having either automated reporting or conveyance processes, and another 46% plan on automating reporting in the near future. There is clear recognition that manually-driven supply chain processes have led to an ongoing lack of supply chain visibility. Digitizing and automating supply chain management using a dedicated technology solution can solve this problem by connecting data from processes and resources across the supply chain network in one centralized place.

Efficiency is one of the key aspects of supply chain management and optimization across all industries. For instance, streamlined shipping that reduces lead times across the final mile of the supply chain leads to improved customer satisfaction. It’s a domino effect: optimizing one point in your organization or supply chain network will positively affect lead times and performance across the rest.

Food production industry is a highly competitive market, hence reducing the production cost is critical for the profitability and competitiveness of the product. Producing food can be a complex process. The product should not only taste and look good but usually other properties such as sensory, kinetic, or physic-chemical properties must be within its specifications.

Here is one applications of AI in the food industry. So, Sorting Food: It can be quite time-consuming to sort fresh produce in order to prepare it for the manufacturing of canned and bagged goods. For example, sorting the size of potatoes can help manufacturers make decisions regarding which ones should be made into French fries, potato chips, or hash browns.

## Case Optimization Intelligent in Food Manufacturing - Code in python [1,2].

If you’re hungry for a mathematical optimization challenge, then try this food manufacturing problem. You’ll learn how to create an optimal multi-period production plan for a product that requires a number of ingredients – each of which has different costs, restrictions, and features. This modeling example is at the intermediate level, where we assume that you know Python and are familiar with the Gurobi Python API. In addition, you should have some knowledge about building mathematical optimization models.

```python
#Code in Python

import numpy as np

import pandas as pd

import gurobipy as gp

from gurobipy import GRB

\# Parameters




months = ["Jan", "Feb", "Mar", "Apr", "May", "Jun"]




oils = ["VEG1", "VEG2", "OIL1", "OIL2", "OIL3"]




cost = {

 ('Jan', 'VEG1'): 110,

 ('Jan', 'VEG2'): 120,

 ('Jan', 'OIL1'): 130,

 ('Jan', 'OIL2'): 110,

 ('Jan', 'OIL3'): 115,

 ('Feb', 'VEG1'): 130,

 ('Feb', 'VEG2'): 130,

 ('Feb', 'OIL1'): 110,

 ('Feb', 'OIL2'): 90,

 ('Feb', 'OIL3'): 115,

 ('Mar', 'VEG1'): 110,

 ('Mar', 'VEG2'): 140,

 ('Mar', 'OIL1'): 130,

 ('Mar', 'OIL2'): 100,

 ('Mar', 'OIL3'): 95,

 ('Apr', 'VEG1'): 120,

 ('Apr', 'VEG2'): 110,

 ('Apr', 'OIL1'): 120,

 ('Apr', 'OIL2'): 120,

 ('Apr', 'OIL3'): 125,

 ('May', 'VEG1'): 100,

 ('May', 'VEG2'): 120,

 ('May', 'OIL1'): 150,

 ('May', 'OIL2'): 110,

 ('May', 'OIL3'): 105,

 ('Jun', 'VEG1'): 90,

 ('Jun', 'VEG2'): 100,

 ('Jun', 'OIL1'): 140,

 ('Jun', 'OIL2'): 80,

 ('Jun', 'OIL3'): 135

}







hardness = {"VEG1": 8.8, "VEG2": 6.1, "OIL1": 2.0, "OIL2": 4.2, "OIL3": 5.0}




price = 150

init_store = 500

target_store = 500

veg_cap = 200

oil_cap = 250




min_hardness = 3

max_hardness = 6

holding_cost = 5

food = gp.Model('Food Manufacture I')

\# Quantity of food produced in each period

produce = food.addVars(months, name="Produce")

\# Quantity bought of each product in each period

buy = food.addVars(months, oils, name = "Buy")

\# Quantity used of each product in each period

consume = food.addVars(months, oils, name = "Use")

\# Quantity stored of each product in each period

store = food.addVars(months, oils, name = "Store")

#1. Initial Balance

Balance0 = food.addConstrs((init_store + buy[months[0], oil]

 == consume[months[0], oil] + store[months[0], oil]

 for oil in oils), "Initial_Balance")




#2. Balance

Balance = food.addConstrs((store[months[months.index(month)-1], oil] + buy[month, oil]

 == consume[month, oil] + store[month, oil]

 for oil in oils for month in months if month != month[0]), "Balance")

#3. Inventory Target

TargetInv = food.addConstrs((store[months[-1], oil] == target_store for oil in oils),"End_Balance")

#4.1 Vegetable Oil Capacity

VegCapacity = food.addConstrs((gp.quicksum(consume[month, oil] for oil in oils if "VEG" in oil)

 <= veg_cap for month in months), "Capacity_Veg")




#4.2 Non-vegetable Oil Capacity

NonVegCapacity = food.addConstrs((gp.quicksum(consume[month, oil] for oil in oils if "OIL" in oil)

 <= oil_cap for month in months), "Capacity_Oil")

#5. Hardness

HardnessMin = food.addConstrs((gp.quicksum(hardness[oil]*consume[month, oil] for oil in oils)

 >= min_hardness*produce[month] for month in months), "Hardness_lower")

HardnessMax = food.addConstrs((gp.quicksum(hardness[oil]*consume[month, oil] for oil in oils)

 <= max_hardness*produce[month] for month in months), "Hardness_upper")

#6. Mass Conservation

MassConservation = food.addConstrs((consume.sum(month) == produce[month] for month in months), "Mass_conservation")




#The objective is to maximize the profit of the company, which includes the revenue as well as the costs for buying and storing the used products.

#0. Objective Function

obj = price*produce.sum() - buy.prod(cost) - holding_cost*store.sum()

food.setObjective(obj, GRB.MAXIMIZE) # maximize profit

food.optimize() 
```

## Analysis

The highest profit we can hope for is $107,842.6$107,842.6. Let's see an optimal solution needed to achieve that figure (there are alternative optimal solutions):

## Purchase Plan

This plan defines the amount of vegetable oil (VEG) and non-vegetable oil (OIL) that we need to purchase during the planning horizon. For example, 659.3 tons of vegetable oil of type VEG1 needs to be bought in June.

```python

rows = months.copy()

columns = oils.copy()

purchase_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)


for month, oil in buy.keys():

 if (abs(buy[month, oil].x) > 1e-6):

 purchase_plan.loc[month, oil] = np.round(buy[month, oil].x, 1)

purchase_plan

```

## Monthly Consumption

This plan determines the amount of vegetable oil (VEG) and non-vegetable oil (OIL) consumed during the planning horizon. For example, 200 tons of vegetable oil of type VEG2 is consumed in January.

```python

In [12]:

rows = months.copy()

columns = oils.copy()

reqs = pd.DataFrame(columns=columns, index=rows, data=0.0)

for month, oil in consume.keys():

 if (abs(consume[month, oil].x) > 1e-6):

 reqs.loc[month, oil] = np.round(consume[month, oil].x, 1)

reqs

```

## Inventory Plan

This plan reflects the amount of vegetable oil (VEG) and non-vegetable oil (OIL) in inventory at the end of each period of the planning horizon. For example, at the end of February we have 500 tons of Non-vegetable oil of type OIL1.

```python
In [13]:

rows = months.copy()

columns = oils.copy()

store_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)

for month, oil in store.keys():

 if (abs(store[month, oil].x) > 1e-6):

 store_plan.loc[month, oil] = np.round(store[month, oil].x, 1)

store_plan

```

### Reference

- H. Paul Williams, Model Building in Mathematical Programming, fifth edition. 2020 Gurobi Optimization, LLC.
- Food Manufacture HYPERLINK "https://gurobi.github.io/modeling" https://gurobi.github.io/modeling-examples/food_manufacturing_1_2/food_manufacture_1.html
- Artificial Intelligent Apply Food Manufacturing. CobuildLab 2021 HYPERLINK "https://www.cobuildlab.com/blog/ai" https://www.cobuildlab.com/blog/ai
- Artificial Intelligent Apply Food Manufacturing and Vehicle Routing Problem. HYPERLINK "https://www.cobuildlab.com/blog/ai" https://www.cobuildlab.com/blog/ai

If you’re hungry for a mathematical optimization challenge, then try this food manufacturing problem. You’ll learn how to create an optimal multi-period production plan for a product that requires a number of ingredients – each of which has different costs, restrictions, and features.

Optimization Intelligent in Food Manufacturing 

# Optimization Intelligent in Food Manufacturing – Part II

## Introduction

There is a HYPERLINK "https://www.math.uni-bielefeld.de/documenta/vol-ismp/25_bixby-robert.pdf" long and rich history of the theoretical development of robust and efficient solvers for optimization problems. However, focusing on practical applications, we will skip that history and move straight to the part of learning how to use programmatic tools to formulate and solve such optimization problems. So, Knowledge of such optimization techniques is extremely useful for data scientists and machine learning (ML) practitioners as discrete and continuous optimization lie at the heart of modern ML and AI systems as well as HYPERLINK "https://www.edx.org/course/optimization-methods-business-analytics-mitx-15-053x" data-driven business analytics processes.

A supply chain encompasses all steps in transferring products from the manufacturer to the end customer. Optimizing this process allows it to function at peak efficiency. To better understand supply chain optimization, it’s important to delve deeper into a supply chain’s three main phases: design, planning, and execution/operation. So, Supply chain network optimization technologies use sophisticated algorithms and analytics to balance supply and demand in such a manner that sufficient raw materials are procured for manufacturing and distribution to meet customer expectations at the highest cost efficiency. Supply chains are complex. To best serve you, we go beyond just managing supply chains to offer complete optimization. It’s imperative you don’t waste money or time on valuable shipments.

Optimization is the science / process behind finding the best solution for a problem with given constraints. We come across optimization problems on a daily basis. These can be for finding the shortest path between your work place and office; maximizing revenues / customer happiness or minimizing costs / debts etc. One of the major goals of the modern enterprise of data science and analytics is to HYPERLINK "https://www.quora.com/Is-optimization-related-to-data-science-And-how" \t "_blank" solve complex optimization problems for business and technology companies to maximize their profit. We come across optimization problems on a daily basis. These can be for finding the shortest path between your work place and office; maximizing revenues / customer happiness or minimizing costs / debts etc. We basically take a real world problem, model it mathematically and then solve it using mathematical techniques with in the constraints. Optimization is useful in Marketing, Manufacturing, Finance, Online advertising, Machine Learning and all fields you can imagine.

There are many excellent optimization packages in Python. Many optimization solvers (commercial and open-source) have Python interfaces for modeling LPs, MILPs, and QPs. Here I’ve selected CPLEX and Gurobi, since they are among the leading commercial solvers, and PuLP, which is a powerful open-source modeling package in Python. I’ll provide a side-by-side tutorial for each of these packages, and I hope it will help you to easily translate your model from one to another. Here, we use gurobipy (Gurobi’s Python API), docplex (the IBM Decision Optimization CPLEX Modeling package for Python), and pulp (an LP/MILP modeler written in Python). For the purpose of this post, I’ll assume that you are familiar with Python, i.e., you know how to install and use Python packages and use Python data structures like lists, tuples and dictionaries. I’ll also assume basic knowledge of linear programming, mixed integer programming, and constrained optimization.

## Case Optimization Intelligent in Food Manufacturing - Code in python [1,2].

If you’re hungry for a mathematical optimization challenge, then try this food manufacturing problem. You’ll learn how to create an optimal multi-period production plan for a product that requires a number of ingredients – each of which has different costs, restrictions, and features.

## Problem Description

A manufacturer needs to refine several raw oils and blend them together to produce a given food product that can be sold. The raw oils required can be divided into two categories:

## Model Formulation

### Sets and Indices

t∈Months={Jan,Feb,Mar,Apr,May,Jun}t∈Months={Jan,Feb,Mar,Apr,May,Jun}: Set of months.

V={VEG1,VEG2}V={VEG1,VEG2}: Set of vegetable oils.

N={OIL1,OIL2,OIL3}N={OIL1,OIL2,OIL3}: Set of non-vegetable oils.

o∈Oils=V∪No∈Oils=V∪N: Set of oils.

### Parameters

Price ∈ R+: Sale price of the final product.

init_store ∈ R+: Initial storage amount in tons.

target_store ∈ R+: Target storage amount in tons.

holding_cost ∈ R+: Monthly cost (in USD/ton/month) of keeping in inventory a ton of oil.

veg_cap ∈ R+: Installed capacity (in tons) to refine vegetable oils.

oil_cap ∈ R+: Installed capacity (in tons) to refine non-vegetable oils.

min_hardness ∈ R+: lowest hardness allowed for the final product.

max_hardness ∈ R+: highest hardness allowed for the final product.

Hardnesso ∈ R+: Hardness of oil o.

costt,o ∈ R+: Estimated purchase price for oil o at month t.

### Decision Variables

Producet ∈ R+: Tons of food to produce at month t.

buyt,o ∈ R+: Tons of oil oo to buy at month t.

consumet,o ∈ R+: Tons of oil oo to use at month t.

storet,o ∈ R+: Tons of oil oo to store at month t

```python

#Code in Python

import numpy as np

import pandas as pd

import gurobipy as gp

from gurobipy import GRB

\# Parameters




months = ["Jan", "Feb", "Mar", "Apr", "May", "Jun"]




oils = ["VEG1", "VEG2", "OIL1", "OIL2", "OIL3"]




cost = {

 ('Jan', 'VEG1'): 110,

 ('Jan', 'VEG2'): 120,

 ('Jan', 'OIL1'): 130,

 ('Jan', 'OIL2'): 110,

 ('Jan', 'OIL3'): 115,

 ('Feb', 'VEG1'): 130,

 ('Feb', 'VEG2'): 130,

 ('Feb', 'OIL1'): 110,

 ('Feb', 'OIL2'): 90,

 ('Feb', 'OIL3'): 115,

 ('Mar', 'VEG1'): 110,

 ('Mar', 'VEG2'): 140,

 ('Mar', 'OIL1'): 130,

 ('Mar', 'OIL2'): 100,

 ('Mar', 'OIL3'): 95,

 ('Apr', 'VEG1'): 120,

 ('Apr', 'VEG2'): 110,

 ('Apr', 'OIL1'): 120,

 ('Apr', 'OIL2'): 120,

 ('Apr', 'OIL3'): 125,

 ('May', 'VEG1'): 100,

 ('May', 'VEG2'): 120,

 ('May', 'OIL1'): 150,

 ('May', 'OIL2'): 110,

 ('May', 'OIL3'): 105,

 ('Jun', 'VEG1'): 90,

 ('Jun', 'VEG2'): 100,

 ('Jun', 'OIL1'): 140,

 ('Jun', 'OIL2'): 80,

 ('Jun', 'OIL3'): 135

}




hardness = {"VEG1": 8.8, "VEG2": 6.1, "OIL1": 2.0, "OIL2": 4.2, "OIL3": 5.0}




price = 150

init_store = 500

target_store = 500

veg_cap = 200

oil_cap = 250




min_hardness = 3

max_hardness = 6

holding_cost = 5

food = gp.Model('Food Manufacture I')

\# Quantity of food produced in each period

produce = food.addVars(months, name="Produce")

\# Quantity bought of each product in each period

buy = food.addVars(months, oils, name = "Buy")

\# Quantity used of each product in each period

consume = food.addVars(months, oils, name = "Use")

\# Quantity stored of each product in each period

store = food.addVars(months, oils, name = "Store")

#1. Initial Balance

Balance0 = food.addConstrs((init_store + buy[months[0], oil]

 == consume[months[0], oil] + store[months[0], oil]

 for oil in oils), "Initial_Balance")

#2. Balance

Balance = food.addConstrs((store[months[months.index(month)-1], oil] + buy[month, oil]

 == consume[month, oil] + store[month, oil]

 for oil in oils for month in months if month != month[0]), "Balance")

#3. Inventory Target

TargetInv = food.addConstrs((store[months[-1], oil] == target_store for oil in oils),"End_Balance")

#4.1 Vegetable Oil Capacity

VegCapacity = food.addConstrs((gp.quicksum(consume[month, oil] for oil in oils if "VEG" in oil) <= veg_cap for month in months), "Capacity_Veg") 

#4.2 Non-vegetable Oil Capacity

NonVegCapacity = food.addConstrs((gp.quicksum(consume[month, oil] for oil in oils if "OIL" in oil)

 <= oil_cap for month in months), "Capacity_Oil")

#5. Hardness

HardnessMin = food.addConstrs((gp.quicksum(hardness[oil]*consume[month, oil] for oil in oils)

 >= min_hardness*produce[month] for month in months), "Hardness_lower")

HardnessMax = food.addConstrs((gp.quicksum(hardness[oil]*consume[month, oil] for oil in oils)

 <= max_hardness*produce[month] for month in months), "Hardness_upper")

#6. Mass Conservation

MassConservation = food.addConstrs((consume.sum(month) == produce[month] for month in months), "Mass_conservation")

#The objective is to maximize the profit of the company, which includes the revenue as well as the costs for buying and storing the used products.

#0. Objective Function

obj = price*produce.sum() - buy.prod(cost) - holding_cost*store.sum()

food.setObjective(obj, GRB.MAXIMIZE) # maximize profit

food.optimize() 

```

## Analysis

The highest profit we can hope for is $107,842.6$107,842.6. Let's see an optimal solution needed to achieve that figure (there are alternative optimal solutions):

## Purchase Plan

This plan defines the amount of vegetable oil (VEG) and non-vegetable oil (OIL) that we need to purchase during the planning horizon. For example, 659.3 tons of vegetable oil of type VEG1 needs to be bought in June.

```python

rows = months.copy()

columns = oils.copy()

purchase_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)


for month, oil in buy.keys():

 if (abs(buy[month, oil].x) > 1e-6):

 purchase_plan.loc[month, oil] = np.round(buy[month, oil].x, 1)

purchase_plan 

```

## Monthly Consumption

This plan determines the amount of vegetable oil (VEG) and non-vegetable oil (OIL) consumed during the planning horizon. For example, 200 tons of vegetable oil of type VEG2 is consumed in January.

```python

In [12]:

rows = months.copy()

columns = oils.copy()

reqs = pd.DataFrame(columns=columns, index=rows, data=0.0)




for month, oil in consume.keys():

 if (abs(consume[month, oil].x) > 1e-6):

 reqs.loc[month, oil] = np.round(consume[month, oil].x, 1)

reqs

```

## Inventory Plan

This plan reflects the amount of vegetable oil (VEG) and non-vegetable oil (OIL) in inventory at the end of each period of the planning horizon. For example, at the end of February we have 500 tons of Non-vegetable oil of type OIL1.

```python

In [13]:

rows = months.copy()

columns = oils.copy()

store_plan = pd.DataFrame(columns=columns, index=rows, data=0.0)

for month, oil in store.keys():

 if (abs(store[month, oil].x) > 1e-6):

 store_plan.loc[month, oil] = np.round(store[month, oil].x, 1)

store_plan 

```

## References

- H. Paul Williams, Model Building in Mathematical Programming, fifth edition. 2020 Gurobi Optimization, LLC.
- Food Manufacture HYPERLINK "https://gurobi.github.io/modeling" https://gurobi.github.io/modeling-examples/food_manufacturing_1_2/food_manufacture_1.html
- Artificial Intelligent Apply Food Manufacturing. CobuildLab 2021 HYPERLINK "https://www.cobuildlab.com/blog/ai" https://www.cobuildlab.com/blog/ai
- Artificial Intelligent Apply Food Manufacturing and Vehicle Routing Problem. CobuildLab 2021

HYPERLINK "https://www.cobuildlab.com/blog/ai" https://www.cobuildlab.com/blog/ai

[5] Optimization Intelligent in Food Manufacturing Part I. CobuildLab 2021

HYPERLINK "https://www.cobuildlab.com/blog/ai" https://www.cobuildlab.com/blog/ai

A supply chain encompasses all steps in transferring products from the manufacturer to the end customer. Optimizing this process allows it to function at peak efficiency. To better understand supply chain optimization, it’s important to delve deeper into a supply chain’s three main phases: design, planning, and execution/operation. 

Optimization Intelligent in Food Manufacturing – Part II

# Optimization Price using Machine Learning

## Introduction

Price optimization is using historical data to identify the most appropriate price of a product or a service that maximizes the company’s profitability. There are numerous factors like demography, operating costs, survey data, etc that play a role in efficient pricing, it also depends on the nature of businesses and the product that is served. The business regularly adds/upgrades features to bring more value to the product and this obviously has a cost associated with it in terms of effort, time, and most importantly companies’ reputation.

As a result, it is important to understand the correct pricing, a little too high, you lose your customers and slight underpricing will result in loss of revenue. Price optimization helps businesses strike the right balance of efficient pricing, achieving profit objectives, and also serves their customers. In this blog, we will take a look at the simplistic price optimization approach and also build a simulator app.

The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of data science helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

So, Predictive analytics is about predicting future phenomena using historical data sets. Therefore, for accurate forecasting, predictive analytics is highly helpful as it shows various mathematical models of predictive forecasting. Also, the predictive analytics tools help compare two roadmaps and decide which is more profitable and efficient. It takes into consideration various factors, both internal and external. This process is known as the optimization of supply chain processes. Therefore, it can be said that the predictive analytics of data science are highly helpful in forecasting, optimization, and visualization of various processes in a supply chain of businesses.

## Challenges in optimizing pricing:

Price optimization for a single product: Price optimization for a single product is to predict changing demand in response to different prices. It helps the business to fix prices that customers are ready to pay and maximize profits. E.g.: In the smartphone market, it is a fine balance of new features and price.

Price optimization for a product family: Any changes in the pricing of one product, may trigger a chain reaction across a product family. Hence, the pricing of product family becomes a daunting task. E.g.: In the ice-cream market, there can be different flavors, sizes as in cups, candy, cones, family packs, or tubs. It’s the same product with varying sizes and prices.

## Benefits of optimizing pricing:

Immediate financial benefits: The businesses can reap instant results by targeting multiple KPI’s whether it is margin, sales conversions, the addition of new customers, or penetrating a new market space, etc., and then review the results to make suitable changes on pricing.

Automation of business process: The price optimizations should be more of a data-driven process. The analysis of historical data, sales trend, demand, etc helps businesses design rules or building ML models or build a hybrid model which is completely automated thus taking away the human error or emotion-based decisions on pricing.

Quick response to changing market trends: The market demand and trend change quite often and sometimes when a competitor launches a similar product at a lower price bracket, it eats into shares of others. In such a scenario, optimizing product prices in a particular product segment or geography helps businesses to tackle this challenge.

Business Case – Code in Python [1]

We will be using a toy data Price.csv. As the dataset is small and clean, there is no need for any kind of pre-processing.

```python

import pandas as pd

import numpy as np

from pandas import DataFrame

import matplotlib.pyplot as plt

import seaborn as sns

from statsmodels.formula.api import ols

import plotly.express as px

import plotly.graph_objects as go




\# Load data and show first 5 rows

df = pd.read_csv('Price.csv')

df.head(5)

#df = pd.read_csv('imports-86.csv')


fig_PriceVsQuantity = px.scatter(df, x="Price", y="Quantity",color='Year',trendline="ols")

fig_PriceVsQuantity.show()

fig_PriceVsQuantity = px.scatter(df, x="Price", y="Quantity",color='Cost',trendline="ols")

fig_PriceVsQuantity.show()

\# fit OLS model

model = ols("Quantity ~ Price", data=df).fit()




Price = list(range(200, 370, 10))

cost = 100

quantity = []

Revenue = []

for i in Price:

 demand = model.params[0] + (model.params[1] * i)

 quantity.append(demand)

 Revenue.append((i-cost) * demand)

profit = pd.DataFrame({"Price": Price, "Revenue": Revenue, "Quantity": quantity})

print(profit)

max_val = profit.loc[(profit['Revenue'] == profit['Revenue'].max())]

print(max_val)

fig_PriceVsRevenue = go.Figure()

fig_PriceVsRevenue.add_trace(go.Scatter(

x=profit['Price'], y=profit['Revenue']))

fig_PriceVsRevenue.add_annotation(x=int(max_val['Price']), y=int(max_val['Revenue']),

 text="Maximum Revenue",

 showarrow=True,

 arrowhead=1)

fig_PriceVsRevenue.update_layout(

 showlegend=False,

 xaxis_title="Price",

 yaxis_title="Revenue")

fig_PriceVsRevenue.add_vline(x=int(max_val['Price']), line_width=2, line_dash="dash", line_color="red", opacity=0.25) 

```

## References

- How to Build A Price Recommender App With Python https://www.analyticsvidhya.com/blog/2021/08/build-a-price-recommender-app-with-python/
- Maximize your Business Profitability with Python https://towardsdatascience.com/maximize-your-business-profitability-with-python-fbefebbdf802
- Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. https://www.cobuildlab.com/blog/ai
- ProfLogit: a Proﬁt Maximizing Logistic Regression Model for Customer Churn Prediction https://www.dataminingapps.com/2018/02/proflogit-a-pro%EF%AC%81t-maximizing-logistic-regression-model-for-customer-churn-prediction/
- Optimization Business Profitability CobuildLab. 2021. https://www.cobuildlab.com/blog/ai

Price optimization is using historical data to identify the most appropriate price of a product or a service that maximizes the company’s profitability. There are numerous factors like demography, operating costs, survey data, etc that play a role in efficient pricing, it also depends on the nature of businesses and the product that is served. The business regularly adds/upgrades features to bring more value to the product and this obviously has a cost associated with it in terms of effort, time, and most importantly companies’ reputation.

Optimization Price using Machine Learning

# Introduction

Setting the right price for a good or service is an old problem in economic theory. There are a vast amount of pricing strategies that depend on the objective sought. One company may seek to maximize profitability on each unit sold or on the overall market share, while another company needs to access a new market or to protect an existing one. Moreover, different scenarios can coexist in the same company for different goods or customer segments.

Price optimization has been used, with significant success, in industries such as hospitality, airline, car rental, and e-commerce retail. One of the first success stories occurred in the early 2000s, when HYPERLINK "http://www.businesstravelnews.com/More-News/Hotel-Rates-Go-Dynamic-Hilton-Intercontinental-End-Fixed-Pricing" \t "_blank" Hilton Hotels Corp and HYPERLINK "http://www.businesstravelnews.com/Strategic-Sourcing/IHG-Doubles-Down-On-Dynamic-Pricing-Push" \t "_blank" InterContinental Hotels Group decided to eliminate fixed rates in favor of a fluid scheme, including dynamic pricing strategies. In those years the prices of the rooms were modified once or twice a day. The current computational power allows prices to change practically in real time.

The hotel industry continues to employ dynamic pricing strategies, based entirely on Machine Learning. Currently, Airbnb proposes a HYPERLINK "https://learnairbnb.com/airbnb-supply-demand-dynamic-airbnb-pricing/" \t "_blank" dynamic price tool that recommends prices to its hosts, considering parameters such as seasonality, the day of the week or special events, and also more sophisticated factors such as photos of the property to be rented or the prices applied in the neighborhood. Other companies such as eBay and Uber have adopted similar approaches.

Amazon is another of the big players when talking about dynamic pricing strategies. To give an idea, in 2012 Amazon was changing prices much more often than its competitors and in 2013, they were performing as much as 2.5 million price changes per day. Changing prices in such a dynamic way is informally known as the Amazon effect. In the retail world, the most popular examples have been in e-commerce, but brick-and-mortar retailers have not been left behind. Although it is difficult to know precisely all the retail companies using Machine Learning to optimize their prices and operating processes, there are nevertheless some known success stories.

So, One of the major goals of the modern enterprise of data science and analytics is to HYPERLINK "https://www.quora.com/Is-optimization-related-to-data-science-And-how" \t "_blank" solve complex optimization problems for business and technology companies to maximize their profit. So, Optimization is the science / process behind finding the best solution for a problem with given constraints. We come across optimization problems on a daily basis. These can be for finding the shortest path between your work place and office; maximizing revenues / customer happiness or minimizing costs / debts etc. We basically take a real world problem, model it mathematically and then solve it using mathematical techniques with in the constraints. Optimization is useful in Marketing, Manufacturing, Finance, Online advertising, Machine Learning and all fields you can imagine.

The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of data science helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

Price optimization helps retailers understand how customers will react to different price strategies for products and services, and set the best prices. Machine Learning models can take key pricing variables into account (e.g. purchase histories, season, inventory, competitors’ pricing), to find the best prices, even for vast catalogs of products or services, that can achieve the set KPIs.

Price optimization is using historical data to identify the most appropriate price of a product or a service that maximizes the company’s profitability. There are numerous factors like demography, operating costs, survey data, etc that play a role in efficient pricing; it also depends on the nature of businesses and the product that is served. The business regularly adds/upgrades features to bring more value to the product and this obviously has a cost associated with it in terms of effort, time, and most importantly companies’ reputation.

As a result, it is important to understand the correct pricing, a little too high, you lose your customers and slight under pricing will result in loss of revenue. Price optimization helps businesses strike the right balance of efficient pricing, achieving profit objectives, and also serves their customers. In this blog, we will take a look at the simplistic price optimization approach and also build a simulator app.

The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of HYPERLINK "https://360digitmg.com/data-science"data science helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

Knowledge of such optimization techniques is extremely useful for data scientists and machine learning (ML) practitioners as discrete and continuous optimization lie at the heart of modern ML and AI systems as well as HYPERLINK "https://www.edx.org/course/optimization-methods-business-analytics-mitx-15-053x" data-driven business analytics processes.

These tools and concepts can also be used to maximize the profit of your Supply Chain by focusing your production on high margin products. So, Optimization to help your local bakery to improve its business profitability by selecting the right items to sell.

## Benefits of optimizing pricing:

**Immediate financial benefits**: The businesses can reap instant results by targeting multiple KPI’s whether it is margin, sales conversions, the addition of new customers, or penetrating a new market space, etc., and then review the results to make suitable changes on pricing.

**Automation of business process**: The price optimizations should be more of a data-driven process. The analysis of historical data, sales trend, demand, etc helps businesses design rules or building ML models or build a hybrid model which is completely automated thus taking away the human error or emotion-based decisions on pricing.

**Quick response to changing market trends**: The market demand and trend change quite often and sometimes when a competitor launches a similar product at a lower price bracket, it eats into shares of others. In such a scenario, optimizing product prices in a particular product segment or geography helps businesses to tackle this challenge.

## Business & Price Case – Code in Python [1, 2]

### We will be using a Price and Products Data

import pandas as pd
import numpy as np
from pandas import DataFrame
import matplotlib.pyplot as plt
import seaborn as sns
from statsmodels.formula.api import ols
import plotly.express as px
import plotly.graph_objects as go
from pulp import \*

# Load data and show first 5 rows
df = pd.read_csv('Price.csv')
df.head(5)

fig_PriceVsQuantity = px.scatter(df, x="Price", y="Quantity",color='Year',trendline="ols")
fig_PriceVsQuantity.show()

fig_PriceVsQuantity = px.scatter(df, x="Price", y="Quantity",color='Cost',trendline="ols")
fig_PriceVsQuantity.show()

# fit OLS model
model = ols("Quantity ~ Price", data=df).fit()

Price = list(range(200, 370, 10))
cost = 100
quantity = []
Revenue = []
for i in Price:
 demand = model.params[0] + (model.params[1] \* i)
 quantity.append(demand)
 Revenue.append((i-cost) * demand)

profit = pd.DataFrame({"Price": Price, "Revenue": Revenue, "Quantity": quantity})
print(profit)
max_val = profit.loc[(profit['Revenue'] == profit['Revenue'].max())]
print(max_val)

fig_PriceVsRevenue = go.Figure()
fig_PriceVsRevenue.add_trace(go.Scatter(
x=profit['Price'], y=profit['Revenue']))
fig_PriceVsRevenue.add_annotation(x=int(max_val['Price']), y=int(max_val['Revenue']),
 text="Maximum Revenue",
 showarrow=True,
 arrowhead=1)

fig_PriceVsRevenue.update_layout(
 showlegend=False,
 xaxis_title="Price",
 yaxis_title="Revenue")

fig_PriceVsRevenue.add_vline(x=int(max_val['Price']), line_width=2, line_dash="dash", line_color="red", opacity=0.25)

# Products 
items = ["A", "B", "C", "D", "E", "F"]
profit = {"A":6, "B":4.4, "C":7.5, "D":0.9, "E":1.2, "F":2.2}
baker = {"A":50, "B":0, "C":45, "D":35, "E":25, "F":0}
oven = {"A":45, "B":0, "C":90, "D":20, "E":45, "F":0}
display = {"A":4, "B":1.5, "C":3, "D":1, "E":1, "F":1}
assist = {"A":0, "B":12, "C":0, "D":0, "E":0, "F":8}

# Initialize Class
model = LpProblem('Maximize Bakery Profits', LpMaximize)

# 2. Define Decision Variables
A = LpVariable('A', lowBound = 0, cat = 'Integer')
B = LpVariable('B', lowBound = 0, cat = 'Integer')
C = LpVariable('C', lowBound = 0, cat = 'Integer')
D = LpVariable('D', lowBound = 0, cat = 'Integer')
E = LpVariable('E', lowBound = 0, cat = 'Integer')
F = LpVariable('F', lowBound = 0, cat = 'Integer')
var_dict = {"A":A, "B":B, "C":C, "D":D, "E":E, "F":F}

# Define Objective Function
model += lpSum([profit[i] * var_dict[i] for i in items])

# 4. Define Constraints 
model += lpSum([var_dict[i]*baker[i] for i in items]) <= 24*60

# Oven
model += lpSum([var_dict[i]*oven[i] for i in items]) <= 48*60
# Assistant
model += lpSum([var_dict[i]*assist[i] for i in items]) <= 4*60

# Display
model += lpSum([var_dict[i]*display[i] for i in items]) <= 100

#Names
dict_name = dict(zip(["A", "B", "C", "D", "E", "F"], 
 ["Lemon Cake", "Sandwich", "Chocolate Cake", "Croissant", "Chocolate Eclair", "Panini"]))

# Solve Model
model.solve()
for v in model.variables():
 print(dict_name[v.name], "=", int(v.varValue))
print("Profit reached: {:,} euros".format(sum(var_dict[i].varValue * profit[i] for i in items)))

## References

- How to Build A Price Recommender App With Python
https://www.analyticsvidhya.com/blog/2021/08/build-a-price-recommender-app-with-python/
- Maximize your Business Profitability with Python
https://towardsdatascience.com/maximize-your-business-profitability-with-python-fbefebbdf802
- Optimization Techniques for Data Scientist and Machine Learning
CobuildLab. 2021. HYPERLINK "https://www.cobuildlab.com/blog/ai"https://www.cobuildlab.com/blog/ai
- ProfLogit: a Proﬁt Maximizing Logistic Regression Model for Customer Churn Prediction
HYPERLINK "https://www.dataminingapps.com/2018/02/proflogit-a-pro%EF%AC%81t-maximizing-logistic-regression-model-for-customer-churn-prediction/"https://www.dataminingapps.com/2018/02/proflogit-a-pro%EF%AC%81t-maximizing-logistic-regression-model-for-customer-churn-prediction/
- Optimization Business Profitability CobuildLab. 2021. HYPERLINK "https://www.cobuildlab.com/blog/ai"https://www.cobuildlab.com/blog/ai

Setting the right price for a good or service is an old problem in economic theory. There are a vast amount of pricing strategies that depend on the objective sought.

Data Analytics - Optimization Business & Price using Machine Learning

# Introduction

In this article, we illustrate the concept of integrating the goal of optimizing a business metric with a machine learning. Specifically, we show a case illustration of how a simple optimization loop can be wrapped around a core ML algorithm for guiding it toward achieving a specific business objective.

Machine Learning in its simplest form is about discerning patterns in data and using that information to answer questions or predict data, in this case, we take a look at how Machine Learning is used to find the optimal price of a product. Retailers have data about many different features such as transactional data, product description,
promotions and marketing campaigns, customer reviews, inventory and supply data, operating costs, seasonality and competitor data. In Brick & Mortar stores the way that products are displayed, the place they occupy on the shelf, location in the store, dimensions of the products, and the distance from the entrance are other features
that can be considered.

Machine Learning is used to parse and process through massive amounts of data and automates the analysis. Analyzing past and current data can be used to anticipate trends early so retailers can make appropriate decisions to adjust prices. Linear regression models and recurrent neural networks (RNNs) can be used to achieve this.

Automating analysis allows retailers to examine the vast amount of their products and optimize prices globally, again looking at different trends or differences in features and parameters.

Retailers can change the price of a product and observe the impact of sales of other products. The predictive accuracy of a Machine Learning algorithm will be higher than an average person.

Machine learning can be used for better inventory management, which can lead to fewer markdowns and minimize promotions. One particular company has used this very successfully to only have sales twice a year and instead, manages their inventory to quickly adapt to changing trends. Ultimately, this allows retailers to avoid a race to the bottom. Consumers can find if another merchant offers the same product at a cheaper price with a few swipes and clicks of their phone. Minimizing markdowns and promotions allows retailers to avoid rashly cutting
prices and decrease the profit margin of items.

k-NN (k-Nearest Neighbor), one of the simplest machine learning algorithms, is non-parametric and lazy in nature. Non-parametric means that there is no assumption for the underlying data distribution i.e. the model structure is determined from the dataset. Lazy or instance-based learning means that for the purpose of model generation, it does not require any training data points and whole training data is used in the testing phase.

In statistics, the k-nearest neighbors algorithm (k-NN) is a non-parametric classification method first. The output depends on whether k-NN is used for classification or regression:

- In k-NN classification, the output is a class membership. An object is classified by a plurality vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors (k is a positive integer, typically small). If k = 1, then the object is simply assigned to the class of that single nearest neighbor.
- In k-NN regression, the output is the property value for the object. This value is the average of the values of k nearest neighbors.

k-NN is a type of classification where the function is only approximated locally and all computation is deferred until function evaluation. Since this algorithm relies on distance for classification, if the features represent different
physical units or come in vastly different scales then normalizing the training data can improve its accuracy dramatically.

Both for classification and regression, a useful technique can be to assign weights to the contributions of the neighbors, so that the nearer neighbors contribute more to the average than the more distant ones. For example, a common weighting scheme consists in giving each neighbor a weight of 1/d, where d is the distance to the neighbor.

The neighbors are taken from a set of objects for which the class (for k-NN classification) or the object property value (for k-NN regression) is known. This can be thought of as the training set for the algorithm, though no explicit training step is required.

A peculiarity of the k-NN algorithm is that it is sensitive to the local structure of the data.

The k-NN algorithm consists of the following two steps:

**Step 1**

In this step, it computes and stores the k nearest neighbors for each sample in the training set.

**Step 2**

In this step, for an unlabeled sample, it retrieves the k nearest neighbors from dataset. Then among these k-nearest neighbors, it predicts the class through voting (class with majority votes wins).

The module, sklearn.neighbors that implements the k-nearest neighbors algorithm, provides the functionality for unsupervised as well as supervised neighbors-based learning methods.

The unsupervised nearest neighbors implement different algorithms (BallTree, KDTree or Brute Force) to find the nearest neighbor(s) for each sample. This unsupervised version is basically only step 1, which is discussed above, and the foundation of many algorithms (KNN and K-means being the famous one) which require the neighbor search. In simple words, it is Unsupervised learner for implementing neighbor searches.

On the other hand, the supervised neighbors-based learning is used for classification as well as regression.

## Unsupervised KNN Learning

As discussed, there exist many algorithms like KNN and K-Means that requires nearest neighbor searches. That is why Scikit-learn decided to implement the neighbor search part as its own “learner”. The reason behind making neighbor search as a separate learner is that computing all pairwise distance for finding a nearest neighbor is obviously not very efficient. Let’s see the module used by Sklearn to implement unsupervised nearest neighbor learning along with example.

Business optimization with a machine learning model

```python

import numpy as np
import pandas as pd
from sklearn.model_selection import cross_val_score, KFold
from sklearn.neighbors import KNeighborsRegressor

dc_listings = pd.read_csv("dc_airbnb.csv")
stripped_commas = dc_listings['price'].str.replace(',', '')
stripped_dollars = stripped_commas.str.replace('$', '')
dc_listings['price'] = stripped_dollars.astype('float')

kf = KFold(5, shuffle=True, random_state=1)

#Create a new instance of the KNeighborsRegressor class and assign to knn
#from sklearn.neighbors import KNeighborsRegressor
model = KNeighborsRegressor()

#Use the cross_val_score() function to perform k-fold cross-validation:
#using the KNeighborsRegressor instance knn,
#using the accommodates column for training,
#using the price column as the target column,
#returning an array of MSE values (one value for each fold).
mses = cross_val_score(model, dc_listings[['accommodates']], dc_listings['price'], scoring='neg_mean_squared_error', cv=kf)

#Assign the resulting list of MSE values to mses. Then, take the absolute value followed by the square root of each MSE value.
#Then, calculate the average of the resulting RMSE values and assign to avg_rmse.
rmses = np.sqrt(np.absolute(mses))
avg_rmse = np.mean(rmses)

print(rmses)
print(avg_rmse)

```

[117.38752023 137.27049219 146.0660213 106.37023494 145.75598127]
130.57004998596955

 

## References

1. Dynamic Pricing: How Pricing Optimization And Revenue Management Benefit From Machine Learning [https://www.scalr.ai/post/dynamic-pricing-price-optimization](https://www.scalr.ai/post/dynamic-pricing-price-optimization)
2. Scikit Learn - KNN Learning [https://www.tutorialspoint.com/scikit_learn/scikit_learn_knn_learning.htm](https://www.tutorialspoint.com/scikit_learn/scikit_learn_knn_learning.htm)
3. Data Analytics - Optimization Business & Price using Machine Learning. CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)
4. Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)
5. Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

In this article, we illustrate the concept of integrating the goal of optimizing a business metric with a machine learning. Specifically, we show a case illustration of how a simple optimization loop can be wrapped around a core ML algorithm for guiding it toward achieving a specific business objective. 

Business optimization with a machine learning model

# Introduction

Over the last decade, machine learning underwent a broad democratization. Countless tutorials, books, lectures, and blog articles have been published related to the topic. While the technical aspects of how to build and optimize models are well documented, very few resources are available on how developing machine learning models fits within a E-Commerce.

So, a typical day in the life of a **Data Scientist in the E-commerce** space is unusually challenging. You can never predict human behavior but you are tasked to do it. Sure, you get to analyze and have random spurts of enlightenment about customer buying patterns and also witness how a marketing decision could overnight change your hypothesis. But nothing comes close to seeing your work save time and increase the engagement experience of a customer while fulfilling your clients’ vision.

E-Commerce is currently one of the fastest and dynamically evolving industries in the world. Its popularity has been growing rapidly with the ease of digital transactions and quick door-to-door deliveries. A major source of big tech companies revenues comes from the interaction of their underlying proprietary algorithms that are heavily powered by data science, so it’s fundamental to understand the methods applied to maintain and grow the number of clients.

Nowadays, most e-commerce platforms collect tons of data from users without hampering customer experience. Collected data is stored in structured or tabulated tables in order to facilitate analysis and interpretation. Not only structured data is stored, but also [non-structured data ](https://medium.com/swlh/which-technology-should-you-learn-according-to-your-profile-e081cf35d72f)such as images, videos or documents, which also have plenty of value at the moment of studying users preferences and are frequently harder to process and analyze.

Data analysis provides these companies with insights and metrics that are constantly changing and that allow them to build even better products. E- commerce stands for electronic commerce and represents the online version of physical retail stores. It allows people from all over the world to purchase, browse and sell products through online platforms. Although it may seem a fairly simple process from the customer standpoint, there are several obstacles to overcome in order to provide a seamless online shopping experience, such as processes related ones which include product ordering, delivery and fair pricing.

However, with the growing number of people looking to shop online, the e-commerce industry is expanding rapidly. This also means that an increasing proportion of traditional businesses are switching or complementing their business model to electronic commerce.

In the context of e-commerce industry evolution, Data Science helps to bring maximum value out of the vast amount of data available in such platforms, and helps to switch focus towards customer engagement and experience. It focuses on:

- Product recommendation for users.
- Analysis of customer trends and behaviors
- Forecasting sales and stock logistics.
- Optimizing product pricing and payment methods.

Each of these applications involves storage and interpretation of large amounts of volumes of data, in which data analysis techniques come in handy. Let’s talk about how machine learning, a type of artificial intelligence (AI) that has been revolutionizing the ecommerce industry. Simply put, machine learning is a method that uses experience to improve performance over a period of time.

## E-Commerce with python [2]

```python
# Imports

import pandas as pd

import matplotlib.pyplot as plt

import matplotlib.gridspec as gridspec

import seaborn as sns

import numpy as np

# Read dataset and preview

data = pd.read_csv('e_commerce.csv')
```

# Exploring data

```python data.info()

# Replace the null features with 0:

data.fillna(0, inplace=True) # Re-check N/A was replaced with 0.

# Group by User ID:

purchases = data.groupby(['User_ID']).sum().reset_index()

print(purchases)

data[data['User_ID'] == 1000001]

purchase_by_age = data.groupby('Age')['Purchase'].mean().reset_index()

print(purchase_by_age)

# Grouping by gender and age

age_and_gender = data.groupby('Age')['Gender'].count().reset_index()

gender = data.groupby('Gender')['Age'].count().reset_index()

# Plot distribution

plt.figure(figsize=(12,9))

plt.pie(age_and_gender['Gender'], labels=age_and_gender['Age'],autopct='%d%%', colors=['cyan', 'steelblue','peru','blue','yellowgreen','salmon','#0040FF'])

plt.axis('equal')

plt.title("Age Distribution", fontsize='20')

plt.show()
```

 

<img>

 

```python

# Plot gender distribution

plt.figure(figsize=(12,9))

plt.pie(gender['Age'], labels=gender['Gender'],autopct='%d%%', colors=['salmon','steelblue'])

plt.axis('equal')

plt.title("Gender Distribution", fontsize='20')

plt.show()

```

 

<img>

 

```python

# Group by occupation:

occupation = data.groupby('Occupation')['Purchase'].mean().reset_index()

# Plot bar chart with line plot:

sns.set(style="white", rc={"lines.linewidth": 3})

fig, ax1 = plt.subplots(figsize=(12,9))

sns.barplot(x=occupation['Occupation'],y=occupation['Purchase'],color='#004488',ax=ax1)

sns.lineplot(x=occupation['Occupation'],y=occupation['Purchase'],color='salmon',marker="o",ax=ax1)

plt.axis([-1,21,8000,10000])

plt.title('Occupation Bar Chart', fontsize='15')

plt.show()

sns.set()

```

 

<img>

 

```python

# Group by product ID

product = data.groupby('Product_ID')['Purchase'].count().reset_index()

product.rename(columns={'Purchase':'Count'},inplace=True)

product_sorted = product.sort_values('Count',ascending=False)

# Plot line plot

plt.figure(figsize=(14,8))

plt.plot(product_sorted['Product_ID'][:10], product_sorted['Count'][:10], linestyle='-', color='purple', marker='o')

plt.title("Best-selling Products", fontsize='15')

plt.xlabel('Product ID', fontsize='15')

plt.ylabel('Products Sold', fontsize='15')

plt.show()

```

 

<img>

 

## References

[1] Data Science and Machine Learning in the E-Commerce Industry: Insider Talks About Tools, Use-Cases, Problems, and More [https://neptune.ai/blog/data-science-and-machine-learning-in-the-e-commerce-industry-tools-use-cases-problems](https://neptune.ai/blog/data-science-and-machine-learning-in-the-e-commerce-industry-tools-use-cases-problems)

[2] Data Science for E-Commerce with Python [https://towardsdatascience.com/data-science-for-e-commerce-with-python-a0a97dd7721d](https://towardsdatascience.com/data-science-for-e-commerce-with-python-a0a97dd7721d)

[3] Data Analytics - Optimization Business & Price using Machine Learning. CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

```

```

Over the last decade, machine learning underwent a broad democratization. Countless tutorials, books, lectures, and blog articles have been published related to the topic. While the technical aspects of how to build and optimize models are well documented, very few resources are available on how developing machine learning models fits within a E-Commerce. 

Machine Learning in the E-Commerce

# Introduction

This article will discuss and analyze the importance of k-fold cross-validation for model prediction in machine learning using the least-squares algorithm for Empirical Risk Minimization (ERM). By the end of this post, you’ll know how to implement the k-fold cross-validation method and understand the benefits and drawbacks that come with it.

The world is constantly changing, humans were living in the Industrial Age, where hand-tools were replaced by powerful machines, and mass production became a possibility. Now, with the rapidly paced development of technology, we are living in the Information Age, where vast amount of information is easily accessible, and those few who harness the potential, will be the ones with an edge to benefit from it. Dynamic Pricing is a strategy in where business set prices of products or services based on time or demand. This is a methodology which makes use of the massive potential of information availability. It is important, as a company, to know and be able to modify prices according to what the consumers are willing to pay. The benefits to business in using this algorithm is to be able to optimally define a pricing strategy to make the most profit possible, by taking into account the product’s interactions, where lowering some prices could mean increasing demand for some products by gaining new customers but at the same time losing presence in some others by having them shifting from different products of the same portfolio.

**Dynamic Pricing** theory has been extensively studied by researchers from a variety of fields over the years. These fields include, among others, economics, marketing, revenue management and telecommunications. In recent years, the rapid development of information technology, the **Internet and E-commerce** has had very strong influence on the development of pricing and revenue management. In an environment where firms compete for the same products in a constantly changing market, the prices need to be dynamic. In literature Dynamic Pricing (DP) is the study of determining an optimal pricing strategy for products or services, in an environment that allows them to be easily and frequently adjusted. These techniques are normally applied by firms selling their merchandise via Internet, or stores with capacity to change their digital price tags. In any case, technology has made possible to adapt to continuous changing circumstances with minimal cost or effort. DP is a terminology which has been gaining popularity over this last decade. As they have been applied in various businesses, it is possible to recognize and relate them with the Airlines and Hotel businesses. These segments of companies are the ones who started using DP techniques to obtain the most possible from the high seasons, by increasing prices as demand increases, while compensating for the low seasons of the year. There exists more than one type of Dynamic Pricing strategies, in this thesis we will describe the most popular ones: Pricing based on value or elasticity, Pricing based on competitors, and Time-based retail pricing.

- **Pricing based on value or elasticity**: The idea is to calculate how many consumers are willing to pay and at each price point, with this method, the strategy could aim to achieve a volume, revenue or profit maximization strategy.
- **Pricing based on competitors**: This strategy is based on monitoring the competition and price the products accordingly. These strategy has its own risks, although the main objective is to stay competitive, many big firms are known to have used this method by setting prices below cost in order to eliminate smaller competitors.
- **Croos-Validation**, sometimes called rotation estimation or out-of-sample testing, is any of various similar model validation techniques for assessing how the results of a statistical analysis will generalize to an independent data set. Cross-validation is a resampling method that uses different portions of the data to test and train a model on different iterations. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice. In a prediction problem, a model is usually given a dataset of known data on which training is run (training dataset), and a dataset of unknown data (or first seen data) against which the model is tested (called the validation dataset or testing set). The goal of cross-validation is to test the model's ability to predict new data that was not used in estimating
it, in order to flag problems like overfitting or selection bias[10] and to give an insight on how the model will generalize to an independent dataset (i.e., an unknown dataset, for instance from a real problem).

One round of cross-validation involves partitioning a sample of data into complementary subsets, performing the analysis on one subset (called the training set), and validating the analysis on the other subset (called the validation set or testing set). To reduce variability, in most methods multiple rounds of cross-validation are performed using different partitions, and the validation results are combined (e.g. averaged) over the rounds to give an estimate of the model's predictive performance.

In summary, cross-validation combines (averages) measures of fitness in prediction to derive a more accurate estimate of model prediction performance.[11] To better visualize the benefits of applying k-fold cross-validation on machine learning, we’ll analyze some problems we may face when estimating a model without doing any type of cross-validation at all.

**Methodology & Pipeline**, To have a clearer overview of the whole process, a pipeline was defined in order to have a good visual reference of the problem’s structure. This pipeline shown in Figure 1, represents the process of the DPS in order to analyze, predict, and optimize the prices. Dynamic Pricing Pipeline with this representation, it becomes more practical to split the problem into its core parts, to further explain the theory behind each, the application, and importance in the process of Dynamic Pricing. As mentioned, the original or full dataset is divided into a subset by store, to have only the information of interest.

Subsequently, in the code example will go deeper into Predictions, taking emphasis in the definition of Machine Learning, types of predictive models and the focus of this work, applying the state-of-the-art, **Croos-Validation** to predict the products demand.

 

<img>

 

Figure 1: Dynamic Pricing Pipeline

## Code Cross-Validation

```python

import numpy as np

import pandas as pd

from sklearn.model_selection import cross_val_score, KFold

from sklearn.neighbors import KNeighborsRegressor

dc_listings = pd.read_csv("dc_airbnb.csv")

stripped_commas = dc_listings['price'].str.replace(',', '')

stripped_dollars = stripped_commas.str.replace('$', '')

dc_listings['price'] = stripped_dollars.astype('float')

# General workflow for performing k-fold cross-validation

# Create a new instance of the KFold class with the following

# 5 folds, shuffle set to True, random seed set to 1 (so we can answer check using the same seed), assigned to the variable

kf.

kf = KFold(5, shuffle=True, random_state=1)

# Create a new instance of the KNeighborsRegressor class and assign to knn

# from sklearn.neighbors import KNeighborsRegressor

model = KNeighborsRegressor()

# Use the cross_val_score() function to perform k-fold cross-validation:

# using the KNeighborsRegressor instance knn,

# using the accommodates column for training,

# using the price column as the target column,

# returning an array of MSE values (one value for each fold).

mses = cross_val_score(model, dc_listings[['accommodates']], dc_listings['price'], scoring='neg_mean_squared_error', cv=kf)

#Assign the resulting list of MSE values to mses. Then, take the absolute value followed by the square root of each MSE value.

#Then, calculate the average of the resulting RMSE values and assign to avg_rmse.

rmses = np.sqrt(np.absolute(mses))

avg_rmse = np.mean(rmses)

print(rmses)

print(avg_rmse)

```

**[117.38752023 137.27049219 146.0660213 106.37023494 145.75598127] 130.57004998596955**

 

## References

[1] Dynamic Pricing: How Pricing Optimization And Revenue Management Benefit From Machine Learning [https://www.scalr.ai/post/dynamic-pricing-price-optimization](https://www.scalr.ai/post/dynamic-pricing-price-optimization)

[2] A. Boer. Dynamic pricing and learning: Historical origins, current research, and new directions. Surveys in Operations Research and Management Science, 20:1–18, 2015. [https://upcommons.upc.edu/bitstream/handle/2117/109814/126762.pdf?sequence=1&isAllowed=y](https://upcommons.upc.edu/bitstream/handle/2117/109814/126762.pdf?sequence=1&isAllowed=y)

[3] Business optimization with a machine learning model CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

This article will discuss and analyze the importance of k-fold cross-validation for model prediction in machine learning using the least-squares algorithm for Empirical Risk Minimization (ERM). By the end of this post, you’ll know how to implement the k-fold cross-validation method and understand the benefits and drawbacks that come with it.

The importance of k-fold cross-validation for model prediction of Dynamic Pricing in ML

# Introduction

Used by a range of industries, dynamic pricing is the action setting your product or services pricing based on the current state of the market demands. With the huge amount of data generated by their customers via analytics and social media, companies can use that data to adjust their pricing in real time with dynamic pricing. Using the data allows companies to understand the current state of their market, make adjustments as needed, and maximize their opportunities to increase their revenue.

### There are three methods of dynamic pricing:

- Demand-based, with prices that change on the basis of demand vs. supply
- Cost-based, changing prices with the cost of doing business and keeping revenue in place‍
- Competitor-based, making decisions on how a company’s competitors price their goods and services.

Done correctly, each of these pricing techniques can benefit a company’s revenue.

## Dynamic Pricing: Pros

<img>

Pricing can be automated as needed, either partly or completely, depending on goals of a product campaign. Dynamic pricing tools use internal and external factors to set prices according to the company’s pricing strategy. The tools may use KPI’s, inventory, demand, and prices from competitors to determine pricing in line with the company strategy.

Businesses that can actively respond to market demand, develop brand awareness with its pricing decisions, and use inventory wisely has a better chance of survival despite current market conditions. Dynamic pricing helps a company increase its competitiveness and recover from a bad move. For example, an airline can recuperate from a lack of sales during periods of low demand or just prior to an impending departure by having a sale on tickets for a particular flight or flights.

## Dynamic Pricing: Cons

The flip side of dynamic pricing is that it can also create customerire. Most customers consider pricing changes normal when theyare looking for and booking flights and hotel accommodations in theirintended destination. Ironically, they don’t feel the same about car rental companies and retail.

A prime example of a customer backlash target is Uber. Their dynamicpricing model changes the cost of a ride according to demand. Customers are warned via the Uber app about price surges, and are given the option to wait until the surge has passed and rates go back down to normal. These surges occur during high-demand periods when more people are looking for a ride—such as New Year’s Eve.

## Racing To The Bottom:

One of today’s market characteristics is price transparency. With just a few clicks, anyone can find the merchant that provides goods and/or services for a lower price. Due to strong competition, many companies are quick to drastically cut their prices to rival their competitors. In doing so, they may create a “race to the bottom” by intentionally charging less and decreasing their own profit margins. Ultimately, the company loses the price war they’ve started on their own.

‍Data-driven decisions on pricing are a better way to handle these challenges. Many tools for dynamic pricing can monitor data and do analyzations on things like:

- Movements in the market
- Demand for each product
- Available inventory
- Prices of competitors
- Digital footprints of each customer
- Website events, i.e.:
- Most viewed pages of products and/or services
- Abandoned shopping carts
- Clicks on the content times
- Determine the most reasonable price to be shown

The tools can also help determine and indicate price limits on everything.

```python

import pandas as pd

import numpy as np
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt

%matplotlib inline

cars = pd.read_csv("imports-85.data")

cars.head()

colnames = ['symboling', 'normalized-losses', 'make', 'fuel-type', 'aspiration', 'num-of-doors', 'body-style', 'drive-wheels', 'engine-location', 'wheel-base', 'length', 'width', 'height', 'curb-weight', 'engine-type', 'num-of-cylinders', 'engine-size', 'fuel-system', 'bore', 'stroke', 'compression-rate', 'horsepower', 'peak-rpm', 'city-mpg', 'highway-mpg', 'price']

cars = pd.read_csv("imports-85.data", names=colnames)

cars.head()

# Data Cleaning
cars = cars.replace("?", np.nan)

to_drop = ["symboling", "make", "fuel-type", "aspiration", "num-of-doors", "body-style", "drive-wheels", "engine-location", "engine-type", "num-of-cylinders", "fuel-system", "engine-size"]

cars_num = cars.drop(to_drop, axis=1)

cars_num.head()

cars_num = cars_num.astype("float")

cars_num.isnull().sum()

cars_num = cars_num.dropna(subset=["price"])

cars_num.isnull().sum()

cars_num = cars_num.fillna(cars_num.mean())

cars_num.isnull().sum()

cars_num.head()

normalized_cars = (cars_num-cars_num.mean())/(cars_num.max()-cars_num.min())

normalized_cars['price'] = cars_num['price']

normalized_cars.head()

#We are going to write a function that uses the KNeighborsRegressor class from scikit-learn. This works a little bit differently, the class actually generates a model that fits the training dataset. It is a regression method using k-nearest neighbors.
#Returns the root mean squared error using KNN

def knn_train_test(features, target_col, df):

#randomize sets

np.random.seed(1)

randomed_index = np.random.permutation(df.index)

randomed_df = df.reindex(randomed_index)

half_point = int(len(randomed_df)/2)

#assign test and training sets

train_df = randomed_df.iloc[0:half_point]

test_df = randomed_df.iloc[half_point:]

#training

knn = KNeighborsRegressor()

knn.fit(train_df[[features]], train_df[[target_col]])

#test

predictions = knn.predict(test_df[[features]])

mse = mean_squared_error(test_df[[target_col]], predictions)

rmse = mse**0.5

return rmse

#We can write a for loop and use the function for each column. That way, we can see the RMSE of each column.

features = normalized_cars.columns.drop('price')

rmse = {}

for item in features:

rmse[item] = knn_train_test(item, 'price', normalized_cars)

results = pd.Series(rmse)

results.sort_values()

# It looks like the 'horsepower' column has the least amount of error. We should definitely keep this list in mind when using the function for multiple features. But first, let's modify the function to include k value or the number of neighbors as a parameter. Then we can loop through a list of K values and features to determine which K value and features are most optimal in our machine learning model.

def knn_train_test2(features, target_col, df, k_values):

#randomize sets

np.random.seed(1)

randomed_index = np.random.permutation(df.index)

randomed_df = df.reindex(randomed_index)

half_point = int(len(randomed_df)/2)

#input k parameter as a list, use function to return a dictionary of dictionaries

k = [1, 3, 5, 7, 9]

features = normalized_cars.columns.drop('price')

feature_k_rmse = {}

for item in features:

feature_k_rmse[item] = knn_train_test2(item, 'price', normalized_cars, k)

feature_k_rmse

best_features = {}

plt.figure(figsize=(10, 12))

for key, value in feature_k_rmse.items():

x = list(value.keys())

y = list(value.values())

plt.legend()

plt.show()

```

 

<img>

 

```python
# This figure is a bit confusing to look at. A better way is to sort the values of the best_features which contains the

features as the key and the average RMSE as the values.

sorted_features_list = sorted(best_features, key=best_features.get)

sorted_features_list

#Now we know which features have the lowest amount of error, we can begin applying the function to multiple features at once.

def knn_train_test3(features, target_col, df):

#randomize sets

np.random.seed(0)

randomed_index = np.random.permutation(df.index)

randomed_df = df.reindex(randomed_index)

k_rmse_features ={}

best_two_features = sorted_features_list[0:2]

best_three_features = sorted_features_list[0:3]

best_four_features = sorted_features_list[0:4]

best_five_features = sorted_features_list[0:5]

k_rmse_features["best_two_rmse"] = knn_train_test3(best_two_features, 'price', normalized_cars)

k_rmse_features["best_three_rmse"] = knn_train_test3(best_three_features, 'price', normalized_cars)

k_rmse_features["best_four_rmse"] = knn_train_test3(best_four_features, 'price', normalized_cars)

k_rmse_features["best_five_rmse"] = knn_train_test3(best_five_features, 'price', normalized_cars)

k_rmse_features
```

```python
# best three features gave us the lowest RMSE. Now, let's try varying the K values. We can further tune our machine learning model by finding the optimal K value to use.

def knn_train_test4(features, target_col, df, k_values):

#randomize sets

np.random.seed(0)

randomed_index = np.random.permutation(df.index)

randomed_df = df.reindex(randomed_index)

#input k parameter as a list, use function to return a dictionary of dictionaries

k = list(range(1,25))

features = [best_three_features, best_four_features, best_five_features]

feature_k_rmse2 = {}

feature_k_rmse2["best_three_features"] = knn_train_test4(best_three_features, 'price', normalized_cars, k)

feature_k_rmse2["best_four_features"] = knn_train_test4(best_four_features, 'price', normalized_cars, k)

feature_k_rmse2["best_five_features"] = knn_train_test4(best_five_features, 'price', normalized_cars, k)

feature_k_rmse2

plt.figure(figsize=(6, 6))

for key, value in feature_k_rmse2.items():

plt.legend()

plt.show()

```

 

<img>

 

## References

[1] Dynamic Pricing: How Pricing Optimization And Revenue Management Benefit From Machine Learning [https://www.scalr.ai/post/dynamic-pricing-price-optimization](https://www.scalr.ai/post/dynamic-pricing-price-optimization)

[2] Integrating business optimization with a machine learning model [https://towardsdatascience.com/integrating-business-optimization-with-a-machine-learning-model-ad0471420693](https://towardsdatascience.com/integrating-business-optimization-with-a-machine-learning-model-ad0471420693)

[3] Business optimization with a machine learning model CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Used by a range of industries, dynamic pricing is the action setting your product or services pricing based on the current state of the market demands. With the huge amount of data generated by their customers via analytics and social media, companies can use that data to adjust their pricing in real time with dynamic pricing. Using the data allows companies to understand the current state of their market, make adjustments as needed, and maximize their opportunities to increase their revenue. 

Predictions Dynamic Pricing uses Machine Learning

# The Importance of Artificial Intelligence in Supply Chain Management

Artificial intelligence (AI) was introduced to develop and create “thinking machines” that are capable of mimicking, learning, and replacing human intelligence. Since the late 1970s, AI has shown great promise in improving human decision-making processes and the subsequent productivity in various business endeavors due to its ability to recognize business patterns, learn business phenomena, seek information, and analyze data intelligently. Despite its widespread acceptance as a decision-aid tool, AI has seen limited application in supply chain management (SCM)

Supply chain management (SCM) is critical in almost every industry today. Still, despite the importance, it hasn`t received the same amount of focus from AI startups and vendors as many other domains. However, given the vast amounts of data collected by industrial logistics, transportation and warehousing, this is an area with a lot of potential.

In an era of greater demand uncertainty, higher supply risk, and increasing competitive intensity, supply chain (SC) excellence often hinges on the organization’s ability to integrate and entire spectrum of end-to-end processes of acquiring materials or components, converting them into finished goods, and delivering them to customers. Since such ability can be enhanced by increased visibility across the end-to-end SC processes, many leading-edge organizations have attempted to enrich their information sources and share real-time information with SC partners. Thus, SC management (SCM) is becoming more information intensive and its focus has been directed toward the substitution of assets (e.g., inventory, warehouses, transportation equipment) with information. Recognizing the increasing significance of information to SC success, SC professionals have explored various ways to better manage information and leverage it to make better business decisions. One of those ways may include artificial intelligence (AI) that has been in existence for decades, but has not been fully utilized in the area of SCM.

One area of AI’s potential application that has not yet been fully explored is the emerging management philosophy of SCM, which requires the comprehension of complex, interrelated decision-making processes and the creation of intelligent knowledge bases crucial for joint problem-solving.

Supply chain is one of the trickiest parts of a business and as the chances of error are quite high, the need for proper synchronization and analytics is very important. Something simple getting out of hand can make the business lose a lot of money as it can create a break or kink in the supply chain. When it comes to strategic and operational parts of the supply chain management, the need for resolving issues on-the-go to keep the chain continuous is very important or every step in the continuation can become a painful point which will block the resources in the wrong places.

This is the reason why data analytics is now taken very seriously in the management of the supply chain. The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of _[data science](https://360digitmg.com/data-science) in the [supply chain](https://360digitmg.com/india/certification-program-in-supply-chain-analytics)_ is helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

Supply chain management is often depicted as having five key components: arrangement, supply of raw materials, assembly, transportation, and returns. While the regulation phase implies promoting a general system for the supply chain, the other four components have some expertise in critical needs to achieve this regulation. Organizations must develop talent in each of the five components to have a competent supply chain and avoid costly bottlenecks. Acknowledging the expanding metrics of data, two experts and researchers highlight the importance of Big Data around the world and the possibility of adding value and improving companies.

Big Data is defined as massive or complex data arrangements, whose scope is an exabyte, from which the sky is the limit. The capacity framework transcends the specialized capacity domain, such as drafting, overseeing, decoding and depicting a traditional framework. As the Business Analytics (BA) team, where the business follows insightful procedures, strategies and data-based logical systems, the term Big Data Analytics data credits depicted by capacity, diversity, and rapidity are measured and regulated by applying thoughts to choose better decisions. These links practice a wide range of intelligent thoughts. Supply Chain Analytics is clearly here to improve data planning limits and supply chain business. This strengthens working with the unique cycle in the union's supply chain.

## AI for the supply chain

Like in any other industry, the current focus on digitalization is transforming also supply chain management. Improving the efficiency of the supply chain is of great importance for many companies. Operating within tough profit margins, even slight improvements can have substantial impact on the bottom line profit.

Examples where data analytics and machine learning can be beneficial for supply chain management is e.g. within demand forecasting and warehouse optimization. Given the vast amounts of data collected by industrial logistics, transportation and warehousing, being able to harness these data to drive operational performance can be a game changer for those that do it correctly.

## Case study: Predictive analytics Efficiency in Supply Chain Management

To illustrate the use of machine learning in the supply chain, I will go through an example case study focused on business case taken from article “Leveraging Machine Learning for Efficiency in Supply Chain Management”, with new contributions at the code in python.

```python

""" Step 1 - Import the required modules"""

import pandas as pd

import numpy as np

from sklearn.tree import DecisionTreeRegressor # Import Decision Tree Algorithm

from sklearn.preprocessing import StandardScaler

from sklearn.svm import SVR #import for support vector regressor

import matplotlib.pyplot as plt

import pyEX as p

```

 

```python

""" Step 2 - Read the data source"""

SourceData=pd.read_excel("Supplier Past Performance.xls") # Load the training data into Pandas

Testdata=pd.read_excel("Defect Predict.xls") # Load the test data

```

 

```python

""" Step 3 - Declare the independent and dependent train data from the sample"""

SourceData_train_independent= SourceData.drop(["Defect Percent"], axis=1) # Drop depedent variable from training dataset

SourceData_train_dependent=SourceData["Defect Percent"].copy() # New dataframe with only independent variable value for training dataset

SourceData_train_independent.head(5)

```

 

<table style="width:100%; border: 1px solid #264A60; border-collapse: collapse;">

<tr>

<th style="border: 1px solid #264A60; padding: 4px; color: #264A60;">ID</th>

<th style="border: 1px solid #264A60; padding: 4px; color: #264A60;">Purchase Order Value</th>

<th style="border: 1px solid #264A60; padding: 4px; color: #264A60;">Purchase Order Quantity</th>

<th style="border: 1px solid #264A60; padding: 4px; color: #264A60;">PO Sent in Advance of Delivery</th>

<th style="border: 1px solid #264A60; padding: 4px; color: #264A60;"></th>

</tr>

<tr>

<td style="border: 1px solid #264A60; padding: 4px;">0</td>

<td style="border: 1px solid #264A60; padding: 4px;">1</td>

<td style="border: 1px solid #264A60; padding: 4px;">820867</td>

<td style="border: 1px solid #264A60; padding: 4px;">536</td>

<td style="border: 1px solid #264A60; padding: 4px;">43</td>

</tr>

<tr>

<td style="border: 1px solid #264A60; padding: 4px;">1</td>

<td style="border: 1px solid #264A60; padding: 4px;">2</td>

<td style="border: 1px solid #264A60; padding: 4px;">156147</td>

<td style="border: 1px solid #264A60; padding: 4px;">674</td>

<td style="border: 1px solid #264A60; padding: 4px;">25</td>

</tr>

<tr>

<td style="border: 1px solid #264A60; padding: 4px;">2</td>

<td style="border: 1px solid #264A60; padding: 4px;">3</td>

<td style="border: 1px solid #264A60; padding: 4px;">328463</td>

<td style="border: 1px solid #264A60; padding: 4px;">846</td>

<td style="border: 1px solid #264A60; padding: 4px;">27</td>

</tr>

<tr>

<td style="border: 1px solid #264A60; padding: 4px;">3</td>

<td style="border: 1px solid #264A60; padding: 4px;">4</td>

<td style="border: 1px solid #264A60; padding: 4px;">650906</td>

<td style="border: 1px solid #264A60; padding: 4px;">736</td>

<td style="border: 1px solid #264A60; padding: 4px;">38</td>

</tr>

<tr>

<td style="border: 1px solid #264A60; padding: 4px;">4</td>

<td style="border: 1px solid #264A60; padding: 4px;">5</td>

<td style="border: 1px solid #264A60; padding: 4px;">661390</td>

<td style="border: 1px solid #264A60; padding: 4px;">648</td>

<td style="border: 1px solid #264A60; padding: 4px;">4</td>

</tr>

</table>

 

```python

""" Step 4 - Scale the independent test and train data"""

sc_X = StandardScaler()

X_train=sc_X.fit_transform(SourceData_train_independent.values) # scale the independent variables

y_train=SourceData_train_dependent # scaling is not required for dependent variable

X_test=sc_X.transform(Testdata.values)

```

 

```python

""" Step 5 - Fit the test data in maching learning model - Support Vector Regressor"""

svm_reg = SVR(kernel="linear", C=1)

svm_reg.fit(X_train, y_train) # fit and train the modelpredictions = svm_reg.predict(X_test)

“””

SVR(C=1, cache_size=200, coef0=0.0, degree=3, epsilon=0.1, gamma='scale', kernel='linear', max_iter=-1, shrinking=True, tol=0.001, verbose=False)

“””

```

 

```python

""" Step 6 - Predictions - Support Vector Regressor"""

y_pred_svm = svm_reg.predict(X_test)

print(y_pred_svm)

```

 

```python

""" Step 7 - Fit the test data in machine learning model - Decision Tree Model"""

tree_reg = DecisionTreeRegressor()

tree_reg.fit(X_train, y_train)

“””

DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=None,

max_features=None, max_leaf_nodes=None,

min_impurity_decrease=0.0, min_impurity_split=None,

min_samples_leaf=1, min_samples_split=2,

min_weight_fraction_leaf=0.0, presort='deprecated',

random_state=None, splitter='best')

“””

```

 

```python

""" Step 8 - Predictions - Decision Tree"""

y_pred_tree = tree_reg.predict(X_test)

print(y_pred_tree)

```

 

```python

""" Step 9 - Graphs Predictions - SVM - Decision Tree"""

plt.plot(SourceData_train_independent['ID'], y_pred_svm, label='SVM',color = 'black')

plt.plot(SourceData_train_independent['ID'], y_pred_tree, label='Decision Tree',color = 'blue')

plt.legend(loc='upper right')

plt.title('Predictions SVR - Decision Tree')

plt.xlabel('Suppliers')

plt.ylabel('Defect Predict')

plt.show()

```

 

<img>

 

## References

[1] Hokey Min. International Journal of Logistics: Research and Applications. Vol. 13, No. 1, February 2010, 13–39. Artificial intelligence in supply chain management: theory and applications. Artificial Intelligence in Supply Chain Management

[2] Artificial Intelligence in Supply Chain Management. [https://towardsdatascience.com/artificial-intelligence-in-supply-chain-management-predictive-analytics-for-demand-forecasting-80%202d512f155](https://towardsdatascience.com/artificial-intelligence-in-supply-chain-management-predictive-analytics-for-demand-forecasting-80%202d512f155)

[3] Supply Chain Analytics with Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[7] Leveraging Machine Learning for efficiency in Supply Chain Management. 2020. [https://www.analyticsvidhya.com/blog/2020/07/leveraging-machine-learning-for-efficiency-in-supply-chain-management/](https://www.analyticsvidhya.com/blog/2020/07/leveraging-machine-learning-for-efficiency-in-supply-chain-management/)

Artificial intelligence (AI) was introduced to develop and create “thinking machines” that are capable of mimicking, learning, and replacing human intelligence. Since the late 1970s, AI has shown great promise in improving human decision-making processes and the subsequent productivity in various business endeavors due to its ability to recognize business patterns, learn business phenomena, seek information, and analyze data intelligently. Despite its widespread acceptance as a decision-aid tool, AI has seen limited application in supply chain management (SCM)

The Importance of Artificial Intelligence in Supply Chain Management

# Data Analytics in Monte Carlo

Monte Carlo, is in fact, the name of the world-famous casino located in the eponymous district of the city-state (also called a Principality) of Monaco, on the world-famous French Riviera.

It turns out that the casino inspired the minds of famous scientists to devise an intriguing mathematical technique for solving complex problems in statistics, numerical computing, and system simulation.

Clearly, Business and Finance domains specifically are highly dominated by Random Variables. Ranging from Stock Returns, Risk involved in your large project, what’s the likelihood that an asset price would move in a specific way, applications of Monte Carlo even range to Astrophysics, even Integral forms which are computationally huge can be solved using Monte Carlo Simulation just by constantly repeating random samples to achieve our result.

Be it any field you’re working on, this simulation can be used to validate the possible outcomes of our decisions, and decisions are integral part of our lives.

## What is Monte Carlo Simulation?

Monte Carlo methods or Monte Carlo Experiments are a broad class of computational algorithms which are used to model the probability of different outcomes in a process that cannot be easily modeled due to introduction of random variable.

Monte Carlo methods are a class of techniques for randomly sampling a probability distribution.

There are many problem domains where describing or estimating the probability distribution is relatively straightforward, but calculating a desired quantity is intractable. This may be due to many reasons, such as the stochastic nature of the domain or an exponential number of random variables.

Instead, a desired quantity can be approximated by using random sampling, referred to as Monte Carlo methods. These methods were initially used around the time that the first computers were created and remain pervasive through all fields of science and engineering, including artificial intelligence and machine learning.

So, The Monte Carlo method is a non-deterministic or statistical numerical method, used to approximate complex mathematical expressions that are expensive to evaluate accurately. The Monte Carlo method provides approximate solutions to a great variety of mathematical problems, making it possible to carry out experiments with samples of pseudo-random numbers on a computer. The method is applicable to any type of problem, be it stochastic or deterministic. Unlike numerical methods that rely on evaluations at N points in M-dimensional space to produce an approximate solution. The current importance of the Monte Carlo method is based on the existence of problems that are difficult to solve by exclusively analytical or numerical methods, but that depends on random factors or can be associated with an artificial probabilistic model (resolution of integrals of many variables, minimization of functions, etc.).

So, Thanks to advances in computer design, Monte Carlo calculations that in other times would have been inconceivable today are presented as affordable for solving certain problems. In these methods the error ~ 1 / √N, where N is the number of trials and, therefore, gaining one decimal place in precision implies increasing N by 100 times.

The basis is the generation of random numbers that we will use to calculate probabilities. In short, getting a good generator of these numbers as well as an adequate statistical set to work on are the first difficulties we will encounter when using this method.

The idea behind the application of the Monte Carlo method is that there are aspects, such as time or cost that cannot be determined exactly, since they are variable points. It is a double-sided variability. On the one hand, the estimates that are carried out are themselves variable, since an action neither always lasts the same nor does it always cost the same; and on the other hand, the risk itself, since they present a different probability of occurring in different situations, as well as an impact that varies from one situation to another.

Monte Carlo allows the analysts in charge of carrying out risk prevention to indicate in what percentage of the random simulations that aspects such as time and cost have been carried out with less than the objectives pursued with the project itself. Thus, if this percentage is lower than the degree of confidence that a certain company or group considers minimally acceptable, it would be a case in which planning is not feasible and it would have to be modified.

Today, it is a technique used in a wide swath of fields:

> _risk analysis, financial engineering, supply chain logistics, healthcare research, drug development statistical learning and modeling, computer graphics, image processing, game design, large system simulations, computational physics, astronomy, etc_.

We demonstrate it in this article with a simple set of Python code, the use Monte Carlo for correlated random variables with Gaussian Copulas.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
from scipy import stats as stats
from scipy.stats import rv_continuous, rv_histogram, norm, beta, multivariate_normal
import warnings
warnings.filterwarnings("ignore")
np.set_printoptions(precision=3, suppress=True)
pd.options.display.float_format = '{:.3f}'.format
N = 10000 # number of random variates to generate
# define a new class pertm_gen: a generator for the PERT distribution

class pertm_gen(rv_continuous):
 '''modified beta_PERT distribution'''

 
 def _shape(self, min, mode, max, lmb):
 s_alpha = 1+ lmb*(mode - min)/(max-min)
 s_beta = 1 + lmb*(max - mode)/(max-min)
 return [s_alpha, s_beta]


 def _cdf(self, x, min, mode, max, lmb):
 s_alpha, s_beta = self._shape(min, mode, max, lmb)
 z = (x - min) / (max - min)
 cdf = beta.cdf(z, s_alpha, s_beta)
 return cdf

 def _ppf(self, p, min, mode, max, lmb):
 s_alpha, s_beta = self._shape(min, mode, max, lmb)
 ppf = beta.ppf(p, s_alpha, s_beta)
 ppf = ppf * (max - min) + min
 return ppf


 def _mean(self, min, mode, max, lmb):
 mean = (min + lmb * mode + max) / (2 + lmb)
 return mean

 def _var(self, min, mode, max, lmb):
 mean = self._mean(min, mode, max, lmb)
 var = (mean - min) * (max - mean) / (lmb + 3)
 return var

 def _skew(self, min, mode, max, lmb):
 mean = self._mean(min, mode, max, lmb)
 skew1 = (min + max - 2*mean) / 4
 skew2 = (mean - min) * (max - mean)
 skew2 = np.sqrt(7 / skew2)
 skew = skew1 * skew2
 return skew

 def _kurt(self, min, mode, max, lmb):
 a1,a2 = self._shape(min, mode, max, lmb)
 kurt1 = a1 + a2 +1
 kurt2 = 2 * (a1 + a2)**2
 kurt3 = a1 * a2 * (a1 + a2 - 6)
 kurt4 = a1 * a2 * (a1 + a2 + 2) * (a1 + a2 + 3)
 kurt5 = 3 * kurt1 * (kurt2 + kurt3)
 kurt = kurt5 / kurt4 - 3 # scipy defines kurtosis of std normal distribution as 0 instead of 3
 return kurt

 def _stats(self, min, mode, max, lmb):
 mean = self._mean(min, mode, max, lmb)
 var = self._var(min, mode, max, lmb)
 skew = self._skew(min, mode, max, lmb)
 kurt = self._kurt(min, mode, max, lmb)
 return mean, var, skew, kurt

pertm = pertm_gen(name="pertm")

# step 0: define your targeted correlation matrix

# targeted correlation matrix: volume, price, material unit cost
c_target = np.array( [[ 1.0, -0.3, 0.2],
 [ -0.3, 1.0, 0.7],
 [ 0.2, 0.7, 1.0]])
# step 1: draw random variates from a multivariate normal distribution 
# with the targeted correlation structure
r0 = [0] * c_target.shape[0] # create vector r with as many zeros as correlation matrix has variables (row or columns)
mv_norm = multivariate_normal(mean=r0, cov=c_target) # means = vector of zeros; cov = targeted corr matrix
rand_Nmv = mv_norm.rvs(N) # draw N random variates

# step 2: convert the r * N multivariate variates to scores 
rand_U = norm.cdf(rand_Nmv) # use its cdf to generate N scores (probabilities between 0 and 1) from the multinormal random variates

# step 3: instantiate the 3 marginal distributions 
# distribution parameters:
min, mode, max, lmb = 8000.0, 12000.0, 18000.0, 4.0 # sales volume, PERT estimae
ms, ss = 20.0, 1.0 # selling price, normally distributed
mm, sm = 13.0, 0.7 # material unit cost, normally distributed 
o = 3.0 # other unit cost, deterministic 

# instantiate the (uncorrelated) marginal distributions, one for each of the input variables
d_P = pertm(min, mode, max, lmb) 
d_N1 = norm(ms, ss) 
d_N2 = norm(mm, sm) 

# draw N random variates for each of the three marginal distributions
# WITHOUT applying a copula
rand_P = d_P.rvs(N) # sales volume
rand_N1 = d_N1.rvs(N) # selling price
rand_N2 = d_N2.rvs(N) # raw material unit cost

# initial correlation structure before applying a copula
c_before = np.corrcoef([rand_P, rand_N1, rand_N2])
c_before

# helper function: plot correlation structure of a pair of random variables
def plotcorr(rand1, rand2, name1, name2):
 min1, max1, min2, max2 = rand1.min(), rand1.max(), rand2.min(), rand2.max()
 h = sns.jointplot(rand1, rand2, kind='kde', space=0, fill=True, xlim=(min1, max1), ylim=(min2, max2), stat_func=None);
 h.set_axis_labels(name1, name2, fontsize=16);

# call plot function: correlation structures BEFORE applying the Gaussian copula
plotcorr(rand_P, rand_N1, "sales volume 1", "selling price 1")
plotcorr(rand_P, rand_N2, "sales volume 1", "material unit cost 1")
plotcorr(rand_N1, rand_N2, "selling price 1", "material unit cost 1")

# step 4: draw N random variates for each of the three marginal disibutions
# and use as inputs the correlated uniform scores we have generated in step 2


rand_P = d_P.ppf(rand_U[:, 0]) # sales volume
rand_N1 = d_N1.ppf(rand_U[:, 1]) # selling price
rand_N2 = d_N2.ppf(rand_U[:, 2]) # raw material unit cost

# correlation structure AFTER applying the copula
c_after = np.corrcoef([rand_P, rand_N1, rand_N2])
c_after

# discrepancy of copula-imposed correlations vs targeted correlations
c_after - c_target

# call plot function: correlation structures after applying the Gaussina copula
plotcorr(rand_P, rand_N1, "sales volume 2", "selling price 2")
plotcorr(rand_P, rand_N2, "sales volume 2", "material unit cost 2")
plotcorr(rand_N1, rand_N2, "selling price 2", "material unit cost 2")

# set up the simulation model:

# input variables: assign the arrays of N random variables to shorter variable names
v = rand_P # sales volume, PERT distributed
p = rand_N1 # selling price, normally distributed
m = rand_N2 # material unit cost, normally distributed

# output variables
GP = v * (p - m - o) # simulation output: Gross Profit
R = v * p # simulation output: Revenues
GM = GP / R # simulation output: Gross Margin
C = R - GP # simulation output: Total Cost

# helper function to plot histogram of an array with N random variates
def plothist(data, name):
 hist = np.histogram(data, bins=100)
 histdist = rv_histogram(hist)

 X = np.linspace(data.min(), data.max(), 100)
 plt.title(name)
 plt.hist(data, density=True, bins=100)
 plt.plot(X, histdist.pdf(X), label="pdf")
 plt.show()

# call the histogram plotter

plothist(GP, "Gross Profit")
plothist(R, "Revenues")
plothist(R, "Gross Margin")

plothist(v, "sales volume")
plothist(p, "selling price")
plothist(m, "material unit cost")
```

 

<img>

 

<img>

 

<img>

 

<img>

 

<img>

 

<img>

 

## References

[1] What is Monte Carlo Integration? [https://www.kdnuggets.com/2020/12/monte-carlo-integration-python.html](https://www.kdnuggets.com/2020/12/monte-carlo-integration-python.html)

[2] Monte Carlo Simulation with Python [https://medium.com/analytics-vidhya/monte-carlo-simulation-with-python-724ec381ae6e](https://medium.com/analytics-vidhya/monte-carlo-simulation-with-python-724ec381ae6e)

[3] Universidad de Alcalá. Que son los Métodos de Montecarlo? [https://www.master-finanzas-cuantitativas.com/metodos-de-montecarlo/](https://www.master-finanzas-cuantitativas.com/metodos-de-montecarlo/)

[4] A Gentle Introduction to Monte Carlo Sampling for Probability [https://machinelearningmastery.com/monte-carlo-sampling-for-probability/](https://machinelearningmastery.com/monte-carlo-sampling-for-probability/)

[5] Artificial Intelligence in Supply Chain Management [https://towardsdatascience.com/artificial-intelligence-in-supply-chain-management-predictive-analytics-for-demand-forecasting-80%202d512f155](https://towardsdatascience.com/artificial-intelligence-in-supply-chain-management-predictive-analytics-for-demand-forecasting-80%202d512f155)

[6] Python Monte Carlo Simulations: Copulas Unchained [https://towardsdatascience.com/python-monte-carlo-simulations-copulas-unchained-ff29d051c0a0](https://towardsdatascience.com/python-monte-carlo-simulations-copulas-unchained-ff29d051c0a0)

[7] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[8] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Monte Carlo, is in fact, the name of the world-famous casino located in the eponymous district of the city-state (also called a Principality) of Monaco, on the world-famous French Riviera.
It turns out that the casino inspired the minds of famous scientists to devise an intriguing mathematical technique for solving complex problems in statistics, numerical computing, and system simulation.
Clearly, Business and Finance domains specifically are highly dominated by Random Variables. Ranging from Stock Returns, Risk involved in your large project, what’s the likelihood that an asset price would move in a specific way, applications of Monte Carlo even range to Astrophysics, even Integral forms which are computationally huge can be solved using Monte Carlo Simulation just by constantly repeating random samples to achieve our result.
Be it any field you’re working on, this simulation can be used to validate the possible outcomes of our decisions, and decisions are integral part of our lives.

Data Analytics in Monte Carlo

# Machine Learning for Efficiency in Supply Chain Management

Machine Learning, Deep Learning and AI are enabling transformational change in all fields from medicine to music. It is helping businesses from procuring to pay, plan to produce, order to cash and record to report, to uncover optimizing opportunities and support decision making which was never possible earlier with traditional business analytics and information technology.

I will avoid complex and subtle details in this article and keep the example simple at this point. We will discuss complex scenarios in the next series in this article. I will be using the programming language Python to implement machine learning algorithms to solve supply chain business challenges in this series, and I am assuming that you have some basic understanding of Python language.

## Business Case

A vendor supplies different materials with wide unit prices and total purchase order values. Sometimes a few parts from suppliers fail to pass the quality checks, but the defect percentage doesn’t seem to be following a trend.

## Objective

The business would like to predict the defective piece percentage in delivery from the supplier. If the defective piece percentage predicted is below the threshold level, then a quality check will not be performed. Let us consider that if the defect piece percentage in the delivery is more than 0.4%, then explicit incoming inspection is needed to ensure the quality of the end product.

The current dynamic and increasingly serious market has expanded the complexities that constrain organizations around the world from starting to take advantage of Big Data regulations to support competition around the world, as it turns out that they are fetching more data determined in dealing with supply chains. Subjective strategies are applied in this review to better understand the wonder of Supply Chain analytics. An inductive methodology of meeting type was applied to obtain new observational data. Fifteen semiorganized meetings were held with administrative professionals in the study of the supply chain, management and big data analytics. The obtained observational data were then analyzed using the topical examination technique by the help of the previous literature and the case studies .Key discoveries in this premise are partially rejected by past reviews and existing articles in terms of the implications, definitions, and uses of the three types of Analytics. Moreover, the discoveries offer new methodologies and perspectives that the latest analytics apply at both basic and operational management levels that shape supply chain management plans.

Supply chain is one of the trickiest parts of a business and as the chances of error are quite high, the need for proper synchronization and analytics is very important. Something simple getting out of hand can make the business lose a lot of money as it can create a break or kink in the supply chain. When it comes to strategic and operational parts of the supply chain management, the need for resolving issues on-the-go to keep the chain continuous is very important or every step in the continuation can become a painful point which will block the resources in the wrong places.

This is the reason why data analytics is now taken very seriously in the management of the supply chain. The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of data science in the supply chain is helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

As mentioned above, supply chain management is all about accurate forecasting where further decisions regarding logistics and production can be taken. This is the reason why predictive analytics can be very useful for the business to make their supply chain more productive. Supply chain forecasting is a tricky situation as there are many variables to take into consideration. Long-term forecasting is still something that can be managed, but short-term forecasting can become quite difficult. This is because the number of variables that come to the scene in short-term forecasting are numerous and hard to track.

Predictive analytics is about predicting future phenomena using historical data sets. Therefore, for accurate forecasting, predictive analytics is highly helpful as it shows various mathematical models of predictive forecasting. Also, the predictive analytics tools help compare two roadmaps and decide which is more profitable and efficient. It takes into consideration various factors, both internal and external. This process is known as the optimization of supply chain processes. Therefore, it can be said that the predictive analytics of data science are highly helpful in forecasting, optimization, and visualization of various processes in a supply chain of businesses.

## Code Machine Learning for Efficiency in Supply Chain Management

This will help to focus on quality checks on only particular purchase orders delivery, to control the end-product quality, and optimize inspection costs. It will also enable us to uncover the variables/parameters which are influencing the defective deliveries of few materials at times and work collaboratively with the supplier to address it.

```python
""" Step 1 - Import the required modules"""
import pandas as pd
from sklearn.tree import DecisionTreeRegressor 
from sklearn.preprocessing import StandardScaler 
from sklearn.svm import SVR
```

```python
""" Step 2 - Read the data source"""
SourceData=pd.read_excel("Supplier Past Performance.xls") # Load the training data into Pandas DataFrame
Testdata=pd.read_excel("Defect Predict.xls") # Load the test data
```

```python
""" Step 3 - Declare the independent and dependent train data from the sample"""
SourceData_train_independent= SourceData.drop(["Defect Percent"], axis=1) # Drop depedent variable from training dataset
SourceData_train_dependent=SourceData["Defect Percent"].copy() # New dataframe with only independent variable value for training dataset
```

```python
""" Step 4 - Scale the independent test and train data"""
sc_X = StandardScaler()
X_train=sc_X.fit_transform(SourceData_train_independent.values) # scale the independent variables
y_train=SourceData_train_dependent # scaling is not required for dependent variable
X_test=sc_X.transform(Testdata.values)
```

```python
""" Step 5 - Fit the test data in maching learning model - Support Vector Regressor"""
svm_reg = SVR(kernel="linear", C=1)
svm_reg.fit(X_train, y_train) # fit and train the model

predictions = svm_reg.predict(X_test)

print("Defect percent prediction by Support Vector model for the order value of 95827 GBP with 851 pallets sent 55 days before delivery data is " ,round(predictions[0],2) , "%")
```

 
Defect percent prediction by Support Vector model for the order value of 95827 GBP with 851 pallets sent 55 days before delivery data is 1.31 %
 

```python
""" Step 6 - Fit the test data in maching learning model - Decision Tree Model"""
tree_reg = DecisionTreeRegressor()
tree_reg.fit(X_train, y_train) # fit and train the model

decision_predictions = tree_reg.predict(X_test) # Predict the value of dependent variable

print("Defect percent prediction by Decision Tree model for the order value of 95827 GBP with 851 pallets sent 55 days before delivery data is " ,round(decision_predictions[0],2) , "%")
```

 
Defect percent prediction by Decision Tree model for the order value of 95827 GBP with 851 pallets sent 55 days before delivery data is 1.43 %
 

## References

[1] Leveraging Machine Learning for Efficiency in Supply Chain Management. 2020. [https://www.analyticsvidhya.com/blog/2020/07/leveraging-machine-learning-for-efficiency-in-supply-chain-management/](https://www.analyticsvidhya.com/blog/2020/07/leveraging-machine-learning-for-efficiency-in-supply-chain-management/)

[2] Artificial Intelligence in Supply Chain Management [https://towardsdatascience.com/artificial-intelligence-in-supply-chain-management-predictive-analytics-for-demand-forecasting-802d512f155](https://towardsdatascience.com/artificial-intelligence-in-supply-chain-management-predictive-analytics-for-demand-forecasting-802d512f155)

[3] Supply Chain Analytics with Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Machine Learning, Deep Learning and AI are enabling transformational change in all fields from medicine to music. It is helping businesses from procuring to pay, plan to produce, order to cash and record to report, to uncover optimizing opportunities and support decision making which was never possible earlier with traditional business analytics and information technology.

I will avoid complex and subtle details in this article and keep the example simple at this point. We will discuss complex scenarios in the next series in this article. I will be using the programming language Python to implement machine learning algorithms to solve supply chain business challenges in this series, and I am assuming that you have some basic understanding of Python language.

Machine Learning for Efficiency in Supply Chain Management

# Deep Learning with TensorFlow and Neural Networks

Machine learning is a subfield of artificial intelligence (AI). The goal of machine learning generally is to understand the structure of data and fit that data into models that can be understood and utilized by people. Although machine learning is a field within computer science, it differs from traditional computational approaches. In traditional computing, algorithms are sets of explicitly programmed instructions used by computers to calculate or problem solve. Machine learning algorithms instead allow for computers to train on data inputs and use statistical analysis in order to output values that fall within a specific range. Because of this, machine learning facilitates computers in building models from sample data in order to automate decision-making processes based on data inputs.

Machine learning is a continuously developing field. Because of this, there are some considerations to keep in mind as you work with machine learning methodologies, or analyze the impact of machine learning processes.

Deep Learning attempts to imitate how the human brain can process light and sound stimuli into vision and hearing. A deep learning architecture is inspired by biological neural networks and consists of multiple layers in an artificial neural network made up of hardware and GPUs. Deep learning uses a cascade of nonlinear processing unit layers in order to extract or transform features (or representations) of the data. The output of one layer serves as the input of the successive layer. In deep learning, algorithms can be either supervised and serve to classify data, or unsupervised and perform pattern analysis. Among the machine learning algorithms that are currently being used and developed, deep learning absorbs the most data and has been able to beat humans in some cognitive tasks. Because of these attributes, deep learning has become the approach with significant potential in the artificial intelligence space Computer vision and speech recognition have both realized significant advances from deep learning approaches. IBM Watson is a well-known example of a system that leverages deep learning.

TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications. So, build and train ML models easily using intuitive high-level APIs like Keras with eager execution, which makes for immediate model iteration and easy debugging. Easily train and deploy models in the cloud, on-prem, in the browser, or on-device no matter what language you use and A simple and flexible architecture to take new ideas from concept to code, to state-of-the-art models, and to publication faster.

## How To Build a Neural Network with TensorFlow

Neural networks are used as a method of deep learning, one of the many subfields of artificial intelligence. They were first proposed around 70 years ago as an attempt at simulating the way the human brain works, though in a much more simplified form. Individual ‘neurons’ are connected in layers, with weights assigned to determine how the neuron responds when signals are propagated through the network. Previously, neural networks were limited in the number of neurons they were able to simulate, and therefore the complexity of learning they could achieve. But in recent years, due to advancements in hardware development, we have been able to build very deep networks, and train them on enormous datasets to achieve breakthroughs in machine intelligence. These breakthroughs have allowed machines to match and exceed the capabilities of humans at performing certain tasks. One such task is object recognition. Though machines have historically been unable to match human vision, recent advances in deep learning have made it possible to build neural networks which can recognize objects, faces, text, and even emotions. In this tutorial, you will implement a small subsection of object recognition—digit recognition. Using TensorFlow (https://www.tensorflow.org/), an open-source Python library developed by the Google Brain labs for deep learning research, you will take hand-drawn images of the numbers 0-9 and build and train a neural network to recognize and predict the correct label for the digit displayed. While you won’t need prior experience in practical deep learning or TensorFlow to follow along with this tutorial, we’ll assume some familiarity with machine learning terms and concepts such as training and testing, features and labels, optimization, and evaluation.

## Code Neural Network with TensorFlow

```python
from tensorflow.keras import models, layers, utils, backend as K
import matplotlib.pyplot as plt
import shap

model = models.Sequential(name="Perceptron", layers=[
 layers.Dense( #a fully connected layer
 name="dense",
 input_dim=3, #with 3 features as the input
 units=1, #and 1 node because we want 1 output
 activation='linear' #f(x)=x
 )
])
model.summary()
```

```bash
 Model: "Perceptron"
_________________________________________________________________
Layer (type) Output Shape Param # 
=================================================================
dense (Dense) (None, 1) 4 
=================================================================
Total params: 4
Trainable params: 4
Non-trainable params: 0
_________________________________________________________________
```

```python
# define the function
import tensorflow as tf
def binary_step_activation(x):
 ##return 1 if x>0 else 0 
 return K.switch(x>0, tf.math.divide(x,x), tf.math.multiply(x,0))

# build the model
model = models.Sequential(name="Perceptron", layers=[
 layers.Dense( 
 name="dense",
 input_dim=3, 
 units=1, 
 activation=binary_step_activation
 )
])

n_features = 10
model = models.Sequential(name="DeepNN", layers=[
 ### hidden layer 1
 layers.Dense(name="h1", input_dim=n_features,
 units=int(round((n_features+1)/2)), 
 activation='relu'),
 layers.Dropout(name="drop1", rate=0.2),
 
 ### hidden layer 2
 layers.Dense(name="h2", units=int(round((n_features+1)/4)), 
 activation='relu'),
 layers.Dropout(name="drop2", rate=0.2),
 
 ### layer output
 layers.Dense(name="output", units=1, activation='sigmoid')
])
model.summary()
```

```bash
Model: "DeepNN"
_________________________________________________________________
Layer (type) Output Shape Param # 
=================================================================
h1 (Dense) (None, 6) 66 
_________________________________________________________________
drop1 (Dropout) (None, 6) 0 
_________________________________________________________________
h2 (Dense) (None, 3) 21 
_________________________________________________________________
drop2 (Dropout) (None, 3) 0 
_________________________________________________________________
output (Dense) (None, 1) 4 
=================================================================
Total params: 91
Trainable params: 91
Non-trainable params: 0
_________________________________________________________________
```

```python
# Perceptron
inputs = layers.Input(name="input", shape=(3,))
outputs = layers.Dense(name="output", units=1, 
 activation='linear')(inputs)
model = models.Model(inputs=inputs, outputs=outputs, 
 name="Perceptron")
model.summary()
# DeepNN
### layer input
inputs = layers.Input(name="input", shape=(n_features,))
### hidden layer 1
h1 = layers.Dense(name="h1", units=int(round((n_features+1)/2)), activation='relu')(inputs)
h1 = layers.Dropout(name="drop1", rate=0.2)(h1)
### hidden layer 2
h2 = layers.Dense(name="h2", units=int(round((n_features+1)/4)), activation='relu')(h1)
h2 = layers.Dropout(name="drop2", rate=0.2)(h2)
### layer output
outputs = layers.Dense(name="output", units=1, activation='sigmoid')(h2)
model = models.Model(inputs=inputs, outputs=outputs, name="DeepNN")
model.summary()
```

```bash
Model: "Perceptron"
_________________________________________________________________
Layer (type) Output Shape Param # 
=================================================================
input (InputLayer) [(None, 3)] 0 
_________________________________________________________________
output (Dense) (None, 1) 4 
=================================================================
Total params: 4
Trainable params: 4
Non-trainable params: 0
_________________________________________________________________
Model: "DeepNN"
_________________________________________________________________
Layer (type) Output Shape Param # 
=================================================================
input (InputLayer) [(None, 10)] 0 
_________________________________________________________________
h1 (Dense) (None, 6) 66 
_________________________________________________________________
drop1 (Dropout) (None, 6) 0 
_________________________________________________________________
h2 (Dense) (None, 3) 21 
_________________________________________________________________
drop2 (Dropout) (None, 3) 0 
_________________________________________________________________
output (Dense) (None, 1) 4 
=================================================================
Total params: 91
Trainable params: 91
Non-trainable params: 0
_________________________________________________________________
```

```python
'''
Extract info for each layer in a keras model.
'''
def utils_nn_config(model):
 lst_layers = []
 if "Sequential" in str(model): #-> Sequential doesn't show the input layer
 layer = model.layers[0]
 lst_layers.append({"name":"input", "in":int(layer.input.shape[-1]), "neurons":0, 
 "out":int(layer.input.shape[-1]), "activation":None,
 "params":0, "bias":0})
 for layer in model.layers:
 try:
 dic_layer = {"name":layer.name, "in":int(layer.input.shape[-1]), "neurons":layer.units, 
 "out":int(layer.output.shape[-1]), "activation":layer.get_config()["activation"],
 "params":layer.get_weights()[0], "bias":layer.get_weights()[1]}
 except:
 dic_layer = {"name":layer.name, "in":int(layer.input.shape[-1]), "neurons":0, 
 "out":int(layer.output.shape[-1]), "activation":None,
 "params":0, "bias":0}
 lst_layers.append(dic_layer)
 return lst_layers

'''
Plot the structure of a keras neural network.
'''
def visualize_nn(model, description=False, figsize=(10,8)):
 ## get layers info
 lst_layers = utils_nn_config(model)
 layer_sizes = [layer["out"] for layer in lst_layers]
 
 ## fig setup
 fig = plt.figure(figsize=figsize)
 ax = fig.gca()
 ax.set(title=model.name)
 ax.axis('off')
 left, right, bottom, top = 0.1, 0.9, 0.1, 0.9
 x_space = (right-left) / float(len(layer_sizes)-1)
 y_space = (top-bottom) / float(max(layer_sizes))
 p = 0.025
 
 ## nodes
 for i,n in enumerate(layer_sizes):
 top_on_layer = y_space*(n-1)/2.0 + (top+bottom)/2.0
 layer = lst_layers[i]
 color = "green" if i in [0, len(layer_sizes)-1] else "blue"
 color = "red" if (layer['neurons'] == 0) and (i > 0) else color
 
 ### add description
 if (description is True):
 d = i if i == 0 else i-0.5
 if layer['activation'] is None:
 plt.text(x=left+d*x_space, y=top, fontsize=10, color=color, s=layer["name"].upper())
 else:
 plt.text(x=left+d*x_space, y=top, fontsize=10, color=color, s=layer["name"].upper())
 plt.text(x=left+d*x_space, y=top-p, fontsize=10, color=color, s=layer['activation']+" (")
 plt.text(x=left+d*x_space, y=top-2*p, fontsize=10, color=color, s="Σ"+str(layer['in'])+"[X*w]+b")
 out = " Y" if i == len(layer_sizes)-1 else " out"
 plt.text(x=left+d*x_space, y=top-3*p, fontsize=10, color=color, s=") = "+str(layer['neurons'])+out)
 
 ### circles
 for m in range(n):
 color = "limegreen" if color == "green" else color
 circle = plt.Circle(xy=(left+i*x_space, top_on_layer-m*y_space-4*p), radius=y_space/4.0, color=color, ec='k', zorder=4)
 ax.add_artist(circle)
 
 ### add text
 if i == 0:
 plt.text(x=left-4*p, y=top_on_layer-m*y_space-4*p, fontsize=10, s=r'$X_{'+str(m+1)+'}$')
 elif i == len(layer_sizes)-1:
 plt.text(x=right+4*p, y=top_on_layer-m*y_space-4*p, fontsize=10, s=r'$y_{'+str(m+1)+'}$')
 else:
 plt.text(x=left+i*x_space+p, y=top_on_layer-m*y_space+(y_space/8.+0.01*y_space)-4*p, fontsize=10, s=r'$H_{'+str(m+1)+'}$')
 
 ## links
 for i, (n_a, n_b) in enumerate(zip(layer_sizes[:-1], layer_sizes[1:])):
 layer = lst_layers[i+1]
 color = "green" if i == len(layer_sizes)-2 else "blue"
 color = "red" if layer['neurons'] == 0 else color
 layer_top_a = y_space*(n_a-1)/2. + (top+bottom)/2. -4*p
 layer_top_b = y_space*(n_b-1)/2. + (top+bottom)/2. -4*p
 for m in range(n_a):
 for o in range(n_b):
 line = plt.Line2D([i*x_space+left, (i+1)*x_space+left], 
 [layer_top_a-m*y_space, layer_top_b-o*y_space], 
 c=color, alpha=0.5)
 if layer['activation'] is None:
 if o == m:
 ax.add_artist(line)
 else:
 ax.add_artist(line)
plt.show()
visualize_nn(model, description=True, figsize=(10,8))
```

 
<img>
 

```python
# define metrics
def Recall(y_true, y_pred):
 true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
 possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
 recall = true_positives / (possible_positives + K.epsilon())
 return recall

def Precision(y_true, y_pred):
 true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
 predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
 precision = true_positives / (predicted_positives + K.epsilon())
 return precision

def F1(y_true, y_pred):
 precision = Precision(y_true, y_pred)
 recall = Recall(y_true, y_pred)
 return 2*((precision*recall)/(precision+recall+K.epsilon()))

# compile the neural network
model.compile(optimizer='adam', loss='binary_crossentropy', 
 metrics=['accuracy',F1])
# define metrics
def R2(y, y_hat):
 ss_res = K.sum(K.square(y - y_hat)) 
 ss_tot = K.sum(K.square(y - K.mean(y))) 
 return ( 1 - ss_res/(ss_tot + K.epsilon()) )

# compile the neural network
model.compile(optimizer='adam', loss='mean_absolute_error', 
 metrics=[R2])
import numpy as np
X = np.random.rand(1000,10)
y = np.random.choice([1,0], size=1000)

# train/validation
training = model.fit(x=X, y=y, batch_size=32, epochs=100, shuffle=True, verbose=0, validation_split=0.3)

# plot
metrics = [k for k in training.history.keys() if ("loss" not in k) and ("val" not in k)] 
fig, ax = plt.subplots(nrows=1, ncols=2, sharey=True, figsize=(15,3))
 
## training 
ax[0].set(title="Training") 
ax11 = ax[0].twinx() 
ax[0].plot(training.history['loss'], color='black') 
ax[0].set_xlabel('Epochs') 
ax[0].set_ylabel('Loss', color='black') 
for metric in metrics: 
 ax11.plot(training.history[metric], label=metric) 
ax11.set_ylabel("Score", color='steelblue') 
ax11.legend()
 
## validation 
ax[1].set(title="Validation") 
ax22 = ax[1].twinx() 
ax[1].plot(training.history['val_loss'], color='black') 
ax[1].set_xlabel('Epochs') 
ax[1].set_ylabel('Loss', color='black') 
for metric in metrics: 
 ax22.plot(training.history['val_'+metric], label=metric) 
ax22.set_ylabel("Score", color="steelblue") 
plt.show()
```

<img>


## References

[1] TensorFlow. [https://www.tensorflow.org/](https://www.tensorflow.org/)

[2]. Machine Learning for Efficiency in Supply Chain Management CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[3] Supply Chain Analytics with Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Techniques for Data Scientist and Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai) 

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Machine learning is a subfield of artificial intelligence (AI). The goal of machine learning generally is to understand the structure of data and fit that data into models that can be understood and utilized by people. Although machine learning is a field within computer science, it differs from traditional computational approaches. In traditional computing, algorithms are sets of explicitly programmed instructions used by computers to calculate or problem solve. Machine learning algorithms instead allow for computers to train on data inputs and use statistical analysis in order to output values that fall within a specific range. Because of this, machine learning facilitates computers in building models from sample data in order to automate decision-making processes based on data inputs.
Machine learning is a continuously developing field. Because of this, there are some considerations to keep in mind as you work with machine learning methodologies, or analyze the impact of machine learning processes.
Deep Learning  attempts to imitate how the human brain can process light and sound stimuli into vision and hearing. A deep learning architecture is inspired by biological neural networks and consists of multiple layers in an artificial neural network made up of hardware and GPUs. Deep learning uses a cascade of nonlinear processing unit layers in order to extract or transform features (or representations) of the data. The output of one layer serves as the input of the successive layer. In deep learning, algorithms can be either supervised and serve to classify data, or unsupervised and perform pattern analysis. Among the machine learning algorithms that are currently being used and developed, deep learning absorbs the most data and has been able to beat humans in some cognitive tasks. Because of these attributes, deep learning has become the approach with significant potential in the artificial intelligence space Computer vision and speech recognition have both realized significant advances from deep learning approaches. IBM Watson is a well-known example of a system that leverages deep learning.
TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications. So, build and train ML models easily using intuitive high-level APIs like Keras with eager execution, which makes for immediate model iteration and easy debugging. Easily train and deploy models in the cloud, on-prem, in the browser, or on-device no matter what language you use and A simple and flexible architecture to take new ideas from concept to code, to state-of-the-art models, and to publication faster.

Deep Learning with TensorFlow and Neural Networks

In the modern day society, supply chain management (SCM) is extremely critical for almost all industries. Supply chain is one of those areas where one can apply technology in order to revolutionize the automation and optimization of different processes. With the vast amount of data that gets collected at every stage in a supply chain, be it transportation, warehousing or be it logistics, it is natural to think of the potential of data science in this field.

Data Science Improve Your Supply Chain Management

Forecasting future demand is essential to making supply chain decisions. Undoubtedly, historical information about demand can be used to forecast future demand and such analysis affects the functioning of the entire supply chain. This section will outline the general principles of supply chain forecasting and the tools used to do so. Demand forecasts form the basis of all supply chain planning. Consider the division of processes performed in the push / pull supply chain (Chopra and Meindl, 2007). Pull processes are initiated by an order placed by the customer, while push processes are initiated and executed while awaiting the order. When considering these two options for push processes, you should plan your level of activity, be it production, transportation, or any other planned activity. With pull processes, you need to plan for the level of available bandwidth and inventory, but not the actual amount of products to be made. In both cases, the first step to be carried out is to forecast customer demand. To be competitive, a company must use a forecast of future demand to determine the quantity of products to be pushed and to determine the production capacity needed in its plants (pull process). Further down the supply chain, another company that produces one of the components also needs forecasts to determine its own production and inventory levels. For the same reason, the suppliers of this company also need forecasts. When each stage of the supply chain makes its own separate forecast, these forecasts are often very different (Chopra and Meindl, 2007; Jaggi and Kadam, 2016). This may result in a mismatch between supply and demand. When all stages of the supply chain work together to produce a shared forecast, it is much more accurate. The resulting forecast accuracy enables supply chains to be both more sensitive and more efficient in serving their customers.

Forecasting Demand in the Supply Chain on Time Series with Machine Learning

# Introduction

Customer Personality Analysis is a detailed analysis of a company’s ideal customers. It helps a business to better understand its customers and makes it easier for them to modify products according to the specific needs, behaviours and concerns of different types of customers. In this article, I’m going to introduce you to a data science project on customer personality analysis with Python.

Data Science and Artificial Intelligence are revolutionizing the world through technical transformations. We can observe many machine learning applications in day-to-day lives, but one of the greatest applications of machine learning is to classify individuals based on their personality traits. Each person on this planet is unique and carries a unique personality. The availability of a high-dimensional and large amount of data has paved the way for increasing marketing campaigns' effectiveness by targeting specific people. Such personality-based communications are highly effective in increasing the popularity and attractiveness of products and services. It increased usage, customer satisfaction, and broader acceptance among users. Some common examples are: Personalization of online advertisement campaigns leads to more revenue and click-through rates.

Personality traits are closely associated with an individual’s behavior and preferences. Hence the fusion of a personality-based approach has primarily increased the Recommender System’s attractiveness.

Personality-based adaption’s are also used to provide personalized visualizations and could even suggest better music recommendations.

<table border="1" style="border-collapse: collapse; width: 100%;">
<tbody>
<tr>
<td colspan="3" style="padding: 10px">
Data Science and Artificial Intelligence are revolutionizing the world through technical
transformations. We can observe many machine learning applications in day-to-day lives, but one of
the
greatest applications of machine learning is to classify individuals based on their personality
traits.
Each person on this planet is unique and carries a unique personality. The availability of a
high-dimensional and large amount of data has paved the way for increasing marketing campaigns'
effectiveness by targeting specific people. Such personality-based communications are highly
effective
in increasing the popularity and attractiveness of products and services. It increased usage,
customer
satisfaction, and broader acceptance among users. Some common examples are: Personalization of
online
advertisement campaigns leads to more revenue and click-through rates.
</td>
</tr>
<tr>
<td style="padding: 10px">
</td>
<td style="padding: 10px">
Personality traits are closely associated with an individual’s behavior and preferences. Hence the
fusion of a personality-based approach has primarily increased the Recommender System’s
attractiveness.
</td>
<td style="padding: 10px">
</td>
</tr>
<tr>
<td style="padding: 10px">
</td>
<td style="padding: 10px">
Personality-based adaption’s are also used to provide personalized visualizations and could even
suggest better music recommendations.
</td>
<td style="padding: 10px">
</td>
</tr>
<tr>
<td style="padding: 10px">
</td>
<td style="padding: 10px">
Personality traits could also solve the “cold start” problem using scientifically validated and
relatively stable latent dimensions of an individual based on personalized systems
</td>
<td style="padding: 10px">
</td>
</tr>
<tr>
<td colspan="3" style="padding: 10px">
With the availability of high-dimensional and fine-grained data about human behavior, it becomes too
handy to research and observe human behavior. Using mobile sensing studies, data collected from our
day-to-day activities have drastically altered how psychologists perform research and undertake
personality assessments. Machine learning models are a boon to researchers and are used to learn
highly complex relationships and evaluate their generalizability and robustness using the resampling
method. It has the potential to transform research and assessment in personality psychology.
Algorithms can handle vast datasets, including thousands of attributes, without succumbing to
co-linearity issues. Moreover, ML algorithms are highly efficient in recognizing patterns in
datasets that humans cannot even perceive. The use of these ML models can lead to better, more
objective, and automated personality assessments.
</td>
</tr>
<tr>
<td colspan="3" style="padding: 10px">
People interact and express their likes, thoughts, feelings, and opinions on social media, capturing
their personality traits. Machine Learning models have been actively using such a wide range of data
to predict individuals. Various supervised machine learning algorithms like Naïve Bayes and Support
Vector Machines are widely used among industries to predict personality traits. Moreover, recently,
researchers have started to apply unsupervised learning methods to identify other psychological
constructs in digital data.
</td>
</tr>
</tbody>
</table>
 

## About this file – Code in Python

Customer Personality Analysis is a detailed analysis of a company’s ideal customers. It helps a business to better understand its customers and makes it easier for them to modify products according to the specific needs, behaviors and concerns of different types of customers.

Customer personality analysis helps a business to modify its product based on its target customers from different types of customer segments. For example, instead of spending money to market a new product to every customer in the company’s database, a company can analyze which customer segment is most likely to buy the product and then market the product only on that particular segment.

```python
# data analysis and wrangling
import pandas as pd
import numpy as np
import random as rnd
# visualization
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
# Clustering
#from yellowbrick.cluster import KElbowVisualizer
from sklearn.cluster import KMeans
from sklearn.preprocessing import LabelEncoder
from sklearn.cluster import AgglomerativeClustering

df = pd.read_csv("marketing_campaign.csv", sep="\t")
df.head()
df.info()
```

```bash
Data columns (total 29 columns):
 # Column Non-Null Count Dtype 
--- ------ -------------- ----- 
 0 ID 2240 non-null int64 
 1 Year_Birth 2240 non-null int64 
 2 Education 2240 non-null object 
 3 Marital_Status 2240 non-null object 
 4 Income 2216 non-null float64
 5 Kidhome 2240 non-null int64 
 6 Teenhome 2240 non-null int64 
 7 Dt_Customer 2240 non-null object 
 8 Recency 2240 non-null int64 
 9 MntWines 2240 non-null int64 
 10 MntFruits 2240 non-null int64 
 11 MntMeatProducts 2240 non-null int64 
 12 MntFishProducts 2240 non-null int64 
 13 MntSweetProducts 2240 non-null int64 
 14 MntGoldProds 2240 non-null int64 
 15 NumDealsPurchases 2240 non-null int64 
 16 NumWebPurchases 2240 non-null int64 
 17 NumCatalogPurchases 2240 non-null int64 
 18 NumStorePurchases 2240 non-null int64 
 19 NumWebVisitsMonth 2240 non-null int64 
 20 AcceptedCmp3 2240 non-null int64 
 21 AcceptedCmp4 2240 non-null int64 
 22 AcceptedCmp5 2240 non-null int64 
 23 AcceptedCmp1 2240 non-null int64 
 24 AcceptedCmp2 2240 non-null int64 
 25 Complain 2240 non-null int64 
 26 Z_CostContact 2240 non-null int64 
 27 Z_Revenue 2240 non-null int64 
 28 Response 2240 non-null int64 
```

```python
# check which columns have null values, it seems only income has null values
df.isnull().sum()
# isolate null values to see if there is any relationship, but it doesnt seem like there is
null_income = df.loc[df.Income.isnull()]
# fill in null income with the median income
med = df.Income.median()
df.Income.fillna(med, inplace=True)
df.isnull().sum()
#convert the column to datetime format
df['Dt_Customer'] = pd.to_datetime(df['Dt_Customer'], infer_datetime_format = True)

#Data Exploration and Visualization
#With the null values filled in and the columns in their appropriate datatype we can now begin exploring the data and forming features. I will create some new features and create some graphs inbetween to visualize relationships and see what other features I might have to create.
#created an age column
df['Age']=2021-df['Year_Birth']

#created a column showing the total amount a customer spent
df['TotalMnt'] = df.MntMeatProducts + df. MntWines + df.MntFruits + df.MntFishProducts + df.MntSweetProducts + df.MntGoldProds
#combined the kidhome and teenhome columns into one column that shows the number of children per household
df['children'] = df.Kidhome + df.Teenhome

#checks how much of a customer's income is spent purchasing these products
df['Income_to_spend'] = round(df['TotalMnt']/df['Income'],3)

#number of purchases the customer made
df['TotalNumPurchases'] = df.NumWebPurchases + df.NumCatalogPurchases + df.NumStorePurchases
#number of purchases that were discounted
df['num_discounted'] = round(df['NumDealsPurchases']/df['TotalNumPurchases'],3)
#check which percent of purchases that were made per platform
df['web_to_total'] = round(df.NumWebPurchases/df.TotalNumPurchases,3)
df['catalog_to_total'] = round(df.NumCatalogPurchases/df.TotalNumPurchases,3)
df['Store_to_total'] = round(df.NumStorePurchases/df.TotalNumPurchases,3)
#fill the na values with 0 because if it is null then that means that there were no purchases for that customer, through that specific channel
df.fillna(0, inplace=True)

#counts the number of campaigns that the customer accepted
df['num_cmp'] = df.AcceptedCmp1 + df.AcceptedCmp2 + df.AcceptedCmp3 + df.AcceptedCmp4 + df.AcceptedCmp5 + df.Response
#graph to see if there is a relationship between total amount spent and number of children
df.boxplot(column = 'TotalMnt', by = 'children')
```

 
<img>
 

```python
#graph to spot outliers
df.boxplot(column = 'Income')
```

 
<img>
 

```python
#removed the one big outlier spotted
df=df.loc[df.Income<df.Income.max()]
#check the distribution of the income, seems relatively normal
sns.histplot(df.Income)
```

 
<img>
 

```python
#check the distribution of amount spent and it seems skewed
sns.histplot(df.TotalMnt) 
```

 
<img>
 

```python
#checked to see if education had a big impact on income and aside from the basic education level, it did no seem like it
sns.boxplot(x=df.Education, y=df.Income)
```

 
<img>
 

```python
#checked the income range between different marital_statuses
df.boxplot(column = 'Income', by = 'Marital_Status')
```

 
<img>
 

```python
#spotted two odd entries
df.Marital_Status.unique()
array(['Single', 'Together', 'Married', 'Divorced', 'Widow', 'Alone','Absurd', 'YOLO'], dtype=object)
#check to see how many inputted those odd entries
df.Marital_Status.value_counts()
Married 864
Together 579
Single 480
Divorced 232
Widow 77
Alone 3
Absurd 2
YOLO 2
#Replaced alone with single, removed absurd and yolo, and combined married and together into one category
df['Marital_Status']=df.Marital_Status.replace({'Alone':'Single'})
df = df.drop(df[df['Marital_Status'] == 'Absurd'].index)
df = df.drop(df[df['Marital_Status'] == 'YOLO'].index)
df['Marital_Status'] = df.Marital_Status.replace({'Married':'Partner', 'Together':'Partner'})

#created a column to check if someone is a parent and if they had a partner
df['isParent'] = df.children.apply(lambda x: 1 if x > 0 else 0)
df['withPartner'] = df.Marital_Status.apply(lambda x: 1 if x == 'Partner' else 0)
#checks for whole family size
df['famsize'] = df.children + df.withPartner + 1

#recheck now that we have removed the other categories
df.boxplot(column = 'Income', by = 'Marital_Status')
```

 
<img>
 

```python
#spotted an outlier and put an age cap to limit it 
df = df.loc[df.Age < 100]
#check for the relationship between total amount spent and total number purhcases
sns.regplot(x = df.TotalMnt, y = df.TotalNumPurchases)
```

 
<img>
 

```python
#check to see if the number of successful campaigns increased sales
sns.barplot(y = 'TotalMnt', x = 'num_cmp', data = df)
```

 
<img>
 

```python
#drop redundant columns
fin_df=df.drop(["Year_Birth", "Z_CostContact", "Z_Revenue",'Kidhome','Teenhome'], axis=1)
df=df.drop(["Year_Birth", "Z_CostContact", "Z_Revenue",'Kidhome','Teenhome'], axis=1)
fin_df

#After visualizing and creating the features it looks like we can now proceed to clustering the customers and we can begin by encoding some of our categorical data

#Encode categorical data 
encode = LabelEncoder()
fin_df['Education']=encode.fit_transform(fin_df['Education'])
fin_df['Marital_Status']=encode.fit_transform(fin_df['Marital_Status'])
fin_df

#Got an error about infinite values and I used this to check where the infinite values were
np.isinf(fin_df.iloc[:]).sum()

#replace infinite values with null and then drop the null rows
fin_df.replace([np.inf, -np.inf], np.nan, inplace=True)
fin_df.dropna(axis = 0,inplace=True)
df.replace([np.inf, -np.inf], np.nan, inplace=True)
df.dropna(axis = 0,inplace=True)
np.all(np.isfinite(fin_df))
True
#scale the values for kmeans clustering
features = ['Income', 'TotalMnt']
fin_df2 = fin_df[features]
fin_df2 = (fin_df2 - fin_df2.mean(axis=0)) / fin_df2.std(axis=0)
#fin_df2 = (fin_df - fin_df.mean(axis=0)) / fin_df.std(axis=0)

#cluster using kmeans
kmeans = KMeans(n_clusters=4, random_state=0)
fin_df["Cluster"] = kmeans.fit_predict(fin_df2)
df['Cluster']=fin_df['Cluster']
#chcek the distribution of the clusters
pl = sns.countplot(x=fin_df["Cluster"])
pl.set_title("Distribution Of The Clusters")
plt.show()
```

 
<img>
 

```python
#check if there is a clear distinction between clusters with reference to income and total amount spent
pl = sns.scatterplot(data = fin_df,x=fin_df["TotalMnt"], y=fin_df["Income"],hue=fin_df["Cluster"])
pl.set_title("Cluster's Profile Based On Income And Spending")
plt.legend()
plt.show()
```

 
<img>
 

```python
#identify the personal attributes of the clusters
Personal = ["Age", "children", "famsize", "isParent", "withPartner"]

for i in Personal:
 plt.figure()
 sns.jointplot(x=fin_df[i], y=fin_df["TotalMnt"], hue =fin_df["Cluster"], kind="kde")
 plt.show()

#Used ratios to better compare the mode of purchase between the clusters
trend = ["web_to_total", "catalog_to_total", "Store_to_total", "NumDealsPurchases", "TotalNumPurchases"]

for i in trend:
 plt.figure()
 sns.boxenplot(y=fin_df[i], x =fin_df["Cluster"])
 plt.show()

#check whether the campaigns were effective
sns.countplot(x=fin_df["num_cmp"],hue=fin_df["Cluster"])
```

 
<img>
 

```python
#check the amount to total ratio to be able to better compare product purchases between clusters
product = ["MntWines", "MntFruits", "MntMeatProducts", "MntFishProducts","MntSweetProducts", "MntGoldProds"]

for i in product:
 plt.figure()
 sns.boxenplot(y=(fin_df[i]/fin_df['TotalMnt']), x=fin_df["Cluster"])
 plt.show()
```

 
<img>
 

```python
df.to_csv('df.csv')
```

It seems that most of the clusters spend most of their money of wine, the one exception to this is cluster 1 who spent the least amount on wine. The second product the customers spent the most on were meats and most of the clusters hover around a similar percent. Cluster 1 seems to have a wide range when purchasing fruits and fish. Cluster 1 along with cluster 3 are also the ones that spend the most on gold compared to the others.

**cluster 0 - high** income and medium expenses, varied ages, may or may not have kids, prefers stores, more liekly to take discounts

**cluster 1 - low** income and low expenses, varied ages, most likely has kids, mainly buys from stores even more than the other clusters, does not buy from catalogs much

**cluster 2 - high** income and high expenses, varied ages, most likely does not have kids, prefers stores, more likely to take part in campaigns

**cluster 3 - medium** income and low expenses, varied ages, most likely has 1-2 children, does not buy from catalogs much, prefers stores, more likely to buy discounted products.

## References

[1] Personality Prediction using Machine Learning [https://www.enjoyalgorithms.com/blog/personality-prediction-using-ml](https://www.enjoyalgorithms.com/blog/personality-prediction-using-ml)

[2] Improve Forecast Quality and Reliability wirh Value-add Forescasting [https://www.logility.com/es/blog/improve-forecast-quality-and-reliability-with-value-add-forecasting-part-1/](https://www.logility.com/es/blog/improve-forecast-quality-and-reliability-with-value-add-forecasting-part-1/)

[3] Customer analysis [https://www.kaggle.com/piercedee/customer-analysis/](https://www.kaggle.com/piercedee/customer-analysis/)

[4] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Forecasting Demand in the Supply Chain on Time Series with Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[7] Deep Learning with TensorFlow and Neural Networks [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Customer Personality Analysis is a detailed analysis of a company’s ideal customers. It helps a business to better understand its customers and makes it easier for them to modify products according to the specific needs, behaviours and concerns of different types of customers. In this article, I’m going to introduce you to a data science project on customer personality analysis with Python.

Customer Personality Analysis and Machine Learning Introduction 

# Introduction

It is often necessary in quantitative analyses to assess bivariate relationships between two or more variables. The most common way of measuring these relationships is with the Pearson product moment correlation coefficient. Correlation matrices of quantitative variables CORR (1990). These matrices are a good way to descriptively assess such relationships, but graphical approaches are often needed to find hidden associations or problems. Correlation matrices are a good way to descriptively assess such relationships, but graphical approaches are often needed to tell the entire story. The Pearson product-moment correlation coefficient r is a standardized measure of the linear relationship between two quantitative variables. Values of the Pearson coefficient range from -1 to 1. A value of r near or equal to zero implies little or no linear relationship between variables x and y. The closer r is to -1 or 1, the stronger the linear relationship. Positive values of r imply that y increases as x increases; negative values imply that y decreases as x increases. A common practice in the initial stages of a project is the generation of a correlation matrix of all quantitative variables as an exploratory way of examining the data. In such a matrix, a correlation coefficient is calculated between each pair of k variables, and these correlations are collectively represented in a square k x k matrix. A correlation matrix is often a good way to descriptively assess the relationship between two variables, but sometimes this information is not enough.

Exploratory Data Analysis or (EDA) is understanding the data sets by summarizing their main characteristics often plotting them visually. This step is very important especially when we arrive at modeling the data in order to apply Machine learning. Plotting in EDA consists of Histograms, Box plot, Scatter plot and many more. It often takes much time to explore the data. Through the process of EDA, we can ask to define the problem statement or definition on our data set which is very important.

This is one such question that everyone is keen on knowing the answer. Well, the answer is it depends on the data set that you are working. There is no one method or common methods in order to perform EDA, whereas in this tutorial you can understand some common methods and plots that would be used in the EDA process.

## Correlation Matrix plots

Correlation is an indication about the changes between two variables. In our previous chapters, we have discussed Pearson’s Correlation coefficients and the importance of Correlation too. We can plot correlation matrix to show which variable is having a high or low correlation in respect to another variable. Correlogram is a graph of correlation matrix. It is very useful to highlight the most correlated variables in a data table. In this plot, correlation coefficients is colored according to the value. Correlation matrix can be also reordered according to the degree of association between variables.

You already know that if you have a data set with many columns, a good way to quickly check correlations among columns is by visualizing the correlation matrix as a heatmap. But is a simple heatmap the best way to do it?For illustration, I’ll use the [Automobile Data Set](http://archive.ics.uci.edu/ml/datasets/automobile), containing various characteristics of a number of cars. You can also find a clean version of the data with header columns [here](https://github.com/drazenz/heatmap/blob/master/autos.clean.csv). Let’s start by making a correlation matrix heatmap for the data set.

To make a regular heatmap, we simply used the Seaborn heatmap function, with a bit of additional styling. For the second kind, there’s no trivial way to make it using Matplotlib or Seaborn. We could use [corrplot from biokit](https://nbviewer.jupyter.org/github/biokit/biokit/blob/master/notebooks/viz/corrplot.ipynb), but it helps with correlations only and isn’t very useful for two-dimensional distributions. Building a robust parametrized function that enables us to make heatmaps with sized markers is a nice exercise in Matplotlib, so I’ll show you how to do it step by step.

We’ll start by using a simple scatter plot with squares as markers. Then we’ll fix some issues with it, add color and size as parameters, make it more general and robust to various types of input, and finally make a wrapper function corrplot that takes a result of DataFrame.corr method and plots a correlation matrix, supplying all the necessary parameters to the more general heatmap function.

If we want to plot elements on a grid made by two categorical axes, we can use a scatter plot. Looks like we’re onto something. But I said it’s just a scatterplot, and there’s quite a lot happening in the previous code snippet. Since the scatterplot requires x and y to be numeric arrays, we need to map our column names to numbers. And since we want our axis ticks to show column names instead of those numbers, we need to set custom ticks and ticklabels. Finally there’s code that loads the dataset, selects a subset of columns, calculates all the correlations, melts the data frame (the inverse of creating a pivot table) and feeds its columns to our heatmap function. You noticed that our squares are placed where our gridlines intersect, instead of being centered in their cells. In order to move the squares to cell centers, we’ll actually move the grid. And to move the grid, we’ll actually turn off major gridlines, and set minor gridlines to go right in between our axis ticks.

[Seaborn](https://python-graph-gallery.com/seaborn/) allows you to make a [correlogram](https://python-graph-gallery.com/correlogram/) or correlation matrix really easily. Correlogram is awesome for exploratory analysis: it makes you quickly observe the relationship between every variable of your matrix. It is easy to do it with seaborn: just call the pairplot() function!

```python
from mpl_toolkits.mplot3d import Axes3D
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt # plotting
import numpy as np # linear algebra
import os # accessing directory structure
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import seaborn as sns
from pandas.plotting import scatter_matrix
```

```python
nRowsRead = 1000 # specify 'None' if want to read whole file
# amazon_trends.csv may have more rows in reality, but we are only loading/previewing the first 1000 rows
df1 = pd.read_csv('amazon_trends.csv', delimiter=',', nrows = nRowsRead)
df1.dataframeName = 'amazon_trends.csv'
nRow, nCol = df1.shape
print(f'There are {nRow} rows and {nCol} columns')
df1.head(5)
There are 1000 rows and 2 columns
nRowsRead = 1000 # specify 'None' if want to read whole file
# mydata.csv may have more rows in reality, but we are only loading/previewing the first 1000 rows
df2 = pd.read_csv('mydata.csv', delimiter=',', nrows = nRowsRead)
df2.dataframeName = 'mydata.csv'
nRow, nCol = df2.shape
print(f'There are {nRow} rows and {nCol} columns')
df2.head(5)
```

```bash
There are 1000 rows and 36 columns
```

```python
corr = df2.corr()
ax = sns.heatmap(
 corr, 
 vmin=-1, vmax=1, center=0,
 cmap=sns.diverging_palette(20, 220, n=200),
 square=True
)
ax.set_xticklabels(
 ax.get_xticklabels(),
 rotation=45,
 horizontalalignment='right'
);
```

Let’s start by making a correlation matrix heatmap for the data set.

 

<img>

 

Great! Green means positive, red means negative. The stronger the color, the larger the correlation magnitude. Now looking at the chart above, think about the following questions:

· Where do your eyes jump first when you look at the chart?

· What’s the strongest and what’s the weakest correlated pair (except the main diagonal)?

· What are the three variables most correlated with price?

If you’re like most people, you’ll find it hard to map the color scale to numbers and vice versa. Distinguishing positive from negative is easy, as well as 0 from 1. But what about the second question? Finding the highest negative and positive correlations mean finding the strongest red and green. To do that I need to carefully scan the entire grid. Try to answer it again and notice how your eyes are jumping around the plot, and sometimes going to the legend.

Now consider the following plot:

```python
def heatmap(x, y, size):
 fig, ax = plt.subplots()
 
 # Mapping from column names to integer coordinates
 x_labels = [v for v in sorted(x.unique())]
 y_labels = [v for v in sorted(y.unique())]
 x_to_num = {p[1]:p[0] for p in enumerate(x_labels)} 
 y_to_num = {p[1]:p[0] for p in enumerate(y_labels)} 
 
 size_scale = 500
 ax.scatter(
 x=x.map(x_to_num), # Use mapping for x
 y=y.map(y_to_num), # Use mapping for y
 s=size * size_scale, # Vector of square sizes, proportional to size parameter
 marker='s' # Use square as scatterplot marker
 )
```

```python
 # Show column labels on the axes
 ax.set_xticks([x_to_num[v] for v in x_labels])
 ax.set_xticklabels(x_labels, rotation=45, horizontalalignment='right')
 ax.set_yticks([y_to_num[v] for v in y_labels])
 ax.set_yticklabels(y_labels)
```

```python
df2 = pd.read_csv('mydata.csv', delimiter=',', nrows = nRowsRead)
df2.dataframeName = 'mydata.csv'
columns = ['Open', 'High', 'Low', 'Close', 'Volume', 'Gain'] 
corr = df2[columns].corr()
corr = pd.melt(corr.reset_index(), id_vars='index') # Unpivot the dataframe, so we can get pair of arrays for x and y
corr.columns = ['x', 'y', 'value']
heatmap(
 x=corr['x'],
 y=corr['y'],
 size=corr['value'].abs()
)
```

If we want to plot elements on a grid made by two categorical axes, we can use a scatter plot. Looks like we’re onto something. But I said it’s just a scatterplot, and there’s quite a lot happening in the previous code snippet. Since the scatterplot requires _x_ and _y_ to be numeric arrays, we need to map our column names to numbers. And since we want our axis ticks to show column names instead of those numbers, we need to set custom _ticks_ and _ticklabels_. Finally there’s code that loads the dataset, selects a subset of columns, calculates all the correlations, _melts_ the data frame (the inverse of creating a pivot table) and feeds its columns to our _heatmap_ function.

 

<img>

 

In the following example, Python script will generate and plot correlation matrix for the Pima Indian Diabetes dataset. It can be generated with the help of corr() function on Pandas DataFrame and plotted with the help of _pyplot_. From the above output of correlation matrix, we can see that it is symmetrical i.e. the bottom left is same as the top right. It is also observed that each variable is positively correlated with each other.

```python
df2 = pd.read_csv('mydata.csv', delimiter=',', nrows = nRowsRead)
df2.dataframeName = 'mydata.csv'
f = plt.figure(figsize=(19, 15))
plt.matshow(df2.corr(), fignum=f.number)
plt.xticks(range(df2.select_dtypes(['number']).shape[1]), df2.select_dtypes(['number']).columns, fontsize=14, rotation=45)
plt.yticks(range(df2.select_dtypes(['number']).shape[1]), df2.select_dtypes(['number']).columns, fontsize=14)
cb = plt.colorbar()
cb.ax.tick_params(labelsize=14)
plt.title('Correlation Matrix', fontsize=16);
```

 

<img>

 

```python
df1 = pd.read_csv('mydata.csv', delimiter=',', nrows = nRowsRead)
df1.dataframeName = 'mydata.csv'
f = plt.figure(figsize=(19, 15))
plt.matshow(df2.corr(), fignum=f.number)
plt.xticks(range(df2.select_dtypes(['number']).shape[1]), df2.select_dtypes(['number']).columns, fontsize=14, rotation=45)
plt.yticks(range(df2.select_dtypes(['number']).shape[1]), df2.select_dtypes(['number']).columns, fontsize=14)
cb = plt.colorbar()
cb.ax.tick_params(labelsize=14)
plt.title('Correlation Matrix', fontsize=16);
```

 

<img>

 

```python
scatter_matrix(df2)
plt.show()
```

This is an example of a scatterplot matrix. The variables are written in a diagonal line from top left to bottom right. Then each variable is plotted against each other. For example, the middle square in the first column is an individual scatterplot of Girth and Height, with Girth as the X-axis and Height as the Y-axis. This same plot is replicated in the middle of the top row. In essence, the boxes on the upper right hand side of the whole scatterplot are mirror images of the plots on the lower left hand. In this scatterplot, it is probably safe to say that there is a correlation between Girth and Volume (Go data! Confirming the obvious) because the plot looks like a line. There is probably less of a correlation between Height and Girth in addition to Height and Volume. More statistical analyses would be needed to confirm or deny this.

 

<img>

 

## References

[1] Amazon Stock Price and Movement Prediction https://www.kaggle.com/aenes95/amazon-stock-price-prediction/

[2] Amazon Stock Price Movement Prediction Using Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[3] Exploratory Data Analysis in Python [https://towardsdatascience.com/exploratory-data-analysis-in-python-c9a77dfa39ce](https://towardsdatascience.com/exploratory-data-analysis-in-python-c9a77dfa39ce)

[4] Exploratory Data Analysis in Python Better Heatmaps and Correlation Matrix Plots in Python [https://towardsdatascience.com/better-heatmaps-and-correlation-matrix-plots-in-python-41445d0f2bec](https://towardsdatascience.com/better-heatmaps-and-correlation-matrix-plots-in-python-41445d0f2bec)

[5] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[7] Forecasting Demand in the Supply Chain on Time Series with Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

It is often necessary in quantitative analyses to assess bivariate relationships between two or more variables. The most common way of measuring these relationships is with the Pearson product moment correlation coefficient. Correlation matrices of quantitative variables CORR (1990). These matrices are a good way to descriptively assess such relationships, but graphical approaches are often needed to find hidden associations or problems. Correlation matrices are a good way to descriptively assess such relationships, but graphical approaches are often needed to tell the entire story. The Pearson product-moment correlation coefficient r is a standardized measure of the linear relationship between two quantitative variables. Values of the Pearson coefficient range from -1 to 1. A value of r near or equal to zero implies little or no linear relationship between variables x and y. The closer r is to -1 or 1, the stronger the linear relationship. Positive values of r imply that y increases as x increases; negative values imply that y decreases as x increases. A common practice in the initial stages of a project is the generation of a correlation matrix of all quantitative variables as an exploratory way of examining the data. In such a matrix, a correlation coefficient is calculated between each pair of k variables, and these correlations are collectively represented in a square k x k matrix. A correlation matrix is often a good way to descriptively assess the relationship between two variables, but sometimes this information is not enough.
Exploratory Data Analysis or (EDA) is understanding the data sets by summarizing their main characteristics often plotting them visually. This step is very important especially when we arrive at modeling the data in order to apply Machine learning. Plotting in EDA consists of Histograms, Box plot, Scatter plot and many more. It often takes much time to explore the data. Through the process of EDA, we can ask to define the problem statement or definition on our data set which is very important.
This is one such question that everyone is keen on knowing the answer. Well, the answer is it depends on the data set that you are working. There is no one method or common methods in order to perform EDA, whereas in this tutorial you can understand some common methods and plots that would be used in the EDA process.

Amazon Stock Price Movement Prediction Using Machine Learning – Exploratory Analysis

# Introduction

Determining the price movement of stocks is a challenging problem to solve because of factors such as industry performance, economic variables, investor sentiment, company news, company performance, and social media sentiment. People can predict the price movement of stocks by applying machine learning algorithms on information contained in historical data, stock candlestick-chart data, and social-media data. However, it is hard to predict stock movement based on a single classifier. In this study, we proposed a multichannel collaborative network by incorporating candlestick-chart and social-media data for stock trend predictions.

The stock market is a major topic in modern life. Investors can easily purchase new stocks and gain significant profits from dividends provided in the company’s bonus program for shareholders. Investors can also trade their own stocks with the other traders in the stock market via stock brokerages and electronic trading platforms. Stock traders wish to buy stocks with values that are expected to increase and sell stocks with values that are expected to decrease. Therefore, stock traders must be able to predict the trends of a stock’s behavior before making a trading decision to buy or sell a stock. The more accurate their prediction about the behavior of a stock is, the more profit they earn. Therefore, developing an automatic algorithm that can accurately predict market trends to help traders maximize profit is essential. However, determining stock market trends is a challenging problem because of factors such as industry performance, company news, company performance, investor sentiment, economic variables, and social media sentiment.

Many researchers have developed various prediction models for stock market prediction to help traders make the correct decision on their stocks. In 1990, Schneburg [38] conducted a study on the German stock market. A machine learning architecture with back-propagation was built based on historical time-series data. In 2007, Khoa [39] proposed a simply recurrent neural network for stock prediction and concluded that this simply recurrent neural network achieves better results compared to back-propagation neural networks due to its “capturing capabilities.” After that, Ou and Wang [20] utilized the Hong Kong stock market to build a stock market prediction model with ten machine learning algorithms. They demonstrated that SVM and LS-SVM achieve better performance than the other methods. In 2016, Chen proposed planar feature representation methods and deep convolutional neural networks to improve the algorithmic trading framework [40]. In the last decade, sentiment analysis has carried great importance because of the huge amount of textual data on news and social media platforms. There is a significant amount of research on mining opinions of users for different application areas [41–45]. Ali performed sentiment analysis to detect transportation entities in the large corpus [42] and to detect traffic accidents to reduce serious injuries [43]. Basiri proposed a sentiment analysis method on the Twitter dataset using the attention-based bidirectional CNN-RNN deep model [44]. Li [45] proposed a bidirectional emotional recurrent unit for exploring the emotion in conversation. Using the same trend for stock prediction, some researchers have started to utilize sentiment analysis to analyze the stock market’s movements. In 2011, J. Bollen proposed a sentiment analysis method to predict the movement of the Dow Jones Industrial Average (DJIA) stock market based on the famous microblogging site Twitter. Khatri and Srivastava [30] explored the relationship between text sentiment from tweets, comments on the Stock Twist website, and the stock price of Facebook, Apple, Google, Oracle, and Microsoft stocks from Yahoo Finance. In 2017, Urolagin used an SVM classifier to perform sentiment classification and predict stock market status based on stock price data from Yahoo Finance and social-media data from Twitter [27]. He explored an association between tweet features and the stock prices of a company. Not long after that, Chakraborty proposed a stock-market-movement prediction model based on tweets’ sentiment of Appple Inc stock [28]. The SVM classifier was used for sentiment classification, while the stock movement prediction model was trained by applying the boosted regression tree. In 2020, Khan [46] utilized algorithms on social media and financial news data to discover the impact of both data on stock movement. Many researchers have found different methods to approach stock data; they transformed the historical time series data into other forms to learn more patterns inside the stock movement. Candlestick charts have been used to visualize the daily price and stock market movement. In 2013, Prado utilized sixteen candlestick patterns to forecast stock movement for Brazilian stocks [26]. Tsai combined candlestick charts with seven different wavelet-based textures to predict stock movement [22]. In 2017, Hu employed a convolutional encoder to learn the candlestick chart patterns to build a decision-making system for the stock market. Our previous work [21] proposed a deep convolutional neural network for stock market prediction based on candlestick charts for two different stock markets (Taiwan50 and Indo10). In 2020, Birogul [25] employed a real-time object detection system (YOLO) to recognize buy–sell objects inside 2D candlestick charts; from these buy–sell objects, the trader can make their decision on the stock. Not long after that, Hung [24] proposed a deep predictor framework for price movement based on candlestick charts. He explored a CNN-autoencoder to acquire the best sub-chart representation and applied recurrent neural networks to predict the stock price movement. Unlike existing methods, we did not solely rely on sentiment analysis of social media posts or explore patterns in the candlestick charts. In this study, we proposed a collaborative network incorporating candlestick charts and social-media data for stock movement prediction. To our knowledge, this is the first study to use both candlestick charts and social sentiment data to predict stock prices. Using both types of data is more effective in predicting stock trends because both types of data can change and affect movements in a stock’s price and in traders’ decisions.

## Amazon Price Prediction

Amazon Inc stock price prediction is an act of determining the future value of Amazon shares using few different conventional methods such as EPS estimation, analyst consensus, or fundamental intrinsic valuation. The successful prediction of Amazon's future price could yield a significant profit. Please, note that this module is not intended to be used solely to calculate an intrinsic value of Amazon and does not consider all of the tangible or intangible factors available from Amazon's fundamental data. We analyze noise-free headlines and recent hype associated with Amazon Inc, which may create opportunities for some arbitrage if properly timed.

It is a matter of debate whether stock price prediction based on information in financial news can generate a strong buy or sell signal. We use our internally-built news screening methodology to estimate the value of Amazon based on different types of headlines from major news networks to social media. The Amazon stock price prediction module provides an analysis of price elasticity to changes in media outlook on Amazon over a specific investment horizon. Using Amazon hype-based prediction, you can estimate the value of Amazon Inc from the perspective of Amazon response to recently generated media hype and the effects of current headlines on its competitors. We also analyze overall investor sentiment towards Amazon using Amazon's options and short interest. It helps to benchmark the overall future attitude of investors towards Amazon using crowd psychology based on the activity and movement of Amazon's stock price. Amazon Operating Margin is quite stable at the moment as compared to the past year. The company's current value of Operating Margin is estimated at 5.76. Cash and Equivalents Turnover is expected to rise to 5.05 this year, although the value of Inventory Turnover will most likely fall to 8.63.

As far as predicting the price of Amazon at your current risk attitude, this probability distribution graph shows the chance that the prediction will fall between or within a specific range. We use this chart to confirm that your returns on investing in Amazon or, for that matter, your successful expectations of its future price, cannot be replicated consistently. Please note, a large amount of money has been lost over the years by many smart people who confused the symmetrical distributions of stock prices, such as prices of Amazon, with the unreliable approximations that try to describe financial returns.

In the context of predicting Amazon's stock value on the day after the next significant headline, we show statistically significant boundaries of downside and upside scenarios based on Amazon's historical news coverage. Amazon's after-hype downside and upside margins for the prediction period are 2,592 and 2,883, respectively. We have considered Amazon's daily market price in relation to the headlines to evaluate this method's predictive performance. Remember, however, there is no scientific proof or empirical evidence that news-based prediction models outperform traditional linear, nonlinear models or artificial intelligence models to provide accurate predictions consistently.

## About this file – Code in Python

Everyone is looking for the best stocks for 2021. Selecting individual stocks can, therefore, be quite tricky. Articles like these are meant to inspire you. The year 2020 has once again shown that things can change very quickly. Investors who do not respond to changes in the world can suddenly be presented with a bad investment result. Today we are going to talk about the Amazon share price forecast; the e-commerce giant. So, the models used in this project are as follows. I'll try to keep it simple this notebook.

> Linear Regression (LR)

> Support Vector Machines (SVR, SVC)

> Long Short Term Memory (LSTM)

```python
import pandas as pd
import math
import numpy as np
import datetime
import matplotlib.pyplot as plt
import time
from sklearn import linear_model
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn import svm
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, LSTM
from sklearn.metrics import r2_score

def readDataFromCSV(file_path):
 data = pd.read_csv(file_path)
 data.reset_index()
 data = data.set_index("Date")
 data.index = pd.to_datetime(data.index)
 return data

def readDataAndWrite(stock_code, file_path):
 ticker = yf.Ticker(stock_code)
 data= ticker.history(period= "max")
 data.to_csv(r""+ file_path)

mydata = readDataFromCSV("mydata.csv")
mydata.columns
mydata

amazon_trends = readDataFromCSV("amazon_trends.csv")
mydata.loc[mydata.index & amazon_trends.index, "Google_Trends"] = amazon_trends["Google_Trends"]
print("Including Google Trends data to my dataset.")
mydata.columns

Index(['Open', 'High', 'Low', 'Close', 'Volume', 'Dividends', 'Stock Splits',
 'Gain', 'Loss', 'Avg_Gain', 'Avg_Loss', 'RSI', 'SMA', 'EMA', 'MACD',
 'Bollinger_Upper', 'Bollinger_Lower', 'ROC', 'PVT', 'Usd_Eur_Close',
 'Usd_Chf_Close', 'Usd_Gbp_Close', 'Usd_Jpy_Close', 'Usd_Cad_Close',
 'Usd_Inf_Close', 'Usd_Rub_Close', 'Usd_Try_Close', 'IRX_Close',
 'JPM_Close', 'BAC_Close', 'Citigroup_Close', 'WFC_Close',
 'NASDAQ_Close', 'SP_500_Close', 'Dow_Jones_Close', 'Google_Trends'],
 dtype='object')
tomorrows_price = mydata["Close"].shift(-1)
tomorrows_price = tomorrows_price[0: -1]
mydata = mydata[0:-1]
print("tomorrows_price is our target value of regression process. We use today's features to predict tomorrow's close price.")

price_movement = mydata["Close"] < tomorrows_price
print("price_movement is our classification target. If it is true close price's of company will increase tomorrow, if it is false it will decrease.")

min_max_scaler = preprocessing.MinMaxScaler()
print(min_max_scaler)

norm_mydata = min_max_scaler.fit_transform(mydata)
norm_tomorrows_price = min_max_scaler.fit_transform(tomorrows_price[:, None])

norm_train_data, norm_test_data, norm_train_target, norm_test_target = train_test_split(norm_mydata, norm_tomorrows_price, test_size = 0.2, random_state = 0, shuffle = False)
norm_train_target_movement = norm_train_target < pd.DataFrame(norm_train_target).shift(-1)
norm_test_target_movement = norm_test_target < pd.DataFrame(norm_test_target).shift(-1)

print("We are splitting our data. 0.8 percent is train data and 0.2 percent is the test data.")

k= int(len(price_movement)* 0.8)
train_price_movement = price_movement[0:k]
test_price_movement = price_movement[k:]
print("Train Price Movement",train_price_movement)
print("Test Price Movement",test_price_movement)



linear_reg = linear_model.LinearRegression()
linear_reg.fit(norm_train_data, norm_train_target)
close_column_index = 3

linear_reg_results = pd.DataFrame()
tmp = linear_reg.predict(norm_test_data)
linear_reg_results["Price_Predictions"] = tmp.reshape(len(tmp))
linear_reg_results["Price_Movement_Predictions"] = linear_reg_results["Price_Predictions"] < linear_reg_results["Price_Predictions"].shift(-1)

plt.figure(figsize = (7, 4))
plt.plot(norm_test_target[0:100], "red", label= 'Real') ## Test Target
plt.plot(linear_reg_results["Price_Predictions"][0:100], "blue", label= 'Predicted') ## Predicted
plt.title('Stock')
plt.xlabel('Time [days]')
plt.ylabel('Normalized Price')
plt.legend(loc= 'best')
plt.show()
```

<img>

```python
print("R2 Metric: ", r2_score(norm_test_target, linear_reg_results["Price_Predictions"]))
print("Price Movement Accuracy: ", accuracy_score(linear_reg_results["Price_Movement_Predictions"], norm_test_target_movement))
```

```bash
R2 Metric: 0.9956152295327649
Price Movement Accuracy: 0.5295774647887324
```

```python
plt.figure(figsize = (7, 4))
plt.title('Stock Movement Prediction')
plt.plot(linear_reg_results["Price_Movement_Predictions"])
plt.xlabel("Days")
plt.show()
```

<img>

```python
linear_reg = linear_model.LinearRegression()
linear_reg.fit(norm_train_data, train_price_movement)
linear_cls_results = pd.DataFrame()
tmp = linear_reg.predict(norm_test_data)
linear_cls_results["Price_Movement_Predictions"] = tmp > 0.5

print("Price Movement Accuracy: ", accuracy_score(linear_cls_results["Price_Movement_Predictions"], norm_test_target_movement))
Price Movement Accuracy: 0.47183098591549294

plt.figure(figsize = (7, 4))
plt.title('Stock Movement Prediction')
plt.plot(linear_cls_results)
plt.xlabel("Days")
plt.show()
```

<img>

```python
#Support Vector Machines
sv_regression = svm.SVR(C=10, degree = 2, kernel= 'sigmoid', tol= 0.1)
sv_regression.fit(norm_train_data, norm_train_target)
svr_results = pd.DataFrame()
tmp = sv_regression.predict(norm_test_data)
svr_results["Price_Predictions"] = tmp.reshape(len(tmp))
svr_results["Price_Movement_Predictions"] = svr_results["Price_Predictions"] < svr_results["Price_Predictions"].shift(-1)

plt.figure(figsize= (7,4))
plt.plot(pd.DataFrame(norm_test_data)[close_column_index][0:10], "purple", label = "Data" )
plt.plot(svr_results["Price_Predictions"][0:10], "blue", label = "Predicted")
plt.plot(norm_test_target[0:10], "red", label = "Real")
plt.title('Stock (10 Days)')
plt.xlabel('Time [days]')
plt.ylabel('Normalized Price')
plt.legend(loc= 'best')
plt.show()

plt.figure(figsize= (7, 4))
plt.plot(norm_test_target, "red", label = "Real")
plt.plot(svr_results["Price_Predictions"], "blue", label= "Predicted")
plt.title('Stock')
plt.xlabel('Time [days]')
plt.ylabel('Normalized Price')
plt.legend(loc= 'best')
plt.show()
```

<img>

```python
plt.figure(figsize = (7, 4))
plt.plot(svr_results["Price_Movement_Predictions"])
plt.title("Stock Movement Prediction")
plt.xlabel("Days")
plt.show()
```

<img>

```python
print("R2 Metric: ", r2_score(norm_test_target, svr_results["Price_Predictions"]))
print("Price Movement Accuracy: ", accuracy_score(svr_results["Price_Movement_Predictions"], norm_test_target_movement))
```

## References

[1] Stock Price Movement Prediction Using Sentiment Analysis and CandleStick Chart. Research Center for Information Technology Innovation, Academia Sinica, Taipei. [https://www.mdpi.com/journal/sensors](https://www.mdpi.com/journal/sensors)

[2] Improve Forecast Quality and Reliability wirh Value-add Forescasting [https://www.logility.com/es/blog/improve-forecast-quality-and-reliability-with-value-add-forecasting-part-1/](https://www.logility.com/es/blog/improve-forecast-quality-and-reliability-with-value-add-forecasting-part-1/)

[3] Amazon Stock Price and Movement Prediction [https://www.kaggle.com/aenes95/amazon-stock-price-prediction/](https://www.kaggle.com/aenes95/amazon-stock-price-prediction/)

[4] Optimization Business Profitability CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Forecasting Demand in the Supply Chain on Time Series with Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Determining the price movement of stocks is a challenging problem to solve because of factors such as industry performance, economic variables, investor sentiment, company news, company performance, and social media sentiment. People can predict the price movement of stocks by applying machine learning algorithms on information contained in historical data, stock candlestick-chart data, and social-media data. However, it is hard to predict stock movement based on a single classifier. In this study, we proposed a multichannel collaborative network by incorporating candlestick-chart and social-media data for stock trend predictions.

Amazon Stock Price Movement Prediction Using Machine Learning

The amount of data is growing exponentially and it is necessary to analyze this huge amount of data to extract useful information from it. This led to emergence the field of data mining. Data mining refers to extracting knowledge from such large amount of databases. Data mining is core of KDD process. KDD is organized process of identified, valid, novel, useful and understandable pattern from large and complex dataset. Data mining tasks can be divided into two categories: descriptive and predictive. Predictive task include classification, regression, time series analysis and descriptive task include clustering, association rule. These techniques can be used in specific areas. Discuss about these technique and their application in various field. Data mining applications include various field such as sales, medicine, finance, marketing, healthcare, banking and insurance. 

Ecommerce Shipping: How to Make Logistics a Competitive  Advantage with Machine Learning – Part II

Many online small businesses might aim to start out with a simplistic approach to shipping. They might even believe that their ecommerce platform has everything you need to manage the whole logistics process. There is no all-in-one platform for ecommerce that will manage logistics, ERP, CRM, and so many other things. The good news is that starting small is fine — you have to walk before you can run. But when you get to that point, take some time to learn some of the ins and outs of logistics. The most successful business owners are using strategic shipping options to differentiate themselves from the competition and increase their profit margins. After the increase in digital shopping due to the coronavirus outbreak, 51% of retail leaders said they’d be increasing investments in logistics and supply chain, per a report from BigCommerce and Retail Dive from late 2020. Of course, while shipping can be a powerful point of differentiation for your brand, you can’t just create a strategy and call it a day. You have to be prepared to execute — both operationally and in terms of pricing strategy. That requires the coordination of multiple teams, from marketing to fulfillment and many others in between. Establishing your shipping policy isn’t just about what options you’ll offer. You have to also ensure that everyone involved in the pipeline understands their part in the process. So let’s look at the absolute must-haves in your shipping strategy, and what you need to know to make it happen.

Ecommerce Shipping: How to Make Logistics a Competitive  Advantage with Machine Learning

# Introduction

The promise of machine learning has shown many stunning results in a wide variety of fields. Natural language processing is no exception of it, and it is one of those fields where machine learning has been able to show general artificial intelligence (not entirely but at least partially) achieving some brilliant results for genuinely complicated tasks.

Now, NLP (natural language processing) is not a new field and neither is machine learning. But the fusion of both the fields is quite contemporary and only vows to make progress. This is one of those hybrid applications which everyone (with a budget smart-phone) comes across daily. For example, take "keyboard word suggestion" into the account, or intelligent auto-completions; these all are the byproducts of the amalgamation of NLP and Machine Learning, and quite naturally these have become the inseparable parts of our lives.

Sentiment analysis is a vital topic in the field of NLP. It has easily become one of the hottest topics in the field because of its relevance and the number of business problems it is solving and has been able to answer. In this tutorial, you will cover this not-so-simple topic in a simple way. You will break down all the little mathematics behind it, and you will study it. You will also build a simple sentiment classifier at the end of this tutorial. Specifically, you will cover:

> Understanding sentiment analysis from a practitioner's perspective
> Formulating the problem statement of sentiment analysis
> Naive Bayes classification for sentiment analysis
> A case study in Python
> How sentiment analysis is affecting several business grounds
> Further reading on the topic

Essentially, sentiment analysis or sentiment classification fall into the broad category of text classification tasks where you are supplied with a phrase, or a list of phrases and your classifier is supposed to tell if the sentiment behind that is positive, negative or neutral. Sometimes, the third attribute is not taken to keep it a binary classification problem. In recent tasks, sentiments like "somewhat positive" and "somewhat negative" are also being considered. Let's understand with an example now.

Consider the following phrases:

> _Titanic is a great movie._ 
> _Titanic is not a great movie._ 
> _Titanic is a movie._

The phrases correspond to short movie reviews, and each one of them conveys different sentiments. For example, the first phrase denotes positive sentiment about the film Titanic while the second one treats the movie as not so great (negative sentiment). Take a look at the third one more closely. There is no such word in that phrase which can tell you about anything regarding the sentiment conveyed by it. Hence, that is an example of neutral sentiment.

Now, from a strict machine learning point of view, this task is nothing but a supervised learning task. You will supply a bunch of phrases (with the labels of their respective sentiments) to the machine learning model, and you will test the model on unlabeled phrases.

In the era of big data, huge number of product reviews has been posted to online social media. Accordingly, mining consumers’ sentiments about products can generate valuable business intelligence for enhancing management’s decision-making. So, With the rapid development of the Internet and data-processing technologies, Internet sentiment analysis can be used to explore many possibilities, from Internet news about products or the influence of product price to the influence of sale behaviour and important brand strategies.

In Electronic commerce, customer reviews play a significant role in purchase making decision. Most of the existing recommendation systems consider the customer reviews, user purchase history and product rating for predicting the recommended product. Since the users interest are varying over time, the existing recommendation systems lack in finding the current relevant items to the customers.

The popularity of the Internet has brought profound influence to electronic commerce. A kind of review-oriented consumption mode is gradually expanding in the market and consumers will refer to the reviews provided by consumers who bought the product in the past. How to accurately analyze users' sentiments from massive data of e-commerce reviews has become one of the key issues for e-commerce platforms. Current standard sentiment analysis classifies overall sentiment of e-commerce reviews without an extended description of the entity. So, Online shopping is becoming one of the most de-manding everyday needs, nowadays. These days people are feeling comfortable shopping online. The number of its customers is increasing day by day as well as raising some problems. The major problem is that the customers can not choose the quality-full product by reading every review of an online product.

This study uses machine learning techniques aimed at comparing the classification in sentiment analysis from the views of E-Commerce customers.. The highest result of this study Random Forest approach with an accuracy of 96%, precision.

## Code E-Commerce in Python

```python
import pandas as pd
import nltk
import unicodedata
import sys
import string
nltk.download('vader_lexicon')
import time
import csv
import matplotlib.pyplot as plt
import seaborn as sns
from nltk.sentiment.vader import SentimentIntensityAnalyzer
import re
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize

df_clothes = pd.read_csv("Womens Clothing E-Commerce Reviews.csv")
df_clothes.head()

#CLeaning and Graph Analysis
df_clothes.dropna(inplace=True)

ax = df_clothes['Division Name'].value_counts().sort_index().plot(kind='bar', fontsize=14, figsize=(12,10))
ax.set_title('NUmber of Reviews by DIvision Name\n', fontsize=20)
ax.set_xlabel('Division Name', fontsize=18)
ax.set_ylabel('Count', fontsize=18);
plt.show()
```

<img>

```python
ax = df_clothes['Department Name'].value_counts().sort_index().plot(kind='bar', fontsize=14, figsize=(12,10))
ax.set_title('NUmber of Reviews by Department Name', fontsize=20)
ax.set_xlabel('Department Name', fontsize=18)
ax.set_ylabel('Count', fontsize=18);
plt.show()
```

<img>

```python
ax = sns.catplot(x='Department Name', y='Rating', kind = 'box', data = df_clothes, height=12, aspect=1)
ax.fig.suptitle('Class Name and Ratings')
```

<img>

```python

df_clothes['Review Text'] = df_clothes['Review Text'].apply(lambda x: (re.sub('@[^\s]+-,','', x)))
df_clothes['Review Text'] = df_clothes['Review Text'].apply(lambda x: re.sub(r'\B#\w*[a-zA-Z]+\w*', '', x))
df_clothes['Review Text'] = df_clothes['Review Text'].str.lower()


stop = stopwords.words('english')
df_clothes["rev_no_stopw"] = df_clothes['Review Text'].apply(lambda x:
 ' '.join([word for word in x.split() if word not in (stop)]))

analyzer= SentimentIntensityAnalyzer()
df_clothes['Compound'] = [analyzer.polarity_scores(a)['compound'] for a in df_clothes["rev_no_stopw"]]
df_clothes['Negative'] = [analyzer.polarity_scores(a)['neg'] for a in df_clothes["rev_no_stopw"]]
df_clothes['Positive'] = [analyzer.polarity_scores(a)['pos'] for a in df_clothes["rev_no_stopw"]]
df_clothes['Neutral'] = [analyzer.polarity_scores(a)['neu'] for a in df_clothes["rev_no_stopw"]]

for a in df_clothes.Compound:
 df_clothes['Analysis'] = df_clothes.Compound.apply(lambda a: 'Positive' if a > 0 else ('Neutral' if a == 0 else 'Negative'))

df_clothes.head()

plt.rcParams['figure.figsize'] = (15, 10)
fig, axes = plt.subplots()
plt.hist(df_clothes['Positive'], bins=30, color="green", alpha=0.6)
plt.xlabel("Positive", fontsize=20)
plt.ylabel("count", fontsize=20)
plt.suptitle("Positive Histogram", fontsize=30)
plt.show()
```

<img>

```python
from PIL import Image
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator

text = " ".join(review for review in df_clothes.rev_no_stopw.astype(str))
print ("There are {} words in the combination of all cells in column BLOOM.".format(len(text)))
```

## There are 4054866 words in the combination of all cells in column BLOOM.

```python
text = text
# word cloud image:
wordcloud = WordCloud().generate(text)
# Display image:
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis("off")
plt.show()
```

<img>

```python
from sklearn.feature_extraction.text import TfidfVectorizer
import sklearn
from sklearn.feature_selection import SelectFromModel
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split

vectorizer = TfidfVectorizer (max_features=1000, min_df=7, max_df=0.7, stop_words=stopwords.words('english'))
processed_features = vectorizer.fit_transform(df_clothes["Review Text"]).toarray()

labels = df_clothes["Analysis"].values
X_train, X_test, y_train, y_test = train_test_split(processed_features, labels, test_size=0.5, random_state=0)

from sklearn.ensemble import RandomForestClassifier
text_classifier = RandomForestClassifier(n_estimators=100, random_state=0)
text_classifier.fit(X_train, y_train)

y_pred = text_classifier.predict(X_test)

from sklearn.metrics import classification_report, confusion_matrix, accuracy_score

print(confusion_matrix(y_test,y_pred))
print(classification_report(y_test,y_pred))
print(accuracy_score(y_test, y_pred)
```

```bash
 precision recall f1-score support

 Negative 0.36 0.01 0.03 281
 Neutral 0.75 0.05 0.09 63
 Positive 0.97 1.00 0.98 9487

 accuracy 0.97 9831
 macro avg 0.69 0.35 0.37 9831
weighted avg 0.95 0.97 0.95 9831

0.9650086461194182
```

## References

[1] Real-time Sentiment Analysis On E-Commerce Application https://ieeexplore.ieee.org/abstract/document/8743331

[2] Amazon Stock Price Movement Prediction Using Machine Learning https://www.cobuildlab.com/blog/ai

[3] Sentiment Analysis of Chinese E-commerce Reviews Based on BERT https://ieeexplore.ieee.org/abstract/document/9442190

[4] A computational intelligence approach to efficiently predicting review ratings in e-commerce https://www.sciencedirect.com/science/article/abs/pii/S1568494616300758

[5] Optimization Business Profitability CobuildLab. 2021. https://www.cobuildlab.com/blog/ai

[6] Predictions Dynamic Pricing uses Machine Learning CobuildLab. 2021. https://www.cobuildlab.com/blog/ai

The promise of machine learning has shown many stunning results in a wide variety of fields. Natural language processing is no exception of it, and it is one of those fields where machine learning has been able to show general artificial intelligence (not entirely but at least partially) achieving some brilliant results for genuinely complicated tasks.
Now, NLP (natural language processing) is not a new field and neither is machine learning. But the fusion of both the fields is quite contemporary and only vows to make progress. This is one of those hybrid applications which everyone (with a budget smart-phone) comes across daily. For example, take "keyboard word suggestion" into the account, or intelligent auto-completions; these all are the byproducts of the amalgamation of NLP and Machine Learning, and quite naturally these have become the inseparable parts of our lives.

Sentiment Analysis in E-commerce Using Machine Learning

The discipline of Machine Learning, residing at the edge and up to the cloud, provides a key enabler to Predictive Maintenance through identifying, monitoring and analyzing critical system variables during operation. In the application of Predictive Maintenance using Machine Learning techniques (including supervised and unsupervised) operators can be alerted before system failure, and in some cases without operator interaction addressed, and avoid costly unplanned downtime.

Machine Learning in Predictive Maintenance towards Smart Manufacturing

# Introduction

There's no area of the enterprise seeing greater positive value from AI investment than business process operations. A recent survey by BI Intelligence revealed that supply chain and operations together are the third most active area implementing AI technology, with over 42% of enterprises that responded already seeing revenue gains from AI investments. In another McKinsey study, enterprises that have invested in AI technology for transportation and logistics increased profit margins by more than 5%.How to ensure efficient supply chain management? This is an open question for many suppliers, distributors, manufacturers, and retailers. Today, amid shifting supply chain market dynamics, changing ways of working, increasingly volatile demand, businesses are wondering how to make their supply chain less vulnerable to disruption. Supply Chain Analytics transforms supply chain activities from guessing, to ones that makes decision using data. An essential tool in Supply Chain Analytics is using optimization analysis to assist in decision making. According to Deloitte, 79% of organizations with high performing supply chains achieve revenue growth that is significantly above average.

## Supply Chain Analytics

Supply chain is one of the trickiest parts of a business and as the chances of error are quite high, the need for proper synchronization and analytics is very important. Something simple getting out of hand can make the business lose a lot of money as it can create a break or kink in the supply chain. When it comes to strategic and operational parts of the supply chain management, the need for resolving issues on-the-go to keep the chain continuous is very important or every step in the continuation can become a painful point which will block the resources in the wrong places.

This is the reason why data analytics is now taken very seriously in the management of the supply chain. The data analytics help in making data-driven and analyzed steps and decisions, thus adding data intelligence to the business. With the use of machine learning and visualization tools and software in the supply chain, not only is the operation becoming easier but the services also get improved. Overall, the use of data science in the supply chain is helping the management and professionals get a clear picture of the operational efficiency and make timely decisions right away.

## Understanding the Supply Chain Management

The supply chain is a well-understood part of a business and is also a part that cannot be ignored regardless of whatever the business is about. The main target of every business when it comes to managing the supply chain is to reduce the overall costs and make the process more reliable and accurate.

## Some of the main steps that are covered in the supply chain of the business are

- Procurement of raw materials
- Inbound logistics
- Inventory of parts
- Manufacturing of goods
- Inventory of finished goods
- Fulfillment order of the goods (customer goods)
- Outbound logistics

All these processes are very different on their own and require separate attention. However, there is no denying that they are entangled with each other and share dependability. For each and every step to be completed successfully there is a need for the preceding one to be completed on time. Failing to complete any one step means the entire supply chain will collapse. To handle every process of the supply chain, there is the need for separate operational channels, management, visibility, and data collection. In all this, visibility plays a very crucial role.

Visibility is a feature that demonstrates the ability of the management to take actions based on the historical data. Therefore, business intelligence and decision making rely on the historical data that is accumulated. Historical data can help in both supply and demand forecasting. It can also help in deciding more efficient operating channels that will save time and in forecasting market demand. Demand forecasting can help to get a clearer picture of how the production line should operate and what the logistics requirements of the future are.

Now that the supply chain is also a data-oriented department, it would not be foolish to say that it is also a department that can rely on data science techniques like predictive analysis.

## Supply Chain and Predictive Analysis

As mentioned above, supply chain management is all about accurate forecasting where further decisions regarding logistics and production can be taken. This is the reason why predictive analytics can be very useful for the business to make their supply chain more productive. Supply chain forecasting is a tricky situation as there are many variables to take into consideration. Long-term forecasting is still something that can be managed, but short-term forecasting can become quite difficult. This is because the number of variables that come to the scene in short-term forecasting are numerous and hard to track.

Predictive analytics is about predicting future phenomena using historical data sets. Therefore, for accurate forecasting, predictive analytics is highly helpful as it shows various mathematical models of predictive forecasting. Also, the predictive analytics tools help compare two roadmaps and decide which is more profitable and efficient. It takes into consideration various factors, both internal and external. This process is known as the optimization of supply chain processes. Therefore, it can be said that the predictive analytics of data science are highly helpful in forecasting, optimization, and visualization of various processes in a supply chain of businesses.

There are many ways in which supply chain management is becoming more and more competitive and efficient with the help of predictive analysis:

## Demand Analytics

Predictive analytics help forecasting future demands on various levels with the help of current sales. It can help with forecasting in a detailed manner at various sale points like retailers, stores, distributors, etc. it can also help in taking into consideration holidays, weather forecasts, and then integrate with promotional events

## Finished Inventory Optimization

With this, predictive analytics helps in giving a clear forecast about how much inventory there must be and how it should be positioned. This makes inventory budgeting easier and more optimized. Also, analytics helps in getting recommendations for safety stock along with customized holding of stock for various customer demands.

## Network Planning

To have a good supply chain and a profitable business, it is important to make sure that the inventory facilities and manufacturing amenities are all properly networked. Analytics takes into consideration the manufacturing units and warehouses available and how it can affect the supply chain with a changing demand. Also, it helps in creating flow paths that can help in fulfilling customer demands of various segments at the lowest cost.

## Replenishment Planning Analytics

Analytics helps in creating a clear plan regarding when and where to ship the products. This allows proper planning which can be easily integrated at various levels like channel, retailer and distributor. Also, the various constraints that are levied on various points of the supply chain can be taken into consideration. This increases the in-store availability of the products and also enhances customer satisfaction with better services.

## Procurement Analytics

One of the most important parts of the supply chain is the very first step and that is to find and procure the best suppliers. Based on the most data, predictive analytics can help in finding the low cost and high-quality supply partners. It takes into consideration the scoring models of cost of suppliers, vendor quality and overall stability of the long- term relationship with the suppliers.

## Transportation Analytics

Lastly, predictive analytics help in predicting and visualizing the best routes of transportation. It takes into consideration various predictive models to find the best shipping routes, backhaul routes, shipment scheduling techniques, and various constraints and compliances of transportation that are to be followed.

## Code Supply Chain Price with Data Analysis

Example of business case for studying Supply Chain Price with Data Analysis with new contributions at the code in python uses the following structure:Supply Chain Shipment Pricing Data - Data Analysis and Modeling,Import Packages, Import CSV file, Check Total Records in CSV file, Check DataType of CSV file, Print first 10 and last 10 recods from DataSet, Total 10 Country wise count with graph, Total Pack Price for Top 15 Countries with graph, First Line Designation Wise Count, Shipment Mode percentage wise Pie Chart, Unquie Manufacturing Site Names, Shipment Mode, Min and Mean value for Air, Top 10 Manufacturing Site for all Shipment Mode with Graph, Top 10 Manufacturing Site for Air Shipment Mode with Graph and Shipment Mode and Pack Price in Bar Plot Graph [2].

```python
import pandas as pd
import numpy as np
from decimal import Decimal
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sn
import os
import plotly.graph_objs as go
import plotly.offline as py
py.init_notebook_mode(connected=True)
pd.options.mode.chained_assignment = None
pd.options.display.max_columns = 9999
pd.options.display.float_format = '{:20,.2f}'.format

DataSet = pd.read_csv('SCMS_Delivery_History_Dataset.csv').fillna(0)

TotalRowCount = len(DataSet)
print("Total Number of Data Count :", TotalRowCount)
Total Number of Data Count : 10324

DataSet.dtypes
```

```bash
ID int64
Project Code object
PQ # object
PO / SO # object
ASN/DN # object
Country object
Managed By object
Fulfill Via object
Vendor INCO Term object
Shipment Mode object
PQ First Sent to Client Date object
PO Sent to Vendor Date object
Scheduled Delivery Date object
Delivered to Client Date object
Delivery Recorded Date object
Product Group object
Sub Classification object
Vendor object
Item Description object
Molecule/Test Type object
Brand object
Dosage object
Dosage Form object
Unit of Measure (Per Pack) int64
Line Item Quantity int64
Line Item Value float64
Pack Price float64
Unit Price float64
Manufacturing Site object
First Line Designation object
Weight (Kilograms) object
Freight Cost (USD) object
Line Item Insurance (USD) float64
dtype: object
```

```python
DataSet.head(5)
DataSet.tail(5)

DataSet = DataSet.dropna()
ItemCount = DataSet["Country"].value_counts().nlargest(10)
print("Top 10 Countries Wise Count \n")
print(ItemCount)
sn.set_context("talk",font_scale=1)
plt.figure(figsize=(22,6))
sn.countplot(DataSet['Country'],order = DataSet['Country'].value_counts().nlargest(10).index)
plt.title('Top 10 Countries Wise Count \n')
plt.ylabel('Total Count')
plt.xlabel('Country Name')
```

```bash
Top 10 Countries Wise Count 

South Africa 1406
Nigeria 1194
Côte d'Ivoire 1083
Uganda 779
Vietnam 688
Zambia 683
Haiti 655
Mozambique 631
Zimbabwe 538
Tanzania 519
Name: Country, dtype: int64
```

<img>

```bash
TotalPrice = DataSet.groupby(['Country'])['Pack Price'].sum().nlargest(15)
print("Total Pack Price for Top 15 Countries\n")
print(TotalPrice)
plt.figure(figsize=(22,6))
GraphData=DataSet.groupby(['Country'])['Pack Price'].sum().nlargest(15)
GraphData.plot(kind='bar')
plt.ylabel('Total Pack Price')
plt.xlabel('Country Name')

Total Pack Price for Top 15 Countries

Country
Nigeria 25,620.72
South Africa 24,318.90
Côte d'Ivoire 22,882.35
Haiti 19,521.30
Uganda 15,664.08
Mozambique 14,092.40
Vietnam 11,963.78
Zambia 11,889.67
Tanzania 11,047.72
Rwanda 7,724.29
Botswana 7,052.54
Zimbabwe 6,563.56
Kenya 6,349.14
Congo, DRC 6,032.41
Guyana 5,659.35
Name: Pack Price, dtype: float64
```

<img>

```python
sn.set_context("talk",font_scale=1)
plt.figure(figsize=(5,6))
sn.countplot(DataSet['First Line Designation'],order = DataSet['First Line Designation'].value_counts().nlargest(10).index)
plt.title('First Line Designation Wise Count \n')
plt.ylabel('Total Count')
plt.xlabel('First Line Designation')
```

<img>

```python
sn.set_context("talk",font_scale=1)
plt.figure(figsize=(5,6))
sn.countplot(DataSet['First Line Designation'],order = DataSet['First Line Designation'].value_counts().nlargest(10).index)
plt.title('First Line Designation Wise Count \n')
plt.ylabel('Total Count')
plt.xlabel('First Line Designation')
```

<img>

```python
UniqueItem = DataSet['Manufacturing Site'].unique()
print("All Unique Manufacturing Site \n")
print(UniqueItem)
```

```bash
All Unique Manufacturing Site 

['Ranbaxy Fine Chemicals LTD' 'Aurobindo Unit III, India'
 'ABBVIE GmbH & Co.KG Wiesbaden' 'Ranbaxy, Paonta Shahib, India'
 'MSD South Granville Australia' "ABBVIE (Abbott) St. P'burg USA"
 'ABBVIE Ludwigshafen Germany' 'Trinity Biotech, Plc'
 'EY Laboratories, USA' 'Cipla, Goa, India' 'BMS Meymac, France'
 'Premier Med. Corp Ltd. India' 'Chembio Diagnostics Sys. Inc.'
 'Inverness Japan' 'Pacific Biotech, Thailand'
 'Standard Diagnostics, Korea' 'GSK Mississauga (Canada)'
 'Gilead(Nycomed) Oranienburg DE' 'Bio-Rad Laboratories'
 'Mylan (formerly Matrix) Nashik' 'Roche Basel' 'GSK Ware (UK)'
 'Novartis Pharma AG, Switzerland' 'BI, Ingelheim, Germany'
 'Not Applicable' 'Ipca Dadra/Nagar Haveli IN' 'MSD, Haarlem, NL'
 'Aspen-OSD, Port Elizabeth, SA' 'ABBVIE (Abbott) Logis. UK'
 'Gland Pharma Ltd Pally Factory' 'GSK Aranda'
 'Hetero Unit III Hyderabad IN' 'ABBVIE (Abbott) France'
 'Strides, Bangalore, India.' 'ABBSP' 'Cipla, Patalganga, India'
 'GSK Cape Town Factory (South Africa)' "MSD Midrand, J'burg, SA"
 'KHB Test Kit Facility, Shanghai China' 'bioLytical Laboratories'
 'Inverness USA' 'Boehringer Ing., Koropi, GR' 'GSK Crawley'
 'OMEGA Diagnostics, UK' 'Roche Madrid'
 'BUNDI INTERNATIONAL DIAGNOSTICS LTD' 'INVERNESS ORGENICS LINE'
 'Novartis Pharma Suffern, USA' 'Micro Labs Ltd. (Brown & Burk), India'
 'Meditab (for Cipla) Daman IN' 'Medopharm Malur Factory, INDIA'
 'ABBVIE (Abbott) Japan Co. Ltd.' 'MSD Elkton USA'
 'Orasure Technologies, Inc USA'
 'Weifa A.S., Hausmanngt. 6, P.O. Box 9113 GrÃ¸nland, 0133, Oslo, Norway'
 'MSD Manati, Puerto Rico, (USA)' 'MSD Patheon, Canada'
 'Emcure Plot No.P-2, I.T-B.T. Park, Phase II, MIDC, Hinjwadi, Pune, India'
 'Alere Medical Co., Ltd.' 'Premier Medical Corporation'
 'ABBVIE Labs North Chicago US' 'Janssen-Cilag, Latina, IT'
 'Aurobindo Unit VII, IN' 'Micro labs, Verna, Goa, India'
 'Orasure Technologies, Inc' 'Mylan, H-12 & H-13, India'
 'Hetero, Jadcherla, unit 5, IN' 'Bristol-Myers Squibb Anagni IT'
 'Cipla Ltd A-42 MIDC Mahar. IN' 'Medochemie Factory A, CY'
 'Boehringer Ingelheim Roxane US' 'BMS Evansville, US'
 'GSK, U1, Poznan, Poland' 'Janssen Ortho LLC, Puerto Rico'
 'Micro Labs, Hosur, India' 'Remedica, Limassol, Cyprus' 'INVERNESS ANY'
 'Guilin OSD site, No 17, China' 'Cipla, Kurkumbh, India'
 'GSK Barnard Castle UK' 'Gland Pharma, Hyderabad, IN' 'Access BIO, L.C.'
 'Human Diagnostic' 'Mepro Pharm Wadhwan Unit II'
 'Ranbaxy per Shasun Pharma Ltd' 'MedMira Inc.'
 'Ranbaxy per Shasun Pharma' 'Macleods Daman Plant INDIA']
```

```python
ItemData=DataSet[DataSet['Shipment Mode']=='Air']
print ("The Max Air Shipment Mode is :",ItemData['Unit of Measure (Per Pack)'].max())
print ("The Min Air Shipment is :",ItemData['Unit of Measure (Per Pack)'].min())
ItemTypeMean = ItemData['Unit of Measure (Per Pack)'].mean()
print ("The Mean Air Shipment is :", round(ItemTypeMean,2))
```

```bash
The Max Air Shipment Mode is : 1000
The Min Air Shipment is : 1
The Mean Air Shipment is : 82.34
```

```python
plt.figure(figsize=(22,6))
TopFiveManufacturingSite=DataSet.groupby('Manufacturing Site').size().nlargest(10)
print(TopFiveManufacturingSite)
TopFiveManufacturingSite.plot(kind='bar')
plt.title('Top 10 Manufacturing Site \n')
plt.ylabel('Total Count')
plt.xlabel('Manufacturing Site Name')
```

```bash
Manufacturing Site
Aurobindo Unit III, India 3172
Mylan (formerly Matrix) Nashik 1415
Hetero Unit III Hyderabad IN 869
Cipla, Goa, India 665
Strides, Bangalore, India. 540
Alere Medical Co., Ltd. 481
Trinity Biotech, Plc 405
ABBVIE Ludwigshafen Germany 366
Inverness Japan 345
ABBVIE (Abbott) Logis. UK 219
dtype: int64
```

<img>

```python
# Top 10 Air Shipment Mode in Bar Chart
ItemData=DataSet[DataSet['Shipment Mode']=='Air']
DataSet[DataSet["Shipment Mode"]=='Air']['Manufacturing Site'].value_counts()[0:10].to_frame().plot.bar(figsize=(22,6))
ItemSupplier = DataSet[DataSet["Shipment Mode"]=='Air']['Manufacturing Site'].value_counts()[0:10]
print("Top 10 Air Manufacturing Site \n")
print(ItemSupplier)
plt.title('Top 10 Air Manufacturing Site\n')
plt.ylabel('Air Count')
plt.xlabel('Manufacturing Site')
```

```bash
Top 10 Air Manufacturing Site 

Aurobindo Unit III, India 1694
Mylan (formerly Matrix) Nashik 695
Alere Medical Co., Ltd. 473
Hetero Unit III Hyderabad IN 417
Trinity Biotech, Plc 383
Inverness Japan 320
ABBVIE Ludwigshafen Germany 285
Cipla, Goa, India 274
Strides, Bangalore, India. 256
ABBVIE (Abbott) Logis. UK 150
Name: Manufacturing Site, dtype: int64
```

<img>

```python
plt.subplots(figsize = (18,6))
plt.xticks(rotation = 90)
sn.barplot('Shipment Mode','Pack Price', data = DataSet)
plt.show()
```

<img>

```bash
Top Country for Pack Price : Nigeria - 25,620.72
Top Shipping Mode : Air
The Max Air Shipment Mode is : 1000
The Min Air Shipment is : 1
The Mean Air Shipment is : 82.35
Top Manufacturing Site : Aurobindo Unit III, India - 3172
Top Air Manufacturing Site : Aurobindo Unit III, India - 1694
```

## References

[1] Leveraging Machine Learning for Efficiency in Supply Chain Management. 2020. https://www.analyticsvidhya.com/blog/2020/07/leveraging-machine-learning-for-efficiency-in-supply-chain-management/

[2] Supply Chain Data Analysis https://www.kaggle.com/divyeshardeshana/

[3] Supply Chain Analytics. CobuildLab 2021 https://www.cobuildlab.com/blog/ai

[4] Supply Chain Analytics with Machine Learning. CobuildLab 2021 https://www.cobuildlab.com/blog/ai

[5] Machine Learning in Predictive Maintenance towards Smart Manufacturing. CobuildLab 2022 https://www.cobuildlab.com/blog/ai

[6] Customer Personality Analysis and Machine Learning. CobuildLab 2022 https://www.cobuildlab.com/blog/ai

There's no area of the enterprise seeing greater positive value from AI investment than business process operations. A recent survey by BI Intelligence revealed that supply chain and operations together are the third most active area implementing AI technology, with over 42% of enterprises that responded already seeing revenue gains from AI investments. In another McKinsey study, enterprises that have invested in AI technology for transportation and logistics increased profit margins by more than 5%.How to ensure efficient supply chain management? This is an open question for many suppliers, distributors, manufacturers, and retailers. Today, amid shifting supply chain market dynamics, changing ways of working, increasingly volatile demand, businesses are wondering how to make their supply chain less vulnerable to disruption. Supply Chain Analytics transforms supply chain activities from guessing, to ones that makes decision using data. An essential tool in Supply Chain Analytics is using optimization analysis to assist in decision making. According to Deloitte, 79% of organizations with high performing supply chains achieve revenue growth that is significantly above average.

Supply Chain Price Data Analysis

# Introduction

The term &quot;Sentiment Analysis&quot; was first defined in 2003 by Nasukawa and Yi [1] as &quot;determining the subjectivity polarity (positive or negative) and polarity strength (strongly positive, mildly positive, weakly positive etc.) of a given review text; in other words - determining the opinion of the writer.&quot; Turney&#39;s pioneering work on Sentiment Analysis [2] applied an unsupervised approach to classify review data into positive class and negative class. This paper focuses on finding sentiments in another type of dataset: financial news, and demonstrating correlation between the sentiments of financial news and stock market variation.

Classical finance theory assumes investors are rational, non-emotional entities and prices represent the equilibrium of expected returns and possible risks. This leaves no scope for investor sentiment to play a role in price determinations. Modern behavioral finance, however, recognizes both sentimental investors and rational investors. [3] states that &quot;Now, the question is no longer, as it was a few decades ago, whether investor sentiment affects stock prices, but rather how to measure investor sentiment and quantify its effect.&quot; Every day, a lot of company news is published that directly affects the investors&#39; behavior. Manually reading this news and labeling it as positive or negative is a very difficult task due to the sheer volume of news generated which is increasing rapidly. Moreover manual evaluation of news may not be completely objective due to factors like reader bias, emotion and fatigue. Automatic sentiment analysis can avoid these pitfalls. The sum total of information received by the investors is reflected through the stock price of the firms. Through this process, information is transformed from a textual form to a numerical form. This process of conversion is very useful, because it allows information to be easily summarized and enables us to compare the sentiments of news with the market returns. There may be variations about the exact meaning of a piece of news, but there cannot be any variation about market returns. The financial news that makes a positive impact on the stock market returns is good and the one that makes a negative impact on stock market returns is bad [4]. In comparison to the work done in sentiment classification applied to the review domain or product reviews, very little work has been done in the field of application of these techniques in the financial domain using unsupervised approach. This paper tries to address this research gap.

## Sentiment Analysis

Over the last years the interest in sentiment analysis has grown rapidly. In the first approaches developed in the sentiment analysis research, terms with high occurrence numbers measured by term frequency or phrases for semantic orientation were tracked and affected the algorithms and models for document classification [5]. Later on, the level was reduced to overview segments of texts and even sentences due to the fact that documents contain multiple entities, which can express their opinions. In this case the identification of subjectivity and objectivity of sentences and their entities is an important task [6]. The finest grade for sentiment research is the entity- or aspect-level, also called feature-level [7]. At the sentence-level, part-of-speech (POS) information are used to extract sentiment polarity from nouns, verbs, adverbs, and adjectives, as well as word sense disambiguation with the help of natural language processing (NLP) techniques. Furthermore n-grams and co-occurrences of words are typical pattern-based methods for sentiment analysis. N-gram, bi-grams, tri-grams or uni-grams have been researched by [8] and [9] with contrary results on sentiment classification of movie and product reviews. The most common methods in sentiment analysis are dictionary-based approaches based on lexical resources [9]. Sets of known positive and negative categorized words are collected to a dictionary, which is extended (automatically or manually) with synonyms and antonyms in further steps [9]. Another approach is to assign terms to emotions, such as &quot;afraid&quot;, &quot;bored&quot;, &quot;happy&quot;, etc. to get a sentiment value based on different categories of emotions [9]. Other lexical resources provide predefined scores for words, word families or n-grams. In addition to lexical affinity and keyword spotting, statistical methods and concept-based techniques are used in the majority of cases [12]. Problems in the sentiment analysis are still existent. Negation handling and word sense disambiguation are difficult and not solved NLP tasks [9]. Moreover opinion spam is a problem which started especially with the rise in popularity of social media [9]. It is vital to detect spam or ironical comments in order to extract true sentiment evaluation and uncover fake reviews, wrong reputation or promotions. A. Financial market information overload Market participants have recently raised growing interest in the field of sentiment analysis. With the development of algorithmic trading systems human-decision making has reached its limit and cannot keep up with processing and execution of orders and decisions. Additionally, the information overload is not manageable without support. To manage all information, analysis methods are applied on different category groups which are expected to hold information about sentiment polarity [10]. Forums, blogs and wikis are seen to be the most representative types of textualdata where users can express their opinion. Also news, research reports and finance-related content generated by firms are analyzed [10]. B. Applying the associaton concept CIMAWA for Sentiment Analysis An innovative method to create a sentiment evaluation is to combine two concepts, namely keyword spotting and word association. The Concept for the Imitation of the Mental Ability of Word Association (CIMAWA) is developed at the Institute of Knowledge Based Systems to measure the association between words [10]. It was designed to simulate the Human Word Association (HWA), based on large collections of text documents.

<img>

The equation (1) describes the strength of association between two words &#39;x &#39; and &#39;y &#39; by taking into account the cooccurrences ((Coocws(x,yi)) in a predefined text window (ws). The damping factor is used to balance the first asymmetric and second symmetric part of the hybrid association formula [15]. By solving the equation for different word combinations one can create all associations (represented, scaled and ordered by the numeric CIMAWA values) from a selected document corpus. As a measure for sentiment analysis in a predefined time period the concept and equation of CIMAWA was adapted to be more flexible to changes of word associations over the time:

<img>

All co-occurrences (Coocws(x,yi)) have a subindex i=1,….,n at keyword yi for an increment operation. Word &#39;x&#39; is a fixed term in the calculation process and will not be changed. The increment operation only counts if &#39;yi &#39; (inside a text window size (ws)) is present in a resource (Ɵ) (e.g. a wordlist), with evaluated entries (Ɵ) and finishes at . For the damping factor values in the interval [0.4 – 0.6] achieved best results in previous studies [11]. For this application damping factor is used.

The formula is applied in temporary changing content. With timestamp information (t), all sub periods (starting with), combined in a maximum time window (t=1) are observed. All calculated CIMAWA associations are multiplied by the valuated wordlist scale, with value entry if keyword is included. Finally, the sum of all multiplication results is divided by the last increment value.

To create a dynamic sentiment value - similar to a moving average - this calculation is done for each document of a document corpus with timestamp information. The time window (T) is moving forward and reviews positive and negative associations in small parts of the whole corpus. The advantage of this extension is the in-depth historical look-back over the time and determination of the changing strength of word associations in combination with the fast processing concept of spotting relevant keywords to filter results. Fast or slow changing sentiment values can be easily created by adjustment of the time window.

## Code Study – Sentiment Analysis Stocks FinViz

Extract stock sentiments from financial news headlines in FinViz website using Python. So, NLTK [12] is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to [over 50 corpora and lexical resources](https://www.nltk.org/nltk_data/) such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active [discussion forum](https://groups.google.com/group/nltk-users).

NLTK has been called &quot;a wonderful tool for teaching, and working in, computational linguistics using Python,&quot; and &quot;an amazing library to play with natural language.&quot;

[Natural Language Processing with Python](https://www.nltk.org/book/) provides a practical introduction to programming for language processing. Written by the creators of NLTK, it guides the reader through the fundamentals of writing Python programs, working with corpora, categorizing text, analyzing linguistic structure, and more.

## Code in Python

```python
# Import libraries
from urllib.request import urlopen, Request
from bs4 import BeautifulSoup
import os
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
# NLTK VADER for sentiment analysis
import nltk
nltk.downloader.download('vader_lexicon')
from nltk.sentiment.vader import SentimentIntensityAnalyzer

news_tables = {}
tickers = ['AMZN', 'TSLA', 'GOOG']

for ticker in tickers:
 url = finwiz_url + ticker
 req = Request(url=url,headers={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:20.0) Gecko/20100101 Firefox/20.0'})
 response = urlopen(req)
 # Read the contents of the file into 'html'
 html = BeautifulSoup(response)
 # Find 'news-table' in the Soup and load it into 'news_table'
 news_table = html.find(id='news-table')
 # Add the table to our dictionary
 news_tables[ticker] = news_table

# Read one single day of headlines for 'AMZN'
amzn = news_tables['AMZN']
# Get all the table rows tagged in HTML with <tr> into 'amzn_tr'
amzn_tr = amzn.findAll('tr')

for i, table_row in enumerate(amzn_tr):
 # Read the text of the element 'a' into 'link_text'
 a_text = table_row.a.text
 # Read the text of the element 'td' into 'data_text'
 td_text = table_row.td.text
 # Print the contents of 'link_text' and 'data_text'
 print(a_text)
 print(td_text)
 # Exit after printing 4 rows of data
 if i == 3:
 break

parsed_news = []
# Iterate through the news
for file_name, news_table in news_tables.items():
 # Iterate through all tr tags in 'news_table'
 for x in news_table.findAll('tr'):
 # read the text from each tr tag into text
 # get text from a only
 text = x.a.get_text()
 # splite text in the td tag into a list
 date_scrape = x.td.text.split()
 # if the length of 'date_scrape' is 1, load 'time' as the only element

 if len(date_scrape) == 1:
 time = date_scrape[0]

 # else load 'date' as the 1st element and 'time' as the second
 else:
 date = date_scrape[0]
 time = date_scrape[1]
 # Extract the ticker from the file name, get the string up to the 1st '_'
 ticker = file_name.split('_')[0]

 # Append ticker, date, time and headline as a list to the 'parsed_news' list
 parsed_news.append([ticker, date, time, text])

parsed_news

# Instantiate the sentiment intensity analyzer
vader = SentimentIntensityAnalyzer()

# Set column names
columns = ['ticker', 'date', 'time', 'headline']

# Convert the parsed_news list into a DataFrame called 'parsed_and_scored_news'
parsed_and_scored_news = pd.DataFrame(parsed_news, columns=columns)

# Iterate through the headlines and get the polarity scores using vader
scores = parsed_and_scored_news['headline'].apply(vader.polarity_scores).tolist()

# Convert the 'scores' list of dicts into a DataFrame
scores_df = pd.DataFrame(scores)

# Join the DataFrames of the news and the list of dicts
parsed_and_scored_news = parsed_and_scored_news.join(scores_df, rsuffix='_right')

# Convert the date column from string to datetime
parsed_and_scored_news['date'] = pd.to_datetime(parsed_and_scored_news.date).dt.date

parsed_and_scored_news.head()
```

| ticker | date | time | headline | neg | neu | pos | compound |
| ------ | --------------- | ------- | ------------------------------------------------- | --- | ----- | ----- | -------- |
| 0 | AMZN 2022-04-23 | 11:33AM | Activision Union Bid Will Go to a Vote, Labor ... | 0.0 | 1.000 | 0.000 | 0.0000 |
| 1 | AMZN 2022-04-23 | 08:00AM | Tech Giants Face Landmark Fines Under EUs New ... | 0.0 | 0.874 | 0.126 | 0.0772 |
| 2 | AMZN 2022-04-23 | 07:53AM | Unions are on the rise. Guess why. | 0.0 | 1.000 | 0.000 | 0.0000 |
| 3 | AMZN 2022-04-22 | 04:47PM | Tech companies gear up to report earnings next... | 0.0 | 1.000 | 0.000 | 0.0000 |
| 4 | AMZN 2022-04-22 | 04:43PM | Whats driving the nationwide push for labor un... | 0.0 | 1.000 | 0.000 | 0.0000 |

```python
plt.rcParams['figure.figsize'] = [10, 6]

# Group by date and ticker columns from scored_news and calculate the mean
mean_scores = parsed_and_scored_news.groupby(['ticker','date']).mean()

# Unstack the column ticker
mean_scores = mean_scores.unstack()

# Get the cross-section of compound in the 'columns' axis
mean_scores = mean_scores.xs('compound', axis="columns").transpose()

# Plot a bar chart with pandas
mean_scores.plot(kind = 'bar')
plt.grid()
```

<img>

## Reference

[1] Sentiment analysis of financial news using unsupervised approach. International Conference on Computational Intelligence and Data Science (ICCIDS 2019). ScienceDirect. Procedia Computer Science 167 (2020) 589–598.

[2] Turney, Peter D. (2002) &quot;Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews.&quot; Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, Pennsylvania: 417-424.

[3] Baker, Malcolm, and Jeffrey Wurgler. (2007) &quot;Investor Sentiment in the Stock Market.&quot; The Journal of Economic Perspectives 21 (2): 129- 15

[4] A. Kirilenko, A. S. Kyle, M. Samadi and T. Tuzun, &quot;The Flash Crash: The Impact of High Frequency Trading on an Electronic Market&quot;, 2011.

[5] P. D. Turney, &quot;Thumbs Up or Thumbs Down?: Semantic Orientation Applied to Unsupervised Classification of Reviews,&quot; Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL), pp.417-424, Philadelphia, July 2002.

[6] J. Wiebe, T. Wilson, R. Bruce, M. Bell, and M. Martin, &quot;Learning Subjective Language,&quot; Computational Linguistics, vol. 30, pp.277–308, September 2004.

[7] M. Hu and B. Liu, &quot;Mining and Summarizing Customer Reviews,&quot; Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD &#39;04, pp.168-177, Seattle, WA, USA, ACM, 2004.

[8] K. Dave, S. Lawrence and D. M. Pennock, &quot;Mining the peanut gallery: Opinion extraction and semantic classification of product reviews,&quot; WWW&#39;03: Proceedings of the 12th international conference on World Wide Web, pp.519-528, New York, NY, USA, ACM, 2003.

[9] B. Pang, L. Lee and S. Vaithyanathan, &quot;Thumbs up? Sentiment classification using machine learning techniques,&quot; Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.79-86, 2002.

[10] B. Lui, &quot;Sentiment Analysis and Opinion Mining,&quot; Morgan &amp; Claypool Publishers, May 2012.

[11] Sentiment Analysis in Financial Markets A Framework to utilize the Human Ability of Word Association for analyzing Stock Market News Reports.2014. IEEE.

[12] Natural Language Toolkit. NLTK. [https://www.nltk.org/](https://www.nltk.org/). 2022.

[13] Optimization Techniques for Data Scientist and Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[14] Optimization Vehicle Routing Problem [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[15] Optimization and Its Importance in the Supply Chain [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

The term "Sentiment Analysis" was first defined in 2003 by Nasukawa and Yi [1] as “determining the subjectivity polarity (positive or negative) and polarity strength (strongly positive, mildly positive, weakly positive etc.) of a given review text; in other words - determining the opinion of the writer.” Turney’s pioneering work on Sentiment Analysis [2] applied an unsupervised approach to classify review data into positive class and negative class. This paper focuses on finding sentiments in another type of dataset: financial news, and demonstrating correlation between the sentiments of financial news and stock market variation.
Classical finance theory assumes investors are rational, non-emotional entities and prices represent the equilibrium of expected returns and possible risks. This leaves no scope for investor sentiment to play a role in price determinations. Modern behavioral finance, however, recognizes both sentimental investors and rational investors. [3] states that “Now, the question is no longer, as it was a few decades ago, whether investor sentiment affects stock prices, but rather how to measure investor sentiment and quantify its effect.” Every day, a lot of company news is published that directly affects the investors’ behavior. Manually reading this news and labeling it as positive or negative is a very difficult task due to the sheer volume of news generated which is increasing rapidly. Moreover manual evaluation of news may not be completely objective due to factors like reader bias, emotion and fatigue. Automatic sentiment analysis can avoid these pitfalls. The sum total of information received by the investors is reflected through the stock price of the firms. Through this process, information is transformed from a textual form to a numerical form. This process of conversion is very useful, because it allows information to be easily summarized and enables us to compare the sentiments of news with the market returns. There may be variations about the exact meaning of a piece of news, but there cannot be any variation about market returns. The financial news that makes a positive impact on the stock market returns is good and the one that makes a negative impact on stock market returns is bad [4]. In comparison to the work done in sentiment classification applied to the review domain or product reviews, very little work has been done in the field of application of these techniques in the financial domain using unsupervised approach. This paper tries to address this research gap.

Sentiment Analysis of Financial using Machine Learning

## Introduction

A supply chain is a network of organizations, individuals, activities, information, and resources involved in the production and sale of a service or product to consumers. The chain begins with the transformation of natural, raw resources and components into a finished product or service, and concludes with the delivery of that product or service to the end customer.

Supply chain optimization refers to the tools and processes by which manufacturing and distribution supply chain performance and efficiency are improved, taking into account all constraints. Supply chain network optimization technologies use sophisticated algorithms and analytics to balance supply and demand in such a manner that sufficient raw materials are procured for manufacturing and distribution to meet customer expectations at the highest cost efficiency. Supply chains are complex. To best serve you, we go beyond just managing supply chains to offer complete optimization. It&#39;s imperative you don&#39;t waste money or time on valuable shipments.

To evaluate a supply chain optimization process, most supply chain managers will look at the bottom line: will it reduce supply chain costs? Will it lower operational expenses? This, as well as increasing revenue, will always be the primary company goals, and a good way to achieve them is to look at inventory holding costs. The cost of storing inventory makes up a major percentage of a product&#39;s total cost. That&#39;s why it&#39;s crucial to properly forecast demand and plan accordingly in order to optimize inventory quantities.

Machine learning would also be very beneficial for proper forecasting. Machine learning automates data analysis as well as analytical model building. In other words, the ability to learn from data and identifying patterns can essentially help improve decision making and performance.

A good example would be eliminating time buffers. If machine learning could predict that the time required for drivers to reach a certain warehouse is one and a half hours rather than two, for instance, then that estimation could be incorporated into [dispatching](https://www.bringg.com/blog/dispatching/dispatch-software/) and route planning, reducing lead times across the last mile of the supply chain. That, of course, means the same resources to perform more deliveries, reducing the related transportation cost. More business, less waste.

So, Optimization is fundamental to any problem related to decision making, whether in engineering or economics. The task of making decisions involves choosing between several alternatives. This choice is going to be governed by our desire to make the &quot;best&quot; decision possible. How good each of the alternatives will be will be described by an objective function or performance index. The theory and optimization methods will help us to select the best alternative according to this objective function. The area of optimization has received great attention in recent years, mainly due to the rapid development of computer science, including the development and availability of extremely user-friendly software tools, high-speed parallel processors, and Artificial Intelligence.

## What is an Optimization Problem?

An optimization problem begins with a set of independent variables, or parameters, and often includes conditions or constraints that define the acceptable values ​​of these variables. Such constraints are called the constraints of the problem. The other essential component of an optimization problem is a single measure of &quot;goodness&quot;, called the objective function, which will also depend on the independent variables. The solution to the optimization problem is going to be given by a set of allowed values ​​for the independent variables, according to the constraints of the problem, in which the objective function assumes an optimal value. In mathematical terms, optimization usually involves maximizing or minimizing the objective function.

In order to apply the mathematical results and numerical techniques of optimization theory to concrete problems, it is necessary to clearly delineate the limits of the system to be optimized, define the quantitative parameters that will be used as criteria based on which the alternatives will be classified to determine the &quot;best &quot; option, select the variables that will be used to characterize or identify alternatives; and define the model that expresses how the variables will be related. A good formulation of the problem is the key to the success of an optimization problem and is highly an art. It is learned through practice and the study of successful applications and is based on the knowledge of the strengths, weaknesses and particularities of the techniques provided by the different optimization methods.

## Classification of Optimization Problems

An important step in any optimization problem is to classify our model, since algorithms for solving optimization problems are usually designed to attack a particular type of problem. Some of the main classifications that we will be able to find are the following:

- **Continuous Optimization versus Discrete Optimization:** Some models only make sense if the variables take values ​​from a discrete set, often a subset of integers, while other models contain variables that can take any real or continuous value. Discrete variable models are discrete optimization problems; while models with continuous variables are continuous optimization problems. Continuous optimization problems tend to be easier to solve than discrete optimization problems; however, improvements in algorithms coupled with advances in computer technology have dramatically increased the size and complexity of discrete optimization problems that can be solved efficiently.
- **Optimization without restrictions versus optimization with restrictions:** Another important distinction that we can find is between the problems in which there are no restrictions on the variables and the problems in which there are restrictions on the variables. Unconstrained optimization problems arise directly in many practical applications; they also arise in the reformulation of constrained optimization problems in which the constraints are replaced by a penalty term in the objective function. Constrained or constrained optimization problems arise from applications where there are explicit constraints on the variables. Constraints on variables can vary widely from simple limits to systems of equalities and inequalities that model complex relationships between variables. Constrained optimization problems can also be classified according to the nature of the constraints they possess (for example, linear, nonlinear, convex) and the smoothness of the functions (for example, differentiable or nondifferentiable).
- **None, one or several objective functions:** Most optimization problems have a single objective function, however, there are interesting cases in which optimization problems have no objective function or multiple objective functions. Feasibility problems are problems in which the objective is to find values ​​for the variables that satisfy the constraints of a model without a particular optimization goal. Complementarity problems often arise in engineering and economics; the objective is to find a solution that satisfies the conditions of complementarity. **Multi-objective optimization problems** arise in many fields, such as engineering, economics, and logistics, when optimal decisions need to be made in the presence of trade-offs between two or more conflicting objectives. In practice, multi-objective problems are often reformulated as single-objective problems, either by forming a weighted combination of the different objectives or by substituting constraints for some of the objectives.
- **Deterministic optimization versus stochastic optimization:** In deterministic optimization, it is assumed that the data for the given problem are known exactly. However, for many real world problems, the data cannot be known precisely for a variety of reasons. The first reason is due to simple measurement error. The second, more fundamental reason is that some data represents information about the future (for example, product demand or price for a future time period) and simply cannot be known with certainty. In optimization under uncertainty, or stochastic optimization, uncertainty is built into the model. The goal is to find a solution that is feasible for all the data and optimal in some sense. Stochastic programming models take advantage of the fact that the probability distributions that govern the data are known or can be estimated; the goal is to find some solution that is feasible for all (or nearly all) possible instances of the data and optimizes the expected performance of the model.

Mathematically, a constrained optimization problem takes the following form:

Min f0(x)

Subject   fi(x)≤bi,i=1,…,m.

Where the vector x=(x1,…,xn) is the optimization of the problem, the function f0:Rn→R is the objective function, the function fi:Rn→R, i=1,…,m; are the inequality constraint functions; and the constants b1,…,m are the limits of the constraints. Within this framework, an important case is that of linear programming, in which the objective function and the constraints are linear. The objective of linear programming is to determine the values ​​of the decision variables that maximize or minimize a linear objective function, and where the decision variables are subject to linear constraints. In general, the goal is to find a point that minimizes the objective function while satisfying the constraints.

**To solve a linear programming problem** , we must follow the following steps:

1. Choose the unknowns or decision variables.

2. Write the objective function based on the data of the problem.

3. Write the constraints in the form of a system of inequalities.

4. Find the set of feasible solutions by graphically representing the constraints.

5. Calculate the coordinates of the vertices of the enclosure of feasible solutions (if they are few).

6. Calculate the value of the objective function in each of the vertices to see in which of them it presents the maximum or minimum value as requested by the problem (the possible non-existence of a solution must be taken into account here).

## Case Study - Supply Chain Optimization with Python

So, **The transportation problem** is a classic linear programming problem in which the **cost of supplying** a series of demand points from a group of supply points **must be minimized** , taking into account the different shipping prices of each supply point supply to each point of demand.

**Code in Python**
```python
# Example of the beer transport problem

# We create the variable prob that contains the data of the problem

prob = pulp.LpProblem(&quot;Beer distribution problem &quot;, pulp.LpMinimize)

# We create a list of breweries or offer nodes

breweries = [&quot;Brewery A&quot;, &quot;Brewery B&quot;]

# dictionary with the supply capacity of each brewery

bid = {&quot;Brewery A&quot;: 1000,

&quot;Brewery B&quot;: 4000}

# We create the list of bars or demand nodes

bars = [&quot;Bar 1&quot;, &quot;Bar 2&quot;, &quot;Bar 3&quot;, &quot;Bar 4&quot;, &quot;Bar 5&quot;]

# dictionary with the demand capacity of each bar

demand = {&quot;Bar 1&quot;:500,

&quot;Bar 2&quot;:900,

&quot;Bar 3&quot;:1800,

&quot;Bar 4&quot;:200,

&quot;Bar 5&quot;:700,}

# List with the transport costs of each node

costs = [ #Bars

#1 2 3 4 5

[2,4,5,2,1],#A Breweries

[3,1,3,2,3] #B

]

# Convert the costs to a PuLP dictionary

costs = pulp.makeDict([breweries, bars], costs,0)

# Create a list of tuples containing all possible transport routes.

routes = [(c,b) for c in breweries for b in bars]

# we create dictionary x that will contain the quantity sent in the routes

x = pulp.LpVariable.dicts(&quot;path&quot;, (breweries, bars),

lowBound = 0,

cat = pulp.LpInteger)

# Add the objective function to the problem

prob += sum([x[c][b]\*costs[c][b] for (c,b) in routes]), \

&quot;Sum\_of\_transportation\_costs&quot;

# Add each brewery&#39;s maximum supply constraint to the problem.

for c in breweries:

prob += sum([x[c][b] for b in bars]) \&lt;= bid[c], \

&quot;Sum\_of\_Products\_that\_come\_out\_of\_breweries\_%s&quot;%c

# We add the minimum demand constraint of each bar

for b in bars:

prob += sum([x[c][b] for c in breweries]) \&gt;= demand[b], \

&quot;Sum\_of\_Products\_into\_Bar%s&quot;%b

# Problem data is exported to .lp file

prob.writeLP(&quot;TransportProblem.lp&quot;)

# Solving the problem.

prob.solve()

# Print the status of the problem.

print(&quot;Status: {}&quot;.format(pulp.LpStatus[prob.status]))

# We print each variable with its optimal solution.

for v in prob.variables():

print(&quot;{0:} = {1:}&quot;.format(v.name, v.varValue))

# Print the optimal value of the objective function

print(&quot;Total transportation cost = {}&quot;.format(prob.objective.value()))
```

**Status: Optimum**

path\_Brewery\_To\_Bar\_1 = 300.0

path\_Brewery\_To\_Bar\_2 = 0.0

path\_Brewery\_To\_Bar\_3 = 0.0

path\_Brewery\_To\_Bar\_4 = 0.0

path\_Brewery\_To\_Bar\_5 = 700.0

route\_Brewery\_B\_Bar\_1 = 200.0

route\_Brewery\_B\_Bar\_2 = 900.0

route\_Brewery\_B\_Bar\_3 = 1800.0

route\_Brewery\_B\_Bar\_4 = 200.0

route\_Brewery\_B\_Bar\_5 = 0.0

Total transportation cost = 8600.0

As we can see, **the optimal solution** we find tells us that we should send from Brewery A, 300 boxs to Bar 1 and 700 boxs to Bar 5; and that from Brewery B we should ship 200 boxs to Bar 1, 900 boxs to Bar 2, 1,800 boxs to Bar 3, and 200 boxs to Bar 4. This way we can **minimize the transportation cost to a total of 8,600$.**

## Reference

[1] Supply Chain Optimization – What Is It? [https://tensoft.com/supply-chain-optimization-what-is-it/](https://tensoft.com/supply-chain-optimization-what-is-it/)

[2] Supply Chaing Optimization: Everything You need To Know [https://www.bringg.com/blog/logistics/supply-chain-optimization/](https://www.bringg.com/blog/logistics/supply-chain-optimization/)

[3] Everything You Need to Know About Supply Chain Optimization. February 20, 2019. [https://www.warehouseanywhere.com/resources/supply-chain-optimization/](https://www.warehouseanywhere.com/resources/supply-chain-optimization/)

[4] Supply Chain Optimization [https://www.omnisci.com/technical-glossary/supply-chain-optimization](https://www.omnisci.com/technical-glossary/supply-chain-optimization)

[5] Supply Chain Optimization Applying Artificial Intelligence [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Optimization Techniques for Data Scientist and Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[7] Optimization Vehicle Routing Problem [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

A supply chain is a network of organizations, individuals, activities, information, and resources involved in the production and sale of a service or product to consumers. The chain begins with the transformation of natural, raw resources and components into a finished product or service, and concludes with the delivery of that product or service to the end customer.

Supply chain optimization refers to the tools and processes by which manufacturing and distribution supply chain performance and efficiency are improved, taking into account all constraints. Supply chain network optimization technologies use sophisticated algorithms and analytics to balance supply and demand in such a manner that sufficient raw materials are procured for manufacturing and distribution to meet customer expectations at the highest cost efficiency. Supply chains are complex. To best serve you, we go beyond just managing supply chains to offer complete optimization. It&#39;s imperative you don&#39;t waste money or time on valuable shipments.

To evaluate a supply chain optimization process, most supply chain managers will look at the bottom line: will it reduce supply chain costs? Will it lower operational expenses? This, as well as increasing revenue, will always be the primary company goals, and a good way to achieve them is to look at inventory holding costs. The cost of storing inventory makes up a major percentage of a product&#39;s total cost. That&#39;s why it&#39;s crucial to properly forecast demand and plan accordingly in order to optimize inventory quantities.

Machine learning would also be very beneficial for proper forecasting. Machine learning automates data analysis as well as analytical model building. In other words, the ability to learn from data and identifying patterns can essentially help improve decision making and performance.

A good example would be eliminating time buffers. If machine learning could predict that the time required for drivers to reach a certain warehouse is one and a half hours rather than two, for instance, then that estimation could be incorporated into [dispatching](https://www.bringg.com/blog/dispatching/dispatch-software/) and route planning, reducing lead times across the last mile of the supply chain. That, of course, means the same resources to perform more deliveries, reducing the related transportation cost. More business, less waste.

So, Optimization is fundamental to any problem related to decision making, whether in engineering or economics. The task of making decisions involves choosing between several alternatives. This choice is going to be governed by our desire to make the &quot;best&quot; decision possible. How good each of the alternatives will be will be described by an objective function or performance index. The theory and optimization methods will help us to select the best alternative according to this objective function. The area of optimization has received great attention in recent years, mainly due to the rapid development of computer science, including the development and availability of extremely user-friendly software tools, high-speed parallel processors, and Artificial Intelligence.

Optimization and Its Importance in the Supply Chain

The traveling salesman problem (TSP) consists of the need to visit many places in the shortest, safest, and least expensive way and then return to the starting point, so that the route does not take too much time, wasting company resources. Usually this problem is presented with the help of a graph or map whose points are expressed by cities and the edges connecting them—roads. This problem has a special feature—it is not possible to solve it any other way than by comparing all possible routes. This feature makes this problem NP-hard. In practice (that is, having a limited amount of available time), heuristic methods are used to solve NP-difficult problem. These methods do not guarantee finding the optimal solution but offer an acceptable approximate solution in a reasonable time. In addition to heuristic methods, which are created to solve one specific problem, there are metaheuristic methods. Their main advantage is that they can be used for more than one problem and do not require prior knowledge of the space available solutions. Most often they combine the features of random search algorithms (random search) and algorithms that use gradient navigation in the space of solutions (for example, in the hill-climbing algorithm). They demand only a measure by which it will be possible to assess how optimal the found solution so. This measure usually has the form of a polynomial function value, whose arguments belong to the searched solution space. The Vehicle Routing Problem (VRP) is a generic name given to a set of problems in which a set of routes for a fleet of vehicles based on one or several depots are to be formed to serve the customers dispersed geographically. The objective of the VRP is to form a route with the lowest cost to serve all customers.

Optimization with Genetic Algorithm to Vehicle Routing Problem

# Introduction

Exploratory Data Analysis refers to the critical process of performing initial investigations on data so as to discover patterns, to spot anomalies, to test hypothesis and to check assumptions with the help of summary statistics and graphical representations. So, Exploratory Data Analysis is a method of evaluating or comprehending data in order to derive insights or key characteristics. [EDA](https://www.analyticsvidhya.com/blog/2022/03/eda-on-superstore-dataset-using-python/) can be divided into two categories: graphical analysis and non-graphical analysis. EDA is a critical component of any data science or machine learning process. You must explore the data, understand the relationships between variables, and the underlying structure of the data in order to build a reliable and valuable output based on it.

Exploratory Data Analysis (EDA) requires a balance of creativity, cleverness, and deep knowledge of statistical analysis. An expert EDA practitioner must also quickly learn and internalize the nuances and needs of varying industries and businesses. This hiring test focuses on feature engineering, data visualization, and statistical analyses to identify candidates who excel at intermediate EDA. So, a strong exploratory data analyst converts business problems to data science problems and uses data and visualizations to create and test solution-driven hypotheses. This screening test presents example datasets across industries and asks candidates to choose actions to solve issues and respond to questions typical for EDA practitioners.

This Exploratory Data Analysis (EDA) test assesses a candidate's ability to work with data from start to finish. This screening test evaluates a candidate's ability to extract key information, develop various kinds of data visualizations to identify trends, as well as understand and interpret various kinds of data types. So, Candidates who perform well on this screening test will have a strong understanding of EDA and a practical ability to perform various kinds of data analysis. This is an intermediate test designed for someone with 2-3 years experience with EDA. Use this hiring test to find the next Exploratory Data Analyst who will leverage your business analytics to increase efficiency and increase revenues.

Exploratory Data Analysis is an approach in analyzing data sets to summarize their main characteristics, often using statistical graphics and other data visualization methods. So, EDA assists Data science professionals in various ways:

> Getting a better understanding of data
> Identifying various data patterns
> Getting a better understanding of the problem statement

Exploratory data analysis (EDA) is an investigative process in which you use summary statistics and graphical tools to get to know your data and understand what you can learn from it. So, with EDA, you can uncover patterns in your data, understand potential relationships between variables, and find anomalies, such as outliers or unusual observations. The goal is to generate interesting questions or hypotheses that you can test using more formal statistical methods.

## How to conduct exploratory data analysis

It can be easier to conduct exploratory data analysis if you break the process down into steps. Here are six key steps that you can follow to conduct EDA:

**1. Observe your dataset**

The first step to conducting exploratory data analysis is to observe your dataset at a high level. Start by determining the size of your dataset, including how many rows and columns it has. This can help you predict any future issues you might have with your data.

**2. Find any missing values**

Once you've observed your dataset, you can start looking for any missing values. When you find missing values, think about what could cause them to be missing. If you can spot a trend in your data, you might be able to replace some missing values with estimates.

**3. Categorize your values**

After you find any missing values, you can categorize your values to help determine what statistical and visualization methods can work with your dataset. You can place your values into these categories:

**Categorical:** Categorical variables can have a set number of values.

**Continuous:** Continuous variables can have an infinite number of values.

**Discrete:** Discrete variables can have a set number of values that must be numeric.

**4. Find the shape of your dataset**

Finding the shape of your dataset is another important step in the EDA process. This step is important because you can gather relevant information about your dataset by observing its shape. The shape of your dataset shows your data's distribution. You can also notice data features like skewness and gaps that can help you learn more about the dataset. It can also help you identify trends in your dataset.

**5. Identify relationships in your dataset**

As you continue to understand your dataset, you can begin to pick out relationships in your dataset. Try to spot any correlations between values. Using scatter plots can make it easier to identify correlations and relationships between values. Be sure to take notes and pick out as many correlations as you can find. As you notice correlations, you can start thinking about why certain values might have correlations.

**6. Locate any outliers in your dataset**

Locating outliers in your dataset is another important step to conducting EDA. Outliers are values in your dataset that are significantly different from the rest of the values. Outliers can be much higher or lower than the other values in a dataset. It's important to identify outliers because they can skew the mean, median, mode or range of a dataset and alter the appearance of a visual representation. You can locate outliers by observing your graphs or sorting your data in numerical order during your EDA.

## Benefits of conducting exploratory data analysis

It's helpful to conduct exploratory data analysis to help you understand a dataset before you start to model it. Some of the benefits of conducting EDA include:

**Organizing a dataset**

One important benefit of conducting exploratory data analysis is that it can help you organize a dataset before you model it. This can help you start to make assumptions and predictions about your dataset. It can also help you make decisions before you model your data.

**Understanding variables**

Another benefit of EDA is that it can help you understand the variables in your dataset. This can help you organize your dataset and begin to pinpoint relationships between variables, which is an integral part of data analysis.

**Identifying relationships between variables**

Conducting EDA can also help you identify the relationships between the variables in your dataset. Identifying the relationships between variables is a critical part of drawing conclusions from a dataset.

**Choosing the right model**

Another important benefit of EDA is helping you choose the right model for your dataset. You can use all of the information that you gain from conducting an EDA to help you choose a data model. It's important to choose the right data model because it can make it easier for everyone in your organization to understand your data. Some commonly used data models that you can choose from include:

**Hierarchical data model:** A hierarchical data model organizes the values of a dataset into a hierarchy resembling a tree.

**Relational data model:** A relational data model organizes data values into tables with relationships.

**Network data model:** A network data model, similar to the hierarchical model, organizes data values into a tree shape, but it also includes a schema, which shows the types of relationships between values.

**Finding patterns in a dataset**

You can also use EDA to help you find patterns in a dataset. Finding patterns in a dataset is important because it can help you make predictions and estimations. This can help your organization plan for the future and anticipate problems and solutions.

It is a good practice to understand the data first and try to gather as many insights from it. EDA is all about making sense of data in hand, before getting them dirty with it.

## Code Study – Exploratory Data Analysis

For this article, we will be doing Customer Churn Prediction. When clients stop doing business with a company, this is known as customer churn or customer attrition.

Because the cost of getting a new customer is usually higher than keeping an existing one, understanding customer churn is critical to a company's success. As a result, churn analysis is the first step in gaining a better understanding of your clients.

## Code in Python

```python
import pandas as pd

import numpy as np

import matplotlib.pyplot as plt

import seaborn as sns

df = pd.read\_csv(&quot;WA\_Fn-UseC\_-Telco-Customer-Churn.csv&quot;)

df.shape

df.head()

df.tail()

df.info()

df.describe()

df.describe(include=&#39;all&#39;)

df.TotalCharges=pd.to\_numeric(df.TotalCharges, errors=&#39;coerce&#39;)

df.isnull().sum()

df.dropna(how=&#39;any&#39;, inplace=True)

df.isnull().sum()

df.boxplot(column=[&quot;Pack Price&quot;])
```

<img>

```python

plt.figure(figsize=(5,5))

sns.countplot(x=df[&#39;Churn&#39;], palette=&#39;Blues&#39;,linewidth=1)

plt.show()
```

<img>

```python
for i, predictor in enumerate(df.drop(columns=[&#39;Churn&#39;,&#39;TotalCharges&#39;, &#39;MonthlyCharges&#39;])):

plt.figure(i)

sns.countplot(data=df, x=predictor, hue=&#39;Churn&#39;)
```

<img>

<img>

<img>

<img>

<img>

<img>

<img>

<img>

<img>

<img>

<img>

<img>

```python
sns.lmplot(data=df, x=&#39;MonthlyCharges&#39;, y=&#39;TotalCharges&#39;, fit\_reg=False)
```

<img>

```python
sns.histplot(data=df, x=&quot;Churn&quot;, hue=&quot;Contract&quot;, multiple=&quot;dodge&quot;, shrink=.8).set(title=&#39;Customer contract distribution&#39;)
```

<img>

```python
plt.pie(df[&#39;PaymentMethod&#39;].value\_counts(), labels=df[&#39;PaymentMethod&#39;].unique(),autopct=&#39;%1.2f%%&#39;)
```

<img>

```python
sns.histplot(data=df, x=&quot;Churn&quot;, hue=&quot;Dependents&quot;, multiple=&quot;dodge&quot;,shrink=.8).set(title=&#39;Dependents distribution&#39;)
```

<img>

```python
sns.histplot(data=df, x=&quot;Churn&quot;, hue=&quot;Partner&quot;, multiple=&quot;dodge&quot;,shrink=.8).set(title=&#39;Partners distribution&#39;)
```

<img>

```python
df.corr()
```

| | SeniorCitizen | tenure | MonthlyCharges | TotalCharges |
| -------------- | ------------- | -------- | -------------- | ------------ |
| SeniorCitizen | 1.000000 | 0.015683 | 0.219874 | 0.102411 |
| tenure | 0.015683 | 1.000000 | 0.246862 | 0.825880 |
| MonthlyCharges | 0.219874 | 0.246862 | 1.000000 | 0.65106 |
| TotalCharges | 0.102411 | 0.825880 | 0.651065 | 1.000000 |
 
## Reference

[1] How To Conduct Exploratory Data Analysis in 6 Steps [https://www.indeed.com/career-advice/career-development/how-to-conduct-exploratory-data-analysis](https://www.indeed.com/career-advice/career-development/how-to-conduct-exploratory-data-analysis)

[2] Exploratory Data Analysis (EDA) in Python [https://www.analyticsvidhya.com/blog/2022/04/exploratory-data-analysis-eda-in-python/](https://www.analyticsvidhya.com/blog/2022/04/exploratory-data-analysis-eda-in-python/)

[3] Optimization Techniques for Data Scientist and Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization Vehicle Routing Problem [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Optimization and Its Importance in the Supply Chain [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[6] Sentiment Analysis of Financial using Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Exploratory Data Analysis refers to the critical process of performing initial investigations on data so as to discover patterns, to spot anomalies, to test hypothesis and to check assumptions with the help of summary statistics and graphical representations. So, Exploratory Data Analysis is a method of evaluating or comprehending data in order to derive insights or key characteristics. EDA can be divided into two categories: graphical analysis and non-graphical analysis. EDA is a critical component of any data science or machine learning process. You must explore the data, understand the relationships between variables, and the underlying structure of the data in order to build a reliable and valuable output based on it.

Exploratory Data Analysis (EDA) in Supply Chain

Data should underpin the decision-making process of all modern
businesses, and restaurants are no exception. Unfortunately, many
restaurants end up shying away from implementing a robust data science
strategy because they simply don't know where to begin. This is a huge
mistake and can be detrimental to their bottom line. Restaurants
can---and should---be collecting and analyzing customer data in
real-time to better understand how guests interact with their brand, and
ultimately drive sales. In this article, I'll break down how restaurants
can implement a data science strategy and what benefits will come of
that effort. So, specifically Pizza's case study.

Data Science in Restaurants

## Introduction

_One of the most challenging questions data scientists face is how data
can bring value to a specific problem. Data scientists encounter various
business requests to solve problems using machine learning and AI_.
However, many of such problems might not be data science problems. By
having the request from the business side and a dataset to explore, a
data scientist should decide if the problem is solvable by data
analytics or not. And if the answer is yes, what is the best way to
tackle the problem.

Exploratory Data Analysis (EDA) is an approach to extract the
information enfolded in the data and summarize the main characteristics
of the data. It is considered to be a crucial step in any data science
project (in Figure 1 it is the second step after problem understanding
in [CRISP](https://blog.camelot-group.com/2018/04/how-artificial-intelligence-ai-is-transforming-logistics-part-1-a-broad-overview/) methodology).
Most people underestimate the importance of data preparation and data
exploration. EDA is essential for a well-defined and structured data
science project and it should be performed before any statistical or
machine learning modeling phase. In this post, we focus on how EDA is
done, and which technologies do we use at Camelot for data exploration
and data visualization.

<img>

_Fig. 1: Data Science Project Flow_

## Data Preparation

_Data preparation is cleaning and organizing the real-world data, which
is known to take up more than 80% of the time of a data scientist's
work. Real-world data or raw data is dirty, full of missing values,
duplicates and in some cases wrong information. Most machine learning
algorithms cannot deal with missing values; hence, data needs to be
converted and cleaned. Common solutions of handling missing values would
be dropping rows, linear interpolation, using mean values etc. Depending
on the importance of the feature and amount of the missing values one of
these solutions can be employed._

## Data Exploration

Having a clean dataset in hand, we need to understand the data,
summarize its characteristics, and visualize it. Understanding the data
is an iterative process between the data science team and the experts
from the business side. It can help both sides to identify and construct
important features, and later to build suitable machine learning models.
An essential part of data exploration is data transformation. Let's
clarify it by an example. Assume a forecasting problem in the field of
logistics for a number of deliveries in different locations by different
suppliers. One data transformation option is "filtering". It is possible
to filter one specific location, or one specific group of suppliers, and
make the forecast on the filtered data to get a fast insight. Another
tactic is called "aggregation". If we have daily data, by making weekly
or even monthly aggregation we can have a new data set which brings us
insights to the existing seasonality and trends. Having the enriched
data, we can visualize it. Data visualization allows us to understand
data quickly. It helps data scientists and business stakeholders to
easily align on processes and data quality. This is an important
feedback loop in the CRISP methodology towards problem understanding.
From a data perspective, it will help to rapidly identify patterns,
detect outliers, and decide how to proceed with the problem in hand.

_Some commonly used plots for EDA are:_

- Histograms: to check the distribution of a specific variable
- Scatter plots: to check the dependency between two variables
- Maps: to show distribution of a variable on a regional or world map
- Feature correlation plot (heatmap): to understand the dependencies
between multiple variables
- Time series plots: to identify trends and seasonality in time
dependent data

_After completing the EDA process, one can start the modeling phase.
Modeling consists of statistical modeling and building machine learning
models. Although, basic statistical analysis is already included in EDA,
the complete statistical modeling is performed in the modeling phase._

## Exploratory Data Analysis of Text Data

_After performing EDA, Sentiment analysis (or opinion mining) is
a natural language processing (NLP) technique used to determine whether
data is positive, negative or neutral. Sentiment analysis is often
performed on textual data to help businesses monitor brand and product
sentiment in customer feedback, and understand customer needs. So,
Sentiment analysis is the process of detecting positive or negative
sentiment in text. It's often used by businesses to detect sentiment in
social data, gauge brand reputation, and understand customers. So,
Sentiment analysis focuses on the polarity of a text (positive,
negative, neutral) but it also goes beyond polarity to detect specific
feelings and emotions (angry, happy, sad, etc), urgency (urgent, not
urgent) and even intentions (interested v. not interested)._

_Depending on how you want to interpret customer feedback and queries,
you can define and tailor your categories to meet your sentiment
analysis needs. In the meantime, here are some of the most popular types
of sentiment analysis:_

## Graded Sentiment Analysis

_If polarity precision is important to your business, you might consider
expanding your polarity categories to include different levels of
positive and negative:_

- Very positive
- Positive
- Neutral
- Negative
- Very negative

_This is usually referred to as graded or fine-grained sentiment
analysis, and could be used to interpret 5-star ratings in a review, for
example:_

- Very Positive = 5 stars
- Very Negative = 1 star

## Emotion detection

_Emotion detection sentiment analysis allows you to go beyond polarity
to detect emotions, like happiness, frustration, anger, and sadness.
Many emotion detection systems use lexicons (i.e. lists of words and the
emotions they convey) or complex machine learning algorithms. One of the
downsides of using lexicons is that people express emotions in different
ways. Some words that typically express anger,
like bad or kill (e.g. your product is so bad or your customer support
is killing me) might also express happiness (e.g. this is bad ass or you
are killing it)._

## Aspect-based Sentiment Analysis

_Usually, when analyzing sentiments of texts you'll want to know which
particular aspects or features people are mentioning in a positive,
neutral, or negative way. That's where aspect-based sentiment
analysis can help, for example in this product review: "The battery
life of this camera is too short", an aspect-based classifier would be
able to determine that the sentence expresses a negative opinion about
the battery life of the product in question._

## Multilingual sentiment analysis

_Multilingual sentiment analysis can be difficult. It involves a lot of
preprocessing and resources. Most of these resources are available
online (e.g. sentiment lexicons), while others need to be created (e.g.
translated corpora or noise detection algorithms), but you'll need to
know how to code to use them. Alternatively, you could detect language
in texts automatically with a language classifier, then train a custom
sentiment analysis model to classify texts in the language of your
choice._

## Why Is Sentiment Analysis Important?

_Since humans express their thoughts and feelings more openly than ever
before, sentiment analysis is fast becoming an essential tool to monitor
and understand sentiment in all types of data. Automatically analyzing
customer feedback, such as opinions in survey responses and social media
conversations, allows brands to learn what makes customers happy or
frustrated, so that they can tailor products and services to meet their
customers' needs. For example, using sentiment analysis to automatically
analyze 4,000+ open-ended responses in your customer satisfaction
surveys could help you discover why customers are happy or unhappy at
each stage of the customer journey._

_Maybe you want to track brand sentiment so you can detect disgruntled
customers immediately and respond as soon as possible. Maybe you want to
compare sentiment from one quarter to the next to see if you need to
take action. Then you could dig deeper into your qualitative data to see
why sentiment is falling or rising._

_The overall benefits of sentiment analysis include:_

- **Sorting Data at Scale**

_Can you imagine manually sorting through thousands of tweets, customer
support conversations, or surveys? There's just too much business data
to process manually. Sentiment analysis helps businesses process huge
amounts of unstructured data in an efficient and cost-effective way._

- **Real-Time Analysis**

_Sentiment analysis can identify critical issues in real-time, for
example is a PR crisis on social media escalating? Is an angry customer
about to churn? Sentiment analysis models can help you immediately
identify these kinds of situations, so you can take action right away._

- **Consistent criteria**

_It's estimated that people only agree around 60-65% of the time when
determining the sentiment of a particular text. Tagging text by
sentiment is highly subjective, influenced by personal experiences,
thoughts, and beliefs. By using a centralized sentiment analysis system,
companies can apply the same criteria to all of their data, helping them
improve accuracy and gain better insights._

_The applications of sentiment analysis are endless. So, to help you
understand how sentiment analysis could benefit your business, let's
take a look at some examples of texts that you could analyze using
sentiment analysis._

## Code Study -- [Sentiment Analysis for EDA](https://programminghistorian.org/en/lessons/sentiment-analysis)

_This code uses sentiment analysis as the basis for an exploratory data
analysis of a large textual corpus. So, we will focus on one tool in the
NLP toolkit: sentiment analysis. Sentiment analysis seeks to quantify
the emotional intensity of words and phrases within a text._

_For this example uses the e-mail correspondence of bankrupt American
energy company Enron. Enron concealed a wide variety of illegal
accounting practices until a federal investigation in 2001 forced it
into bankruptcy. At the time, the Enron Scandal was the largest collapse
of a publicly traded company in history. In 2001, the company started
showing signs of financial strain that didn't align with the company's
financial disclosures to that point. The publicly traded Enron stocks
dropped from their mid-2000 high of $90.75 to less than a dollar in
November 2001, which led stockholders to sue the company. A subsequent
U.S. Securities and Exchange Commission (SEC) investigation revealed
that Enron executives committed fraud and accounting malpractice on a
massive scale. Enron declared bankruptcy in December of that year. In
the years that followed, several executives faced criminial convictions
for their role in the scandal._

## Code in Python -- Example Programing Historian [2]

```python
# below is the sentiment analysis code rewritten for sentence-level analysis
# note the new module -- word_tokenize!
import nltk.data
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from nltk import sentiment
from nltk import word_tokenize

# Next, we initialize VADER so we can use it within our Python script
sid = SentimentIntensityAnalyzer()

# We will also initialize our 'english.pickle' function and give it a short name

tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')

message_text = '''It seems to me we are in the middle of no man's land with respect to the following: Opec production speculation, Mid east crisis and renewed tensions, US elections and what looks like a slowing economy (?), and no real weather anywhere in the world. I think it would be most prudent to play the markets from a very flat price position and try to day trade more aggressively. I have no intentions of outguessing Mr. Greenspan, the US. electorate, the Opec ministers and their new important roles, The Israeli and Palestinian leaders, and somewhat importantly, Mother Nature. Given that, and that we cannot afford to lose any more money, and that Var seems to be a problem, let's be as flat as possible. I'm ok with spread risk (not front to backs, but commodity spreads). The morning meetings are not inspiring, and I don't have a real feel for everyone's passion with respect to the markets. As such, I'd like to ask John N. to run the morning meetings on Mon. and Wed. Thanks. Jeff'''

# The tokenize method breaks up the paragraph into a list of strings. In this example, note that the tokenizer is confused by the absence of spaces after periods and actually fails to break up sentences in two instances. How might you fix that?

sentences = tokenizer.tokenize(message_text)

# We add the additional step of iterating through the list of sentences and calculating and printing polarity scores for each one.

for sentence in sentences:
 print(sentence)
 scores = sid.polarity_scores(sentence)
 for key in sorted(scores):
 print('{0}: {1}, '.format(key, scores[key]), end='')
 print()
```

The output should look like this:

```bash
It seems to me we are in the middle of no man's land with respect to the following: Opec production speculation, Mid east crisis and renewed tensions, US elections and what looks like a slowing economy (?
compound: -0.5267, neg: 0.197, neu: 0.68, pos: 0.123,
), and no real weather anywhere in the world.
compound: -0.296, neg: 0.216, neu: 0.784, pos: 0.0,
I think it would be most prudent to play the markets from a very flat price position and try to day trade more aggressively.
compound: 0.0183, neg: 0.103, neu: 0.792, pos: 0.105,
I have no intentions of outguessing Mr. Greenspan, the US.
compound: -0.296, neg: 0.216, neu: 0.784, pos: 0.0,
electorate, the Opec ministers and their new important roles, The Israeli and Palestinian leaders, and somewhat importantly, Mother Nature.
compound: 0.4228, neg: 0.0, neu: 0.817, pos: 0.183,
Given that, and that we cannot afford to lose any more money, and that Var seems to be a problem, let's be as flat as possible.
compound: -0.1134, neg: 0.097, neu: 0.823, pos: 0.081,
I'm ok with spread risk (not front to backs, but commodity spreads).
compound: -0.0129, neg: 0.2, neu: 0.679, pos: 0.121,
The morning meetings are not inspiring, and I don't have a real feel for everyone's passion with respect to the markets.
compound: 0.5815, neg: 0.095, neu: 0.655, pos: 0.25,
As such, I'd like to ask John N. to run the morning meetings on Mon.
compound: 0.3612, neg: 0.0, neu: 0.848, pos: 0.152,
and Wed.
compound: 0.0, neg: 0.0, neu: 1.0, pos: 0.0,
Thanks.
compound: 0.4404, neg: 0.0, neu: 0.0, pos: 1.0,
Jeff
compound: 0.0, neg: 0.0, neu: 1.0, pos: 0.0,
```

## Reference

[1] Exploratory Data Analysis -- An Important Step in Data Science https://blog.camelot-group.com/2019/03/exploratory-data-analysis-an-important-step-in-data-science/

[2] [Sentiment Analysis for Exploratory Data Analysis](https://programminghistorian.org/en/lessons/sentiment-analysis) https://programminghistorian.org/en/lessons/sentiment-analysis

[3] Exploratory Data Analysis (EDA) in Supply Chain [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[4] Optimization and Its Importance in the Supply Chain [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[5] Sentiment Analysis of Financial using Machine Learning [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

One of the most challenging questions data scientists face is how data can bring value to a specific problem. Data scientists encounter various business requests to solve problems using machine learning and AI. However, many of such problems might not be data science problems. By having the request from the business side and a dataset to explore, a data scientist should decide if the problem is solvable by data analytics or not. And if the answer is yes, what is the best way to tackle the problem.

Exploratory Data Analysis use Sentiment Analysis

## Introduction

Before you start [data
analysis](https://www.simplilearn.com/tutorials/data-analytics-tutorial/what-is-data-analytics) or
run your data through a [machine learning
algorithm](https://www.simplilearn.com/10-algorithms-machine-learning-engineers-need-to-know-article),
you must clean
your [data](https://www.simplilearn.com/what-is-data-article) and make
sure it is in a suitable form. Further, it is essential to know any
recurring patterns and significant correlations that might be present in
your data. The process of getting to know your data in depth is
called [Exploratory Data
Analysis.](https://www.simplilearn.com/exploratory-data-analysis-article)
So, Exploratory Data Analysis is an integral part of working with data.
In this tutorial titled 'All the ins and outs of exploratory data
analysis,' you will explore how to perform exploratory data analysis on
different data types.

Exploratory Data Analysis is a data analytics process to understand the
data in depth and learn the different data characteristics, often with
visual means. This allows you to get a better feel of your data and find
useful patterns in it.

Exploratory Data Analysis (EDA) is an approach to extract the
information enfolded in the data and summarize the main characteristics
of the data. It is considered to be a crucial step in any data science
project (in Figure 1 it is the second step after problem understanding
in [CRISP](https://blog.camelot-group.com/2018/04/how-artificial-intelligence-ai-is-transforming-logistics-part-1-a-broad-overview/) methodology).
Most people underestimate the importance of data preparation and data
exploration. EDA is essential for a well-defined and structured data
science project and it should be performed before any statistical or
machine learning modeling phase. In this post, we focus on how EDA is
done, and which technologies do we use at Camelot for data exploration
and data visualization.

<img>

_Fig. 1: Data Science Project Flow_

## Data Preparation

Data preparation is cleaning and organizing the real-world data, which
is known to take up more than 80% of the time of a data scientist's
work. Real-world data or raw data is dirty, full of missing values,
duplicates and in some cases wrong information. Most machine learning
algorithms cannot deal with missing values; hence, data needs to be
converted and cleaned. Common solutions of handling missing values would
be dropping rows, linear interpolation, using mean values etc. Depending
on the importance of the feature and amount of the missing values one of
these solutions can be employed.

## Data Exploration

Having a clean dataset in hand, we need to understand the data,
summarize its characteristics, and visualize it. Understanding the data
is an iterative process between the data science team and the experts
from the business side. It can help both sides to identify and construct
important features, and later to build suitable machine learning models.
An essential part of data exploration is data transformation. Let's
clarify it by an example. Assume a forecasting problem in the field of
logistics for a number of deliveries in different locations by different
suppliers. One data transformation option is "filtering". It is possible
to filter one specific location, or one specific group of suppliers, and
make the forecast on the filtered data to get a fast insight. Another
tactic is called "aggregation". If we have daily data, by making weekly
or even monthly aggregation we can have a new data set which brings us
insights to the existing seasonality and trends. Having the enriched
data, we can visualize it. Data visualization allows us to understand
data quickly. It helps data scientists and business stakeholders to
easily align on processes and data quality. This is an important
feedback loop in the CRISP methodology towards problem understanding.
From a data perspective, it will help to rapidly identify patterns,
detect outliers, and decide how to proceed with the problem in hand.

Some commonly used plots for EDA are:

- Histograms: to check the distribution of a specific variable
- Scatter plots: to check the dependency between two variables
- Maps: to show distribution of a variable on a regional or world map
- Feature correlation plot (heatmap): to understand the dependencies
between multiple variables
- Time series plots: to identify trends and seasonality in time
dependent data

After completing the EDA process, one can start the modeling phase.
Modeling consists of statistical modeling and building machine learning
models. Although, basic statistical analysis is already included in EDA,
the complete statistical modeling is performed in the modeling phase.

# Code Study -- Credit Card Fraud Detection

In this modern era where everything is online, making a transaction is
not safe anymore, so it is duty of every bank to check the transaction
details and analyses them whether it's a fraud transaction or not which
is not an easy task and very time consuming in the era where time is
money here comes Data Science and Machine Learning algorithm to rescue
us from this vital problem. This project includes to give a complete
view how to solve this problem using very sophisticated Machine Learning
algorithm with Credit Card Fraud Detection, which includes modelling
with the past records of Credit Card Transactions with the ones who have
the Fraud Transactions. Here we want to classify all the correct Fraud
Transactions to avoid any misclassifications. In this project, first we
have focused in data pre-processing and data transformation, then we
have performed various Classification algorithm to get a model having
maximum accuracy score.

```python
Code [1,2]
import warnings
warnings.filterwarnings('ignore')

#Other's
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from plotly.subplots import make_subplots
import plotly.graph_objects as go
pd.set_option('display.max_columns', 300) #Setting column display limit
plt.style.use('ggplot') #Applying style to graphs

df1 = pd.read_csv("application_data.csv")
df1.head()

df1.shape
(307511, 122)

df1.info(verbose = True)
df1.describe()
(df1.isnull().sum()/len(df1)*100).sort_values(ascending = False).head(50)

null_col = df1.isnull().sum().sort_values(ascending = False)
null_col = null_col[null_col.values >(0.35*len(df1))]


#Plotting Bar Graph for null values greater than 35%
plt.figure(figsize=(20,4))
null_col.plot(kind='bar', color="#4CB391")
plt.title('List of Columns & null counts where null values are more than 35%')
plt.xlabel("Null Columns",fontdict={"fontsize":12,"fontweight":5}) #Setting X-label and Y-label
plt.ylabel("Count of null values",fontdict={"fontsize":12,"fontweight":5})
plt.show()
```

<img>

```python
len(null_col)
49

label = list(null_col.index.values) #Making list of column names having null values greater than 35%
df1.drop(labels = label,axis=1,inplace = True) #Droping those columns
df1.shape
(307511, 73)

null = (df1.isnull().sum()/len(df1)*100).sort_values(ascending = False).head(50)
null.head(30)
```

```bash
OCCUPATION_TYPE 31.345545
EXT_SOURCE_3 19.825307
AMT_REQ_CREDIT_BUREAU_YEAR 13.501631
AMT_REQ_CREDIT_BUREAU_MON 13.501631
AMT_REQ_CREDIT_BUREAU_WEEK 13.501631
AMT_REQ_CREDIT_BUREAU_DAY 13.501631
AMT_REQ_CREDIT_BUREAU_HOUR 13.501631
AMT_REQ_CREDIT_BUREAU_QRT 13.501631
NAME_TYPE_SUITE 0.420148
OBS_30_CNT_SOCIAL_CIRCLE 0.332021
DEF_30_CNT_SOCIAL_CIRCLE 0.332021
OBS_60_CNT_SOCIAL_CIRCLE 0.332021
DEF_60_CNT_SOCIAL_CIRCLE 0.332021
EXT_SOURCE_2 0.214626
AMT_GOODS_PRICE 0.090403
AMT_ANNUITY 0.003902
CNT_FAM_MEMBERS 0.000650
DAYS_LAST_PHONE_CHANGE 0.000325
NAME_INCOME_TYPE 0.000000
FLAG_OWN_REALTY 0.000000
TARGET 0.000000
FLAG_EMAIL 0.000000
FLAG_PHONE 0.000000
FLAG_CONT_MOBILE 0.000000
NAME_CONTRACT_TYPE 0.000000
FLAG_WORK_PHONE 0.000000
FLAG_EMP_PHONE 0.000000
FLAG_MOBIL 0.000000
CODE_GENDER 0.000000
FLAG_OWN_CAR 0.000000
```

```python
df1.AMT_REQ_CREDIT_BUREAU_YEAR.fillna(df1.AMT_REQ_CREDIT_BUREAU_YEAR.mode()[0],inplace = True) #AMT_REQ_CREDIT_BUREAU_YEAR

df1.AMT_REQ_CREDIT_BUREAU_MON.fillna(df1.AMT_REQ_CREDIT_BUREAU_MON.mode()[0],inplace = True) #AMT_REQ_CREDIT_BUREAU_MON

df1.AMT_REQ_CREDIT_BUREAU_WEEK.fillna(df1.AMT_REQ_CREDIT_BUREAU_WEEK.mode()[0],inplace = True) #AMT_REQ_CREDIT_BUREAU_WEEK

df1.AMT_REQ_CREDIT_BUREAU_DAY.fillna(df1.AMT_REQ_CREDIT_BUREAU_DAY.mode()[0],inplace = True) #AMT_REQ_CREDIT_BUREAU_DAY

df1.AMT_REQ_CREDIT_BUREAU_HOUR.fillna(df1.AMT_REQ_CREDIT_BUREAU_HOUR.mode()[0],inplace = True) #AMT_REQ_CREDIT_BUREAU_HOUR

df1.AMT_REQ_CREDIT_BUREAU_QRT.fillna(df1.AMT_REQ_CREDIT_BUREAU_QRT.mode()[0],inplace = True) #AMT_REQ_CREDIT_BUREAU_QRT

df1.NAME_TYPE_SUITE.fillna(df1.NAME_TYPE_SUITE.mode()[0],inplace = True) #NAME_TYPE_SUITE

df1.OBS_30_CNT_SOCIAL_CIRCLE.fillna( df1.OBS_30_CNT_SOCIAL_CIRCLE.mode()[0],inplace = True) #OBS_30_CNT_SOCIAL_CIRCLE

df1.DEF_30_CNT_SOCIAL_CIRCLE.fillna( df1.DEF_30_CNT_SOCIAL_CIRCLE.mode()[0],inplace = True) #DEF_30_CNT_SOCIAL_CIRCLE

df1.OBS_60_CNT_SOCIAL_CIRCLE.fillna( df1.OBS_60_CNT_SOCIAL_CIRCLE.mode()[0],inplace = True) #OBS_60_CNT_SOCIAL_CIRCLE

df1.DEF_60_CNT_SOCIAL_CIRCLE.fillna( df1.DEF_60_CNT_SOCIAL_CIRCLE.mode()[0],inplace = True) #DEF_60_CNT_SOCIAL_CIRCLE

df1.CNT_FAM_MEMBERS.fillna(df1.CNT_FAM_MEMBERS.mode() , inplace = True) #CNT_FAM_MEMBERS

df1.DAYS_LAST_PHONE_CHANGE.fillna(df1.DAYS_LAST_PHONE_CHANGE.mode()[0],inplace = True) #DAYS_LAST_PHONE_CHANGE

df1.EXT_SOURCE_2.fillna(df1.EXT_SOURCE_2.median() , inplace = True) #EXT_SOURCE_2

df1.EXT_SOURCE_3.fillna(df1.EXT_SOURCE_3.median() , inplace = True) #EXT_SOURCE_3

(df1.isnull().sum()/len(df1)*100).sort_values(ascending=False)
```

```bash
OCCUPATION_TYPE 31.345545
AMT_GOODS_PRICE 0.090403
AMT_ANNUITY 0.003902
CNT_FAM_MEMBERS 0.000650
AMT_REQ_CREDIT_BUREAU_YEAR 0.000000
 ...
FLAG_DOCUMENT_3 0.000000
FLAG_DOCUMENT_4 0.000000
FLAG_DOCUMENT_5 0.000000
FLAG_DOCUMENT_6 0.000000
SK_ID_CURR 0.000000
```

```python
df1[["DAYS_BIRTH","DAYS_EMPLOYED","DAYS_REGISTRATION","DAYS_ID_PUBLISH","DAYS_LAST_PHONE_CHANGE"]]

day_cols = [i for i in df1 if i.startswith('DAYS')]
day_cols
```

```bash
['DAYS_BIRTH',
 'DAYS_EMPLOYED',
 'DAYS_REGISTRATION',
 'DAYS_ID_PUBLISH',
 'DAYS_LAST_PHONE_CHANGE']
```

```python
df1[day_cols]= abs(df1[day_cols])
print(df1['DAYS_BIRTH'].unique())
print(df1['DAYS_EMPLOYED'].unique())
print(df1['DAYS_REGISTRATION'].unique())
print(df1['DAYS_ID_PUBLISH'].unique())
print(df1['DAYS_LAST_PHONE_CHANGE'].unique())
```

```bash
[ 9461. 16765. 19046. ... 7951. 7857. 25061.]
[ 637. 1188. 225. ... 12971. 11084. 8694.]
[ 3648. 1186. 4260. ... 16396. 14558. 14798.]
[2120. 291. 2531. ... 6194. 5854. 6211.]
[1134. 828. 815. ... 3988. 3899. 3538.]
```

```python
df1['FLAG_OWN_CAR'] = np.where(df1['FLAG_OWN_CAR']=='Y', 1 , 0)
df1['FLAG_OWN_REALTY'] = np.where(df1['FLAG_OWN_REALTY']=='Y', 1 , 0)
df1[['FLAG_OWN_CAR','FLAG_OWN_REALTY']].head()
```

```bash
FLAG_OWN_CAR FLAG_OWN_REALTY
0 0 1
1 0 0
2 1 1
3 0 1
4 0 1
```

```python
df1.CODE_GENDER.value_counts()
df1.loc[df1.CODE_GENDER == 'XNA','CODE_GENDER'] = 'F'
df1.CODE_GENDER.value_counts()
```

```bash
F 202452
M 105059
```

```python
df1.ORGANIZATION_TYPE.value_counts().head()
```

```bash
Business Entity Type 3 67992
XNA 55374
Self-employed 38412
Other 16683
Medicine 11193
```

```python
df1[['ORGANIZATION_TYPE','NAME_INCOME_TYPE']].head(30)
df1.NAME_INCOME_TYPE.value_counts() #Check the counts for each in NAME_INCOME_TYPE
```

```bash
Working 158774
Commercial associate 71617
Pensioner 55362
State servant 21703
Unemployed 22
Student 18
Businessman 10
Maternity leave 5
```

```python
df1['ORGANIZATION_TYPE'] = df1['ORGANIZATION_TYPE'].replace('XNA', 'Pensioner')
df1['OCCUPATION_TYPE'].fillna('Pensioner' , inplace = True)
df1['AMT_INCOME_TYPE'] = pd.qcut(df1.AMT_INCOME_TOTAL, q=[0, 0.2, 0.5, 0.8, 0.95, 1], labels=['VERY_LOW', 'LOW', "MEDIUM", 'HIGH', 'VERY_HIGH'])
df1['AMT_INCOME_TYPE'].head(11)
```

```bash
0 MEDIUM
1 HIGH
2 VERY_LOW
3 LOW
4 LOW
5 VERY_LOW
6 MEDIUM
7 VERY_HIGH
8 LOW
9 LOW
10 LOW
Name: AMT_INCOME_TYPE, dtype: category
```

```python
df1['AMT_CREDIT_TYPE'] = pd.qcut(df1.AMT_CREDIT, q=[0, 0.2, 0.5, 0.8, 0.95, 1], labels=['VERY_LOW', 'LOW', "MEDIUM", 'HIGH', 'VERY_HIGH'])
df1['AMT_CREDIT_TYPE'].head(11)
```

```bash
0 LOW
1 HIGH
2 VERY_LOW
3 LOW
4 LOW
5 LOW
6 VERY_HIGH
7 VERY_HIGH
8 HIGH
9 LOW
10 MEDIUM
Name: AMT_CREDIT_TYPE, dtype: category
```

```python
df1['DAYS_BIRTH']= (df1['DAYS_BIRTH']/365).astype(int) # Converting
df1['DAYS_BIRTH'].unique()
array([25, 45, 52, 54, 46, 37, 51, 55, 39, 27, 36, 38, 23, 35, 26, 48, 31,
 50, 40, 30, 68, 43, 28, 41, 32, 33, 47, 57, 65, 44, 64, 21, 59, 49,
 56, 62, 53, 42, 29, 67, 63, 61, 58, 60, 34, 22, 24, 66, 69, 20])

df1['AGE_GROUP']=pd.cut(df1['DAYS_BIRTH'],
 bins=[19,25,35,60,100], labels=['Very_Young','Young', 'Middle_Age', 'Senior_Citizen']) #Binning
df1[['DAYS_BIRTH','AGE_GROUP']].head()
DAYS_BIRTH AGE_GROUP
0 25 Very_Young
1 45 Middle_Age
2 52 Middle_Age
3 52 Middle_Age
4 54 Middle_Age

df1.info()

df1['NAME_CONTRACT_TYPE'] = df1['NAME_CONTRACT_TYPE'].astype('category')
df1['CODE_GENDER'] = df1['CODE_GENDER'].astype('category')
df1['NAME_TYPE_SUITE'] = df1['NAME_TYPE_SUITE'].astype('category')
df1['NAME_INCOME_TYPE'] = df1['NAME_INCOME_TYPE'].astype('category')
df1['NAME_EDUCATION_TYPE'] = df1['NAME_EDUCATION_TYPE'].astype('category')
df1['NAME_FAMILY_STATUS'] = df1['NAME_FAMILY_STATUS'].astype('category')
df1['NAME_HOUSING_TYPE'] = df1['NAME_HOUSING_TYPE'].astype('category')
df1['OCCUPATION_TYPE'] = df1['OCCUPATION_TYPE'].astype('category')
df1['WEEKDAY_APPR_PROCESS_START'] = df1['WEEKDAY_APPR_PROCESS_START'].astype('category')
df1['ORGANIZATION_TYPE'] = df1['ORGANIZATION_TYPE'].astype('category')
unwanted=['FLAG_MOBIL', 'FLAG_EMP_PHONE', 'FLAG_WORK_PHONE', 'FLAG_CONT_MOBILE',
 'FLAG_PHONE', 'FLAG_EMAIL','REGION_RATING_CLIENT','REGION_RATING_CLIENT_W_CITY','FLAG_EMAIL', 'REGION_RATING_CLIENT',
 'REGION_RATING_CLIENT_W_CITY', 'FLAG_DOCUMENT_2', 'FLAG_DOCUMENT_3','FLAG_DOCUMENT_4', 'FLAG_DOCUMENT_5', 'FLAG_DOCUMENT_6',
 'FLAG_DOCUMENT_7', 'FLAG_DOCUMENT_8', 'FLAG_DOCUMENT_9','FLAG_DOCUMENT_10', 'FLAG_DOCUMENT_11', 'FLAG_DOCUMENT_12',
 'FLAG_DOCUMENT_13', 'FLAG_DOCUMENT_14', 'FLAG_DOCUMENT_15','FLAG_DOCUMENT_16', 'FLAG_DOCUMENT_17', 'FLAG_DOCUMENT_18',
 'FLAG_DOCUMENT_19', 'FLAG_DOCUMENT_20', 'FLAG_DOCUMENT_21']

df1.drop(labels=unwanted,axis=1,inplace=True)

numerical_col = df1.select_dtypes(include='number').columns
len(numerical_col)
```

```bash
35
```

```python
fig , axes = plt.subplots(nrows=7, ncols=5, constrained_layout=True) # Plot Configuration
fig.subplots_adjust(left= 0, bottom=0, right=3, top=12, wspace=0.09, hspace=0.3)


for ax, column in zip(axes.flatten(),numerical_col): #Using For loop

 sns.boxplot(df1[column],ax=ax) #Ploting
```

<img>

## Reference

[1] Exploratory Data Analysis Use Sentiment Analysis [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[2] Exploratory Data Analysis (EDA) -- Credit Card Fraud Detection
Case Study https://www.analyticsvidhya.com/blog/2022/03/exploratory-data-analysis-eda-credit-card-fraud-detection-case-study/

[3] Exploratory Data Analysis (EDA) in Supply Chain [[https://www.cobuildlab.com/blog/ai]{.underline}](https://www.cobuildlab.com/blog/ai)

[4] Optimization and Its Importance in the Supply Chain [[https://www.cobuildlab.com/blog/ai]{.underline}](https://www.cobuildlab.com/blog/ai)

Before you start data analysis or run your data through a machine learning algorithm, you must clean your data and make sure it is in a suitable form. Further, it is essential to know any recurring patterns and significant correlations that might be present in your data. The process of getting to know your data in depth is called Exploratory Data Analysis. So, Exploratory Data Analysis is an integral part of working with data. In this tutorial titled ‘All the ins and outs of exploratory data analysis,’ you will explore how to perform exploratory data analysis on different data types.
Exploratory Data Analysis is a data analytics process to understand the data in depth and learn the different data characteristics, often with visual means. This allows you to get a better feel of your data and find useful patterns in it.       
Exploratory Data Analysis (EDA) is an approach to extract the information enfolded in the data and summarize the main characteristics of the data. It is considered to be a crucial step in any data science project (in Figure 1 it is the second step after problem understanding in CRISP methodology). Most people underestimate the importance of data preparation and data exploration. EDA is essential for a well-defined and structured data science project and it should be performed before any statistical or machine learning modeling phase. In this post, we focus on how EDA is done, and which technologies do we use at Camelot for data exploration and data visualization.

Exploratory Data Analysis use in The Bank Marketing

# Introduction

Exploratory Data Analysis refers to the critical process of performing
initial investigations on data so as to discover patterns, to spot
anomalies, to test hypothesis and to check assumptions with the help of
summary statistics and graphical representations. Exploratory Data
Analysis is a data analytics process to understand the data in depth and
learn the different data characteristics, often with visual means. This
allows you to get a better feel of your data and find useful patterns in
it. So, It is a good practice to understand the data first and try to
gather as many insights from it. EDA is all about making sense of data
in hand, before getting them dirty with it.

Think of it as fishing for information about an app before you actually
go ahead and download. We need to know how many people have downloaded
it, the rating and reviews so that we can take the next step. The tool
of choice for exploratory data analysis is usually Pandas, which is an
open-source Python library for data manipulation and drum roll, data
analysis. Seaborn is a Python data visualization library.

# Code Study -- Clients would subscribe to a term deposit

To share my understanding of the concept and techniques I know, I'll
take an example of Bank Marketing data set which is available on UCI
Machine Learning Repository and try to catch hold of as many insights
from the data set using EDA. To starts with, I imported necessary
libraries (for this example pandas, numpy, matplotlib and seaborn) and
loaded the data set. The data set contains data about a marketing
campaign conducted by a Portuguese banking institution to determine if
clients would subscribe to a term deposit.

Code [3]

```python
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns

bank_data=pd.read_csv("bank.csv")
bank_data.head(5)
bank_data.shape
(4521, 17)

bank_data.info()
```

```bash
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4521 entries, 0 to 4520
Data columns (total 17 columns):

 # Column Non-Null Count Dtype
--- ------ -------------- -----
 0 age 4521 non-null int64
 1 job 4521 non-null object
 2 marital 4521 non-null object
 3 education 4521 non-null object
 4 default 4521 non-null object
 5 balance 4521 non-null int64
 6 housing 4521 non-null object
 7 loan 4521 non-null object
 8 contact 4521 non-null object
 9 day 4521 non-null int64
 10 month 4521 non-null object
 11 duration 4521 non-null int64
 12 campaign 4521 non-null int64
 13 pdays 4521 non-null int64
 14 previous 4521 non-null int64
 15 poutcome 4521 non-null object
 16 y 4521 non-null object
dtypes: int64(7), object(10)
memory usage: 600.6+ KB
```

```python
bank_data.describe()
```

| age | balance | day | duration | campaign | pdays | previous | |
| ----- | ----------- | ------------ | ----------- | ----------- | ----------- | ----------- | --- |
| count | 4521.000000 | 4521.000000 | 4521.000000 | 4521.000000 | 4521.000000 | 4521.000000 | |
| mean | 41.170095 | 1422.657819 | 15.915284 | 263.961292 | 2.793630 | 39.766645 | |
| std | 10.576211 | 3009.638142 | 8.247667 | 259.856633 | 3.109807 | 100.121124 | |
| min | 19.000000 | -3313.000000 | 1.000000 | 4.000000 | 1.000000 | -1.000000 | |
| 25% | 33.000000 | 69.000000 | 9.000000 | 104.000000 | 1.000000 | -1.000000 | |
| 50% | 39.000000 | 444.000000 | 16.000000 | 185.000000 | 2.000000 | -1.000000 | |
| 75% | 49.000000 | 1480.000000 | 21.000000 | 329.000000 | 3.000000 | -1.000000 | |
| max | 87.000000 | 71188.000000 | 31.000000 | 3025.000000 | 50.000000 | 871.000000 | |

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.barplot(x=bank_data['y'], y=bank_data['age'])
```

<img>

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.barplot(x=bank_data['y'], y=bank_data['balance'])
```

<img>

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.barplot(x=bank_data['y'], y=bank_data['campaign'])
```

<img>

```python
s=(bank_data.dtypes == 'object')
objectcols=list(s[s].index)
bank_data_object=bank_data[objectcols]
bank_data_object.head(5)

sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(bank_data['job'])
```

<img>

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(bank_data['marital'])
```

<img>

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(bank_data['education'])
```

<img>

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(bank_data['housing'])
```

<img>

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(bank_data['loan'])
```

<img>

```python
plt.figure(figsize=(14,7))
cor=bank_data.corr()
sns.heatmap(cor, annot=True)
plt.show()
```

<img>

```python
(bank_data['y']=='yes').sum()
```

```bash
521
```

```python
sns.set_style('whitegrid')
plt.figure(figsize=(14,7))
sns.countplot(bank_data['y'])
```

<img>

## Reference

[1] Exploratory Data Analysis Use Sentiment Analysis Exploratory Data
Analysis use in The Bank Marketing [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

[2] Exploratory Data Analysis (EDA) -- Credit Card Fraud Detection
Case Study https://www.analyticsvidhya.com/blog/2022/03/exploratory-data-analysis-eda-credit-card-fraud-detection-case-study/

[3] Exploratory Data Analysis (EDA) in Supply Chain Exploratory data analysis on the bank marketing data set with Pandas and Seaborn https://medium.com/swlh/exploratory-data-analysis-on-the-bank-marketing-data-set-with-pandas-and-seaborn-72e5c05e0076

[4] Optimization and Its Importance in the Supply Chain
[https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

Exploratory Data Analysis refers to the critical process of performing initial investigations on data so as to discover patterns, to spot anomalies, to test hypothesis and to check assumptions with the help of summary statistics and graphical representations. Exploratory Data Analysis is a data analytics process to understand the data in depth and learn the different data characteristics, often with visual means. This allows you to get a better feel of your data and find useful patterns in it.  So, It is a good practice to understand the data first and try to gather as many insights from it. EDA is all about making sense of data in hand, before getting them dirty with it.
Think of it as fishing for information about an app before you actually go ahead and download. We need to know how many people have downloaded it, the rating and reviews so that we can take the next step. The tool of choice for exploratory data analysis is usually Pandas, which is an open-source Python library for data manipulation and drum roll, data analysis. Seaborn is a Python data visualization library.

Machine Learning and Artificial Intelligence technologies, they have the opportunity to analyze data that originates beyond the bank office. Financial companies collect and store more and more user data in order to revise their strategies, improve user experience, prevent fraud, and mitigate risks. In this article, we will talk about how Artificial Intelligence and Machine Learning are used as well as the benefits and risks of these solutions.

Bank Marketing with Machine Learning

The world is getting “smarter” every day, and to keep up with consumer expectations, companies are increasingly using machine learning algorithms to make things easier. You can see them in use in end-user devices (through face recognition for unlocking smartphones) or for detecting credit card fraud (like triggering alerts for unusual purchases).
Within Artificial Intelligence (AI) and Machine Learning (ML), there are two basic approaches: supervised learning and unsupervised learning. The main difference is one uses labeled data to help predict outcomes, while the other does not. However, there are some nuances between the two approaches, and key areas in which one outperforms the other. This post will clarify the differences so you can choose the best approach for your situation.

Bank Marketing with Machine Learning - Part II

The supply chain is the set of processes that takes place from when an order is placed until it has been delivered and paid for. This chain is made up of different suppliers and companies that supply the next level from the origin of the product to the delivery of the final product. Logistics has evolved throughout history to become "the part of the process that manages the supply chain that is responsible for the planning, implementation and efficient control of the flow of materials or finished products and services, as well as of the flow of related information, from the origin to its destination, meeting the needs of the client and with the minimum operating cost” (Center of Logistics). The first level of the supply chain is the supply of it, the suppliers are in charge of delivering the raw material through the process called purchase. So, when a company starts a new project, the first step that must be carried out is the selection of suppliers. This process stands out for being a fundamental step in logistics and supply chain management since it affects the following links in the chain as well as the quality of the final product and therefore customer satisfaction, which, as stated above, is the main purpose of logistics. Today, the high number of suppliers that can be found for the same component and the great variety of criteria and aspects that can be taken into account for their evaluation makes the task grow exponentially more difficult. In addition, there are not a few cases in which the criteria contain a subjective character, generating significant differences in the result of the evaluation. It is true that there is no globally accepted methodology for the selection of suppliers. This is because companies have different needs, they seek to satisfy clients with different approaches and therefore each case of selection and evaluation is unique. However, there are several researchers who propose different solutions to this problem. 

Multi-criteria Decision Analysis Intelligent for Supply Chain

Optimization plays an essential role in many scientific areas, such as engineering, data analytics, and deep learning. These fields are fast-growing and their concepts are employed for various purposes, for instance gaining insights from large data sets or fitting accurate prediction models. Whenever an algorithm has to handle a significantly large amount of data, and efficient implementation in a suitable programming language is important. Python has become the programming language of choice for the above-mentioned research areas over the last few years, not only because it is easy to use but also because good community support exists. Python is a high-level, cross-platform, and interpreted programming language that focuses on code readability. A large number of high-quality libraries are available and support for any kind of scientific computation is ensured. These characteristics make Python an appropriate tool for many research and industry projects where the investigations can be rather complex.

Optimization MultiObjectives

# Introduction

So, there is something that is called genetic algorithm. That is
computerized abstraction of evolution that is used for optimizing
mathematical functions. NGSA(Non-dominated Sorting Genetic Algorithm) is
extension of this where you could make a multi-criteria optimization.
Think of it as a generalization of genetic algorithm. For example, you
could simultaneously optimize ship capacity and maximum velocity by its
structural characteristics. So,what NSGA, in essence, adds extra to
plain genetic algorithm, is this operator of domination which makes it
possible to compare duos ( (1,5) ) or triplets ( (1, 7, 8) ) etc. It
orders solutions in something called fronts, sets where you could tell
one set is better than the other, which is not trivial when you have
more than one criteria. Therefore, making hierarchy of sets of
solutions. Notice that in one criteria problem you could do this just by
putting operators of more, less or equal between solutions. Also notice
that NSGA can work on the one-criteria problems.

The algorithm follows the general outline of a genetic algorithm with a
modified mating and survival selection. In NSGA-II, first, individuals
are selected frontwise. By doing so, there will be the situation where a
front needs to be split because not all individuals are allowed to
survive. In this splitting front, solutions are selected based on
crowding distance.

<img>

The crowding distance is the Manhatten Distance in the objective space.
However, the extreme points are desired to be kept every generation and,
therefore, get assigned a crowding distance of infinity.

<img>

Furthermore, to increase some selection pressure, NSGA-II uses a binary
tournament mating selection. Each individual is first compared by rank
and then crowding distance. There is also a variant in the original C
code where instead of using the rank, the domination criterium between
two solutions is used.

## Types NSGA

Computer scientists made 3 versions of this algorithm called NSGA1
(NSGA), NSGA2 and NSGA3. From this 3 algorithms, I think that NSGA2 is
the best one, and in a study of these algorithms on university
(University of Zagreb) we really got best results with NSGA2 on standard
functions made for testing algorithms for multi-criteria optimizations
(but as far as I know not with verified statistical significance).

<img>

In NSGA1 authors tried to simulate evolution by including stochasticity
in the structure of algorithm. That turned out to be the worst. In
NSGA2, I think that authors tried to be as natural as possible. To just
abstract out the evolution, which turned out to be the best. The NSGA1
failed, I think, because stochasticity is already embedded in evolution,
you don't need to add it extra. In NSGA3 authors tried to make a
mathematical model of evolution, which turned out second. Basically, I
think it was supposed to happen, because they were trying to make
evolution by mathematical "chemistry". And they already had NSGA2 which
was pure evolution. You could maybe make the same parallel between NSGA2
and NSGA3 and between Superman and Bizarro from DC Universe. :) Of
course, I think you could maybe make a more perfect mathematical model
of evolution and thus making NSGA3 more performance similar to NSGA2. By
mathematical model of evolution in NSGA3 I think that they were making
lines and triangles to abstract out evolution, and in NSGA2 they just
put evolution's philosophy into algorithm, without this extra step.

Perhaps if one could make NSGA3.1 to have same performance as NSGA2, one
could tell that they have perfectly modeled evolution with mathematical
"chemistry".

## Code de Example

```python
from pymoo.algorithms.moo.nsga2 import NSGA2
from pymoo.problems import get_problem
from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter

problem = get_problem("zdt1")

algorithm = NSGA2(pop_size=100)

res = minimize(problem,
 algorithm,
 ('n_gen', 200),
 seed=1,
 verbose=False)

plot = Scatter()
plot.add(problem.pareto_front(), plot_type="line", color="black", alpha=0.7)
plot.add(res.F, facecolor="none", edgecolor="red")
plot.show()
```

<img>

```python
from pymoo.algorithms.moo.nsga2 import NSGA2
from pymoo.problems import get_problem
from pymoo.operators.crossover.pntx import TwoPointCrossover
from pymoo.operators.mutation.bitflip import BitflipMutation
from pymoo.operators.sampling.rnd import BinaryRandomSampling
from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter

problem = get_problem("zdt5")

algorithm = NSGA2(pop_size=100,
 sampling=BinaryRandomSampling(),
 crossover=TwoPointCrossover(),
 mutation=BitflipMutation(),
 eliminate_duplicates=True)

res = minimize(problem,
 algorithm,
 ('n_gen', 500),
 seed=1,
 verbose=False)

Scatter().add(res.F).show()
```

<img>

## Reference

\[1\] Optimization Multiobjective [https://ksra.eu/pymoo-multi-objective-optimization-in-pyhon/](https://ksra.eu/pymoo-multi-objective-optimization-in-pyhon/)

\[2\] Optimization Multiobjective Part I [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

\[3\] Optimization and Its Importance in the Supply Chain [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

So, there is something that is called genetic algorithm. That is
computerized abstraction of evolution that is used for optimizing
mathematical functions. NGSA(Non-dominated Sorting Genetic Algorithm) is
extension of this where you could make a multi-criteria optimization.
Think of it as a generalization of genetic algorithm. For example, you
could simultaneously optimize ship capacity and maximum velocity by its
structural characteristics. So,what NSGA, in essence, adds extra to
plain genetic algorithm, is this operator of domination which makes it
possible to compare duos ( (1,5) ) or triplets ( (1, 7, 8) ) etc. It
orders solutions in something called fronts, sets where you could tell
one set is better than the other, which is not trivial when you have
more than one criteria. Therefore, making hierarchy of sets of
solutions. Notice that in one criteria problem you could do this just by
putting operators of more, less or equal between solutions. Also notice
that NSGA can work on the one-criteria problems.

Optimization MultiObjectives - NSGA

The manufacturing industry has faced growing challenges in recent years. Technologies such as Cloud computing, Cyber-physical systems, Internet of Things, blockchain, edge computing are the terminologies used to describe Industry 4.0. So, The Manufacturers have benefitted from data‐driven innovations for demand planning and logistics management of their supply chains, but tracking production across entire processes and managing the supply chain as an integrated platform is still an urgent need. Hence, there is a demand to further explore the futuristic technologies for intelligent manufacturing and supply chain management since the Supply Chain is one of the most sensitive components of a business and the need for proper synchronization and analytics is key because the possibilities of making mistakes are significantly high: anything that goes out of control, however simple, can cause a disruption in the supply chain, thus making the business to lose a lot of money. When it comes to strategic and operational parts of the Supply Chain Management, the need for resolving issues on-the-go to keep the chain continuous is very important or every step in the continuation can become a painful point which will block the resources in the wrong places. This is the reason why Artificial intelligence (AI) and Machine Learning [1] now play a leading role in enhancing the quality of the manufacturing process. With the use of machine learning in the supply chain, not only is the operation becoming easier but the services also get improved. So, Artificial intelligence and Machine learning [1]. In fact, the importance of analytics has been generally perceived and played a critical role in Supply Chain Management (SCM).

Supply Chain towards Smart Management

## **Introduction**

The process of finding optimal values for the specific parameters of a given system to fulfill all design requirements while considering the lowest possible cost is referred to as an optimization. Optimization problems can be found in all fields of science.

Conventional optimization algorithms (Deterministic algorithms) have some limitations such as:single-based solutions, converging to local optima,unknown search space issues. To overcome these limitations, many scholars and researchers have developed several metaheuristics to address complex/unsolved optimization problems. Example: Particle Swarm Optimization, Grey wolf optimization, Ant colony Optimization, Genetic Algorithms, Cuckoo search algorithm, etc.

So, In computational science, **particle swarm optimization** (PSO) is a computational method that optimizes a problem by iteratively trying to improve a candidate solution with regard to a given measure of quality. There are multiple ways that one can take to either minimize or maximize any function so that the optimal value can be found out. You can find several optimisation solutions on the internet but in the end, no one solution is the best for all. Everyone has its own advantage and disadvantages. The one that we are going to discuss here is the PSO or the Particle Swarm Optimization. You can also say that it is a field of optimisation which belongs to the nature-inspired way of computing. It is a type of algorithm which searches for the optimal solution in a definite space in a straightforward manner.

The solution works by the logic where a candidate solution is worked upon to make it better by a specific measure of quality and then the operation is performed iteratively until a better solution is not available and thus reaching an optimal state itself. The language here will be Python and we will see a hands-on implementation of it using a python package "PySwarms". We will cover the following topics here :

1. PSO: Particle Swarm Optimization

2. The inner workings

3. Variants or types of PSO

4. Implementing PSO with PySwarms

## **What is Particle Swarm Optimization (PSO)?**

In the 1990s there were a number of studies conducted on the topic of the social behavior of animal clusters. These research studies concluded with one of the learnings being that some animal groups such as birds or fishes are able to pass on their knowledge or communicate with others in the group. This sometimes helped them survive attacks or get an advantage over other animal groups when evading predators. Based on all of this research, Kennedy and Eberhart proposed a PSO algorithm. In 1995, this PSO algorithm was proposed as a metaheuristic algorithm. This algorithm worked to optimise nonlinear continuous functions based on the concept of animal swarm intelligence exhibited in flocks and shoals.

<img>

####

#### **Convergence of Particle Swarm Optimization**

In a PSO convergence, it does not matter how a swarm will operate but the convergence to a specific local optimum when various personal bests which let's say P or the best-known position of the swarm or also known as G will reach a local optimum for the problem. The ability which a PSO algorithm has so that it can exploit and explore can get affected by the topology structure. So for different structures, the algorithm can converge in a different manner because of the shape of the topology which is generally either a local ring or a global star. The topology structure defines the factors like searching, information sharing, speed, and even the direction in which each particle should follow.

<img>

A PSO which has a global star structure where all particles are connected with each other has one benefit of the shortest average distance but a local ring structure where one particle is connected with the two nearest ones has the highest average distance in the whole swarm as shown in the above image. The figure on the left is the global star structure whereas the left one is the local ring structure. Each group consists of 16 particles and you can see why the average distance differs for both cases.

#### **Adaptive mechanism**

In an Adaptive mechanism, one can implement it with the required trade-off between convergence and divergence or exploitation and exploration. An APSO or Adaptive Particle Swarm Optimisation can outperform a regular PSO. With a faster convergence time, an APSO could also execute any global search across the whole search space.

## **Variants of Particle Swarm Optimization**

PSO algorithms can be of different types, even simple ones. The particles and velocities can be initiated in different ways. Update the Swarm and then set values for Pi and G and so forth.

#### **Gradient Particle Swarm Optimization**

We can construct gradient-based PSOs by combining the efficiency of the PSO at exploring many local minimums with the gradient-based local search algorithms. This helps us to accurately calculate a local minimum. The PSO algorithm can be used for gradient-based PSO also where several local minima are discovered along with a deep basin of attraction for a deep local minimum. The deep minimum can then be confirmed using the local gradient-based search techniques.

<img>

#### **Hybrid Particle Swarm Optimization**

In order to increase and make the optimization process better, newer and more advanced types of PSO variations are being tested and used and are an ongoing field of study. A Hybrid PSO is where a normal PSO is combined with another optimization technique which helps to make it better. One example is Biogeography based optimization while also implementing an effective learning mechanism that will improve the existing PSO and enhance it.

<img>

## **Implementing Particle Swarm Optimization**

PySwarms is a python based tool that helps with swarm optimisation. Researchers, practitioners, and students alike use this tool in order to apply the PSO algorithm using a high-level interface. PySwarms is the best tool to integrate swarm optimisation with basic optimization. So, thiis tool allows you to implement and use a number of many-particle swarm optimisation techniques. It's also extremely user-friendly and adaptable to different projects.

```python
# Import all the modules
import numpy as np
# Importing PySwarms
import pyswarms as ps
from pyswarms.utils.functions import single_obj as fx
import matplotlib.pyplot as plt
from pyswarms.utils.plotters import plot_contour, plot_surface
from pyswarms.utils.plotters.formatters import Designer
from pyswarms.utils.plotters.formatters import Mesher

# Set-up hyperparameters
options = {'c1': 0.5, 'c2': 0.3, 'w':0.9}
# Call instance of PSO
optimizer = ps.single.GlobalBestPSO(n_particles=10, dimensions=2, options=options)
# Perform optimization
cost, pos = optimizer.optimize(fx.sphere, iters=1000)
```

```bash
2022-10-08 16:52:51,220 - pyswarms.single.global_best - INFO - Optimize for 1000 iters with {'c1': 0.5, 'c2': 0.3, 'w': 0.9}
pyswarms.single.global_best: 100%|██████████|1000/1000, best_cost=2.72e-44
2022-10-08 16:52:52,796 - pyswarms.single.global_best - INFO - Optimization finished | best cost: 2.7227863224794666e-44, best pos: [-3.55167758e-23 1.61141000e-22]
```

```python
m = Mesher(func=fx.sphere)
animation = plot_contour(pos_history=optimizer.pos_history,mesher=m,mark=(0,0))
```

<img>

```python
# The preprocessing
pos_history_3d = m.compute_history_3d(optimizer.pos_history)
# Adjusting the figure
d = Designer(limits=[(-1,1), (-1,1), (-0.1,1)], label=['x-axis', 'y-axis', 'z-axis'])
animation3d = plot_surface(pos_history=pos_history_3d, # The cost_history that we computed
 mesher=m, designer=d, # Various Customizations
 mark=(0,0,0)) # Mark the minima
```

<img>

## **Reference**

\[1\] Multi-objective Optimization in Python [https://pymoo.org/](https://pymoo.org/)

\[2\] Optimization MultiObjectives - NSGA <https://www.cobuildlab.com/blog/ai>

\[3\] Optimization Multiobjective Part I [https://www.cobuildlab.com/blog/ai](https://www.cobuildlab.com/blog/ai)

\[4\] Implementing Particle Swarm Optimization using Python [Particle Swarm Optimization with Python - Analytics
Vidhya](https://www.analyticsvidhya.com/blog/2021/11/implementing-a-particle-swarm-optimization-with-python/)

The process of finding optimal values for the specific parameters of a given system to fulfill all design requirements while considering the lowest possible cost is referred to as an optimization. Optimization problems can be found in all fields of science.

Particle Swarm Optimization using Python

Regression analysis is a statistical method to model the relationship between a dependent (target) and independent (predictor) variables with one or more independent variables. More specifically, Regression analysis helps us to understand how the value of the dependent variable is changing corresponding to an independent variable when other independent variables are held fixed. It predicts continuous/real values such as **temperature, age, salary, price,** etc.

### Implementation of Linear Regression in Machine Learning

Linear regression is employed in varied ways in which a number of them are listed as:

- Sales prognostication
- Risk analysis
- Housing applications
- Finance applications

The process used for implementing the statistical regression whereas exploitation it in many ways in which some are mentioned below:

- Loading the data
- Exploring the data
- Slicing the data
- Train and split data
- Generate the model
- Evacuate the accuracy

### Advantages and Disadvantages of Linear Regression

There are several advantages and disadvantages of linear regression:

#### Advantages:

- Linear regression performs well when the data set is linearly separable. We can use it to find the nature of the relationship between the variables.
- It is easier to implement, interpret and very efficient to train.
- It is prone to over-fitting but it can be easily avoided using some dimensionality reduction techniques, regularization techniques, and cross-validation.
- It has the extrapolation beyond the specific data set.

#### Disadvantages:

- Linear assumption: It assumes that the relationship between the input and the output is linear.
- Remove noise: It assumes that the input and the output variables are not noisy.
- Remove collinearity: It will over-fit the data when we have highly correlated input variables.
- Gaussian distributions: It will create a lot of reliable predictions if the input and output variables have a Gaussian distribution.
- Resize inputs: It usually creates a lot of reliable predictions if we tend to use resize input variables exploitation standardization or social control.
- Susceptible to outliers: It is very sensitive to outliers. So, the outliers need to be removed before applying the linear regression to the data set.

## Why do we use Regression Analysis?

As mentioned above, Regression analysis helps in the prediction of a continuous variable. There are various scenarios in the real world where we need some future predictions such as weather condition, sales prediction, marketing trends, etc., for such case we need some technology which can make predictions more accurately. So for such case we need Regression analysis which is a statistical method and used in machine learning and data science. Below are some other reasons for using Regression analysis:

- Regression estimates the relationship between the target and the independent variable.
- It is used to find the trends in data.
- It helps to predict real continuous values.
- By performing the regression, we can confidently determine the most important factor, the least important factor, and how each factor is affecting the other factors.

## Example - Code in TensorFlow

We'll be learning how to get started using TensorFlow to create and train a model when the result we need is a single number, like maybel a price or a probability or in this case miles per gallon. So we want our model to look at an example from our data and a predict a number. Specifically today we're going to look at a training model to tell us the miles per gallon of cars from the 1970s by looking at data, like their weight, the number of cylinders, the horsepower.

Miles per gallon is a single number. We're going to use Keras, it's a high level API for deep learning and it's very user friendly. So, in a regression problem, the aim is to predict the output of a continuous value, like a price or a probability. Contrast this with a classification problem, where the aim is to select a class from a list of classes (for example, where a picture contains an apple or an orange, recognizing which fruit is in the picture).

To do this, you will provide the models with a description of many automobiles from that time period. This description includes attributes like cylinders, displacement, horsepower, and weight.

```python
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

# Make NumPy printouts easier to read.
np.set_printoptions(precision=3, suppress=True)
import tensorflow as tf

from tensorflow import keras
from tensorflow.keras import layers

print(tf.__version__)

url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data'
column_names = ['MPG', 'Cylinders', 'Displacement', 'Horsepower', 'Weight',
 'Acceleration', 'Model Year', 'Origin']

raw_dataset = pd.read_csv(url, names=column_names,
 na_values='?', comment='\t',
 sep=' ', skipinitialspace=True)
dataset = raw_dataset.copy()
dataset.tail()
```

### Clean the data

The dataset contains a few unknown values:

```bash
dataset.isna().sum()
MPG 0
Cylinders 0
Displacement 0
Horsepower 6
Weight 0
Acceleration 0
Model Year 0
Origin 0
dtype: int64
```

Drop those rows to keep this initial tutorial simple:

```python
dataset = dataset.dropna()
```

The "Origin" column is categorical, not numeric. So the next step is to one-hot encode the values in the column with [pd.get_dummies](https://pandas.pydata.org/docs/reference/api/pandas.get_dummies.html).

Note: You can set up the tf.keras.Model to do this kind of transformation for you but that's beyond the scope of this tutorial. Check out the [Classify structured data using Keras preprocessing layers](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/structured_data/preprocessing_layers.ipynb) or [Load CSV data](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/load_data/csv.ipynb)
tutorials for examples.

```python
dataset['Origin'] = dataset['Origin'].map({1: 'USA', 2: 'Europe', 3: 'Japan'})

dataset = pd.get_dummies(dataset, columns=['Origin'], prefix='', prefix_sep='')
dataset.tail()
```

### Split the data into training and test sets

Now, split the dataset into a training set and a test set. You will use the test set in the final evaluation of your models.

```python
train_dataset = dataset.sample(frac=0.8, random_state=0)
test_dataset = dataset.drop(train_dataset.index)
```

### Inspect the data

Review the joint distribution of a few pairs of columns from the training set.

The top row suggests that the fuel efficiency (MPG) is a function of all the other parameters. The other rows indicate they are functions of each other.

```python
sns.pairplot(train_dataset[['MPG', 'Cylinders', 'Displacement', 'Weight']], diag_kind='kde')
```

<img>

Let's also check the overall statistics. Note how each feature covers a very different range:

```python
train_dataset.describe().transpose()
```

### Split features from labels

Separate the target value---the "label"---from the features. This label is the value that you will train the model to predict.

> Code

> Text

```python
train_features = train_dataset.copy()
test_features = test_dataset.copy()

train_labels = train_features.pop('MPG')
test_labels = test_features.pop('MPG')
```

## Normalization

In the table of statistics it's easy to see how different the ranges of
each feature are:

```python
train_dataset.describe().transpose()\[\[\'mean\', \'std\'\]\]
```

It is good practice to normalize features that use different scales and
ranges.

One reason this is important is because the features are multiplied by
the model weights. So, the scale of the outputs and the scale of the
gradients are affected by the scale of the inputs.

Although a model _might_ converge without feature normalization,
normalization makes training much more stable.

Note: There is no advantage to normalizing the one-hot features---it is done here for simplicity. For more details on how to use the preprocessing layers, refer to the [Working with preprocessinglayers](https://www.tensorflow.org/guide/keras/preprocessing_layers) guide and the [Classify structured data using Keras preprocessing layers](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/structured_data/preprocessing_layers.ipynb) tutorial.

### The Normalization layer

The tf.keras.layers.Normalization is a clean and simple way to add
feature normalization into your model.

The first step is to create the layer:

```python
normalizer = tf.keras.layers.Normalization(axis=-1)
```

Then, fit the state of the preprocessing layer to the data by calling Normalization.adapt:

```python
normalizer.adapt(np.array(train_features))
```

Calculate the mean and variance, and store them in the layer:

```python
print(normalizer.mean.numpy())

[[ 5.478 195.318 104.869 2990.252 15.559 75.898 0.178 0.197
 0.624]]
```

When the layer is called, it returns the input data, with each feature independently normalized:

```python
first = np.array(train_features\[:1\])

with np.printoptions(precision=2, suppress=True):

print(\'First example:\', first)

print()

print(\'Normalized:\', normalizer(first).numpy())
```

```bash
First example: [[ 4. 90. 75. 2125. 14.5 74. 0. 0. 1. ]]
Normalized: [[-0.87 -1.01 -0.79 -1.03 -0.38 -0.52 -0.47 -0.5 0.78]]
```

### Linear regression with one variable

Begin with a single-variable linear regression to predict 'MPG' from 'Horsepower'. Training a model with tf.keras typically starts by defining the model architecture. Use a tf.keras.Sequential model, which [represents a sequence of steps](https://www.tensorflow.org/guide/keras/sequential_model).

There are two steps in your single-variable linear regression model:

- Normalize the 'Horsepower' input features using the tf.keras.layers.Normalization preprocessing layer.
- Apply a linear transformation (
- y=mx+b
- ) to produce 1 output using a linear layer (tf.keras.layers.Dense).

The number of _inputs_ can either be set by the input_shape argument, or automatically when the model is run for the first time.

First, create a NumPy array made of the 'Horsepower' features. Then, instantiate the tf.keras.layers.Normalization and fit its state to the horsepower data:

```python
horsepower = np.array(train_features\[\'Horsepower\'\])

horsepower_normalizer = layers.Normalization(input_shape=\[1,\],
axis=None)
horsepower_normalizer.adapt(horsepower)
```

Build the Keras Sequential model:

```python
horsepower_model = tf.keras.Sequential([
 horsepower_normalizer,
 layers.Dense(units=1)
])

horsepower_model.summary()
```

```bash
Model: "sequential"
_________________________________________________________________
 Layer (type) Output Shape Param #
=================================================================
 normalization_1 (Normalizat (None, 1) 3
 ion)

 dense (Dense) (None, 1) 2

=================================================================
Total params: 5
Trainable params: 2
Non-trainable params: 3
_________________________________________________________________
```

This model will predict 'MPG' from 'Horsepower'.

Run the untrained model on the first 10 'Horsepower' values. The output won't be good, but notice that it has the expected shape of (10,
1):

```python
horsepower_model.predict(horsepower\[:10\])
```

Once the model is built, configure the training procedure using the Keras Model.compile method. The most important arguments to compile are the loss and the optimizer, since these define what will be optimized (mean_absolute_error) and how (using the tf.keras.optimizers.Adam).

```python
horsepower_model.compile(
 optimizer=tf.keras.optimizers.Adam(learning_rate=0.1),
 loss='mean_absolute_error')
```

Use Keras Model.fit to execute the training for 100 epochs:

```python
%%time
history = horsepower_model.fit(
 train_features['Horsepower'],
 train_labels,
 epochs=100,
 # Suppress logging.
 verbose=0,
 # Calculate validation results on 20% of the training data.
 validation_split = 0.2)
```

```bash
CPU times: user 5.46 s, sys: 194 ms, total: 5.66 s
Wall time: 8.69 s
```

Visualize the model's training progress using the stats stored in the history object:

```python
hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
hist.tail()

def plot_loss(history):
 plt.plot(history.history['loss'], label='loss')
 plt.plot(history.history['val_loss'], label='val_loss')
 plt.ylim([0, 10])
 plt.xlabel('Epoch')
 plt.ylabel('Error [MPG]')
 plt.legend()
 plt.grid(True)
```

<img>

Collect the results on the test set for later:

```python
test_results = {}

test_results['horsepower_model'] = horsepower_model.evaluate(
 test_features['Horsepower'],
 test_labels, verbose=0)
```

Since this is a single variable regression, it's easy to view the model's predictions as a function of the input:

```python
x = tf.linspace(0.0, 250, 251)
y = horsepower_model.predict(x)
```

```bash
8/8 [==============================] - 0s 2ms/step
```

```python
def plot_horsepower(x, y):
 plt.scatter(train_features['Horsepower'], train_labels, label='Data')
 plt.plot(x, y, color='k', label='Predictions')
 plt.xlabel('Horsepower')
 plt.ylabel('MPG')
 plt.legend()
plot_horsepower(x, y)
```

<img>

Regression analysis is a statistical method to model the relationship between a dependent (target) and independent (predictor) variables with one or more independent variables. More specifically, Regression analysis helps us to understand how the value of the dependent variable is changing corresponding to an independent variable when other independent variables are held fixed. It predicts continuous/real values such as temperature, age, salary, price, etc.

Regression Analysis in Machine learning

In this articles, we'll be going over some of the main concepts in TensorFlow tool. We'll be looking at tensors, variables, and the current API

Machine Learning TensorFlow for SPY ETF

One of the most advanced models out there to forecast time series is the Long Short-Term Memory (LSTM) Neural Network.

PyTorch and LSTM to Predict Stock Prices 

<title-3 style=text-align:undefined;>Introduction</title-3>The use of artificial intelligence has been on the rise in many industries, as AIs assist us in many aspects of our lives. A new AI that is making the headlines now is the ChatGPT, which has been famed to help with coding trading strategies and indicators, as well as suggesting the direction of financial markets in the future. What do you know about ChatGPT trading strategies? So, ChatGPT trading strategies are strategies generated with the help of the ChatGPT AI that is currently taking the world by storm. The AI can help you create technical indicators, trading strategies based on specified indicators, and even trading algos, as well as offer sentiment analysis and prediction models that can help you make better trading decisions. So, In this paper, we answer some questions about ChatGP trading strategies. ChatGPT is a natural language processing (NLP) tool developed by OpenAI that allows users to generate human-like text based on a given prompt. It can be used for a variety of applications, including language translation, conversation simulation, and content creation. In this article, we will explore how ChatGPT can be used in the context of stock trading, specifically in the areas of analysis, strategy development, and risk management. At the end of the article, we make an example of a strategy made by ChatGPT for Amibroker. ChatGPT is an AI developed by OpenAI. It is designed to assist with tasks such as answering questions, giving information, and engaging in conversation. The AI is based on the GPT (Generative Pre-training Transformer) architecture and is trained on a massive dataset of human-generated answers. The model is able to generate human-like text in response to a given prompt and can be fine-tuned for specific tasks such as language translation, question answering, and text summarization. It was created to be helpful and provide an accurate resource for its users. The model uses a technique called “attention mechanism” to focus on specific parts of the input while generating the output. It also uses a technique called beam search to generate multiple hypotheses and then select the most likely one as the final output. ChatGPT has several advantages, including:<ul style=text-align:undefined;><li>Answer basic questions: ChatGPT can answer basic questions in many areas and can even write near-perfect codes for different trading strategies in accordance with your input for the problem you want to be solved.</li><li>Fine-tuning capabilities: ChatGPT can be fine-tuned to perform specific tasks, such as language translation or question answering, by training it on a smaller dataset of task-specific examples.</li><li>Large model size: It is based on a large model, which means it has a lot of parameters and can learn from a large amount of data, leading to better performance on many tasks.</li><li>Flexibility: The AI can be used for a variety of natural language processing tasks, such as conversation and language translation.</li><li>Beam search: With beam search, ChatGPT can generate multiple hypotheses, which improves the quality of the generated text.</li></ul>What are some of the top trading strategies used with ChatGPT?There are no specific strategies to use with ChatGPT; it is a multi-purpose AI. What you do with it depends on what you want. Having said that, here are some ways you can use the tool: <ul style=text-align:undefined;><li>Fundamental analysis: You can use ChatGPT to extract insights from financial news articles, social media posts, and other unstructured data sources. These insights can then be used in conjunction with other trading strategies to inform investment decisions.</li><li>Sentiment analysis: ChatGPT can be fine-tuned to perform sentiment analysis on financial news articles, social media posts, and other unstructured data sources. This can be used to identify positive or negative sentiment about a particular stock or market trend.</li><li>Technical analysis: You can ask ChatGPT to code any technical indicator or strategy you want for any trading platform, and it will write a near-perfect code for you. However, you need to be familiar with the language to know when you need to make the necessary modifications for the code to work properly. The wording of your input matters a lot in how ChatGPT understands the problem to provide the anticipated solution.</li></ul> Example 1: AnalysisOne way that ChatGPT can be used in stock trading is for analysis purposes. For example, let’s say that you are interested in investing in a particular company, but you want to get a better understanding of its financial health and future prospects. You could use ChatGPT to generate a summary of the company’s financial statements and key metrics, such as earnings per share, price-to-earnings ratio, and debt-to-equity ratio.To do this, you would first need to gather the necessary data and input it into ChatGPT. This could include the company’s income statement, balance sheet, and cash flow statement, as well as data on industry benchmarks and competitors. You would then provide ChatGPT with a prompt, such as “Please provide a summary of the company’s financial health and future prospects based on the data provided.” ChatGPT would then analyze the data and generate a summary that includes key highlights and insights. For example, it might highlight the company’s strong revenue growth or high profitability, but also note any potential risks or challenges it may face in the future. This summary could be used as a starting point for further research and analysis, helping you to make more informed investment decisions. Example 2: Strategy DevelopmentAnother way that ChatGPT can be used in stock trading is for strategy development. Let’s say that you want to create a trading strategy that focuses on identifying and capitalizing on trends in the market. You could use ChatGPT to help generate ideas for specific indicators and tactics to use in your strategy. To do this, you would provide ChatGPT with a prompt such as “Please suggest indicators and tactics for a trend-following trading strategy.” ChatGPT would then analyze data on historical market trends and generate a list of potential indicators and tactics to consider, such as moving averages, relative strength index, and breakout trades. This list could serve as a starting point for further research and testing, helping you to refine and optimize your strategy over time. Example 3: Risk ManagementIn addition to analysis and strategy development, ChatGPT can also be used for risk management in stock trading. For example, let’s say that you are concerned about the potential impact of market volatility on your portfolio. You could use ChatGPT to generate a list of potential risk mitigation strategies to consider.To do this, you would provide ChatGPT with a prompt such as “Please suggest strategies for managing risk in a volatile market.” ChatGPT would then analyze data on market conditions and generate a list of strategies such as diversification, stop-loss orders, and hedging.This list could help you to identify and implement risk management measures that are most appropriate for your portfolio and risk tolerance. <title-4 style=text-align:justify;>STEP-BY-STEP: USING CHATGPT TO WRITE SCRIPTS AND TRADING ALGORITHMS</title-4>To use ChatGPT to write options trading scripts, you will first need to access the model through an API or by using OpenAI’s GPT-3 Playground. Find that link here: <a href="https://beta.openai.com/playground" style=text-align:undefined;>https://beta.openai.com/playground</a>. You’ll also want to get comfortable with a brokerage that allows direct plug-ins of a script into their interface, so that you can fully automate your trading process and get the most out of the trading algorithm. Once you have access to the model and have become familiar with your brokerage of choice, you can input your desired trading strategy and parameters, and ChatGPT will generate code based on your inputs. Then, you can test the script using historical market data and make any necessary adjustments before using it in live trading. Let’s look at a few example scripts and trading algorithms that we’ve built to shine some light on the process. <title-4 style=text-align:justify;>QQQ STRATEGY: BULLISH WHEN THE RSI IS OVERSOLD</title-4>For a simple starter script, let’s look at the RSI rule. So, for the uninitiated, RSI — or the relative strength indicator — attempts to depict when a security is oversold or overbought, and by how much. A simple rule of thumb for reading RSI is that when the RSI is below 30, a security is oversold, and prone to bounce. Likewise, when the RSI is over 70, the security is overbought, and prone to take a pause. With that in mind, let’s make a rudimentary script that does just that, using QQQ as the target, and at-the-money options as the tool. Simple — a python script you could plug directly into a brokerage for auto-execution of trades when the RSI fits the parameters described. When this strategy was backtested on SPY using the 15 minute timeframe it carried a <a href="https://trendspider.com/blog/5-popular-day-trading-strategies-exposed/" style=text-align:undefined;>win rate of 66% according to TrendSpider</a> (See Fig.1) <title-4 style=text-align:justify;>SPY STRATEGY: MACD CROSSOVER WITH BOLLINGER BAND CONFIRMATION</title-4>The headline of this strategy may sound complicated, but it’s relatively simple. The MACD indicator — which stands for moving average convergence divergence, is essentially two different moving averages measured against once another. So, With that in mind, let’s ask ChatGPT to write a nuanced MACD strategy, using another indicator — Bollinger Bands — as a confirmation. (See Fig.2) This is where ChatGPT shines — no matter how complex the ask, AI is capable of getting it done. For those of us who understand and are willing trade nuanced strategies like this, it can be a pain to hunt down these opportunities and pounce on them immediately. With ChatGPT, one could theoretically set this script up on multiple high-liquidity tickers — just like setting hunting traps. Notably, there are areas where ChatGPT hasn’t yet broken through — strategies that still need a human touch. For instance, you could tell ChatGPT to create a strategy that hunts for trend reversals with 100% average volume, however, ChatGPT still doesn’t understand the nuance of reversal candles and technical patterns (though you could theoretically define those items). <img> Fig. 1 Bullish when the RSI is Oversold <img> Fig.2 MACD Crossover with Bollinger Band In conclusion, ChatGPT is a powerful NLP tool that can be used in a variety of ways in stock trading, including analysis, strategy development, and risk management. By gathering data and providing ChatGPT with a prompt, you can generate insights and ideas that can help you make more informed investment decisions and manage risk more effectively. As with any tool, it is important to use ChatGPT in conjunction with other sources of information. <title-4 style=text-align:justify;>References</title-4>[1] ChatGPT Trading Strategies (How To Use AI To Make Strategies)ChatGPT Trading Strategies (How To Use AI To Make Strategies) – Quantified Strategies For Traders And Investors [2] Machine Learning for Trading - XGBoost - <a href="https://cobuildlab.com/blog/ai" style=text-align:undefined;>https://www.cobuildlab.com/blog/ai</a> [3] <a href="https://cobuildlab.com/blog/ai/sentiment-analysis-of-financial-using-machine-learning" style=text-align:undefined;>Sentiment Analysis of Financial using Machine Learning</a> - <a href="https://cobuildlab.com/blog/ai" style=text-align:undefined;>https://www.cobuildlab.com/blog/ai</a> [4] <a href="https://cobuildlab.com/blog/ai/Machine-Learning-TensorFlow-for-SPY-ETF" style=text-align:undefined;>Machine Learning TensorFlow for SPY ETF</a> - <a href="https://cobuildlab.com/blog/ai" style=text-align:undefined;>https://www.cobuildlab.com/blog/ai</a><a href="https://www.cobuildlab.com/blog/ai" style=text-align:undefined;></a><a href="https://www.cobuildlab.com/blog/ai" style=text-align:undefined;></a>

The use of artificial intelligence has been on the rise in many industries, as AIs assist us in many aspects of our lives. 