Project Outline<\/h1>\n\n\n\n
After a series of conversations with ChatGPT regarding the models that can be used for stock price prediction, the methodology to be followed, etc., it gave me an overwhelming number of choices to choose from. But I decided to keep it simple and wanted to follow the most sought-after method of predicting stock prices which is through using an LSTM model.<\/p>\n\n\n\n
I then asked ChatGPT for help drafting an outline for an extensive LSTM Python project for predicting stock prices and this is what it generated:<\/p>\n\n\n\n
1. Obtaining data
2. Linear Regression
3. A basic LSTM model
4. Improving the LSTM model
5. Hyperparameter Tuning<\/p>\n\n\n\n
This was not the initial outline generated by the bot, but after conversing for a while, it came up with this, which I think is pretty straightforward and easy to grasp. The entire article will follow this outline and apart from the first section of obtaining data, all the other sections will be solely driven by ChatGPT alone.<\/p>\n\n\n\n
Register & Get Data<\/span><\/a><\/p>\n\n\n\n To build a successful ML model which can predict with a higher level of accuracy, data of quality and reliability must be preferred over anything else. This is where EODHD<\/a> comes into play. EOD Historical Data (EODHD) is a reliable provider of financial APIs covering a huge variety of market data ranging from historical data to economic and financial news data.<\/p>\n\n\n\n For this project, we\u2019ll be predicting Microsoft’s stock prices for which we need its time series data for a long period. Thus, I went with EODHD’s end-of-day or historical data API. I used the following code to extract Microsoft’s historical data from 2010 to the present:<\/p>\n\n\n\n There isn’t anything complicated with this code. For the smooth working of the code, make sure to replace “YOUR API KEY” with your secret EODHD API key which you can obtain after creating an account with them<\/a>. We are saving the extracted data so that it can be easily imported and fed into the upcoming prediction program rather than making API calls every single time. One thing which I would like to add is that though the API has a data coverage of 30+ years, I\u2019ve decided to keep it 13 as it\u2019s more than enough for our program. This is the final dataframe:<\/p>\n\n\n\n Instead of directly jumping into building an LSTM model, I wanted to take things slowly with ChatGPT to get a glimpse of the results generated by the code given by the bot. So I asked it to write me a Python program to build a basic Linear Regression model using Scikit Learn for the purpose of predicting stock prices. It initially gave me the steps to be followed for building the regression model and then wrote this sample code:<\/p>\n\n\n\n This code is surprisingly well-formatted and incorporates all the conventional procedures. One of the best things about this code are the comments which are included in each and every process, making it easier to understand what is going on. It also gave a short brief about the code to explain the whole process. Now, these are the results of the code:<\/p>\n\n\n\n One small thing I would do with this code is to play with the model parameters to ascertain the most ideal model for our case but this is a topic for another day because the focus here is to get an understanding of ChatGPT\u2019s code and its results.<\/p>\n\n\n\n Now it\u2019s time to up the game and ask ChatGPT to generate a Python program to build an LSTM model for predicting stock prices. But before moving on, here is a short description of LSTM: Long Short-Term Memory (LSTM) is a sophisticated neural network and a powerful tool for analyzing sequential data. It selectively stores or forgets information to understand complex sequences of data. This makes it ideal for tasks like speech recognition, natural language processing, and analyzing time-series data. Here\u2019s the code generated by ChatGPT to build a predictive LSTM model:<\/p>\n\n\n\n This code is very simple yet amazingly clear and again well-formatted with wonderful comments for each step of the code. ChatGPT followed a very straightforward way of using TensorFlow to build an LSTM model which I think is perfectly fine because that\u2019s what I asked for and I love the evaluation process that comes along with a graph. I did customize the plot by adding some additional parameters to beautify it but other than that, I didn’t touch a single line of code. Here\u2019s the result:<\/p>\n\n\n\n The LSTM model which we just built and used for prediction is very simplistic in nature and can be improved in various aspects. So I asked ChatGPT to give me some suggestions to improve the code for a better LSTM model and here\u2019s what it got to say about it:<\/p>\n\n\n\n This is awesome! It just gave us a complete guideline to be followed in order to improve an AI model, specifically LSTM in our case. I then asked ChatGPT to include all these suggestions in the previous code and it returned me this updated version:<\/p>\n\n\n\n This code is a sort of extended version of the previous code with all those little changes suggested by ChatGPT. According to ChatGPT: It increases the number of LSTM layers, increases the number of neurons in each layer, adds dropout layers, and adjusts the learning rate. It also uses more data by training on the entire historical data and uses a different loss function, mean squared logarithmic error (MSLE), which is more appropriate for time series data.<\/em><\/p>\n\n\n\n In my opinion, there aren’t any significant changes to the code, but that is acceptable because it is not necessary to bring in a great number of modifications to improve an existing model. Here is a side-by-side comparison of the results of both models (left: previous model, right: improved model):<\/p>\n\n\n\n By observing the results, we can conclude that there are some great improvements made in the performance of the LSTM model. The R2 score of the new model has increased from 0.61 to 0.91 (yowza!) indicating higher prediction accuracy, and the MSE, MSLE, and MAE scores are all significantly reduced making them even more closer to zero which is again, a good sign of a better model. Henceforth, ChatGPT really did a great job in improving our model.<\/p>\n\n\n\n No AI & ML project is complete without hyperparameter tuning. Here\u2019s a little background about hyperparameter tuning: The process which involves adjusting the settings of a machine learning model to improve its performance is called hyperparameter tuning. These settings, known as hyperparameters, are set before training the model and include parameters such as learning rate, number of layers and neurons, etc. The primary objective of fine-tuning these settings is to boost the model\u2019s accuracy in predictions. Here\u2019s the code ChatGPT generated when asked to perform hyperparameter tuning on the previously generated model:<\/p>\n\n\n\n Just like all the other programs generated before, this code also follows a clean structure, neatly written comments, and a pretty well-known procedure of hyperparameter tuning which is to define a “param_grid” and pass the grid to the “GridSearchCV” function to perform a Grid Search and ascertain the best model parameters. These are the best parameters according to the hyperparameter tuning process by ChatGPT:<\/p>\n\n\n\n ChatGPT did not stop there and went further to use the best parameters to build a model and make predictions. This graph is the final result of the model derived from hyperparameter tuning:<\/p>\n\n\n\n <\/p>\n\n\n\n1. Obtaining Data using EODHD<\/h1>\n\n\n\n
def get_historical_data(symbol, start, end):\n api_key = 'YOUR API KEY'\n api_response = requests.get(f'https:\/\/eodhistoricaldata.com\/api\/eod\/{symbol}?api_token={api_key}&fmt=json&from={start}&to={end}').json()\n df = pd.DataFrame(api_response).drop('close', axis = 1)\n df.columns = ['Date', 'Open', 'High', 'Low', 'Close', 'Volume']\n df.Date = pd.to_datetime(df.Date)\n return df\n\ndf = get_historical_data('MSFT', '2010-01-01', '2023-07-23')\ndf.to_csv('msft.csv')\n\ndf.tail()<\/pre>\n\n\n\n![\"\"](\"https:\/\/eodhistoricaldata.com\/financial-academy\/wp-content\/uploads\/2023\/07\/image-3.png\")
<\/figure>\n\n\n\n2. Linear Regression<\/h1>\n\n\n\n
import pandas as pd\nimport numpy as np\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.metrics import r2_score, mean_squared_error\nimport matplotlib.pyplot as plt\n\n# Get the stock data\ndf = pd.read_csv('msft.csv').dropna()\ndf = df.set_index('Date')\n\n# Add a column for the target variable\ndf['Price'] = df['Close']\n\n# Define the features and target variable\nfeatures = ['Open', 'High', 'Low', 'Volume']\ntarget = 'Price'\n\n# Split the data into training and testing sets\ntrain_size = 0.8\ntrain_data = df[:int(train_size * len(df))]\ntest_data = df[int(train_size * len(df)):]\n\n# Normalize the data\nscaler = StandardScaler()\ntrain_data[features] = scaler.fit_transform(train_data[features])\ntest_data[features] = scaler.transform(test_data[features])\n\n# Train the model\nmodel = LinearRegression()\nmodel.fit(train_data[features], train_data[target])\n\n# Make predictions on the test data\npredictions = model.predict(test_data[features])\n\n# Compute the R^2 score and RMSE\nr2 = r2_score(test_data[target], predictions)\nrmse = np.sqrt(mean_squared_error(test_data[target], predictions))\n\nprint(f'R^2 score: {r2:.4f}')\nprint(f'RMSE: {rmse:.4f}')\n\n# Plot the predicted and actual values\nplt.plot(test_data[target].values, label='Actual')\nplt.plot(predictions, label='Predicted')\nplt.legend()\nplt.show()<\/pre>\n\n\n\n![\"\"](\"https:\/\/eodhistoricaldata.com\/financial-academy\/wp-content\/uploads\/2023\/07\/image-5.png\")
<\/figure>\n\n\n\n3. A basic LSTM model<\/h1>\n\n\n\n
import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import *\nfrom sklearn.preprocessing import MinMaxScaler\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import LSTM, Dense, Dropout\n\n# Load the stock price data\ndf = pd.read_csv('msft.csv').dropna()\ndf = df.set_index('Date')\n\n# Preprocess the data\nscaler = MinMaxScaler()\ndf_scaled = scaler.fit_transform(df['Close'].values.reshape(-1, 1))\n\n# Define the training and testing data\ntrain_size = int(len(df_scaled) * 0.8)\ntrain_data = df_scaled[:train_size, :]\ntest_data = df_scaled[train_size:, :]\n\n# Define the function to create a sliding window dataset\ndef create_dataset(dataset, time_steps=1):\n X_data, y_data = [], []\n for i in range(len(dataset)-time_steps-1):\n X_data.append(dataset[i:(i+time_steps), 0])\n y_data.append(dataset[i + time_steps, 0])\n return np.array(X_data), np.array(y_data)\n\n# Define the time steps and create the sliding window dataset\ntime_steps = 60\nX_train, y_train = create_dataset(train_data, time_steps)\nX_test, y_test = create_dataset(test_data, time_steps)\n\n# Reshape the data for LSTM input\nX_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))\nX_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))\n\n# Define the LSTM model architecture\nmodel = Sequential()\nmodel.add(LSTM(units=64, return_sequences=True, input_shape=(X_train.shape[1], 1)))\nmodel.add(Dropout(0.2))\nmodel.add(LSTM(units=64, return_sequences=True))\nmodel.add(Dropout(0.2))\nmodel.add(LSTM(units=64, return_sequences=False))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(units=1))\n\n# Compile the model\nmodel.compile(optimizer='adam', loss='mean_squared_error')\n\n# Train the model\nmodel.fit(X_train, y_train, epochs=50, batch_size=64, validation_data=(X_test, y_test), verbose=1)\n\n# Make predictions\ny_pred = model.predict(X_test)\n\n# Inverse transform the predicted and actual values\ny_pred = scaler.inverse_transform(y_pred)\ny_test = y_test.reshape(y_pred.shape[0], 1)\ny_test = scaler.inverse_transform(y_test)\n\n# Evaluate the model\nmse = mean_squared_error(y_test, y_pred)\nmsle = mean_squared_log_error(y_test, y_pred)\nmae = mean_absolute_error(y_test, y_pred)\nr2 = r2_score(y_test, y_pred)\n\nprint('MSE: ', mse)\nprint('MSLE: ', msle)\nprint('MAE: ', mae)\nprint('R-squared: ', r2)\n\n# Plot the predicted vs actual values\nplt.figure(figsize=(10, 6))\nplt.style.use('fivethirtyeight')\nplt.plot(y_test, label='Actual', linewidth = 3, alpha = 0.4)\nplt.plot(y_pred, label='Predicted', linewidth = 1.5)\nplt.xlabel('Days')\nplt.ylabel('Stock Price')\nplt.title('LSTM: Actual vs Predicted')\nplt.legend()\nplt.show()<\/pre>\n\n\n\n![\"\"](\"https:\/\/eodhistoricaldata.com\/financial-academy\/wp-content\/uploads\/2023\/07\/image-9.png\")
<\/figure>\n\n\n\n4. Improving the LSTM model<\/h1>\n\n\n\n
\n
import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import *\nfrom sklearn.preprocessing import MinMaxScaler\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import LSTM, Dense, Dropout\nfrom tensorflow.keras.callbacks import EarlyStopping\n\n# Obtain stock data for Microsoft\ndf = pd.read_csv('msft.csv').dropna()\n\n# Extract the closing prices\nclose_prices = df[[\"Date\", \"Close\"]].copy()\nclose_prices.columns = [\"ds\", \"y\"]\nclose_prices.set_index(\"ds\", inplace=True)\n\n# Normalize the data\nscaler = MinMaxScaler(feature_range=(0, 1))\nclose_prices_scaled = scaler.fit_transform(close_prices)\n\n# Create training and testing data\ntrain_size = int(len(close_prices) * 0.8)\ntrain_data = close_prices_scaled[:train_size]\ntest_data = close_prices_scaled[train_size:]\n\n# Define the number of time steps to look back\nlook_back = 60\n\n# Create training and testing data for the LSTM model\ndef create_data(data):\n X, Y = [], []\n for i in range(look_back, len(data)):\n X.append(data[i - look_back : i, 0])\n Y.append(data[i, 0])\n X, Y = np.array(X), np.array(Y)\n X = np.reshape(X, (X.shape[0], X.shape[1], 1))\n return X, Y\n\nX_train, Y_train = create_data(train_data)\nX_test, Y_test = create_data(test_data)\n\n# Define the LSTM model\nmodel = Sequential()\n\nmodel.add(LSTM(units=64, return_sequences=True, input_shape=(X_train.shape[1], 1)))\nmodel.add(Dropout(0.2))\n\nmodel.add(LSTM(units=128, return_sequences=True))\nmodel.add(Dropout(0.2))\n\nmodel.add(LSTM(units=64))\nmodel.add(Dropout(0.2))\n\nmodel.add(Dense(units=1))\n\n# Compile the model\nmodel.compile(optimizer=\"adam\", loss=\"mean_squared_error\")\n\n# Define the early stopping criteria\nearly_stop = EarlyStopping(monitor=\"val_loss\", patience=10)\n\n# Fit the model to the training data\nhistory = model.fit(X_train, Y_train, epochs=100, batch_size=32, validation_data=(X_test, Y_test), callbacks=[early_stop])\n\n# Make predictions on the test data\ny_pred = model.predict(X_test)\n\n# Inverse the scaling of the data\ny_pred = scaler.inverse_transform(y_pred)\nY_test = scaler.inverse_transform([Y_test])\n\n# Evaluate the model\nmse = mean_squared_error(Y_test[0], y_pred)\nmsle = mean_squared_log_error(Y_test[0], y_pred)\nmae = mean_absolute_error(Y_test[0], y_pred)\nr2 = r2_score(Y_test[0], y_pred)\n\nprint('MSE: ', mse)\nprint('MSLE: ', msle)\nprint('MAE: ', mae)\nprint('R-squared: ', r2)\n\n# Plot the predicted vs actual values\nplt.figure(figsize=(10, 6))\nplt.style.use('fivethirtyeight')\nplt.plot(Y_test.flatten(), label=\"Actual\", linewidth = 3, alpha = 0.4)\nplt.plot(y_pred.flatten(), label=\"Predicted\", linewidth = 1.5)\nplt.title(\"MSFT Stock Price Prediction\")\nplt.xlabel(\"Days\")\nplt.ylabel(\"Stock Price\")\nplt.legend()\nplt.show()<\/pre>\n\n\n\n![\"\"](\"https:\/\/eodhistoricaldata.com\/financial-academy\/wp-content\/uploads\/2023\/07\/Screenshot-2023-07-24-182215.png\")
<\/figure>\n<\/div>\n\n\n\n![\"\"](\"https:\/\/eodhistoricaldata.com\/financial-academy\/wp-content\/uploads\/2023\/07\/Screenshot-2023-07-24-182451.png\")
<\/figure>\n<\/div>\n<\/div>\n\n\n\n5. Hyperparameter Tuning<\/h1>\n\n\n\n
import numpy as np\nimport pandas as pd\nfrom sklearn.preprocessing import MinMaxScaler\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, LSTM, Dropout\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.wrappers.scikit_learn import KerasRegressor\nfrom sklearn.model_selection import GridSearchCV, TimeSeriesSplit\n\n# Download the data\ndf = pd.read_csv('msft.csv').dropna()\ndf = df.set_index('Date')\n\n# Scale the data\nscaler = MinMaxScaler()\nscaled_data = scaler.fit_transform(df['Close'].values.reshape(-1, 1))\n\n# Define the training and test sets\ntrain_size = int(len(scaled_data) * 0.8)\ntrain_data = scaled_data[:train_size]\ntest_data = scaled_data[train_size:]\n\n# Define the function to create the LSTM model\ndef create_model(neurons=50, layers=2, dropout=0.2, learning_rate=0.001):\n model = Sequential()\n for i in range(layers):\n model.add(LSTM(units=neurons, return_sequences=True, input_shape=(train_data.shape[1], 1)))\n model.add(Dropout(dropout))\n model.add(LSTM(units=neurons))\n model.add(Dropout(dropout))\n model.add(Dense(units=1))\n optimizer = Adam(learning_rate=learning_rate)\n model.compile(optimizer=optimizer, loss='mean_squared_error')\n return model\n\n# Define the parameter grid for hyperparameter tuning\nparam_grid = {\n 'neurons': [50, 100, 200],\n 'layers': [2, 3],\n 'dropout': [0.2, 0.3],\n 'learning_rate': [0.001, 0.01]\n}\n\n# Define the time series split for cross-validation\ntscv = TimeSeriesSplit(n_splits=3)\n\n# Create the KerasRegressor for hyperparameter tuning\nmodel = KerasRegressor(build_fn=create_model, epochs=50, batch_size=32, verbose=0)\n\n# Perform the hyperparameter tuning\ngrid = GridSearchCV(estimator=model, param_grid=param_grid, cv=tscv, scoring='neg_mean_squared_error', n_jobs=-1)\ngrid_result = grid.fit(train_data, train_data)\n\n# Print the best parameters and score\nprint(\"Best parameters: \", grid_result.best_params_)\nprint(\"Best score: \", np.sqrt(-grid_result.best_score_))\n\n# Use the best model to make predictions on the test data\nbest_model = grid_result.best_estimator_.model\npredictions = best_model.predict(test_data.reshape(-1, 1))\npredictions = scaler.inverse_transform(predictions)\n\n# Plot the predictions and actual values\nimport matplotlib.pyplot as plt\n\n# Plot the predicted vs actual values\nplt.figure(figsize=(10, 6))\nplt.style.use('fivethirtyeight')\nplt.plot(pd.to_datetime(df.index[train_size:]), df['Close'][train_size:], label='Actual', linewidth = 3, alpha = 0.4)\nplt.plot(pd.to_datetime(df.index[train_size:]), predictions, label='Predicted', linewidth = 1.5)\nplt.title(f\"MSFT Stock Price Prediction\")\nplt.xlabel(\"Date\")\nplt.ylabel(\"Stock Price\")\nplt.legend()\nplt.show()<\/pre>\n\n\n\nBest parameters: {'dropout': 0.2, 'layers': 2, 'learning_rate': 0.001, 'neurons': 100}\nBest score: 0.0047933462470134474<\/pre>\n\n\n\n![\"\"](\"https:\/\/eodhistoricaldata.com\/financial-academy\/wp-content\/uploads\/2023\/07\/image-10.png\")
<\/figure>\n\n\n\n