FOMC Monetary Policy Analysis v2

This is the second version of a project I've decided to do in Artificial Intelligence and Deep Learning. I've started getting quite a passion for the subject and its potential applications. This uses natural language processing (NLP) for binary classification from a multi-layer perceptron (MLP) i.e. standard neural network. It attempts to predict the EUR/USD appreciation/depreciation from the Fed's Monetary Policy Report release date up until the next report release to Congress.

With this second version of my project, I again set the optimizer to Adam. I've read some research that suggests at this point in time it is the best optimizer in dealing with gradient descent as it realizes both benefits of AdaGrad and RMSProp. I then proceeded to incorporate the Hyperas library to assist in optimizing my hyper-parameters. Those parameters chosen for optimization are listed below:

Best performing model chosen hyper-parameters:
{'Dense': 64, 'Dense_1': 16, 'Dropout_1': 0.3838088604298333, 'Dense_2': 8, 'Dropout': 0.9128294469805703}

With those hyper-parameters, I was able to get a 60% level of accuracy which does make it statistically significant but not ultimately ideal....in my opinion.

I also did some research with recurrent neural networks such as LSTM and GRU. This did achieve a similar level of accuracy, but they seemed to be more resource intensive than the standard. I also did some research and testing with a 1D convolutional neural network to see what output I could achieve. This did not lead me to higher levels of accuracy, but was certainly less resource intensive in general.

If there are issues with accessing my Gihub repo below, I have a zipped file with my code, model, and datasets here: Repo Copy

Please see my Github for code and datasets related to this project.

This is my deep learning output from Keras.

Using TensorFlow backend.
Found 8784 unique tokens.
Shape of data tensor: (34, 20000)
Shape of label tensor: (34,)
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
embedding_1 (Embedding)      (None, 20000, 256)        5120000
_________________________________________________________________
flatten_1 (Flatten)          (None, 5120000)           0
_________________________________________________________________
dense_1 (Dense)              (None, 64)                327680064
_________________________________________________________________
dropout_1 (Dropout)          (None, 64)                0
_________________________________________________________________
dense_2 (Dense)              (None, 16)                1040
_________________________________________________________________
dropout_2 (Dropout)          (None, 16)                0
_________________________________________________________________
dense_3 (Dense)              (None, 8)                 136
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 9
=================================================================
Total params: 332,801,249
Trainable params: 332,801,249
Non-trainable params: 0
_________________________________________________________________
Train on 30 samples, validate on 4 samples
Epoch 1/50
 - 26s - loss: 0.6981 - acc: 0.5333 - val_loss: 0.6970 - val_acc: 0.7500
Epoch 2/50
 - 6s - loss: 2.4526 - acc: 0.4000 - val_loss: 0.7367 - val_acc: 0.7500

.........................................................................

 - 5s - loss: 1.5206 - acc: 0.5667 - val_loss: 0.6844 - val_acc: 0.5000
Epoch 45/50
 - 4s - loss: 1.1919 - acc: 0.6000 - val_loss: 0.6859 - val_acc: 0.5000
Epoch 46/50
 - 4s - loss: 3.1487 - acc: 0.6000 - val_loss: 0.6850 - val_acc: 0.5000
Epoch 47/50
 - 4s - loss: 2.0293 - acc: 0.5000 - val_loss: 0.6863 - val_acc: 0.5000
Epoch 48/50
 - 4s - loss: 1.0979 - acc: 0.5000 - val_loss: 0.6977 - val_acc: 0.5000
Epoch 49/50
 - 5s - loss: 3.5089 - acc: 0.4333 - val_loss: 0.7059 - val_acc: 0.5000
Epoch 50/50
 - 4s - loss: 1.0606 - acc: 0.6000 - val_loss: 0.7121 - val_acc: 0.5000
5/5 [==============================] - 0s 67ms/step
Test loss: 0.6752365827560425
Test accuracy: 0.6000000238418579

FOMC Monetary Policy Analysis v1

This is the first project I've decided to do in Artificial Intelligence. I've started getting quite a passion for the subject and its potential applications. This uses natural language processing (NLP) for binary classification from a multi-layer perceptron (MLP) i.e. standard neural network. It attempts to predict the EUR/USD appreciation/depreciation from the Fed's Monetary Policy Report release date up until the next report release to Congress.

Since this is my first project, the model is quite elementary. I was able to get an 80% level of accuracy on my cross validation set, but it doesn't provide anything statistically significant on the test set (It doesn't provide a higher level of accuracy than 50%). Never-the-less, by using more advanced features in Keras and Hyperas/Hyperopt, I've already gotten a statistically significant result on my test set with another model I'm currently working on. I will be publishing that soon.

If there are issues with accessing my Gihub repo below, I have a zipped file with my code, model, and datasets here: Repo Copy

Please see my Github for code and datasets related to this project.

This is my deep learning output from Keras.

Using TensorFlow backend.
Found 8879 unique tokens.
Shape of data tensor: (35, 20000)
Shape of label tensor: (35,)
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 20000, 150)        3000000   
_________________________________________________________________
flatten_1 (Flatten)          (None, 3000000)           0         
_________________________________________________________________
dense_1 (Dense)              (None, 32)                96000032  
_________________________________________________________________
dense_2 (Dense)              (None, 16)                528       
_________________________________________________________________
dense_3 (Dense)              (None, 4)                 68        
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 5         
=================================================================
Total params: 99,000,633
Trainable params: 99,000,633
Non-trainable params: 0
_________________________________________________________________
Train on 30 samples, validate on 5 samples
Epoch 1/10
2018-07-28 17:44:28.161931: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2
30/30 [==============================] - 4s 150ms/step - loss: 0.6951 - acc: 0.4000 - val_loss: 0.9779 - val_acc: 0.2000
Epoch 2/10
30/30 [==============================] - 3s 116ms/step - loss: 0.4317 - acc: 0.7667 - val_loss: 1.1679 - val_acc: 0.2000
Epoch 3/10
30/30 [==============================] - 3s 115ms/step - loss: 0.1396 - acc: 0.9333 - val_loss: 0.9146 - val_acc: 0.2000
Epoch 4/10
30/30 [==============================] - 3s 108ms/step - loss: 0.0227 - acc: 1.0000 - val_loss: 0.7182 - val_acc: 0.4000
Epoch 5/10
30/30 [==============================] - 3s 108ms/step - loss: 0.0084 - acc: 1.0000 - val_loss: 0.6026 - val_acc: 0.8000
Epoch 6/10
30/30 [==============================] - 3s 107ms/step - loss: 0.0033 - acc: 1.0000 - val_loss: 0.5490 - val_acc: 0.8000
Epoch 7/10
30/30 [==============================] - 3s 107ms/step - loss: 0.0014 - acc: 1.0000 - val_loss: 0.5170 - val_acc: 0.8000
Epoch 8/10
30/30 [==============================] - 3s 108ms/step - loss: 6.2672e-04 - acc: 1.0000 - val_loss: 0.4956 - val_acc: 0.8000
Epoch 9/10
30/30 [==============================] - 3s 106ms/step - loss: 2.9860e-04 - acc: 1.0000 - val_loss: 0.4828 - val_acc: 0.8000
Epoch 10/10
30/30 [==============================] - 3s 110ms/step - loss: 1.5072e-04 - acc: 1.0000 - val_loss: 0.4755 - val_acc: 0.8000
5/5 [==============================] - 0s 24ms/step
Test loss: 0.4754864275455475
Test accuracy: 0.800000011920929
4/4 [==============================] - 0s 23ms/step
Test loss: 7.971192359924316
Test accuracy: 0.5