← back

Artificial Neural Network for XOR Problem

We learned how an Artificial Neural Network can be used to solve the XOR problem.

The XOR function takes 2 inputs - if the inputs are the same, it outputs 0 (false), and if the inputs are different, it outputs 1 (true).

input 1	input 2	output
0	0	0
0	1	1
1	0	1
1	1	0

Early AI couldn't process this until a "hidden layer" was added.

I decided to make my own Artificial Neural Network and test it on the XOR problem.

import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# define the XOR input and output data
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])

# build the neural network model
model = Sequential()
model.add(Dense(2, activation='relu')) # hidden layer with 2 neurons
model.add(Dense(1, activation='sigmoid')) # output layer with 1 neuron
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# train the model
model.fit(X, y, epochs=10000, verbose=0)

# evaluate the model
loss, accuracy = model.evaluate(X, y)
print(f"accuracy: {accuracy * 100:.2f}%")

# make predictions
predictions = model.predict(X)
predictions = np.round(predictions).astype(int)

print("predictions:")
for i in range(len(X)):
    print(f"input: {X[i]} => predicted output: {predictions[i]}, actual output: {y[i]}")

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 53ms/step - accuracy: 1.0000 - loss: 0.0013
accuracy: 100.00%
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 19ms/step
predictions:
input: [0 0] => predicted output: [0], actual output: [0]
input: [0 1] => predicted output: [1], actual output: [1]
input: [1 0] => predicted output: [1], actual output: [1]
input: [1 1] => predicted output: [0], actual output: [0]

model.weights

[<Variable path=sequential/dense/kernel, shape=(2, 2), dtype=float32, value=[[-3.234061  3.3131413]
  [ 3.2338295  -3.3131545]]>,
 <Variable path=sequential/dense/bias, shape=(2,), dtype=float32, value=[-0.00017982  -0.00036467]>,
 <Variable path=sequential/dense_1/kernel, shape=(2, 1), dtype=float32, value=[[ 4.1856656 ]
  [ 3.9753308 ]]>,
 <Variable path=sequential/dense_1/bias, shape=(1,), dtype=float32, value=[-5.779532]>]

def sigmoid(x):
  return 1 / (1 + np.exp(-x))

I drew a diagram of the model.

I tested the weights and biases of the model.

X1	X2	H1 input	H1 output (ReLU)	H2 input	H2 output (ReLU)	y input	y output (sigmoid)
0	0	0 * -3.234061 + 0 * 3.2338295 + -0.00017982 = -0.00018	negative value outputs 0	0 * 3.3131413 + 0 * -3.3131545 + -0.00036467 = -0.000365	negative value outputs 0	0 * 4.1856656 + 0 * 3.9753308 + -5.779532 = -5.78	round(sigmoid(-5.78)) = 0
0	1	0 * -3.234061 + 1 * 3.2338295 + -0.00017982 = 3.234	3.234	0 * 3.3131413 + 1 * -3.3131545 + -0.00036467 = -3.314	negative value outputs 0	3.234 * 4.1856656 + 0 * 3.9753308 + -5.779532 = 7.757	round(sigmoid(7.757)) = 1
1	0	1 * -3.234061 + 0 * 3.2338295 + -0.00017982 = -3.234	negative value outputs 0	1 * 3.31314137 + 0 * -3.3131545 + -0.00036467 = 3.313	3.313	0 * 4.1856656 + 3.313 * 3.9753308 + -5.779532 = 7.391	round(sigmoid(7.391)) = 1
1	1	1 * -3.234061 + 1 * 3.2338295 + -0.00017982 = -0.000411	negative value outputs 0	1 * 3.3131413 + 1 * -3.3131545 + -0.00036467 = -0.000378	negative value outputs 0	0 * 4.1856656 + 0 * 3.9753308 + -5.779532 = -5.78	round(sigmoid(-5.78)) = 0

The hidden layer and the weights and biases do produce the correct outputs.

This activity really made it clear to me how the "hidden layers" of an Artificial Neural Network work. I will certainly be using machine learning models such as this in my career as a data scientist!