I believe this is because it just gives the answer with the highest probability as if it is correct and we don't get to see the alternative possibilities. It is giving the answer as 100% even if internally the answer only has a 40% probability of being correct as there is just much uncertainty in the response. I wish we were able to see more of this under the hood but I can see how people wouldn't relate to the answers the same way.
The other weakness seems to be that it has to treat int/floats as strings and has no concept of time. It is understandably bad with the ordering of events. A good example is to ask it what advanced civilizations were around 10,000 years ago. It told me ancient Egypt was an advanced civilization around 10,000 years ago but that does make sense as the highest probability answer when there is really no good answer.
That regression problem it has to treat everything in the document as a string and then it is also keeping state in the chat window. I think if you tell it read that document the information at the end of the document will muddle the probability of the answers from the start of the document. It gave me this proof through dialog with it but I have no idea if it is correct, totally over my head.
"Sure! To prove that H is a projection matrix, we need to show that it has the following properties:
Symmetric: HT = H
Idempotent: H × H = H
Proof:
Symmetric:
Let's start by showing that H is symmetric. To do this, we need to show that HT = H. We have:
HT = (X(XTX)-1XT)T
= (XT)T((XTX)-1)T XT [using the transpose rule (AB)T = BT AT]
= X(XTX)-1XT [since (XTX)-1 is symmetric]
Therefore, we have HT = X(XTX)-1XT = H, which proves that H is symmetric.
Idempotent:
Next, let's show that H is idempotent, i.e., H × H = H. We have:
H × H = X(XTX)-1XT × X(XTX)-1XT
= X(XTX)-1(XT × X)(XTX)-1XT [using the associative property of matrix multiplication]
= X(XTX)-1XT [since (XTX)-1(XTX) = In, the identity matrix]
Therefore, we have H × H = X(XTX)-1XT = H, which proves that H is idempotent.
Since H satisfies both properties of being symmetric and idempotent, it is indeed a projection matrix."
It can show the proof in numpy too
import numpy as np
# Generate some random data
X = np.random.rand(10, 3)
y = np.random.rand(10)
# Calculate the OLS estimator of the coefficients
beta_hat = np.linalg.inv(X.T @ X) @ X.T @ y
# Calculate the hat matrix
H = X @ np.linalg.inv(X.T @ X) @ X.T
# Calculate the residuals
residuals = (np.eye(len(y)) - H) @ y
# Check if H is symmetric
print(np.allclose(H, H.T))
# Check if H is idempotent
print(np.allclose(H @ H, H))
It does print true true but I wish it would give a confidence score.
Note also the tone of confidence. Chatgpt seems never to answer "I don't know" or "I'm not that good with linear algebra so I can't answer your question." In this way she resembles the biggest bullshitters (aka best salesmen) I have come across in my years in this field.
.