[Machine Learning][Andrew ng's lesson] 标准表达式 Normal Equation

jkwRay

墨衍会员 · AI 创作全网分发码龄10年

关注

原创最新推荐文章于 2026-09-14 16:25:51 发布 345 阅读 AI 写同款· GEO 优化 ›

限时加码！20+主流AI编程工具免费用购周边加赠Coding Plan Lite，Claude Code、Cursor等即刻畅享，学习进阶更高效！阅读详情

Normal Equation :

According to Features Matrix X and the Result Matrix y

and the H(x) turns out to be

\sum i = 0 n X i * θ i (X o = 1)

$\sum_{i=0}^n Xi * θi (Xo = 1)$

From pinv(X’*X)*X’*y to get the Correct θ Matrix.
Theory: To set the partial derivative(θj to J(θ)) equal to 0.

(In the same time ,it is not necessary to use Feature Scaling)

    h = 0 + 5*X1 + 7*X2

    X = [1 2 4; 1 8 6; 1 3 7]
    y = [38;82;64]
    pinv(X' * X) * X' * y :

    ans =
        -1.2221e-012
        5.0000e+000
        7.0000e+000

Disadvantage:
Need to compute pinv(X’X) O(n*n*n)
Slow if n is very large
Notice:
Even the X’*X can’t be inversed. Using pinv() instead of using inv() will compute the answers correctly.

Expand

It’s obviously less possible that a matrix can’t be inversed, but it can happen.
The Reasons are probably below:
1. Some features are same or have linear connection. For example, mile and feet are same, so we may know θx + 3.33θy = AExactNumber, but we can’t decide the θx and θy’s value, because there are too many possibilities.
2. The number of data (m) is less than the number of features. For example, we have 10 datas and 100 features. However ,we have a technique to solve this problem, which can be used to get h(x) which is made up of many variables from small datas.

Technique Explanation:

This technique will also solve the problem of overfitting by plusing a penality to the size of θ in J(θ).
(NOTICE! We only penalize θ from 1 to n, we don’t penalize θ0)
So the J may like this
这里写图片描述
And the normal equation will change to

newM = eye(n + 1);
newM(1,1) = 0;
pinv(X'*X + λ*newM)*X'*y

It can be proved that the Matrix which is to plus a new Matrix(λ >0) must has inverse Matrix.
So, we can get correct θ from small datas now!(λ>0)

收录于

日常学习

机器学习

标准表达式

标签

#机器学习 #标准表达式 #数学 #收敛

本内容遵循 CC 4.0 BY-SA 版权协议

聚类在 RAG 里的落地细节 RAG 主检索一般还是向量相似度（ANN），聚类解决「向量分布混乱、上下文割裂、检索冗余、长文档分片语义漂移」问题。常见聚类算法：K-Means、DBSCAN、HDBSCAN、层次聚类；RAG 场景，K-Means 适合已知类数的场景。阅读详情

[Machine Learning][Andrew ng's lesson] 标准表达式 Normal Equation

Normal Equation :

相关推荐