今天簡單介紹Xaier Initialization這個方法這個方法的目的是在初始化神經網路參數的時候穩定梯度避免梯度爆炸或是梯度消失可以看到下方算式代表的是一層神經網路參數W乘以輸入x得到輸出yW是矩陣 x是向量n_in n_out代表輸入的維度跟輸出的維度我們的目標就是讓輸出的變異數等於輸入的變異數(見第二個算式)藉此來穩定我們signal 傳遞我們假設W跟x是獨立變數 所以變異數就可以拆開變成相乘接下來W代表初始化根據正態分布 均值0 變異數sigma平方x則是輸入 輸入通常都會正則化 代表均值0 變異數1接著帶入rearange得出sigma平方=1/_in接著可以看到下面第一行我們要將損失函數做一個反向傳播的動作那這邊我們關注的就會是損失函數梯度變異數是否跟輸入的變異數一樣我們用chain rule來反向到xi 輸入層第二行就是對xi 偏微分其他變成0 剩下Wji接下來對損失函數梯度指定它為小delta套上變異數 跟昨天一樣 輸入跟神經網路參數獨立所以可以變成相乘接下來解開sigma得到結果你會發現說正向傳播跟反向傳播怎麼一個分母是輸入維度一個是輸出維度 那無法同時滿足兩個要求所以就有學者提出第二張最終的公式參數wji 就會從正態分布均值0標準差為根號變異數當中提取數字 Source link

On the lookout for the perfect RPA (Robotic Course of Automation) software? Here is a fast breakdown of three trade leaders –…

If you wish to efficiently launch a product, one of the vital vital issues you are able to do is to make…

Are you a Data Analyst trying to break into knowledge science? If that’s the case, this put up is for you. Many…

Stand an opportunity to win a NVIDIA Jetson Orin Nano Tremendous Developer package for participationSo, just like final yr, the world’s greatest…

AI is remodeling prostate most cancers imaging by bettering prognosis and therapy planning. Here is what you have to know: Key Advantages…

Are you prepared to combine enterprise with a ardour for well being and wellness? Baya Bar, the #1 Açai Bowl Franchise, gives…

DBeaver is probably the most highly effective open-source SQL IDE, however there are a number of options folks don’t learn about. On…

“We’ve been form of caught with outdated notions of what equity and bias means for a very long time,” says Divya Siddarth,…

شماره خاله تهران شماره خاله اصفهان شماره خاله شیراز شماره خاله کرج شماره خاله کرمانشاه شماره خاله کرمان شماره خاله رفسنجان شماره…