【導(dǎo)讀】一種從單個(gè)觀察中抽象出通用規(guī)則的方法。目標(biāo)是下次可以快速地解決類似的問(wèn)題。記憶——通過(guò)保存結(jié)果和避免從零開(kāi)始解。1983年美國(guó)Illinois大學(xué)的DeJong提出。解釋為什么一個(gè)想法是好的比提出一個(gè)想法容。一旦理解了一件事，就可以泛化并重用于其它。通過(guò)對(duì)一棵證明樹(shù)的常量變量化,EBL可以同時(shí)。給定一個(gè)例子，使用背景知識(shí)構(gòu)建一棵證明。去掉所有與目標(biāo)中變量真正無(wú)關(guān)的條件。1)對(duì)所產(chǎn)生的結(jié)論的推理過(guò)程作詳細(xì)說(shuō)明，以增加系統(tǒng)的可接受性；的缺陷和錯(cuò)誤的概念；3)對(duì)初學(xué)的用戶進(jìn)行訓(xùn)練。預(yù)先用自然語(yǔ)言寫(xiě)好，并插。遍歷目標(biāo)樹(shù)，通過(guò)總結(jié)與結(jié)。論是如何得到的；明確表示控制知識(shí)，即用元。知識(shí)概括地描述，與領(lǐng)域規(guī)則完全分開(kāi)。提供問(wèn)題求解策略的解釋。目標(biāo)驅(qū)動(dòng)-極大地提高了速度?？刹僮餍?一個(gè)子目標(biāo)是可操作的，意思。可操作性和通用性之間的平衡。對(duì)EBL學(xué)習(xí)效率的實(shí)驗(yàn)分析。者選自領(lǐng)域知識(shí)易于評(píng)測(cè)的謂詞。訓(xùn)練實(shí)例的泛化是對(duì)目標(biāo)概念給以充分定義，并且滿。足可操作性標(biāo)準(zhǔn)。

　　

【正文】 onlinear dimensionality reduction: ? But it is very difficult to optimize deep autoencoders using backpropagation. ? Deep Autoencoders (Hinton amp。 Salakhutdinov, 2020) ? We now have a much better way to optimize them: ? First train a stack of 4 RBM’s ? Then “unroll” them. ? Then finetune with backprop. 1000 neurons 500 neurons 500 neurons 250 neurons 250 neurons 30 1000 neurons 28x28 28x28 12344321WWWWWWWWTTTTlinear units 2020/11/4 89 大數(shù)據(jù)挖掘 史忠植 自編碼神經(jīng)網(wǎng)絡(luò) 2020/11/4 大數(shù)據(jù)挖掘 史忠植 90 自編碼神經(jīng)網(wǎng)絡(luò) 限制玻爾茲曼機(jī) ? We restrict the connectivity to make learning easier. ? Only one layer of hidden units. ? We will deal with more layers later ? No connections between hidden units. ? In an RBM, the hidden units are conditionally independent given the visible states. ? So we can quickly get an unbiased sample from the posterior distribution when given a datavector. ? This is a big advantage over directed belief s hidden i j visible 2020/11/4 91 大數(shù)據(jù)挖掘 史忠植 深度信念網(wǎng)絡(luò) Deep Belief Networks ? Stacking RBMs to from Deep architecture ? DBN with l layers of models the joint distribution between observed vector x and l hidden layers h. ? Learning DBN: fast greedy learning algorithm for constructing multilayer directed works on layer at a time v h1 h2 h3 1P2PLP2020/11/4 92 大數(shù)據(jù)挖掘 史忠植 深度信念網(wǎng)絡(luò) Deep Belief Network(DBN) h2 data h1 h3 2W3W1WRBM RBM RBM 2020 toplevel neurons 500 neurons 500 neurons 28x28 pixel image (784 neurons) DBNs are stacks of restricted Boltzmann machines forming deep (multilayer) architecture. 2020/11/4 93 大數(shù)據(jù)挖掘 史忠植 Deep Belief Net (DBN) amp。 Deep Neural Net (DNN) ? DBN: Undirected at top two layers which is an RBM。 directed Bayes (topdown) at lower layers (good for synthesis and recognition) ? DNN: Multilayer perceptron (bottom up) + unsupervised pretraining w. RBM weights (good for recognition only) 94 n Layers Visible Layer Labels MANY Hidd n Layers 2020/11/4 大數(shù)據(jù)挖掘 史忠植 Deep Belief Net (DBN) amp。 Deep Neural Net (DNN) 2020/11/4 大數(shù)據(jù)挖掘 史忠植 95 First train a stack of three models each of which has one hidden layer. Each model in the stack treats the hidden variables of the previous model as data. Then pose them into a single Deep Belief Network. Then add outputs and train the DNN with backprop. Hinton, Deng, Yu, Mohamed, Dahl… etc. IEEE Sig. Proc. Mag. (Nov 2020) 語(yǔ)音識(shí)別 96 1989 2020 1999 2020/11/4 大數(shù)據(jù)挖掘 史忠植 To estimate the word error rate (WER), the correct and the recognized sentence must be first aligned. Then the number of substitutions (S), deletions (D), and insertions (I) can be estimated. The WER is defined as ?????? = 100% （ ?? + ?? +??） / ?? 語(yǔ)音識(shí)別 97 1989 2020 2020/11/4 大數(shù)據(jù)挖掘 史忠植 語(yǔ)音識(shí)別 2020/11/4 大數(shù)據(jù)挖掘 史忠植 98 DNNHMM (replacing GMM only。 longer MFCC/filterback windows w. no transformation) 99 Model tied triphone states directly Many layers of nonlinear feature transformation + SoftMax 大數(shù)據(jù)挖掘 史忠植 2020/11/4 CDDNNHMM: Architecture 100 2020/11/4 大數(shù)據(jù)挖掘 史忠植 圖像識(shí)別 2020/11/4 大數(shù)據(jù)挖掘 史忠植 101 聲音與視頻重構(gòu) 2020/11/4 大數(shù)據(jù)挖掘 史忠植 102 多任務(wù)學(xué)習(xí) 2020/11/4 大數(shù)據(jù)挖掘 史忠植 103 深度學(xué)習(xí)網(wǎng)站 ? (Bengio’s group) ? labs/ ? d_Readings ? ? ? (Andrew Ng’s group) ? Google+ Deep Learning munity 104 2020/11/4 大數(shù)據(jù)挖掘 史忠植 University of Toronto Machine Learning Group (Geoff Hinton, Rich Zemel, Ruslan Salakhutdinov, Brendan Frey, Radford Neal) Universit233。 de Montr233。al Lisa Lab (Yoshua Bengio, Pascal Vincent, Aaron Courville, Roland Memisevic) New York University – Yann Lecun‘s and Rob Fergus‘ group Stanford University – Andrew Ng‘s group UBC – Nando de Freitas‘s group Google Research – Jeff Dean, Samy Bengio, Jason Weston, Marc’Aurelio Ranzato, Dumitru Erhan, Quoc Le et al Microsoft Research – Li Deng et al SUPSI – IDSIA (Schmidhuber’s group) UC Berkeley – Bruno Olshausen‘s group University of Washington – Pedro Domingos‘ group IDIAP Research Institute Ronan Collobert‘s group University of California Merced – Miguel A. CarreiraPerpinan‘s group University of Helsinki Aapo Hyv228。rinen‘s Neuroinformatics group Universit233。 de Sherbrooke – Hugo Larochelle‘s group University of Guelph – Graham Taylor‘s group University of Michigan – Honglak Lee‘s group Technical University of Berlin – KlausRobert Muller‘s group Baidu – Kai Yu‘s group Aalto University – Juha Karhunen‘s group U. Amsterdam – Max Welling‘s group U. California Irvine – Pierre Baldi‘s group Ghent University – Benjamin Shrauwen‘s group University of Tennessee – Itamar Arel‘s group IBM Research – Brian Kingsbury et al University of Bonn – Sven Behnke’s group Gatsby Unit @ University College London – Maneesh Sahani, YeeWhye Teh, Peter Dayan Last modified on April 10, 2020, at 1:27 pm by Caglar Gulcehre 深度學(xué)習(xí)研究組 2020/11/4 105 大數(shù)據(jù)挖掘 史忠植 2020/11/4 史忠植 智能科學(xué) 106 Thank You Intelligence Science