鏈€杩戯紝鍏充簬鑱婂ぉ鏈哄櫒浜?strong>ChatGPT锛圕hat Generative Pre-Training Transformer锛夌殑鐩稿叧璁ㄨ甯嵎浜嗗悇涓ぞ浜ゅ钩鍙般€?/p>
浠婂ぉ姝ｅソ鏄簩鏈堜簩锛屽浜庝负浠€涔堣鐞嗗彂杩欎釜闂锛孋hatGPT鏄繖鏍峰洖绛旂殑锛?/p>
涓婇潰杩欎釜鍥炵瓟鎸烘甯哥殑锛屼絾杩介棶鈥滅悊浠€涔堝彂鍨嬧€濈殑鏃跺€欙紝瀹冪殑鍥炵瓟娌℃湁澶鎯婂枩銆?/p>
浣咰hatGPT绌剁珶鏄€滀綍鏂圭鍦ｂ€濓紝涓轰粈涔堣兘寮曠垎璇濋锛熶粖澶╂垜浠氨鏉ユ墥涓€鎵掋€?/p>
ChatGPT鍜屽畠鐨勨€滀笁鍏勫紵鈥?/strong>
ChatGPT鐨勪腑鏂囧叏绉颁负鈥滈璁粌鐢熸垚鑱婂ぉ妯″瀷鈥?/strong>锛屾槸涓€绉嶈嚜鐒惰瑷€澶勭悊妯″瀷銆傚湪ChatGPT寮曞彂杞板姩涔嬪墠锛?GPT1锛?018骞存帹鍑猴級 銆丟PT2锛?019锛夈€?GPT3锛?020锛夋ā鍨嬫棭宸茬浉缁ч棶涓栵紝瀹冧滑閮芥槸鐢監penAI鍥㈤槦鐮斿彂鐨勩€?/p> 鍏朵腑锛孏PT1棣栧厛瑕佸湪澶ч噺鐨勬暟鎹笂杩涜棰勮缁?/strong>锛坧re鈥攖raining锛夛紝涔嬪悗鍦ㄦ斁鍒版洿鍔犵粏鍖栨ā鍨嬩笂杩涜寰皟銆?/p> 鎵€璋?strong>寰皟 GPT1鍦?strong>鏂囨湰钑村惈銆佹枃妗ｅ綊绫汇€侀棶绛斻€佽瑷€鐩镐技搴?/strong>绛変笅娓镐换鍔′腑琛ㄧ幇浼樺紓銆?/p> 鎵€璋撴枃鏈暣鍚紝鍗充竴鍙ヨ瘽涓殣鍚潃鍙︿竴鍙ヨ瘽鐨勫惈涔夈€備緥濡傦紝鈥滃姫鍔涚殑浜虹粓鑳借幏寰楁垚鍔熲€濓紝杩欏彞璇濅腑钑村惈鐫€鈥滄垚鍔熸槸涓€浠跺叿鏈夌Н鏋佹剰涔夌殑浜嬫儏锛屼汉浠€氬父浼氭湡寰呭畠鐨勫彂鐢熲€濄€?/p> 鏂囨湰鍒嗙被锛岄【鍚嶆€濅箟锛屽氨鏄皢鏂囨湰鍒嗘垚涓嶅悓鐨勭被鍒€傝瑷€鐩镐技搴︼紝鎸囦袱鍙ヨ瘽涓涔夋槸鍚︽帴杩戙€?/p> GPT1鎷ユ湁鍙傛暟1.17浜夸釜銆侴PT2鍦℅PT1鐨勫熀纭€涓婄簿杩涗簡妯″瀷锛屽弬鏁伴噺澧炲姞鑷?5浜匡紝棰勮缁冩暟鎹泦澧炲姞鑷?0GB銆?strong>澧為噺宸ㄥぇ鐨勫弬鏁板拰棰勮缁冧娇寰楁暣涓ā鍨嬬殑閫氱敤鎬ф洿濂斤紝杩涜€岀渷鐣ヤ簡妯″瀷寰皟鐨勮繃绋嬨€?/strong> 濡傛灉璇碐PT1鐩稿綋浜庝竴涓湪寰皟涔嬪悗鑳藉鎴愪负涓嶅悓棰嗗煙鐨勪笓瀹讹紝GPT2灏辨槸涓嶇敤寰皟锛岀洿鎺ヨ兘澶熷湪鍚勪釜棰嗗煙澶ф樉韬墜鐨勨€滃叏鑳介珮鎵嬧€濄€?/p> GPT3鐨勭粨鏋勪笌GPT2鐩稿樊涓嶅ぇ锛屽尯鍒富瑕佸湪浜嶨PT3鐨勫弬鏁板鍔犺嚦1750浜?/strong>锛屾嫢鏈夋洿鍔犳捣閲忕殑棰勮缁冩暟鎹€斺€旂害45TB銆傝繖浜涢璁粌鏁版嵁鍖呮嫭涔︾睄銆佹潅蹇楃敋鑷充笓涓氳鏂囩瓑绛夈€?/p> 鍦℅PT3涔嬪悗锛屽張鍑虹幇浜咷PT3.5锛?strong>濡備粖澶х伀鐨凜hatGPT姝ｆ槸鍩轰簬GPT3.5鏋舵瀯寮€鍙戠殑銆?/strong> 瑙傚療GPT瀹舵棌鐨勫彂灞曞巻绋嬶紝鍙傛暟閲忓拰璁粌闆嗚倝鐪煎彲瑙佸湴涓嶆柇澧炲姞銆?/strong> 杩欏拰浜虹被瀛︿範鐨勮繃绋嬩綍鍏剁浉浼笺€備汉绫绘兂瑕佸仛鍑轰竴浜涘垱閫狅紝棣栧厛瑕佹敹闆嗚冻澶熷鐨勬劅鎬ф潗鏂欍€傛兂瑕佹垚涓哄皬璇村锛岄鍏堣璇诲埆浜虹殑灏忚锛屼簡瑙ｅ皬璇寸殑鍐欎綔妯″紡銆傝寰楄秺澶氾紝璁よ瘑灏辫秺娣卞埢銆?/p> 涓嶄粎鏄亰澶╂満鍣ㄤ汉锛屽墠闃靛瓙澶х儹鐨凙I缁樼敾杞欢锛屼篃鏄€氳繃瀛︿範澶ч噺璇枡鍜屽浘鍍忚祫鏂欙紝浠庤€屽畬鎴愮敓鎴愮粯鐢荤殑浠诲姟銆?/p> AI缁樼敾鐨勪换鍔″彨text-to-image锛堟枃瀛楀埌鍥剧墖锛夈€傚畠棣栧厛瑕佸涔犳枃瀛椾笌鍥惧儚涔嬮棿鐨勫叧绯汇€傛瘮濡傦紝褰撶敤鎴疯緭鍏モ€滅嫍鈥濈殑姒傚康锛岃鎯宠AI缁樼敾鐢诲嚭鈥滅嫍鈥濈殑鍥惧儚锛岄鍏堝氨寰楄瀹冪煡閬撯€滅嫍鈥濊繖涓€璇嶈涓庘€滅嫍鈥濈殑褰㈣薄涔嬮棿鐨勮仈绯伙紝鍏舵锛岃繕蹇呴』鐞嗚В鈥滅嫍鈥濈殑姒傚康锛屽惁鍒欏彲鑳芥妸鐙楃敾鎴愮尗鎴栬€呰€侀紶銆?/p> 鍥炬簮锛欿aggle Dogs vs. Cats Redux: Kernels Edition | Kaggle 鍥犳锛屽ぇ閲忓涔犮€佸弽澶嶇籂閿欐垚浜嗚缁傾I蹇呬笉鍙皯鐨勮繃绋嬨€傝繖涓€鐐瑰拰浜虹被瀛︿範涔熸瀬鍏剁浉浼笺€傚浜庡効绔ユ潵璇达紝瀛︿範鍐欌€滅尗鈥濃€滅嫍鈥濅笌璁よ瘑銆佸尯鍒嗗浘鍍忎腑鐨勭尗銆佺嫍閮芥槸蹇呰鐨勮繃绋嬨€?/p> ChatGPT浠儗鍚庣殑鎶€鏈ā鍨?/strong> 鏃犺鏄棭鏈熺殑GPT锛岃繕鏄綋涓嬪ぇ鐑殑ChatGPT锛屽畠浠兘鍩轰簬涓€绉嶅叧閿妧鏈細Transformer銆?017骞达紝璋锋瓕鎻愬嚭浜員ransformer妯″瀷锛屼腑鏂囧悕绉颁负鈥滃彉褰㈣€呪€濄€?/p> 
Transformer鏁翠綋缁撴瀯锛岀敱缂栫爜锛圗ncoder锛夊拰瑙ｇ爜锛圖ecoder锛変袱閮ㄥ垎鏋勬垚锛?/p>
鍥炬簮锛氱煡涔嶡鍒濊瘑CV
Transformer鏈変粈涔堢敤锛熷畠鐨勫伐浣滄祦绋嬪彲浠ョ畝鍗曠悊瑙ｆ垚锛屽綋鎴戜滑鍦ㄥ仛鏂囨湰缈昏瘧浠诲姟鏃讹紝杈撳叆杩涘幓涓€涓腑鏂囷紝缁忚繃杩欎釜Transformer妯″瀷鍚庯紝杈撳嚭鏉ョ炕璇戣繃鍚庣殑鑻辨枃銆?/p>
鍥炬簮锛氱煡涔嶡Robin.Q7
Transformer妯″瀷閲屾湁涓€涓噸瑕佺殑妯″紡锛?strong>娉ㄦ剰鍔涙満鍒讹紙Attention锛?/strong>锛屼竴鑸敱Query锛堟煡璇級銆並ey锛堝叧閿瓧锛夈€乂alue锛堝€硷級绛夐儴鍒嗙粍鎴愩€備笁涓儴鍒嗗悇鑷鐫€杈撳叆閮ㄥ垎鎻愬彇淇℃伅锛岀粡杩囧眰灞傜疮璁★紝鏈€缁堝叧娉ㄥ埌杈撳叆涓叧閿殑閮ㄥ垎锛屼粠鑰屽畬鎴愪换鍔°€?/p>
QKV璁＄畻鍥?/p>
鍥炬簮锛氥€夾ttention Is All You Need銆?/p>
Transformer妯″瀷鍑虹幇鐨勬剰涔夋繁杩滐紝鐗瑰埆鏄畠鐨勬敞鎰忓姏鏈哄埗锛屽悗缁嚭鐜扮殑鑷劧璇█澶勭悊妯″瀷澶氭槸鍦ㄥ畠鐨勫熀纭€涓婃敼閫犵殑銆俆ranformer妯″瀷琚箍娉涘簲鐢ㄤ簬鑷劧璇█澶勭悊銆佽绠楁満瑙嗚绛夐鍩熴€?/p>
ChatGPT璧㈠湪浜嗗摢鍎匡紵
褰撲笅锛岃闊虫満鍣ㄤ汉濡俿iri銆佸皬鐖卞悓瀛︾瓑宸茬粡娣卞叆鎴戜滑鐨勭敓娲汇€侰hatGPT鐨勫嚭鐜帮紝璁╂垜浠湅鍒颁簡鑱婂ぉ鏈哄櫒浜虹殑鏇村ぇ鍙兘銆?/p>
灏忕埍鍚屽銆乻iri鍙互鎵ц濡傛挱鏀炬瓕鏇蹭箣绫荤殑鎸囦护锛岃兘澶熷簲瀵逛竴浜涚畝鍗曠殑闂瓟锛屼絾鏄紝闅忕潃瀵硅瘽娆℃暟澧炲锛岀敤鎴峰緢蹇氨浼氬彂瑙夛紝鑷繁鏄湪鍜屾満鍣ㄥ璇濓紝瀹冪粡甯镐細缁欏嚭涓€浜涗护浜哄摥绗戜笉寰楃殑鍥炵瓟锛岃€屾棤娉曠粰鍑烘洿杩戜技浜庝汉鐨勫洖绛斻€?/p>
ChatGPT鍙互缁欏嚭鏇村姞缁嗚吇銆佷汉鎬у寲鐨勫洖绛旓紝涓庡畠浜よ皥锛屼細鏇存帴杩戜簬鍜屼汉瀵硅瘽銆傞櫎姝や箣澶栵紝ChatGPT鑳藉鐢熸垚涓€浜涗笓涓氶鍩熺殑闂瑙ｇ瓟銆佸洖搴旇亰澶╃敋鑷宠嚜鍔ㄧ敓鎴愯鏂囷紝鐞嗚В璇箟鐨勮兘鍔涙洿寮恒€?/p>
涓轰粈涔圕hatGPT鑳芥湁杩欐牱鐨勪紭鍔匡紵
棣栧厛锛屽綋鐒舵槸寮€澶存彁鍒扮殑锛孋hatGPT鎷ユ湁瑙勬ā鏇村ぇ鐨勮缁冩暟鎹拰鍙傛暟锛屽涔犺祫鏂欎赴瀵屻€傚叾娆★紝ChatGPT鐨凾ransformer妯″瀷鏄彲浠?strong>璁板綍鏃堕棿搴忓垪鐨勶紝瀹冭兘璁板綍鐢ㄦ埛鍦ㄤ笂涓€鍒昏杩囩殑璇濓紝浠庤€屼娇寰楀璇濇湁杩炵画鎬э紝鑰屼笉鏄満姊板湴鍥炵瓟闂銆?/p>
浣嗗畠鏈夋椂鍊欎篃浼氫竴鏈缁忓湴鐬庤鍏亾銆?/p>
鍥炬簮锛氫腑鍥芥櫘娉曞井淇″叕浼楀彿
涔嬫墍浠ヤ細鍑虹幇杩欑鎯呭喌锛屼竴鏂归潰鏄洜涓哄畠瀛︿範鐨勫噯纭€у皻鏈変笉瓒筹紝瀵硅涔夌殑鐞嗚В鍙兘鏈夊亸宸紝浠ヨ嚦浜庘€滃樊涔嬫鍘橈紝璋箣鍗冮噷鈥濄€傚彟涓€鏂归潰锛屽畠鐨勮缁冩潗鏂欒妯″簽澶э紝鏃犳硶缁濆鍦颁繚璇佸涔犲埌鐨勭煡璇嗘湰韬殑姝ｇ‘鎬с€?/p>
鎬荤粨涓嬫潵锛孋hatGPT鍙槸涓€涓彁渚涗究鍒╃殑宸ュ叿锛岃€屾垜浠兘鍋氱殑锛屽氨鏄湪鍚堥€傜殑鑼冨洿鍐咃紝鍚堢悊鍦拌繍鐢ㄥ畠锛屼粠鑰岃緟鍔╂垜浠洿楂樻晥鍦板畬鎴愬伐浣溿€?/p>
鍙傝€冩枃鐚?/p>
[1]聽聽Ashish Vaswani, Noam Shazeer, Niki Parmar et al.Attention Is All You Need. Computation and Language (cs.CL); Machine Learning (cs.LG).
[2]聽Alec Radford, Karthik Narasimhan, Tim Salimans et al. Improving Language Understanding by Generative Pre-Training.
[3] Alec Radford , Jeffrey Wu,Rewon Child et al.Language Models are Unsupervised Multitask Learners.2019. OpenAI blog, 1(8), p.9.
[4]聽Tom B. Brown锛孊enjamin Mann锛孨ick Ryder et al. Language Models are Few-Shot Learners.2020.
浣滆€咃細鏉庨湝姘わ紝绉戞櫘绉戝够鍒涗綔鑰呫€佽绠楁満绉戠爺宸ヤ綔鑰?/p>
缂栬緫锛氫竴浜虹櫧
楦ｈ阿锛氫笂娴蜂氦閫氬ぇ瀛﹁绠楁満绉戝涓庡伐绋嬬郴鍓暀鎺?鍚存ⅵ鐜?涓烘湰鏂囨彁渚涚瀛︽寚瀵?/p>
