Tibari Bwongo Bumwe, Naye Ebingi: Enkora ya Mixture of Experts (MoE) Ekyongerera AI Obwengye n’Obwangu

Nk’ensimbi za AI zikura mu maani, n’okukura kwaazo n’okuguma kukaba kukuru muno. Okuzikozesa n’okuzihugurisa bisaba amaani menshi g’ekyuma, ekiyikora eby’amaani n’okuba bingi omu kugarukamu n’okubihendaho. Naye enshonga ey’omugasha ey’omunda eyetwa Mixture of Experts (MoE) erikuhindura ebyo.

Ahubwo ya bwongo bumwe obukuru obushaba kumanya byonna, omu MoE model nki ekikundi ky’abahanga ab’enkora zitandikanye. Buli buuzo bwija, model ebushemererwa okureetera abahanga abarikukwataho kurungi kurusha abandi.

Enkora Enkuru Ekyahera: Model Ey’Eyebisi, Ey’obuguru Bumwe

Mu mateeka gakare, Large Language Models zaali “dense” (zitarikuhwaho). Ekyo kisoboka ngu buli bwo ubibuza buuzo—ahabw’obwangu oba obukome—network yonna ya neurons (eyine billions y’obuhangi) n’eyaka yose okugyenda okukora ekyo buuzo.

Kitekateke nki professor omu asa kumanya buli subject. Obu omubuza “2+2 ki?”, abanza kukozesa obwongo bwe bwona—obwonga obumanya physics, Shakespeare, n’ebya kicweka kya Greece—okuguha ansa y’okubanza. N’obugumu muno, naye n’obuhendeka bukuru.

Enkora Empya: Ekikundi kya Mixture of Experts (MoE)

Model ya MoE ekora nka “divide and conquer.” Erimu ebicweka bibiri ebikulu:

1. Abahanga (Experts): Ahubwo ya network emwe, model etandikirwa mu bisinga ebito ebirukwatiraho ebikora bitandikanye, ebiyetwa “experts.” Buli expert asobora kuba omuhangi mu kintu eky’omutwe—nki okwandika, okucodinga, okutunula eby’obuhangwa, oba okutekereza n’obwenge.
2. Router: Kino nikyo kicweka ekirimu oburinganiza. Nki project manager oba receptionist. Kyo kikora kyonka kwekureeba buuzo obwija n’okuteera abahanga abakwataho kurungi omu ekyo kibazo.

Enkora yaayo:

Obu obuuzire MoE model buuzo nka “Wandika Python script eyokuhandiika eby’amateka g’ensimbi,” ebyo nibikora n’okugyenda n’omurongo ogu:

1. Router akatunuulira buuzo bwahweho.
2. Akarondora ngu ekyo kibazo kikenaga kumanya eby’okucodinga na financial history.
3. Akahita akoresha “Python Coding Expert” n’“Financial History Expert” bokka.
4. Abandi abahanga—nki “Poetry Expert” oba “Biology Expert”—basigara, batakora, okusasira energy.
5. Abahanga abahindikirwe bakora hamwe, bakakora ansa ya nyuma.

Enkora Eyahindura Omuzanyo: Ebihango bya MoE

Eyi architecture ey’omugasha erimu ebihango ebikuru ebirikuhindura ensi ya AI:

• Obwangu n’Okwehendeka: Kubanga akatundu katono k’amodel niko kakozesibwa buli bwo omukozesa abibuza, MoE models nizo zangu kandi zihehenda bike kurusha dense models ezikuru muno. Kino nikyo kintu kyazo ekirungi enyo.
• Okukura Ahahango: Abahandi n’obuhanga bashobora kuziyongera obw’omanyi n’obushobozi (n’okuteka abahanga abangi) obutazihindura kuba zirabura mu bwangu. Osobora kuba na model eyine trillion parameters, naye ikozesa nka 200 billion buli mirimu, n’eyikora nk’eyangwe.
• Okwegyerera Obuhanga: Kubanga buli expert ashobora kwetegyereza mu domain emwe, model esobora kugira obumanyi obuzibu mu bintu ebihingwire, ekyongera ku quality y’ebyavudemu mu bibuuzo bikomye.

Models nka Mistral’s Mixtral 8x7B na Grok ya xAI zirikuzimbwa aha architecture eyo. Nizo zirikutegereza ensi ya AI eyirikuba n’obumanyi obw’omugisha n’obwangu obuhambane, okutuleetera ku ntambwe emwe aha AI ey’obuhangwa n’ey’abanitu kwetegyereza.