Imibuzo Evame Ukubuzwa
Imibuzo Evame Ukubuzwa mayelana nedatha yokwenziwa
Kuyaqondakala! Ngenhlanhla, sinazo izimpendulo futhi silapha ukuze sisize. Hlola imibuzo yethu evame ukubuzwa.
Sicela uvule umbuzo ongezansi bese uchofoza izixhumanisi ukuze uthole ulwazi olwengeziwe. Ingabe unombuzo onzima kakhulu ongashiwongo lapha? Buza ochwepheshe bethu ngokuqondile!
Imibuzo ebuzwa kakhulu
Idatha yokwenziwa isho idatha ekhiqizwe ngokuzenzakalelayo esikhundleni sokuqoqwa emithonjeni yomhlaba wangempela. Ngokuvamile, kuyilapho idatha yoqobo iqoqwa kukho konke ukusebenzisana kwakho nabantu (amaklayenti, iziguli, njll.) futhi ngazo zonke izinqubo zakho zangaphakathi, idatha yokwenziwa ikhiqizwa i-algorithm yekhompyutha.
Idatha yokwenziwa ingase isetshenziselwe ukuhlola nokuhlola amamodeli endaweni elawulwayo, noma ukuvikela ulwazi olubucayi ngokukhiqiza idatha efana nedatha yomhlaba wangempela kodwa engaqukethe noma yiluphi ulwazi olubucayi. Idatha yokwenziwa ngokuvamile isetshenziswa njengenye idatha ebucayi yobumfihlo futhi ingasetshenziswa njengedatha yokuhlola, ukuhlaziya noma ukuqeqesha umshini wokufunda.
Ukuqinisekisa ukuthi idatha yokwenziwa ibamba ikhwalithi yedatha efanayo nedatha yasekuqaleni kungaba inselele, futhi ngokuvamile kuncike esimweni esithile sokusetshenziswa nezindlela ezisetshenziswa ukukhiqiza idatha yokwenziwa. Ezinye izindlela zokukhiqiza idatha yokwenziwa, njengamamodeli akhiqizayo, zingakhiqiza idatha efana kakhulu nedatha yoqobo. Umbuzo obalulekile: ungakubonisa kanjani lokhu?
Kunezindlela ezithile zokuqinisekisa ikhwalithi yedatha yokwenziwa:
- Amamethrikhi ekhwalithi yedatha ngombiko wethu wekhwalithi yedatha: Enye indlela yokuqinisekisa ukuthi idatha yokwenziwa ibamba ikhwalithi yedatha efanayo nedatha yoqobo ukusebenzisa amamethrikhi ekhwalithi yedatha ukuze uqhathanise idatha yokwenziwa kwedatha yoqobo. Lawa mamethrikhi angasetshenziswa ukukala izinto ezifana nokufana, ukunemba, nokuphelela kwedatha. Isofthiwe ye-Syntho yayihlanganisa umbiko wekhwalithi yedatha enamamethrikhi ekhwalithi yedatha ahlukahlukene.
- Ukuhlola kwangaphandle: njengoba ikhwalithi yedatha yedatha yokwenziwa uma kuqhathaniswa nedatha yoqobo ibalulekile, muva nje senze ukuhlola nochwepheshe bedatha be-SAS (umholi wemakethe kuzibalo) ukuze sibonise ikhwalithi yedatha yedatha yokwenziwa ngu-Syntho uma kuqhathaniswa nedatha yangempela. U-Edwin van Unen, uchwepheshe wezibalo wakwa-SAS, uhlole amasethi edatha okwenziwa avela kwa-Syntho ngokuhlolwa okuhlukahlukene kokuhlaziya (AI) futhi wabelane ngemiphumela. Buka isifinyezo esifushane saleyo vidiyo lapha.
- Ukuhlola nokuhlola uwedwa: idatha yokwenziwa ingahlolwa futhi ihlolwe ngokuyiqhathanisa nedatha yomhlaba wangempela noma ngokuyisebenzisela ukuqeqesha amamodeli okufunda omshini nokuqhathanisa ukusebenza kwayo namamodeli aqeqeshwe kudatha yomhlaba wangempela. Kungani ungahloli ikhwalithi yedatha yedatha yokwenziwa uwedwa? Buza ochwepheshe bethu mayelana namathuba alokhu lapha.
Kubalulekile ukuqaphela ukuthi idatha yokwenziwa ayinakuqinisekisa ukuthi izofana ngo-100% nedatha yasekuqaleni, kodwa ingaba seduze ngokwanele ukuze ibe wusizo esimweni esithile sokusetshenziswa. Lesi simo esithile sokusetshenziswa singase sibe izibalo ezithuthukisiwe noma amamodeli okufunda omshini wokuqeqesha.
'Ukungaziwa' kwakudala akuhlali kuyisixazululo esingcono kakhulu, ngoba:
- Ingozi yobumfihlo – uyohlale unayo
ingozi yobumfihlo. Ukusebenzisa lezo
amasu we-classic anonymization
kwenza kube nzima kuphela, kodwa hhayi
akunakwenzeka ukukhomba abantu ngabanye. - Icekela phansi idatha – kulapho wena
veza igama, uvikela kangcono
ubumfihlo bakho, kodwa nakakhulu wena
chitha idatha yakho. Akukhona lokhu
ufuna ama-analytics, ngoba
idatha echithiwe izoholela kokubi
ukuqonda. - Kuthatha isikhathi – kuyisixazululo
lokho kuthatha isikhathi esiningi, ngoba
lawo masu asebenza ngokwehlukile
ngesethi yedatha ngayinye kanye nohlobo ngalunye lwedatha.
Idatha yokwenziwa ihlose ukuxazulula zonke lezi ziphutha. Umehluko uyamangalisa kangangokuthi senze ividiyo ngakho. Buka lapha.
imibuzo ejwayelekile ukubuzwa
Idatha Yokwenziwa
Ngokuvamile, iningi lamaklayenti ethu asebenzisa idatha yokwenziwa:
- Ukuhlolwa kwesofthiwe nokuthuthukiswa
- Idatha yokwenziwa yezibalo, ukuthuthukiswa kwemodeli nokuhlaziya okuthuthukile (i-AI ne-ML)
- Amademo omkhiqizo
Iwele ledatha yokwenziwa liwumfanekiso okhiqizwe yi-algorithm wedathasethi yomhlaba wangempela kanye/noma isizindalwazi. Nge-Synthetic Data Twin, i-Syntho ihlose ukulingisa idathasethi yoqobo noma isizindalwazi esiseduze kakhulu nedatha yasekuqaleni ukuze kwakhe ukumelelwa okungokoqobo kokwangempela. Ngewele ledatha yokwenziwa, sihlose ikhwalithi yedatha yokwenziwa ephezulu uma kuqhathaniswa nedatha yoqobo. Lokhu sikwenza ngesofthiwe yethu yokwenziwa yedatha esebenzisa amamodeli we-AI wesimanje. Lawo mamodeli e-AI akhiqiza amaphoyinti edatha amasha ngokuphelele futhi awafanekisele ngendlela yokuthi silondoloze izici, ubudlelwano namaphethini ezibalo edatha yoqobo ngokwezinga lokuthi ukwazi ukulisebenzisa njengokungathi kuyidatha yoqobo.
Lokhu kungasetshenziselwa izinjongo ezihlukahlukene, njengamamodeli okufunda nokuqeqeshwa komshini, ukulingisa izimo zocwaningo nokuthuthukiswa, nokudala izindawo ezingokoqobo zokuqeqeshwa nokufunda. Amawele edatha yokwenziwa angasetshenziswa ukudala idatha engokoqobo nemele engasetshenziswa esikhundleni sedatha yomhlaba wangempela uma ingatholakali noma uma ukusebenzisa idatha yomhlaba wangempela kungenakusebenza noma okungekho emthethweni ngenxa yemithetho eqinile yobumfihlo bedatha.
Yebo siyakwenza. Sinikeza izici ezihlukahlukene zokwenziwa zedatha yokwenziwa ezingeza inani kanye nezici zokwandisa, okuhlanganisa abaklolodayo, ukuze siyise idatha yakho ezingeni elilandelayo.
Idatha mbumbulu kanye nedatha yokwenziwa ekhiqizwe yi-AI zombili izinhlobo zedatha yokwenziwa, kodwa ikhiqizwa ngezindlela ezihlukene futhi zisebenzisa izinjongo ezihlukile.
Idatha ye-Mock iwuhlobo lwedatha yokwenziwa eyenziwa ngesandla futhi evame ukusetshenziselwa izinjongo zokuhlola nokuthuthukisa. Ngokuvamile isetshenziselwa ukulingisa ukuziphatha kwedatha yomhlaba wangempela endaweni elawulwayo futhi ngokuvamile isetshenziselwa ukuhlola ukusebenza kwesistimu noma uhlelo lokusebenza. Ngokuvamile kulula, kulula ukuyikhiqiza, futhi ayidingi amamodeli ayinkimbinkimbi noma ama-algorithms. Imvamisa, omunye obhekisisayo ubuye agcone idatha njengokuthi "idatha eyimbumbulu" noma "idatha yomgunyathi".
Idatha yokwenziwa ekhiqizwe yi-AI, ngakolunye uhlangothi, ikhiqizwa kusetshenziswa amasu obuhlakani bokwenziwa, njengokufunda komshini noma amamodeli akhiqizayo. Isetshenziselwa ukudala idatha engokoqobo nemele engasetshenziswa esikhundleni sedatha yomhlaba wangempela lapho ukusebenzisa idatha yomhlaba wangempela kungenakusebenza noma okungekho emthethweni ngenxa yemithetho eqinile yobumfihlo. Ivamise ukuba yinkimbinkimbi futhi idinga izinsiza zokubala kakhulu kunedatha mbumbulu eyenziwa mathupha. Njengomphumela, ingokoqobo kakhulu futhi ilingisa idatha yoqobo eduze ngangokunokwenzeka.
Kafushane, idatha mbumbulu yenziwa mathupha futhi ngokuvamile isetshenziselwa ukuhlola nokuthuthukiswa, kuyilapho idatha yokwenziwa ekhiqizwe yi-AI idalwa kusetshenziswa amasu obuhlakani bokwenziwa futhi isetshenziselwa ukudala idatha emele nengokoqobo.
Ikhwalithi yedatha
Ukuqinisekisa ukuthi idatha yokwenziwa ibamba ikhwalithi yedatha efanayo nedatha yasekuqaleni kungaba inselele, futhi ngokuvamile kuncike esimweni esithile sokusetshenziswa nezindlela ezisetshenziswa ukukhiqiza idatha yokwenziwa. Ezinye izindlela zokukhiqiza idatha yokwenziwa, njengamamodeli akhiqizayo, zingakhiqiza idatha efana kakhulu nedatha yoqobo. Umbuzo obalulekile: ungakubonisa kanjani lokhu?
Kunezindlela ezithile zokuqinisekisa ikhwalithi yedatha yokwenziwa:
- Amamethrikhi ekhwalithi yedatha ngombiko wethu wekhwalithi yedatha: Enye indlela yokuqinisekisa ukuthi idatha yokwenziwa ibamba ikhwalithi yedatha efanayo nedatha yoqobo ukusebenzisa amamethrikhi ekhwalithi yedatha ukuze uqhathanise idatha yokwenziwa kwedatha yoqobo. Lawa mamethrikhi angasetshenziswa ukukala izinto ezifana nokufana, ukunemba, nokuphelela kwedatha. Isofthiwe ye-Syntho yayihlanganisa umbiko wekhwalithi yedatha enamamethrikhi ekhwalithi yedatha ahlukahlukene.
- Ukuhlola kwangaphandle: njengoba ikhwalithi yedatha yedatha yokwenziwa uma kuqhathaniswa nedatha yoqobo ibalulekile, muva nje senze ukuhlola nochwepheshe bedatha be-SAS (umholi wemakethe kuzibalo) ukuze sibonise ikhwalithi yedatha yedatha yokwenziwa ngu-Syntho uma kuqhathaniswa nedatha yangempela. U-Edwin van Unen, uchwepheshe wezibalo wakwa-SAS, uhlole amasethi edatha okwenziwa avela kwa-Syntho ngokuhlolwa okuhlukahlukene kokuhlaziya (AI) futhi wabelane ngemiphumela. Buka isifinyezo esifushane saleyo vidiyo lapha.
- Ukuhlola nokuhlola uwedwa: idatha yokwenziwa ingahlolwa futhi ihlolwe ngokuyiqhathanisa nedatha yomhlaba wangempela noma ngokuyisebenzisela ukuqeqesha amamodeli okufunda omshini nokuqhathanisa ukusebenza kwayo namamodeli aqeqeshwe kudatha yomhlaba wangempela. Kungani ungahloli ikhwalithi yedatha yedatha yokwenziwa uwedwa? Buza ochwepheshe bethu mayelana namathuba alokhu lapha.
Kubalulekile ukuqaphela ukuthi idatha yokwenziwa ayinakuqinisekisa ukuthi izofana ngo-100% nedatha yasekuqaleni, kodwa ingaba seduze ngokwanele ukuze ibe wusizo esimweni esithile sokusetshenziswa. Lesi simo esithile sokusetshenziswa singase sibe izibalo ezithuthukisiwe noma amamodeli okufunda omshini wokuqeqesha.
Yebo kunjalo. Idatha yokwenziwa ibamba ngisho namaphethini obungazi ukuthi akhona kudatha yoqobo.
Kodwa musa nje ukuthatha izwi lethu ngakho. Ochwepheshe bezibalo be-SAS (umholi wemakethe yomhlaba wonke ekuhlaziyeni) benze ukuhlola (AI) kwedatha yethu yokwenziwa futhi bayiqhathanisa nedatha yoqobo. Ufuna ukwazi? Buka i- wonke umcimbi lapha noma ubuke inguqulo emfushane mayelana ikhwalithi yedatha lapha.
Yebo siyakwenza. Inkundla yethu yenzelwe izizindalwazi futhi ngenxa yalokho, ukulondolozwa kobuqotho obuyinkomba phakathi kwamasethi edatha ku-datgabase.
Ufuna ukwazi okwengeziwe ngalokhu?
Inqubomgomo
Cha asikwenzi. Singasebenzisa kalula i-Syntho Engine endaweni noma efwini lakho eliyimfihlo nge-docker.
Cha. Sithuthukise inkundla yethu ngendlela yokuthi ingafakwa kalula endaweni ethembekile yekhasimende. Lokhu kuqinisekisa ukuthi idatha ayisoze yashiya indawo ethembekile yekhasimende. Izinketho zokusebenzisa zendawo ethembekile yekhasimende "zikhona endaweni" futhi "indawo yamafu yekhasimende (ifu langasese)".
Ongakukhetha: I-Syntho isekela inguqulo esingethwe "kumafu we-Syntho".
Cha. I-Syntho Engine iyinkundla yokuzisiza. Njengomphumela, ukukhiqiza idatha yokwenziwa nge-Syntho Engine kungenzeka ngendlela yokuthi end-to-end inqubo, u-Syntho akakwazi ukubona futhi akadingeki ukucubungula idatha.
Yebo sikwenza lokhu ngombiko wethu we-QA.
Lapho uhlanganisa idathasethi, kubalulekile ukubonisa ukuthi umuntu akakwazi ukuphinda akhombe abantu ngabanye. Ku le vidiyo, u-Marijn wethula izilinganiso zobumfihlo ezisembikweni wethu wekhwalithi ukuze abonise lokhu.
Umbiko we-QA ka-Syntho uqukethe ezintathu imboni-izinga amamethrikhi okuhlola ubumfihlo bedatha. Umbono ngemuva kwemethrikhi ngayinye umi kanje:
- Idatha yokwenziwa (S) izoba “seduze ngangokunokwenzeka”, kodwa “ingabi seduze kakhulu” nedatha eqondiwe (T).
- Idatha yokubamba ekhethwe ngokungahleliwe (H) inquma ibhentshimakhi yokuthi "isondele kakhulu".
- A isixazululo esiphelele ikhiqiza idatha yokwenziwa entsha esebenza njengedatha yoqobo, kodwa engakaze ibonwe ngaphambilini (= H).
Esinye sezimo zokusebenzisa esigqanyiswa ngokukhethekile yi-Dutch Data Protection Authority sisebenzisa idatha yokwenziwa njengedatha yokuhlola.
Injini ye-Syntho
I-Syntho Engine ithunyelwa ngesitsha se-Docker futhi ingafakwa kalula futhi ixhunywe endaweni oyithandayo.
Izinketho ezingase zisetshenziswe zihlanganisa:
- On-premise
- Noma yiliphi ifu (eliyimfihlo).
- Noma iyiphi enye indawo
I-Syntho ikuvumela ukuthi uxhume kalula kusizindalwazi sakho, izinhlelo zokusebenza, amapayipi edatha noma amasistimu wefayela.
Sisekela izixhumi ezihlukahlukene ezihlanganisiwe ukuze ukwazi ukuxhumana nomthombo-imvelo (lapho kugcinwa khona idatha yoqobo) kanye nendawo okuyiwa kuyo (lapho ufuna ukubhala khona idatha yakho yokwenziwa) ukuze uthole end-to-end indlela edidiyelwe.
Izici zokuxhuma esizisekelayo:
- Xhuma futhi udlale nge-Docker
- Izixhumi zedatha engu-20+
- 20+ izixhumi zesistimu yefayela
Ngokwemvelo, isikhathi sokukhiqiza sincike kusayizi wesizindalwazi. Ngokwesilinganiso, ithebula elinamarekhodi angaphansi kwesigidi esisodwa lihlanganiswa ngaphansi kwemizuzu emi-1.
Ama-algorithms wokufunda komshini we-Syntho angenza izici zibe ngcono kangcono ngamarekhodi ebhizinisi atholakalayo, okunciphisa ubungozi bobumfihlo. Kunconywa isilinganiso esincane sekholomu kumugqa esingu-1:500. Isibonelo, uma ithebula lakho lomthombo linamakholomu angu-6, kufanele liqukathe ubuncane bemigqa engu-3000.
Lutho neze. Nakuba kungase kuthathe umzamo othile ukuqonda ngokugcwele izinzuzo, ukusebenza kanye nokusetshenziswa kwamacala edatha yokwenziwa, inqubo yokuhlanganisa ilula kakhulu futhi noma ubani onolwazi oluyisisekelo lwekhompyutha angayenza. Ukuze uthole ukwaziswa okwengeziwe mayelana nenqubo yokuhlanganisa, hlola leli khasi or cela i-demo.
I-Syntho Engine isebenza kangcono kudatha yethebula ehleliwe (noma yini equkethe imigqa namakholomu). Ngaphakathi kwalezi zakhiwo, sisekela izinhlobo zedatha ezilandelayo:
- Idatha yezakhiwo ifomethwe kumathebula (isigaba, inombolo, njll.)
- Izihlonzi eziqondile kanye ne-PII
- Amasethi edatha amakhulu nezizindalwazi
- Idatha yendawo yendawo (njenge-GPS)
- Idatha yochungechunge lwesikhathi
- Imininingo egciniwe yamathebula amaningi (enobuqotho obuyinkomba)
- Vula idatha yombhalo
Ukusekelwa kwedatha okuyinkimbinkimbi
Eduze kwazo zonke izinhlobo ezivamile zedatha yethebula, i-Syntho Engine isekela izinhlobo zedatha eziyinkimbinkimbi kanye nezakhiwo zedatha eziyinkimbinkimbi.
- Uchungechunge lwesikhathi
- Multi-table database
- Vula umbhalo
Cha, sithuthukise inkundla yethu ukuze sinciphise izidingo zokubala (isb. ayikho i-GPU edingekayo), ngaphandle kokuphazamisa ukunemba kwedatha. Ngaphezu kwalokho, sisekela ukukala okuzenzakalelayo, ukuze umuntu akwazi ukuhlanganisa imininingwane emikhulu.
Yebo. Isofthiwe ye-Syntho yenzelwe isizindalwazi esiqukethe amathebula amaningi.
Mayelana nalokhu, i-Syntho ithola ngokuzenzakalelayo izinhlobo zedatha, izikimu namafomethi ukuze kukhuliswe ukunemba kwedatha. Ngolwazi olugciniwe olunamathebula amaningi, sisekela okuzenzakalelayo kobudlelwano betafula kanye nokuhlanganisa ukuze kugcinwe ubuqotho obuyinkomba.
Idatha iyenziwe, kodwa ithimba lethu lingokoqobo!
Xhumana noSyntho futhi omunye wochwepheshe bethu uzoxhumana nawe ngesivinini sokukhanya ukuze ahlole inani ledatha yokwenziwa!