Ƙimar Amfani da kamanceceniya a cikin Masu Samar da Bayanan Haɓaka: Zurfin Zurfin Fasaha da Nazarin Kwatancen
Gabatarwa
A zamanin dijital na yau, sanin sirrin bayanai ya ƙaru sosai. Masu amfani suna ƙara gane bayanan su azaman sawun yatsa na dijital na musamman, suna haifar da haɗari ga keɓancewar su a yayin da aka keta bayanan. Wannan damuwa yana ƙara haɓaka da ƙa'idodi kamar GDPR, waɗanda ke ba masu amfani damar neman goge bayanan su. Yayin da ake buƙata da yawa, wannan doka na iya zama mai tsada sosai ga kamfanoni yayin da aka rage damar samun bayanai; ƙuntatawa waɗanda galibi suna ɗaukar lokaci da albarkatu don shawo kan su.
Teburin Abubuwan Ciki
Menene masu samar da bayanai na roba?
Shigar da bayanan roba, mafita ga wannan rikice-rikice. Masu samar da bayanan roba suna ƙirƙirar bayanan bayanai waɗanda ke kwaikwayi ainihin bayanan mai amfani yayin da suke kiyaye ɓoyewa da sirri. Wannan hanyar tana samun karɓuwa a cikin masana'antu, daga kiwon lafiya zuwa kuɗi, inda keɓaɓɓen keɓaɓɓu.
Yadda za a zabi hanyar samar da bayanan roba daidai?
A cikin yanayi daban-daban na samar da bayanan roba, akwai wadatattun hanyoyin da ake da su, kowanne yana neman kulawa tare da iyawar sa na musamman. Zaɓi hanyar da ta fi dacewa don takamaiman aikace-aikacen yana buƙatar cikakken fahimtar halaye na kowane zaɓi. Wannan yana buƙatar cikakken kimanta na'urorin samar da bayanai na roba daban-daban bisa ƙayyadaddun ma'auni masu kyau don yanke shawara mai fa'ida.
Abin da ke biyo baya shine ƙaƙƙarfan nazarin kwatancen Injin Syntho tare da sanannen tsarin buɗe tushen tushe, Rukunin Bayanai na Synthetic (SDV). A cikin wannan bincike, mun yi amfani da ma'auni da yawa da aka saba amfani da su kamar amincin ƙididdiga, daidaiton tsinkaya da alaƙa mai canzawa.
Ma'aunin Ƙididdigar Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwaƙwalwa )
Kafin gabatar da kowane takamaiman ma'auni, dole ne mu yarda cewa akwai akidu da yawa game da kimanta bayanan Synthetic, kowannensu yana ba da haske ga wani bangare na bayanai. Tare da wannan a zuciyarsa, nau'ikan nau'ikan guda uku masu zuwa sun fito da mahimmanci kuma cikakke. Waɗannan ma'auni suna ba da haske kan fannoni daban-daban na ingancin bayanai. Waɗannan nau'ikan sune:
- Ma'aunin Ƙididdiga na Ƙididdiga: Binciken ainihin fasalulluka na bayanan, kamar hanyoyi da bambance-bambance, don tabbatar da bayanan roba sun yi daidai da bayanan kididdiga na asali na bayanan.
- Daidaiton Hasashen: Yin nazarin aikin ƙirar ƙirƙira bayanan roba, horar da bayanan asali, da kimantawa akan bayanan roba (Train Real - Test Synthetic, TRTS) da kuma mataimakin (Train Synthetic - Test Real, TSTR)
- Dangantaka Mabambanta: Wannan rukunin haɗin gwiwar ya haɗa da:
- Daidaiton Siffar: Muna tantance yadda bayanan da aka yi amfani da su ke kula da alaƙa tsakanin masu canji ta amfani da ƙididdiga masu alaƙa. Sanannen ma'auni kamar Kuskuren Ma'anar Ma'anar Ma'ana (PMSE) zai kasance irin wannan.
- Bayanin Juna: Muna auna dogaron juna tsakanin masu canji don fahimtar zurfin waɗannan alaƙa fiye da alaƙa kawai.
Binciken Kwatanta: Injin Syntho vs. Madadin Buɗe-Source
An gudanar da nazarin kwatancen ta hanyar amfani da daidaitaccen tsarin kimantawa da dabarun gwaji iri ɗaya a cikin kowane ƙira, gami da Injin Syntho da ƙirar SDV. Ta hanyar haɗa bayanan bayanai daga tushe iri ɗaya da ƙaddamar da su ga gwaje-gwajen ƙididdiga iri ɗaya da kimanta ƙirar ƙirar na'ura, muna tabbatar da kwatanta gaskiya da rashin son zuciya. Sashen da ke biye da cikakkun bayanai game da aikin kowane janareta na bayanai na roba a cikin kewayon ma'aunin da aka gabatar a sama.
Dangane da bayanan da aka yi amfani da su don kimantawa, mun yi amfani da Ƙididdigar Ƙididdigar Manya ta UCI wanda sanannen ma'aunin bayanai ne a cikin al'ummar koyon injin. Mun share bayanan kafin duk horon sannan muka raba bayanan zuwa saiti biyu ( horo da saiti don gwaji). Mun yi amfani da tsarin horon don samar da sabbin wuraren bayanai miliyan 1 tare da kowane samfurin kuma mun kimanta ma'auni daban-daban akan waɗannan bayanan da aka samar. Don ƙarin ƙididdigar koyan inji, mun yi amfani da saitin riƙewa don kimanta ma'auni kamar waɗanda ke da alaƙa da TSTR da TRTS.
An gudanar da kowane janareta tare da sigogi na asali. Kamar yadda wasu samfuran, kamar Syntho, na iya yin aiki a waje akan kowane bayanan tabular, ba a yi gyara mai kyau ba. Neman madaidaicin hyperparameters ga kowane samfurin zai ɗauki lokaci mai mahimmanci, kuma Teburin 2 ya riga ya nuna babban bambanci tsakanin ƙirar Syntho da waɗanda aka gwada.
Abin lura ne cewa sabanin sauran samfuran a cikin SDV, Gaussian Copula Synthesizer yana dogara ne akan hanyoyin ƙididdiga. Sabanin haka, sauran sun dogara ne akan hanyoyin sadarwa na jijiyoyi kamar su Samfuran Sadarwar Sadarwa (GAN) da bambance-bambancen auto-encoders. Wannan shine dalilin da ya sa Gaussian Copula za a iya gani a matsayin tushen tushen duk samfuran da aka tattauna.
results
Ingancin Bayanai
Hoto 1. Kallon sakamakon inganci na asali don duk samfura
Ana iya samun abubuwan da aka tattauna a baya ga abubuwan da ke faruwa da kuma wakilci a cikin bayanai a cikin Hoto 1 da Table 1. Anan, kowane ma'auni da ake amfani da shi za a iya fassara shi kamar haka:
- Makin Ingancin Gabaɗaya: Gabaɗaya kimanta ingancin bayanan roba, haɗa abubuwa daban-daban kamar kamannin ƙididdiga da halayen bayanai.
- Siffofin Rukuni: Yana tantance ko bayanan roba suna kiyaye sifar rarraba iri ɗaya da ainihin bayanan kowane shafi.
- Juyin Halitta Biyu: Yana kimanta alaƙa ko alaƙa tsakanin nau'ikan ginshiƙai a cikin bayanan roba idan aka kwatanta da ainihin bayanai.
Gabaɗaya, ana iya lura cewa Syntho yana samun maki mai yawa a duk faɗin hukumar. Da farko, lokacin kallon ingancin bayanai gabaɗaya (aka kimanta tare da ɗakin karatu na awo na SDV) Syntho na iya samun sakamako sama da 99% (tare da riko da sifar shafi na 99.92% da madaidaicin siffar shafi biyu na 99.31%). Wannan shi ne yayin da SDV ke samun sakamako mafi girma na 90.84% (tare da Gaussian Copula, yana da siffar ginshiƙi na 93.82% da madaidaicin siffar shafi na 87.86%).
Tebura 1. Tambari wakilcin ingancin makin kowane saitin bayanai da aka samar a kowane samfuri
Rufin Bayanai
Tsarin Rahoton Ganewa na SDV yana kawo mana hankali cewa bayanan da aka samar da SDV (a kowane yanayi) sun ɓace fiye da 10% na jeri; A cikin yanayin Triplet-Based Variational Autoencoder (TVAE), adadin adadin bayanai iri ɗaya ma sun ɓace idan aka kwatanta da ainihin bayanan. Ba a samar da irin wannan gargaɗin tare da sakamakon da aka samu ta amfani da Syntho ba.
Hoto 2. hangen nesa na matsakaicin ma'aunin aiki mai hikima na ginshiƙi don kowane ƙira
A cikin nazarin kwatancen, makircin Hoto 2 yana misalta cewa SDV ta adana mafi kyawun sakamako a cikin ɗaukar hoto tare da wasu samfuran su (wato tare da GaussianCopula, CopulaGAN, da GAN Conditional Tabular GAN - CTGAN). Duk da haka, yana da mahimmanci a haskaka cewa amincin bayanan Syntho ya zarce na ƙirar SDV, saboda rashin daidaituwa a cikin ɗaukar hoto a cikin nau'ikan nau'ikan da jeri kaɗan ne, yana nuna bambancin 1.1% kawai. Sabanin haka, samfuran SDV suna nuna babban bambanci, kama daga 14.6% zuwa 29.2%.
Ma'aunin da aka wakilta a nan, ana iya fassara su kamar haka:
- Rufin Rubutun: Yana auna kasancewar duk rukunoni a cikin bayanan roba idan aka kwatanta da ainihin bayanai.
- Rufe Kewaye: Yana ƙididdige yadda kewayon ƙimar da ke cikin bayanan roba ya yi daidai da wancan a ainihin bayanan.
Tebura 2. Matsakaicin tambura na matsakaicin ɗaukar hoto na nau'in sifa da aka ba kowane samfuri
mai amfani
Ci gaba zuwa batun amfani da bayanan roba, batun ƙirar horarwa akan bayanan ya zama dacewa. Don samun daidaito kuma daidaitaccen kwatance tsakanin duk tsarin mun zaɓi tsohowar Gradient Boosting Classifier daga ɗakin karatu na SciKit Learn, ganin an yarda da shi a matsayin ingantaccen samfuri tare da saitunan waje.
Ana horar da nau'ikan nau'ikan nau'ikan nau'ikan guda biyu, ɗaya akan bayanan roba (na TSTR) ɗaya kuma akan ainihin bayanan (na TRTS). Ana ƙididdige samfurin da aka horar da bayanan roba ta hanyar amfani da saitin gwajin riƙewa (wanda ba a yi amfani da shi ba yayin tsara bayanan roba) kuma ana gwada samfurin da aka horar akan bayanan asali akan bayanan roba.
Hoto 3. gani na yanki a karkashin curve (Auc) scores a kowane hanya kowane samfurin
Sakamakon da aka gani a sama yana nuna fifikon samar da bayanan Synthetic ta injin Syntho idan aka kwatanta da sauran hanyoyin, ganin babu bambanci tsakanin sakamakon da aka samu ta hanyoyi daban-daban (yana nuna babban kamance tsakanin bayanan roba da na ainihi). Hakanan, layin ja mai dige-dige da ke cikin makircin shine sakamakon da aka samu ta hanyar kimanta aikin tushe na gwajin Train Real, Test Real (TRTR) don samar da tushe don ma'aunin da aka lura. Wannan layin yana wakiltar darajar 0.92, wanda shine yankin da ke ƙarƙashin ƙirar ƙirar (AUC) ta hanyar ƙira da aka horar da shi akan bayanan gaske kuma an gwada shi akan ainihin bayanan.
Tebura 3. Tambayoyi wakilci na maki AUC da TRTS da TSTR suka samu daidai da kowane samfuri.
Kwatanta-hikimar lokaci
A zahiri, yana da mahimmanci a yi la'akari da lokacin da aka kashe don samar da waɗannan sakamakon. Hoton da ke ƙasa yana kwatanta wannan kawai.
Hoto 5. Kallon lokacin da aka ɗauka don horarwa da yin aiki roba data tsara na maki miliyan ɗaya tare da samfuri tare da kuma ba tare da GPU ba.
Hoto na 5 yana kwatanta lokacin da aka ɗauka don samar da bayanan roba a cikin saituna daban-daban guda biyu. Na farkon wanda (a nan ake kira Ba tare da GPU ba), gwajin gwajin da aka yi amfani da shi tare da Intel Xeon CPU tare da muryoyin 16 masu gudana a 2.20 GHz. Gwaje-gwajen da aka yiwa alama a matsayin "gudu tare da GPU" sun kasance akan tsarin tare da AMD Ryzen 9 7945HX CPU tare da muryoyin 16 da ke gudana a 2.5GHz da NVIDIA GeForce RTX 4070 Laptop GPU. Kamar yadda aka sani a cikin Hoto 2 da kuma a cikin Teburin 2 da ke ƙasa, ana iya lura cewa Syntho yana da sauri da sauri wajen samar da bayanan roba (a cikin al'amuran biyu) wanda ke da mahimmanci a cikin aiki mai ƙarfi.
Tebura 5. Tamburan wakilcin lokacin da aka ɗauka roba data tsara na maki miliyan ɗaya tare da kowane samfuri tare da kuma ba tare da GPU ba
Sakamakon binciken ya nuna mahimmancin ingantaccen kimantawa wajen zabar ingantacciyar hanyar samar da bayanai. Injin Syntho, tare da tsarin sa na AI, yana nuna kyakkyawan ƙarfi a cikin wasu ma'auni, yayin da kayan aikin buɗaɗɗen tushe kamar SDV ke haskaka haɓakarsu da haɓakar al'umma.
Yayin da fannin bayanan roba ke ci gaba da ingantawa, muna ƙarfafa ku da ku yi amfani da waɗannan ma'auni a cikin ayyukanku, bincika ƙaƙƙarfan su, da kuma raba abubuwan da kuka samu. Kasance tare don saƙon da ke gaba inda za mu zurfafa zurfafa cikin wasu ma'auni kuma mu haskaka misalan ainihin aikace-aikacen su.
A ƙarshen rana, ga waɗanda ke neman gwada ruwa akan bayanan roba, zaɓin buɗe tushen da aka gabatar zai iya zama zaɓi mai ma'ana da aka ba da dama; duk da haka, ga ƙwararrun masu haɗa wannan fasaha ta zamani a cikin tsarin ci gaban su, duk wata dama ta ingantawa dole ne a dauki nauyin haɓaka kuma a kauce wa duk wani cikas. Don haka yana da mahimmanci a zaɓi mafi kyawun zaɓi da ke akwai. Tare da nazarin da aka bayar a sama ya zama sananne cewa Syntho kuma tare da wannan Injin Syntho kayan aiki ne mai ƙarfi ga masu aiki.
Game da Syntho
Syntho yana ba da dandamalin samar da bayanan roba mai kaifin baki, yana ba da damar nau'ikan bayanan roba da yawa da hanyoyin tsarawa, yana ƙarfafa ƙungiyoyi don canza bayanai cikin hankali zuwa gasa mai gasa. Bayanan roba da aka samar da AI ɗinmu yana kwaikwayon tsarin ƙididdiga na bayanan asali, yana tabbatar da daidaito, sirri, da sauri, kamar yadda ƙwararrun waje kamar SAS suka tantance. Tare da fasalulluka masu fa'ida masu kaifin basira da taswirar taswira, ana kiyaye mahimman bayanai yayin da ake kiyaye amincin ra'ayi. Dandalin mu yana ba da damar ƙirƙira, gudanarwa, da sarrafa bayanan gwaji don mahallin da ba samarwa ba, ta amfani da hanyoyin samar da bayanan roba na tushen doka don al'amuran da aka yi niyya. Bugu da ƙari, masu amfani za su iya samar da bayanan roba ta hanyar tsari kuma su sami bayanan gwaji na gaske don haɓaka cikakken gwaji da yanayin haɓaka cikin sauƙi.
Kuna son ƙarin koyo aikace-aikace masu amfani na bayanan roba? Jin kyauta don tsarin demo!
Game da marubutan
Injiniyan Injiniya Software
roham dalibi ne na farko a Jami'ar Fasaha ta Delft kuma ƙwararren Injiniyan Software ne a Syntho.
Masanin ilimin injiniya
Mihai ya kammala karatunsa na PhD daga jami'ar Jami'ar Bristol akan batun Koyon Ƙarfafa Ƙarfafawa na Hierarchical da aka yi amfani da su ga Robotics kuma yana da Injiniya Learning Machine at Syntho.
Ajiye jagorar bayanan roba yanzu!
- Menene bayanan roba?
- Me yasa kungiyoyi suke amfani da shi?
- Ƙimar ƙara ƙarar bayanan abokin ciniki na roba
- Yadda za'a fara
