Tea@programming.dev to Technology@lemmy.worldEnglish · 8 months agoWikipedia is giving AI developers its data to fend off bot scrapersenterprise.wikimedia.comexternal-linkmessage-square9linkfedilinkarrow-up195arrow-down13
arrow-up192arrow-down1external-linkWikipedia is giving AI developers its data to fend off bot scrapersenterprise.wikimedia.comTea@programming.dev to Technology@lemmy.worldEnglish · 8 months agomessage-square9linkfedilink
minus-squareMonument@lemmy.sdf.orglinkfedilinkEnglisharrow-up13·8 months agoNo, because there isn’t a single IP range or user agent, and many developers are going to lengths to defeat anti-scraping measures, which include user agent spoofing as well as vpns and the like to mask the source of the traffic.
No, because there isn’t a single IP range or user agent, and many developers are going to lengths to defeat anti-scraping measures, which include user agent spoofing as well as vpns and the like to mask the source of the traffic.