86thumbs
  • Communities
  • Create Post
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
fattyfoods@feddit.nl to Open Source@lemmy.ml · 1 day ago

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

external-link
message-square
97
fedilink
  • cross-posted to:
  • technology@beehaw.org
513
external-link

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

fattyfoods@feddit.nl to Open Source@lemmy.ml · 1 day ago
message-square
97
fedilink
  • cross-posted to:
  • technology@beehaw.org
Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.
  • unexposedhazard@discuss.tchncs.de
    link
    fedilink
    arrow-up
    97
    ·
    edit-2
    24 hours ago

    Non paywalled link https://archive.is/VcoE1

    It basically boils down to making the browser do some cpu heavy calculations before allowing access. This is no problem for a single user, but for a bot farm this would increase the amount of compute power they need 100x or more.

    • exu@feditown.com
      link
      fedilink
      English
      arrow-up
      4
      ·
      6 hours ago

      It inherently blocks a lot of the simpler bots by requiring JavaScript as well.

    • Mubelotix@jlai.lu
      link
      fedilink
      arrow-up
      60
      ·
      23 hours ago

      Exactly. It’s called proof-of-work and was originally invented to reduce spam emails but was later used by Bitcoin to control its growth speed

      • JackbyDev@programming.dev
        link
        fedilink
        English
        arrow-up
        1
        ·
        4 hours ago

        It’s funby that older captchas could be viewed as proof of work algorithms now because image recognition is so good. (From using captchas.)

      • RandomTester@lemmybefree.net
        link
        fedilink
        arrow-up
        3
        ·
        10 hours ago

        it wasn’t made for bitcoin originally? didn’t know that!

        • 0xebfe@lemmy.today
          link
          fedilink
          arrow-up
          9
          arrow-down
          1
          ·
          9 hours ago

          Originally called hashcash: http://hashcash.org/

          • daw@feddit.org
            link
            fedilink
            arrow-up
            2
            ·
            5 hours ago

            TIL

          • RandomTester@lemmybefree.net
            link
            fedilink
            arrow-up
            8
            ·
            8 hours ago

            you know it’s old when it doesn’t have ssl

    • lazynooblet@lazysoci.al
      link
      fedilink
      English
      arrow-up
      6
      ·
      23 hours ago

      Thank you for the link. Good read

Open Source@lemmy.ml

opensource@lemmy.ml

remote_follow_modal_title

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !opensource@lemmy.ml

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

  • Open Source Initiative
  • Free Software Foundation
  • Electronic Frontier Foundation
  • Software Freedom Conservancy
  • It’s FOSS
  • Android FOSS Apps Megathread

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

  • !libre_culture@lemmy.ml
  • !libre_software@lemmy.ml
  • !libre_hardware@lemmy.ml
  • !linux@lemmy.ml
  • !technology@lemmy.ml

Community icon from opensource.org, but we are not affiliated with them.

community_visibility: public
globe

public_blurb

  • 664 users / day
  • 1.52K users / week
  • 3.26K users / month
  • 10.5K users / 6 months
  • number_of_local_subscribers
  • 38.7K subscribers
  • 2.3K Posts
  • 36.5K Comments
  • Modlog
  • mods:
  • Evan@lemmy.ml
  • kevincox@lemmy.ml
  • CrypticCoffee@lemmy.ml
  • Lettuce eat lettuce@lemmy.ml
  • UI: unknown version
  • BE: 0.19.5
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org