this post was submitted on 01 Jul 2023
392 points (100.0% liked)
News
19 readers
12 users here now
Breaking news and current events worldwide.
founded 1 year ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I just find it interesting how the fixation on API and things of that nature is happening all of a sudden across the different platforms.
You can say it's Musk being Musk or Reddit setting up for their IPO, but a similar thing has been going on with instagram as well. They had rate limiting for a long time for those without accounts. Then back in May it started to get worse. At the same time they began banning/suspending accounts for merely just existing. The common theme was those accounts only had email attached to them.
It just seems there's a real push to isolate things and Twitter and Instagram really want verification.
It's not that interesting. There is an on going AI (llm) craze that requires participants to acquire huge amounts of "clean" data. Reddit and Twitter are platforms that can provide this. So why not cash on it? Also free money is no more so they are all looking to find ways to penny pinch.
That doesn't explain it IMO. That data is already freely available and LLMs don't need the latest data. So data scraping for AI makes up a miniscule amount of API usage.
I think it has more to do with inflation. Investors aren't putting their money into risky tech companies as much, and so these companies (twitter, reddit, etc) have to prove they can be profitable for once, and they probably are going to fail to do this. It does mean we can't have nice things for free anymore like API access.
its about the big datasets required to train systems to interact with and generate content as human beings... as usual, its a profit motive.
I think it has to do with OpenAI. They launched the fastest growing app of all time. ChatGPT wouldn’t have been possible without massive amounts of scraping. Reddit and Twitter sees this, and feel like they’re owed a share of their success.
If someone can make a successful product by scraping Reddit and Twitter, they better pay for it. Especially considering each request costs money for Reddit and Twitter.
Just my guess.