this post was submitted on 18 Sep 2024
64 points (100.0% liked)

TechTakes

1398 readers
120 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago
MODERATORS
top 15 comments
sorted by: hot top controversial new old
[–] UnseriousAcademic@awful.systems 36 points 1 month ago (2 children)

Man I feel this, particularly the sudden shutting down of data access because all the platforms want OpenAI money. I spent three years building a tool that pulled follower relation data from Twitter and exponentially crawled it's way outwards from a few seed accounts to millions of users. Using that data it was able to make a compressed summary network, identify community structures, give names to the communities based on words in user profiles, and then use sampled tweet data to tell us the extent to which different communities interacted.

I spent 8 months in ethics committees to get approval to do it, I got a prototype working, but rather than just publish I wanted to make it accessible to the academic community so I spent even more time building an interface, making it user friendly, improving performance, making it more stable etc.

I wanted to ensure that when we published our results I could also say "here is this method we've developed, and here you can test it and use it too for free, even if you don't know how to code". Some people at my institution wanted me to explore commercialising but I always intended to go open source. I'm not a professional developer by any means so the project was always going to be a janky academic thing, but it worked for our purposes and was a new way of working with social media data to ask questions that couldn't be answered before.

Then the API got put behind a $48K a month paywall and the project was dead. Then everywhere else started shutting their doors too. I don't do social media research anymore.

After my own heart right here. I followed some version of Luca Hammer's guide to categorise everyone I followed on Twitter into communities, then created rss feeds of them using nitter. It was fascinating seeing how they clustered together. I think I still have an old gephi file with that output. I did this before Musk bought Twitter, since I knew he was going to wreck it.

Basically, I would have killed for this tool.

(I'm now wondering if anyone's published a guide on this for bluesky.)

I would wager that, more than the costs of serving these API calls, preserving the opacity of the resultant network is probably part of the advantage these companies get from locking down their APIs. Given how much flak they already get for the mental and social damage done by social media and Twitter specifically, I suspect they're very happy to preserve as much of the black boxiness as they can so they can point to the value users get and their ad revenue and say that all the costs are unfortunate coincidents rather than central problems with the paradigm.

[–] Soyweiser@awful.systems 16 points 1 month ago

Well that really sucks, another project stopped because of all this bullshit. :(

[–] dgerard@awful.systems 8 points 1 month ago
[–] dgerard@awful.systems 7 points 1 month ago

This was actually posted in June, but it became hot news yesterday for some reason

[–] Serinus@lemmy.world 5 points 1 month ago (2 children)

It's an excellent read on a safe site. I appreciate OP doing the opposite of clickbait, but if this interests you at all, check it out.

[–] froztbyte@awful.systems 19 points 1 month ago (1 children)

this post reads like an amazon product review

[–] Soyweiser@awful.systems 11 points 1 month ago (1 children)

It reminded me of those blog reply spam comments.

[–] Serinus@lemmy.world 8 points 1 month ago

It's a matter of time until AI starts misspelling everything to try to fit in better.

[–] V0ldek@awful.systems 7 points 1 month ago (3 children)

but if this interests you at all, check it out.

ye that's how most normal people use the internet? what's the alternative strategy, checking it out if it doesn't interest you?

[–] UnseriousAcademic@awful.systems 6 points 1 month ago

To be fair I've spent an inordinate amount of time looking at stuff on the Internet that doesn't interest me. Especially since my workplace moved their employee training online.

[–] froztbyte@awful.systems 6 points 1 month ago

staring at it suspiciously with eyes narrowed, until it scampers

[–] Serinus@lemmy.world 4 points 1 month ago (1 children)

The alternative is reading the headline and skipping the article.

The punchline is in the title, yes, but the article is still worth reading. Maybe I didn't phrase that well.

[–] dgerard@awful.systems 12 points 1 month ago

I try for clickbait that delivers