13 replies · 526 views · thread synced · 3 days ago
· View on torn.com
About this thread
Posts archived:14 / 14 posts (100%) · the total is Torn's reply count + the opening post at the last fetch
Counted by TornLife from the archived posts.
Archived posts
14
Discussion span
→
People posting
8
Likes on archived posts
51
Posts by staff, officers and moderators
5
Authority score
84 / 100
Historical score
58 / 100
Story score
40 / 100
Engagement score
63 / 100
Most-liked replies
Chedburn[1]Admin
· 17 likes · I guess the real answer is getting web sockets setup, so scripts / apps can listen in for live updates rather than just continuously fetching. I'm just not sure how to go about that, but I can start discussing it …
DeKleineKobini[2114440]Committee
· 5 likes · The big culprit for the events being spammed is DoctorN. Since it's no longer maintained, it will keep spamming the API until everyone stops using it.
MightyGoober[812478]
· 3 likes · Step 1 would be to stop/block/not accept DocTorn requests (Check for comment=doctorn ?). As said above, it hasn't been updated and is dying slowly. You might as well save the CPU and force those users to find TornTools.
On the user API, we have the 'events' and 'newevents' selections. 'events' provided the most recent 100 events (or 1000 if a 'from' timestamp was provided), and 'newevents' provided up to 1000 of events that the user has not seen on-site yet (since they last viewed their events page).
Since the update today, moving events to the cloud, it seems some scripts are requesting 100 events, or even the full 1,000 events every 30 seconds, resulting in players hitting their daily read caps of 100k and being locked out of the events page.
I've just changed these limits to 25 as standard, and 100 if a from timestamp is supplied, or when viewing only new events as a temporary measure to resolve this.
But to be honest, I'm not really sure how best to handle this long-term. We obviously don't want scripts constantly re-reading the same hundred events over and over again every 30 seconds (which I think a lot are currently doing?). We could maybe implement 'limit' into events so the timestamp of the most recent event can be checked, or provide timestamps of when the user's last event / message / whatever was received on a new selection, then a full call to pull new events can be made only when that timestamp changes?
Quick and dirty, how about the ability to specify a &stream=XXX parameter? Each time you use &stream it will store the most recent timestamp read against that key/stream/selection combo and then automatically use that timestamp as the &from parameter in the next selection?
This could apply to a number of selections and allow less frequent polling if it limited results to higher quantities.
It seems like this is an issue with how we (as developers) use the API, not how the API should be managed.
Based on my understanding, scripts should start using newevents rather than the old "events" endpoint to pull event data (for notifications and such), so changing something on the API side is not the solution.
This does break backward compatibility (rip DoctorN as DDK stated, however, it's been dead for years) but really is the best solution I see.
The other solution I can think of is to swap the endpoint names - I.E. the events endpoint returns the newevents, and make a new endpoint (eventsfull or something) that would return all the events. This would theoretically keep backward compatibility, while also giving us the full events if we need it
I guess the real answer is getting web sockets setup, so scripts / apps can listen in for live updates rather than just continuously fetching. I'm just not sure how to go about that, but I can start discussing it with the team again to prioritize it.
I would love to see this as an addition to the API but I definitely don't want to see the newevents/events portions of the API removed as in some cases this solution is just not necessary.
But in terms of making notifications/events, this would be AWESOME. would be even more awesome if we could also get the log as a WebSocket stream (log and events) - because watching log has a similar issue.
Step 1 would be to stop/block/not accept DocTorn requests (Check for comment=doctorn ?). As said above, it hasn't been updated and is dying slowly. You might as well save the CPU and force those users to find TornTools.
Isn’t DocTorn so old that its last updates predates requests comments?
Regardless, a solution could be to implement *something* (changing the endpoint name for example) that fundamentally breaks DocTorn and would force the users to upgrade to something else. It’s not pretty, but it’d be effective nonetheless. Just my 2c on the matter.
As much as I hate to say this, I completely agree. DocTorn needs to stop being used. I don't think blocking comments is the right solution, I think putting a newsticker that using doctorn will cause issues with your events, so disable it. Manually blocking one comment is just not a good idea, so it would be best if it was on the user to uninstall it
To avoid this in the future, instead of just a "comments" section, you ask for "scriptName" and "version"?
So request ends up being
scriptname=torntools&version=1.2
Something you don't like about a script/version? BLOCK.
Edit: and to avoid anyone putting anything in those fields, you could have scripters register their names with Torn. Some kind of public key/private key system for example.
That's assuming this is all a big enough deal to implement future-proof avenues for big tools that may need to be dealt with.
Ask yourself, Will TornTools be maintained forever? What do you do when all those users are hitting the API with broken requests? (again, or in addition to this)
This won't be the last time you'll want to retire a certain way the api is accessed.
There is absolutely no way to have a decent applications identification.
Query parameters like that are just text, change it to YATA and abuse the api as much as you like. Literally holding YATA hostage. Since DoctorN (and TornTools and ReTorn) run on the client, doing something like a private key or something also wouldn't work as it would be exposed for those tools.
I agree, I can't see other clean frameworks to solve this issue in a sustainable manner, though it won't stop old tools from using the old ways.
I find newevents a good alternative (for tools that are updated), and limiting to 25 events a good threshold to mitigate the stress on the db (for deprecated tools still being used or lazy developers like me that don't follow the updates :( ).
I guess you thought about it already but increasing the cache timeout on the "event" selection (like drastically) can 1/ force developers to move to newevents 2/ force users to ditch their old tools 3/ lower the number of requests on your db anyway.
As for websockets being a priority, only you can tell as we can't appreciate the load that these requests represent. I'm definitely excited about it :)
Increasing the cache timeout is a good idea except that it would hurt newer scripts that want/need to be able to pull all the events (I'm thinking some sort of 3rd party event viewer or something idk). I almost kinda would like to see the limit set back to what it was which would basically force users of old deprecated tools to update.