API Bug

In this instance though, it's not relying on "third party programs" that seems to be the issue, it's that the API - part of Torn (ie: not a third party program) - was giving incorrect data. What people were using to process that data isn't really relevant.

Edit: Just saw this in the post above : "Per Chedburn: "The API is an external service totally separate from Torn game", so.. I guess it's not "part of Torn", but it's still hardly "third party"

AMEN!

Exactly what this guy said, oh my god. Thank you. I was just racking my brain, trying to put this to words. I read what was said this morning, and I literally laughed out loud at the concept of players with our horrible sense of entitlement, with our audacity of expecting the game to continue to work the way it has. People put too much work in this stuff to be blamed for Torn's mistakes, and there were plenty of mistakes we had stomached this week already.

imo it makes no sense to restore chains - i remember 2-3 years back on valentines we had cases of peoples chains dropping purely because they couldn't hit as actual servers were inoperable (not api ones) with 20-30s between each turn. helvete lost a 10k that way and were told to suck it up, as were a bunch of other factions. (39th and PnB to memory)

its a bit shaky logic if an API SLA is created for chain restores (owing to non-functionality of 3rd party notification tools) but no parlance is given towards direct server lag as causality of chain collapse. one of those stance has to go.

chains were restored due to server lag at the start of this last event. i think due in part because of systems put in place after the instance you mentioned.

i hope this instance leads to systems being put in place that stops the api spewing out bad data for hours in the future. as said, no data is better than bad data. making people aware of that was why i started the thread.

job done.

This is the most valid response.

As I understand it, the root issue is the API sending invalid or incorrect data. Imagine telling your customer they're at fault for a broken service you provide. "Do not rely on the data provided by our services and monitor it yourself". What is the counter argument to this?

I do appreciate the conversation and transparency from Torn thus far.

They might be getting sick of the "damned if you do, damned if you don't" situation.

Torn community isn't gonna be happy either way - whether they restored the chains, or whether they didn't, it'll come back and bite them in the ass sooner or later.

From what i can see, people get annoyed with the lack of "Hello everyone, we are very sorry but <This Error> has happened, we are working hard to resolve it and will have an update in X hours" , yes - stuff goes wrong and breaks, yes - knowing how long its going to take to fix is almost impossible until you know the extent of the issue. if people don't know somethings wrong how can they work around it or accommodate it. unless its just me. Also totally agree with boogie that restored chains etc shouldn't happen, although that is tough for factions involved (don't know who they are). due to the precedent it sets. Just my 5 cents.

TL;DR Communicate More = Happier Customer/players

I'm sorry bogie, I disagree with you here as well.

I sympathize with those affected as sure it is very frustrating to lose a chain, especially ones that high, it would piss me off too but I have to make a decision here that would affect a much longer term than the short term quick and easy decision to restore these now. This isn't out of spite, malice or contempt for any of you, but rather one that needs to be done for the consideration of Torn as a whole.

There is nothing wrong with resetting a 5 minute timer because of a data issue FROM Torn. Again, if the API was down, your stance would be fair. But instead, the API, which is Torn's and managed by Torn, sent misleading information, which was more information than anyone knowledgeable of the situation shared with potentially affected users.

A fight auto-timing out is not the same as Torn making data available that there is 3 minutes left when there is really 30 seconds. It is literally misleading and users would have been able to adjust accordingly and the responsibility of the chain would have been on them if there would have been notification or action. It is entirely possible to shift responsibility through accurate communication.

Does Torn not have the ability to put a banner across the top of the page warning customers of API issues? Torn can email me every 3 months with a +250e newsletter- couldn't a communication go out through the same channel that there is a system issue? Even if the customers didn't read the email, it would have still shown that there was effort in place to take action for an internal system issue that was likely to cause significant downstream issues. Surely there is an administrator role/function that would allow every user to receive an in-game message, right? If faction leaders can email entire factions, surely Torn staff has some communication tool available to them too.

This is Torn's fault and not a risk of playing the game. No one can prevent Torn from taken an inactive position but no one had the opportunity to succeed. We were set up to fail and we failed, quite predictably. The issue was known, no action was taken, no preventative measures taken, nothing disabled, and everyone was left to their own devices.

100% preventable. If nothing else- the API should have been disabled so that it was no longer misleading, instead of being left to run for over 5 hours.

You run a business with customers handling products that are set to be shipped every five minutes. It's a reoccurring transaction that needs to be manually completed every 5 minutes by a worker. You implement a third party system that can ping the worker at regular intervals to ensure they can still manually process each order every 5 minutes. The third party system goes down, but your worker doesn't check the 5 minute timer interval.

The customer has now decided to leave your business because you couldn't continue to complete your order in the manner specified. While the third party system is partially to blame, you as the business, knew your obligation to this 5 minute timer window. You became lazy and compliant once the automated system for notification was brought in. That's a failure on the company, not the system to only have 1 level of fail safe.

What's the usual value for the timeout field? Shouldn't the API clients have detected that there was something wrong with the data?

I know the code I write that uses the API usually assumes that the data is OK, until I run into a problem (bloody cache!), at which point I fix the code, by adding checks on the data.

So I think the problem can be reframed as: the API is not guaranteed to work 100% of the time and the clients did not perform enough sanity checks on the data. At least not enough checks to turn a 99% availability service into a 99.99% one. Which makes the problem a client problem, and not a server one.

So... "Git Gud Clients"?

I'll make your analogy even better

You run a business with customers employees handling products that are set to be shipped every five minutes to Partner A. It's a reoccurring transaction that needs to be manually completed every 5 minutes by a worker. You implement a third party system that can ping the worker at regular intervals to ensure they can still manually process each order every 5 minutes by the API provided by Partner A. The third party system goes down was not working properly because Partner A's API was giving wrong information, but your employee doesn't check the 5 minute timer interval.

The Partner A has now decided to leave your business because you couldn't continue to complete your order in the manner specified. While the third party system is partially to blame for trusting Partner A's API, you as the business, knew your obligation to this 5 minute timer window. You became lazy and compliant once the automated system for notification was brought in. That's a failure on the company, not the system to only have 1 level of fail safe.

Now when you ask why Partner A's API was sending wrong information which(partially) led to this situation, the response from Partner A is you shouldn't have trusted our API and watched our dashboard manually. Then you wonder why even provide an API if it can't be trusted.

I don't believe anyone who writes client code assumes the data sent by API is not true, hell if you can't trust the data sent by an API why use that API? It's not like it's a third-party api which may or may not send accurate information

At this point I guess we should just start the 'Petition for Elon Musk to buy torn and fire Boogeyman' Load image: grin

[image: www.fintwitfame.com]

#shitpost

As with all pieces of technology, nothing is 100% fail proof. At the end of the day, the obligation was to the 5 minute timer window. The chain in this case, does not require you to use the API provided, hence why equating Partner A as the party that provided the API access is not correct. You had the ability to complete your obligations to the client in the manner provided, and you attempted to work around manual work by automation. Makes sense, but believing that anything is 100% fail safe, is just setting yourself up for failure.

I'm not saying it isn't a shitty situation, but to also come in here and expect Torn to revert your own failure when it affects more than just this one situation is just being biased. If a trader updates prices based on the API, and made trades based on those calls, should Torn also reimburse them when they could have manually checked those prices were correct? Potentially, but will they? Probably not as there are tons and tons of specific situations that could be argued they failed as a result of the API.

100% I'd rather have bogie over Elon Musk. I have disagreed with bogie in this situation but bogie has been consistent in his position and clear in his communication. :-P

I agree with this the fact is that at the time our chain died we had 1x watcher out of 3 slots the team where expected to fill, our own fail safes where not inplace and relying on a flawed system, if the third party system failed it would be liable contractually so in this case i disagree.

I have said always that people here are not to rely on alerts as we have just had a period where most add ons relying on API have experienced issues throughout halloween.

But the fact is - It is a flawed system .. Torn in itself could not cope with an event on this scale but it continues to make things easier for new players and draw more in .

The response to this is it would set future precedents from the community manager... but ched already stated this is UNPRECEDENTED. on his own announcement .

The precedent of shafting the player base was set about 15 years ago XD so i am not going to cry about it , i think any older torn player already knows this. For the new guys , people in our faction I think they have learned a valuable lesson about what to expect from the game.

I don't believe anyone who writes client code assumes the data sent by API is not true

You'd better believe it. lol

The API is (was?) over-zealous with caching, It may have changed, I haven't tried recently. If you ask for historical data, ie use the timestamp or from/to parameters, then more often than not, after the first call, you will get the cached result and the timestamp will be ignored. Either that or I really screwed up. In any case, you'd better believe that my code accounted for that possibility.

APIs are wonderful, but whether it's Torn's or others (including ones you pay for), they are rarely as straightforward and simple as you think they would be.