Graveyard

sus dump, try again

‎

I'm not sure why you're doing this. Torn has a feature built in for this (and it does pull everyone accurately), and no one has indicated how this would be helpful.

What uses do you see people getting from this?

If you are using a scraper that auto-advances to the next page - that is highly illegal. If you want to get user IDs, just leave an incremental user ID API caller run for a couple of days. Or ask someone who has done so before. Or instead use factions to filter out batches of people at a time...

I'm pretty sure that scraping is fine so long as it's not being used in a way that's abusive. Perhaps we could have a mod clarify but I don't really see any purpose to doing this in the first place.

I agree it makes no sense to do it, but auto-advancing is the issue here. Scraping in itself is perfectly fine, as long as it is not automated, i.e. triggered by a manual user input. From what the OP described, it sounds like he is using a script to go through all the pages and auto-scrape the IDs.

‎

Fair enough.

My initial thoughts as to why it might not be pulling all data is because he's probably getting rate limited if it's auto advancing like he says.

Torn's held a pretty solid 50k active users a day for awhile now.

‎

yeah this makes no sense and is not very useful for anything, do actually useful data scrapes like feddings or something

‎

Your sentence isn't even a real sentence so nobodys going on that dodgy link

If you are manually advancing, it should take ages to go through the 2k or so pages of users. In that case, the list will change several times as you go, as new users meet the criteria and some fall off because they no longer meet it. This method of gathering a list of users is inefficient (unless it is automated, which is illegal) and serves little to no purpose. If you want a more representative list of active users, just use a list of factions and make 1 API call / faction to get a bunch of users at once and with no question about legality. If you need a list of all faction IDs, i provide an up to date version here: chedslist. The reasoning is that users who are active in the game are also in a faction 99.9% of the time.