The Thread Thread 2005-03-13 05:51:23
I couldn't help myself. I had to make a stat thread of sorts for the BBS.
This project of mine started last June when I was on a ban, actually. I needed something to do with my free time, so I decided to write a bot to crawl the BBS and gather information about the threads on it. But right around that time, the BBS layout changed and I didn't feel it was worth my effort to fix up my crawler accordingly. I just this week finally took it out and reworked all the kinks.
So, here is what I have for pretty much every post -- I did make a couple changes in the middle of the run, so what I have doesn't apply to searches on the entire list of threads, but if you want to see all the threads you've made, I'll make sure those are up to date.
- the topic ID (i.e. /bbs/topic.php?this_number_here)
- the title of the topic
- the date and time the topic was posted
- the userid of whoever made the topic*
- the forum in which the post was made**
- the icon used for the topic
- whether the topic was locked when I crawled it
- the post ID of the first post in the topic (the number if you hit the reply with quote button)
*There was an error up until probably topic ID 180,000 or so, where if the first poster didn't have a profile, the parser said that the first person to post WITH a profile created that topic... but, when I make a list of any certain user's posts, I fix that. But do not expect me to go over the 167,399 topics of which I'm uncertain.
**Forums 9 and 11 both are restricted, so I assumed all restricted topics to be in forum 9.
Now I realize that this may be construed by some as a "spamometer" of sorts, and keeping this in mind, I will never post any statistics that I think will encourage spamming. In fact, to get this out of the way now, I'll just say that P-Bot has the most topics, and P-Bot is the only BBS member who should be putting out a thread a day anyway. I'm not sure offhand what else might be considered a spam statistic, but I am never obligated to give anyone the statistics they ask for.
The first statistics I'd like to get out of the way are the topics vs. point in time stats, so here are topics on given points in time. First, total topics per day of the week:
It looks like BBS members do go out most often on Fridays after all :)
Posts by hour of the day:
That seems to somewhat accurately reflect BBS activity. Want your thread to get good exposure? I'd say you should probably post it in the early evening, around 4 or 5 PM EST. This way, there will be many people online but your topic will be less likely to get pushed down to the 2nd page by other posts.
Finally for the activity-relevant stats that I can think of, here are the topics by month, but only for 2002-2004, because 2001 and 2005 add bias to some of the months:
This set of data might not be so worthwhile, considering how Newgrounds' userbase is always increasing. But, August and December at the top makes sense for people being on break.
If you've read this far, I'd like to say thanks. But what next? Surely if this was the whole topic, it would hardly be worth an entire thread. So, I'll tell you what I'm going to do for you.
You can see the statistics I have up above, and you can figure out what I can make from them. I expect the most common request to be "give me a list of all the threads I've made." As long as the moderators say this is okay, then I'll gladly give you a list of your threads. No requesting lists of other people's threads. (Moderators may be an exception to this rule, but we'll worry about that later if it comes up at all.)
Next, you might need to find a thread that won't come up in the search box for whatever reason (<4 letters, too common of a word). I made a topic called "Tell me why" a while back, and it's impossible to find in the search because "tell" is too common of a word and is not returned as a result. Also, in case anyone is wondering, here are numbers on some poorly named threads:
If you can think of any other ways of organizing any of these stats, please make a request for them in this thread. I'll finish this post up with a couple of random other ideas I had.
I don't necessarily know how to use the text search in MySQL flawlessly, but as far as I can tell, Idiot-Finder is leading the Photoshop threads with 41 topics starting with the word Photoshop, with the next closest contender being TehBanStick with 9.
Last, of the 242,588 topics I have indexed, 18,813 have been deleted and 611 are in the moderators' forum. And disregarding any topics from the last week, while the mods can still lock them, we have 20,916 of our 221,068 topics locked. That's nearly 1 in 10 topics. Good work, moderators! (This is probably the only time I will post that stat because the lock information will need refreshing, which I do not plan on doing... right now was the freshest that will ever be.)
As far as closing notes go, don't expect there to be any kind of web interface for this. It won't be worth my effort to code. Don't expect open access to this database either, at least not without convincing me somehow :P
It is now going on 5 AM here, so I should go to sleep. Seriously, if you really read it all up to here, thank you. I hope to have some questions on this data when I wake up! :)