Vocabularies of forum-goers

[Gabu]

I'm interested in quite a few things surrounding language, and one item I find interesting is people's choice of language.  I was bored for most of yesterday, so on a complete whim, given that you can find the complete record of any user's posts, I wrote a C# program that takes in a user's ID and then tallies every single word that that user has used and how many times the user has used it, as well as producing a count of the total number of unique words that that user has said.

Essentially, how it works is it loads each webpage containing the records of the user's posts and then uses a bunch of regular expressions to strip away everything except for the posts' content, remove quotes of other users, and then strip away everything except for spaces and words.  Then it splits each post into an array of words and either increments the number of instances of the word if it's already in the list or adds it to the list if it's a new word.

It's not perfect, and I'll be trying to make it a bit better, as, currently, it smashes together words separated by hyphens, forward slashes, etc. into one giant word, and, consequently, very slightly inflates a user's total word count.  Other than that, it works perfectly fine, however.

I've tested it on myself and on the top five posters, and below are the results.  I'll list the total words used as well as the top twenty-five words used (and how many times they were used).  Given that very common words like articles, prepositions, etc. are not terribly interesting and are always at the top since you can't form many sentences without them, I'll omit those.

The results:

A18

Total words used: 15990

people - 1686
states - 1343
state - 1268
government - 841
right - 770
federal - 750
president - 707
kerry - 697
amendment - 662
congress - 661
income - 651
constitution - 607
republican - 591
support - 563
actually - 546
rights - 545
taxes - 537
money - 502
democrats - 495
point - 492
united - 488
social - 446
election - 445
party - 443
court - 430

Other notable words:

constitutional - 253
unconstitutional - 229

The constitutional + unconstitutional combination would have been in 22nd place.

Al

Total words used: 21996

people - 1024
labour - 974
party - 862
south - 813
state - 671
election - 645
actually - 642
north - 617
county - 605
probably - 532
seats - 501
right - 474
kerry - 394
areas - 383
liberal - 383
candidate - 380
voters - 366
interesting - 365
democrats - 363
tories - 352
district - 349
government - 338
problem - 332
democratic - 327
elections - 321
years - 320

Other notable words:

union - 139

StatesRights

Total words used: 15100

people - 583
right - 559
believe - 505
state - 441
kerry - 436
florida - 401
states - 382
party - 372
agree - 362
south - 323
democrats - 321
government - 292
president - 281
years - 264
support - 236
election - 209
course - 205
great - 204
history - 202
republican - 201
money - 198
southern - 191
world - 191
wrong - 190
problem - 189

Other notable words:

lincoln - 185 (28th place)

Boss Tweed

Total words used: 11582

kerry - 994
edwards - 488
election - 294
people - 285
right - 266
president - 247
state - 239
party - 219
liberal - 207
candidate - 182
states - 179
against - 165
clinton - 159
republican - 153
south - 140
reagan - 139
senate - 139
conservative - 135
mccain - 132
votes - 131
primary - 127
polls - 122
years - 122
chance - 114
voted - 113

No notable words.

Gustaf

Total words used: 15169

people - 1048
right - 719
states - 637
election - 609
party - 590
kerry - 576
agree - 446
state - 432
sweden - 360
thought - 340
forum - 326
least - 324
democrats - 298
guess - 293
republican - 292
republicans - 278
political - 264
voters - 260
close - 257
president - 249
voted - 249
conservative - 247
course - 247
liberal - 242
believe - 236

No notable words.

Gabu

Total words used: 16010

people - 1454
votes - 957
given - 607
election - 583
senate - 537
right - 488
person - 466
section - 452
amendment - 421
party - 421
voting - 413
favor - 393
debate - 387
legislation - 381
agree - 378
president - 371
simply - 362
senators - 352
believe - 349
least - 337
government - 336
states - 324
enough - 313
state - 309
kerry - 298

Other notable words:

canada - 204

---

So far, Al has the largest utilized vocabulary, and Boss Tweed has the least.  Given that Boss Tweed has less unique words used than posts, I can only assume that he's had a very healthy amount of very short posts and of repeating himself.   A18 also has less unique words used than posts.  The rest have more.

I'm not entirely sure what to divine from these results, but they're nevertheless interesting to look at.

Any comments?  If anyone would like to know about a specific word for anyone, or would like to see the entire, gigantic list of words for anyone, let me know; I'll either post it (for the first option) or email it to you (for the second option).  Additionally, if anyone would like me to do this for themselves, just say the word; I'd be more than happy to.   Please be patient, however; given that many people have hundreds of pages of posts (or in A18's case, over a thousand), it takes my program a while to process all of the information.

[Gabu]

How come "Bush" isn't on any of those lists? I would think he would be used pretty frequently.

Oops.  I think you just found a bug; I made it just knock off every word less than five letters long in the list of words used (though they are still accounted for in the total number of words used) because so many four-letter words are very common.  I never even thought about the fact that "Bush" is only four letters long.

I won't do those six again unless someone really wants me to, but I'll make "Bush" an exception to the "over four letters" rule, given that that definitely is a very valid word to keep track of.

Hmmm... can you do me?

NO I CANNOT

Oh, all right, you've talked me into it.

You're one of the ones with hundreds of pages of posts, though, so the program will be a while.  I'll post here with the results when they're in.

[Gabu]

I'd appreciate if you did me.

Okay, I'll start a run for you, as well.

[Gabu]

Could you give me a shot? I wonder how high Potts will be on my list.

I'd like to see my results, if possible.

I'd like to see my results Gabu. Please, pretty please.

Okay, I added you three to the queue, as well as migrendel, just for kicks (HIGH SCORE!!!!!11).

EDIT: Somehow missed Akno's post; I added him as well.

[Gabu]

id be curious to see my results.

i wonder how many times i used the word 'horrible'

Let's find out.    Added you.

[Gabu]

You guys are turning this into a full-time job for Gabu.

Hah, no, my computer's doing all the work.  I just open a new instance of the program, enter the username and ID, and off it goes.

Wanna give it a whirl?

[Gabu]

Sure, why not, if you don't mind.  Still, I'm afraid of what might come up.

Well, let's find out.

Wanna give it a whirl?

If you don't mind, I would like to do so. My predictions are:

- Constitution (or a variant thereof)
- Rights
- People

Okay, I added you, too.

To everyone else: I have to go for about two hours, but I'll leave the programs running.  Probably a good thing, too, given that nine instances of it kind of use up a lot of CPU power.

[Gabu]

I'm back, but unfortunately, I didn't account for the fact that a response from the website might not be received (it must have gone down for a while or something), and as such, all of the programs crashed.   Oops.

I now have it fixed such that it will not crash when that happens, and will instead simply keep trying until a response is received.

I'm running the first eight again.  Any more than that and my computer starts to seriously lag from the CPU usage.  I'll get to the rest in time, however, so don't worry.

[Gabu]

Can you do it for me?

And can you post it here, in case you plan on PMing them, please?

I'll be posting all of them here, unless explicitly told not to.

[Gabu]

I should up my rate of "typeos" to increase my word count.

Yes, unfortunately, the program has no way of differentiating between typos and actual unique words, so if a person makes a ton of typos, just adjust their total number of unique words accordingly.

[Gabu]

I'd be interested to see how I stack up...but the queue is pretty damn big.

I'll still take you down as one in line; I will get through everyone, I promise.

[Gabu]

Okay, we've got the first two results in; not surprisingly, they're MHS2002 and WalterMitty, who had the smallest number of posts.

So, without further ado...

MHS2002

Total words used: 7314

spade - 258
amendment - 242
president - 239
passes - 230
senate - 227
state - 218
colin - 188
supersoulty - 188
hughento - 185
wixted - 175
senator - 174
march - 161
think - 141
republican - 139
election - 133
democrat - 132
right - 130
debate - 121
harry - 118
clause - 116
virginia - 111
masterjedi - 105
keystone - 103
district - 101
states - 101

Other notable words:

potts - 60

WalterMitty

Total words used: 9056

bush - 473
think - 458
kerry - 408
people - 307
state - 274
republican - 210
republicans - 160
president - 153
party - 146
democrats - 140
support - 139
right - 138
believe - 128
governor - 126
however - 117
edwards - 111
states - 111
democrat - 109
election - 109
opebo - 104
course - 102
senate - 94
voted - 90
liberal - 89
horrible - 85

MHS seems to enjoy talking about other forum members more than political matters.  And to WalterMitty, yes, "horrible" did indeed just barely make it into the top 25.

I'll start the next two now.  Results will be posted in the order that they come in, not necessarily in the order that requests arrived, although if you request earlier, your run will get started earlier.

[Gabu]

Me next?

I'll put you in the queue after the last person, but it unfortunately would not be fair to everyone else if I did you immediately next.

[Gabu]

Ebowed

Total words used: 9952

people - 429
think - 426
abortion - 321
party - 302
support - 294
right - 274
death - 251
believe - 248
harry - 219
penalty - 218
Ebowed - 216
senate - 209
rights - 203
district - 200
governor - 200
agree - 197
election - 193
government - 193
against - 181
voted - 177
southeast - 169
state - 166
liberal - 163
states - 157
oppose - 153

Apparently, Ebowed likes talking about himself quite a bit.   "Abortion" is surprisingly high, as well.  I didn't think he talked about it that much...

[Gabu]

Colin Wixted

Total words used: 10400

party - 1120
people - 546
senate - 497
think - 426
atlasia - 424
election - 365
believe - 323
atlasian - 322
senator - 283
votes - 279
president - 275
district - 264
state - 248
candidate - 244
government - 243
states - 240
support - 232
national - 218
republican - 216
great - 210
against - 208
right - 199
parties - 183
union - 178
thank - 174

Colin Wixted likes partying.

[Gabu]

How did you access the web? wget?

These are the relevant lines of C#:

using System;
using System.Net;

...

      Console.Write("Making request to webpage...");
      WebRequest request = WebRequest.Create(new Uri("https://uselectionatlas.org/FORUM/index.php?action=profile;u=" + userId + ";sa=showPosts;start=" + ((page - 1) * 15)));
      Console.WriteLine(" Done.");
      Console.Write("Waiting for response...");
      WebResponse response = request.GetResponse();
      Console.WriteLine(" Response received.");

where "userId" and "page" are parameters to a function that does this processing.

Some of this is actually in a try...catch block to catch exceptions relating to a failure to connect, but I left that out for simplicity's sake.

[Gabu]

harry - 219
Ebowed - 216

I talk about abortion and the death penalty way too much, don't I?

Hey, I just posted what the program gave me; make from it what you will.

[Gabu]

I need to talk about Lincoln more.

Yes, and "tyrant" is way down there at 22; you need to vent your anger more often to boost those numbers.

[Gabu]

AkSaber

Total words used: 2686

alaska - 40
state - 27
bracket - 20
disagree - 20
social - 19
anchorage - 18
government - 18
states - 18
control - 16
guess - 16
thought - 16
agree - 14
economic - 14
people - 14
voted - 14
always - 13
dakota - 13
republican - 13
south - 13
bush - 12
favorite - 12
hampshire - 12
north - 12
place - 12
right - 12

I think I can guess what state this guy is from.

[Gabu]

Could you search results for me Gabu?

Yes, yes I can.

...After everyone before you has had their run started, however.

[Gabu]

i'm going to be going offline in ~20 minutes, so unless you are already running me, feel free to bump me back for a couple of hours.

I don't think I've gotten to you yet, but you're close to the top of the list.

It won't be until an hour or two (or three) after I begin your run that the results come in, however.

[Gabu]

Okay, I was away for a bit, but now we've got a whole slew of results.

Jake

Total words used: 16091

people - 469
party - 449
think - 380
right - 329
mortality - 317
bush - 315
president - 297
senate - 297
support - 276
maybe - 274
state - 264
government - 246
abortion - 207
senator - 205
election - 199
democrats - 198
against - 197
states - 197
money - 181
school - 179
kerry - 174
candidate - 173
initiative - 173
district - 170
years - 167

Can't think of any comment to add.

Joe Republic

Total words used: 11040

rolls - 1066
harry - 680
battalion - 553
loses - 519
think - 377
people - 376
frodo - 355
casey - 326
attacking - 320
battalions - 314
reese - 308
defending - 304
invades - 299
leaving - 272
kemperor - 271
conquers - 260
party - 260
general - 245
placing - 229
state - 209
states - 205
nation - 198
votes - 198
right - 193
south - 192

Other notable words:

taft - 44

It appears that the USA Risk posts have been so repetitive that they dominated everything else.

Emsworth

Total words used: 10226

senate - 335
amendment - 295
government - 268
billion - 264
constitution - 257
state - 246
think - 225
governor - 220
right - 206
states - 202
section - 198
president - 197
voting - 193
clause - 180
court - 180
rights - 179
party - 176
question - 176
election - 175
votes - 175
people - 155
federal - 151
senator - 148
constitutional - 142
senators - 134

Emsworth appears to like the Constitution and the Senate.

nclib

Total words used: 6885

bush - 245
conservative - 210
states - 210
liberal - 196
state - 172
kerry - 162
disagree - 159
legal - 155
think - 151
agree - 150
voted - 148
option - 134
democratic - 130
social - 127
party - 121
people - 118
republican - 114
women - 111
support - 108
district - 96
rights - 94
right - 93
democrat - 91
abortion - 87
issues - 81

For being a liberal, the first two are kinda funny.  I suppose they were likely said in a bad tone, however.

CheeseWhiz

Total words used: 3947

governor - 197
party - 136
midwest - 130
election - 95
think - 95
region - 89
cheesewhiz - 83
ballot - 74
district - 74
proposition - 74
section - 68
people - 67
amendment - 66
senate - 66
article - 64
constitution - 57
southeast - 54
clause - 51
ilikeverin - 51
voting - 51
pacific - 49
votes - 49
support - 48
believe - 46
candidates - 46

CheeseWhiz appears to be another one who likes to talk about himself.

Apologies to Supersoulty; for whatever reason, yours is taking a while.  Rest assured that it is coming along, however.

[Gabu]

Four more comin' at you.

REPRESENT

Akno21

Total words used: 13929

people - 764
think - 740
senate - 462
right - 398
president - 389
senator - 384
party - 375
harry - 359
election - 356
support - 352
bush - 348
liberal - 325
governor - 308
against - 304
kerry - 297
democrat - 274
district - 268
state - 268
government - 267
states - 264
great - 249
democrats - 240
candidate - 238
democratic - 229
votes - 222

Max Power

Total words used: 5810

think - 117
people - 108
party - 98
kerry - 91
casey - 89
reese - 82
right - 80
thanks - 79
support - 78
attack - 70
president - 64
running - 59
senator - 59
believe - 54
governor - 53
votes - 46
liberalpa - 45
economic - 44
social - 44
clinton - 42
district - 42
warner - 42
agree - 40
election - 40
state - 40

bullmoose88

Total words used: 6700

think - 167
party - 155
republican - 147
state - 147
liberal - 143
right - 125
conservative - 124
though - 96
bush - 95
democrats - 94
people - 91
social - 85
government - 82
democratic - 80
county - 79
states - 71
score - 66
democrat - 63
republicans - 63
results - 63
while - 62
bucks - 60
libertarian - 60
result - 59
selectsmart - 59

Milk_and_cereal

Total words used: 9497

think - 417
people - 397
government - 232
bush - 193
state - 177
right - 155
libertarian - 153
george - 147
states - 137
party - 118
liberal - 114
money - 113
voted - 113
point - 105
president - 93
republican - 89
opebo - 88
social - 87
against - 86
johnson - 82
least - 80
republicans - 80
clinton - 78
enough - 78
votes - 77

[Gabu]

Four more - and one of them's finally Supersoulty!

Supersoulty

Total words used: 19282

think - 1207
people - 1198
bush - 898
party - 528
state - 508
president - 492
believe - 469
right - 465
kerry - 456
states - 401
election - 366
government - 362
point - 353
years - 336
republican - 318
world - 306
support - 299
senate - 290
country - 263
thought - 259
least - 246
thank - 242
great - 230
nation - 230
clinton - 228

dazzleman

Total words used: 17685

think - 2831
people - 2596
right - 886
democrats - 764
liberal - 711
democratic - 683
republican - 644
black - 629
blacks - 616
women - 608
agree - 606
liberals - 586
bush - 583
party - 580
believe - 535
problem - 533
president - 527
support - 509
political - 497
white - 497
great - 496
point - 491
school - 491
states - 489
state - 485

"Liberal" plus "liberals" alone get well into third place with 1297 uses in total.  Add in "Democrats" and "Democratic" and it's then very close to first place with 2744 uses.

I think you may have a fixation here, dazzleman.

Also notable is the copious amount of times black people and women are mentioned.  Not sure what to make of that part.  Maybe dazzleman is fond of demographics?

hughento

Total words used: 20234

think - 681
people - 633
australia - 490
votes - 481
party - 399
state - 384
right - 329
whilst - 326
states - 322
leads - 313
election - 295
support - 291
south - 273
senate - 263
bush - 254
world - 253
believe - 252
nation - 251
least - 243
president - 242
system - 237
country - 236
three - 236
great - 225
melbourne - 225

I'm going to take a wild stab and guess that this guy is Australian.

He also apparently likes the number three ().

I should also note that he's misspelled "Melbourne" at least five times:

melborn - 1
melboune - 1
melbounre - 1
melbourbne - 1
melbourn - 1

Shame on you.

Flyers2006

Total words used: 12723

think - 761
people - 531
bush - 483
philly - 431
schwartz - 427
brown - 415
kerry - 344
northeast - 344
philadelphia - 315
right - 315
section - 275
melissa - 273
republican - 267
liberal - 265
hoeffel - 263
state - 259
santorum - 245
issues - 242
catholic - 231
conservative - 228
agree - 209
social - 205
democrat - 198
allyson - 183
school - 183

Holy Jesus, Flyers, you need to stop talking about Pennsylvania.

I think that that wraps up everyone who has requested one of these so far.  For kicks, I'm going to do ones for migrendel and opebo now.

[Gabu]

This should be interesting. I wanna see mine.

Okay, I started one for you.