Kilg.us – Fantasy Stat Tracker …Tracker

A blog about the development of Kilg.us – The Fantasy Baseball Stat Tracker

Stat Updates

July 19th, 2012

Over the last few weeks I’ve received a number of messages that some players that haven’t played in the big leagues are showing season stats for 2012. All of that data comes from MLB, so if they show season stats in their data feeds, Kilg.us shows it as well. Usually these are spring training stats that just haven’t been updated.

I have made an effort to go through and delete any season stats for players that have been reported. If you see this behavior for any additional players, please let me know via email or in comments here.

A quick apology…

April 18th, 2012

As I mentioned yesterday, the old Kilg.us system for generating and sending email boxscores would run up against usage rates on a daily basis, leaving some messages unsent. With the move to SendGrid, our cap is raised, which is a good thing. I removed the limits from Kilg.us. What I failed to identify was that all of those unsent email boxscores were still sitting in queue (almost a full year’s worth…). When I removed the limit in Kilg.us and told it to send, boy did it ever!

I logged into SendGrid this morning expecting to see ~470 boxscores sent this morning. Instead the count was nearly 4000! I frantically shut down the process to try to reduce the irritation any more than had already been caused. But a lot of messages went out to people that may not have been expecting them. For that, I am very sorry!

For some reason, these email boxscores have been the single biggest headache with Kilg.us over the years.

Anyway, I’m back in the code tweaking things up in hopes of avoiding another disaster tomorrow.

For all those that have emailed me over the last year about a missed boxscore email, I hope it showed up today! /badjoke

Seriously, though, sorry about the extra messages. I’ll try to make sure it doesn’t happen again.


Update

I’ve made three changes to try to avoid this happening again:

1) I’ve cleared the email queue so nothing old even exists now.

2) I’ve capped the number of email boxscores Kilg.us will send tomorrow at 500. This is slightly higher than the number I think should be sent, so hopefully everyone gets their proper messages. If something goes wrong, though, the impact will be minimal.

3) I’ve updated the script that selects the emails to send to have it double-check that it is only pulling email boxscores generated in the current day.

Hopefully these three actions will prevent this from happening tomorrow. Nonetheless, I’ll be keeping a very close eye on things in the morning.

SendGrid Roll-out Successful

April 17th, 2012

Last night I upgraded the email sending mechanism in Kilg.us to use SendGrid. This morning all email boxscores look to have gone out as planned.

As a number people have noted, “as planned” may be less that what we all would like. One of the things SendGrid will help me do is see how many emails are being generated, sent, delivered, read, etc. This will allow me to better manage how daily boxscore emails work.

The first issue to address are those users that have signed up for boxscore emails but haven’t received them. The old email provider had a much lower limit on the number of messages that could be sent per batch and per day. This lead to some less than ideal compromises in designing the system. Basically, rather than just flooding the email server with all the boxscore emails, I had to throttle the process, building a queuing system and sending out small batches of emails. As the system neared its daily limits, some people’s emails would go un-sent.

In addition to the migration to SendGrid last night, this morning I have updated the queuing and sending system. It now queues everything the same, but tomorrow will send one large batch of messages (using an anti-flood function to ensure the email server doesn’t get overwhelmed). This SHOULD ensure that everyone receives every email they have requested! We’ll check back in tomorrow morning to see if it worked.

As a tack on, I switched this WordPress installation to also use SendGrid which should make it less likely that I miss comments and questions people post on here (sorry if I missed your question or comment!).

Moving to SendGrid

April 16th, 2012

In the next day or so Kilg.us will be moving its email handling to SendGrid. This should give me better transparency into what messages are being send, received, and bounced back. Right now things are a bit of a manual process which makes it difficult to diagnose one-off errors (particularly with daily boxscore emails).

If all goes well, I’ll probably make the change tonight. Theoretically, there should be no change for anyone but me. Let’s hope it works out that way!

I love Kilgus! Can you include [STATX]?

April 6th, 2012

This is by far the most common topic when people email me about Kilg.us. I’m thrilled that so many people are enjoying the tool (we start the year with over 1800 members, nearly 500 users per day), but these are bummer emails to receive.

Unfortunately, 95% of the time the answer: No, sorry.

As a free tool, Kilg.us needs to rely on whatever free stats we can find. As it turns out, MLB publishes some limited live data (I believe for their GameDay feature) that we’ve been able to tap. While it is great that the data is free, the bummer is that it isn’t comprehensive.

Kilg.us is already pulling every stat available from MLB and even calculating a whole series more that aren’t made available in the data feed.

If you’re favorite stat isn’t included, it’s almost certainly because we just don’t have access to it. At the start of each season, I double-check the feeds to see if any new stats are available. This year, there is nothing new.

So, if there is something that can be calculated with the data already in Kilg.us, let me know and I’ll be happy to add it! If you want a new counting stat added, though, I can’t add it.

Maybe someday an angel investor will throw a pile of money at Kilg.us and we can afford to purchase a real data feed (~$25k/year). Until then, we just have to make due with what we have.

Hopefully it’s enough to be useful to you!

(That said, don’t forget to click “Select Stats” for each of your teams to see all the stats that are available: currently 32 categories!)

First 2012 Boxscore Emails

April 6th, 2012

From this end, it looks like all daily boxscore emails were sent out properly. We had a handful of bounce-backs (expected after 6 months of inactivity) but no fatal errors.

If you received an email boxscore that you don’t think you should have, or didn’t receive one that you think you should have, please double-check your account settings and if you see an error, please let me know.

2012 Plans

April 5th, 2012

I’m a little late getting this notice out, but Kilg.us will be up and running for the 2012 season. This evening I intend to enable daily boxscore emails so those should start showing up in inboxes tomorrow morning. Be sure to update your subscriptions (Manage My Settings) so you receive emails for the teams that matter to you!

Best of luck with your 2012 season!

Internet Explorer Sign-up or Log-in Bug

May 7th, 2011

Over the last couple weeks, I have received numerous emails from people having trouble signing-in to their accounts. With a little digging I was able to find a consistent theme: they were all using Internet Explorer 8 or 9.

Specifically what was happening was that a person would type their existing username into the sign-in form and would then be prompted to create an account. As everyone is probably aware, Kilg.us uses the same form for logging in and signing up. If you enter an existing username, it asks you to log-in. If you enter a username that doesn’t exist, it asks you to sign-up.

To manage this functionality, Kilg.us generates an AJAX request after you enter your username then move focus to the password field. This is creates a “change” event on the username field. When that event happens, Kilg.us asks the database whether the username you entered exists. The database returns a response in the form of XML. The XML has one message: the username is valid (it exists in the database) or it is invalid (it doesn’t exist in the database).

Once the XML response is returned, some JavaScript parses it to read whether it says “valid” or “invalid”. If the value is “invalid” the form asks you to create an account. The JavaScript to read this follows standard practices to navigate the document object model (DOM) and find the value in question. This works in Firefox, Chrome, Safari, Konqueror, Mobile Safari, Opera, Android Browser, Blackberry Browser, etc., etc. Apparently, how Internet Explorer handles the XML is different. It doesn’t create all the parameters that the standard approach would.

This shouldn’t come as a surprise to anyone.

What I ultimately found is that Internet Explorer will parse the value out and save it in a “text” parameter for the XML object. Rather than navigating the DOM in a standard fashion, I’m now taking a shortcut and checking if that “text” parameter exists on the XML object. If it does, I read it in as the returned value. If it doesn’t (obviously, all the other browsers to generate this rogue parameter), I continue to navigate the tree and get the value in the proper manner. This only works because the only value of the XML is the “valid” or “invalid” message.

The end result: the log-in/sign-up form logic should now be working again in IE. And still working in every other browser. If you find that not to be the case, please let me know (comment, email, Facebook, Tweet).

New Stat: Total Bases

May 5th, 2011

Now that Kilg.us is migrated and–seemingly–stable, I can actually make some improvements!

First up are some stats additions. The easiest request was to include total bases. Because this is calculated based on data we already have, I was able to implement it pretty quickly. To enable Total Bases for any of your teams, visit the “Select Stats” page for that team. You will see a checkbox for “TB.” Checking that box and saving your changes should add a TB column right after strikeouts.

The most commonly requested stats that Kilg.us does not currently have are games played. While this data isn’t conveniently available in the MLB XML data Kilg.us uses, I think I have determined a way to deduce whether a player has played in the day’s game(s) or not. That will take significantly more effort, though, so it will have to wait. Perhaps this weekend I’ll find some time.

Database back up and running

May 4th, 2011

I have moved Kilg.us back to referencing the database on the new servers. Once again, things seem to be going well. Hopefully that will still be the case in a couple hours!

I did not attempt to copy over the old database. There is a lot of complexity involved in merging databases and I just didn’t have the patience after yesterday. As a result, if you added or removed players, moved them on or off the bench, or manipulated a team or your account in the last 24 hours, those changes will be lost. You will need to re-apply any changes.

To the couple people that signed up for Kilg.us today–I’m sorry, but you’ll need to do it again! I promise this isn’t a regular occurrence. In fact, I’ve never deleted an account before!

The only outstanding issue that I am aware of from the migration is the appearance of players that are not on the bench. They are rendering as gray and “benched” right now, but the team is still calculating their stats in totals. I’m working on that now and assume it will be a quick fix.

If you notice any other anomalies, please comment here, email me, or post to Facebook or Twitter (@kilgustracker).

**UPDATE 10:40PM**

It appears that batter stats are being collected properly for individuals, but tallied improperly for teams.

It appears that pitcher stats are not being collected properly for individuals, but tallies are correct for teams (based on day-old data).

Still investigating.

**UPDATE 11:19PM**

The culprits seem to be the stored procedures that Kilg.us tries to use for storing stats. All the procedures seem to be failing with the move to the new server. Luckily, Kilg.us included fallback string concatenation mechanisms for exactly this situation. I have disabled all the stored procedure calls and everything now seems to be working.