Kilg.us – Fantasy Stat Tracker …Tracker

I’ve been getting a flood of emails about spring training statistics showing up for players that haven’t played in the big leagues yet this year.

Kilg.us collects all of its stats for an XML feed that MLB publishes. Every time you request the stats for your team(s), Kilg.us checks to see what the latest stats are and loads them into the system.

Unfortunately, it seems that MLB leaves springs stats listed for all players that appear in the spring but are then sent to the minors (or DL). Until they appear in a big league game, MLB will continue to show springs stats in their feed.

This is just how it works.

If you have a player on your team that is showing spring stats, please move that player to your bench. The player’s individual stats will still show from spring, but this will remove the player’s stats from your team totals.

The 2012 baseball season has come to a close. As such, Kilg.us will no longer be sending out daily box score emails.

It’s been a tumultuous year for Kilg.us. In particular, we experienced a lot of accuracy issues with our data source (free MLB.com published XML feeds) and I heard more than ever about the lack of this stat or that stat.

I stopped using Kilg.us for my personal needs a couple years ago (I originally built it as a personal tool). Going into 2013, I’m on the fence as to whether to continue maintaining Kilg.us. Obviously there are costs associated with keeping the servers up and running (thanks to all of you that have donated to help cover hosting costs!), but even more important is the time it takes to field email questions and comments, fix things when they go wrong, and monitor the performance of servers, databases, and email services. We’ll see how that goes in a few months when the excitement of a new season ramps up.

In the meantime, keep your chins up. Baseball is only a few cold months away!

Over the last few days I’ve received a lot of emails about Kilg.us not properly updating some player’s stats. Reports include missing stats, delayed reporting of stats, and out-dated season stats. I’ve not been able to investigate every report (timeliness is obviously critical in a real-time stat tracker!), but in each instance I’ve been able to check, the MLB data source is, in fact, reporting the incorrect, incomplete, or out-dated states.

Unfortunately, there is nothing I can do about the incorrect stats feed. This is one of the down-sides of using a free, unsupported stats feed. We just have to live with whatever it provides us.

It seems like most people have had success checking later in the evening or first thing in the morning. Hopefully that will suffice in getting everyone the stats they need.

With any luck, there is just a glitch in the MLB.com stats feeds and they will right themselves soon. In the four or so years Kilg.us has been up and running, though, I haven’t seen this happen before, so I’m not quite sure what to expect.

Over the last few weeks I’ve received a number of messages that some players that haven’t played in the big leagues are showing season stats for 2012. All of that data comes from MLB, so if they show season stats in their data feeds, Kilg.us shows it as well. Usually these are spring training stats that just haven’t been updated.

I have made an effort to go through and delete any season stats for players that have been reported. If you see this behavior for any additional players, please let me know via email or in comments here.

As I mentioned yesterday, the old Kilg.us system for generating and sending email boxscores would run up against usage rates on a daily basis, leaving some messages unsent. With the move to SendGrid, our cap is raised, which is a good thing. I removed the limits from Kilg.us. What I failed to identify was that all of those unsent email boxscores were still sitting in queue (almost a full year’s worth…). When I removed the limit in Kilg.us and told it to send, boy did it ever!

I logged into SendGrid this morning expecting to see ~470 boxscores sent this morning. Instead the count was nearly 4000! I frantically shut down the process to try to reduce the irritation any more than had already been caused. But a lot of messages went out to people that may not have been expecting them. For that, I am very sorry!

For some reason, these email boxscores have been the single biggest headache with Kilg.us over the years.

Anyway, I’m back in the code tweaking things up in hopes of avoiding another disaster tomorrow.

For all those that have emailed me over the last year about a missed boxscore email, I hope it showed up today! /badjoke

Seriously, though, sorry about the extra messages. I’ll try to make sure it doesn’t happen again.

Update

I’ve made three changes to try to avoid this happening again:

1) I’ve cleared the email queue so nothing old even exists now.

2) I’ve capped the number of email boxscores Kilg.us will send tomorrow at 500. This is slightly higher than the number I think should be sent, so hopefully everyone gets their proper messages. If something goes wrong, though, the impact will be minimal.

3) I’ve updated the script that selects the emails to send to have it double-check that it is only pulling email boxscores generated in the current day.

Hopefully these three actions will prevent this from happening tomorrow. Nonetheless, I’ll be keeping a very close eye on things in the morning.

Last night I upgraded the email sending mechanism in Kilg.us to use SendGrid. This morning all email boxscores look to have gone out as planned.

As a number people have noted, “as planned” may be less that what we all would like. One of the things SendGrid will help me do is see how many emails are being generated, sent, delivered, read, etc. This will allow me to better manage how daily boxscore emails work.

The first issue to address are those users that have signed up for boxscore emails but haven’t received them. The old email provider had a much lower limit on the number of messages that could be sent per batch and per day. This lead to some less than ideal compromises in designing the system. Basically, rather than just flooding the email server with all the boxscore emails, I had to throttle the process, building a queuing system and sending out small batches of emails. As the system neared its daily limits, some people’s emails would go un-sent.

In addition to the migration to SendGrid last night, this morning I have updated the queuing and sending system. It now queues everything the same, but tomorrow will send one large batch of messages (using an anti-flood function to ensure the email server doesn’t get overwhelmed). This SHOULD ensure that everyone receives every email they have requested! We’ll check back in tomorrow morning to see if it worked.

As a tack on, I switched this WordPress installation to also use SendGrid which should make it less likely that I miss comments and questions people post on here (sorry if I missed your question or comment!).

In the next day or so Kilg.us will be moving its email handling to SendGrid. This should give me better transparency into what messages are being send, received, and bounced back. Right now things are a bit of a manual process which makes it difficult to diagnose one-off errors (particularly with daily boxscore emails).

If all goes well, I’ll probably make the change tonight. Theoretically, there should be no change for anyone but me. Let’s hope it works out that way!

This is by far the most common topic when people email me about Kilg.us. I’m thrilled that so many people are enjoying the tool (we start the year with over 1800 members, nearly 500 users per day), but these are bummer emails to receive.

Unfortunately, 95% of the time the answer: No, sorry.

As a free tool, Kilg.us needs to rely on whatever free stats we can find. As it turns out, MLB publishes some limited live data (I believe for their GameDay feature) that we’ve been able to tap. While it is great that the data is free, the bummer is that it isn’t comprehensive.

Kilg.us is already pulling every stat available from MLB and even calculating a whole series more that aren’t made available in the data feed.

If you’re favorite stat isn’t included, it’s almost certainly because we just don’t have access to it. At the start of each season, I double-check the feeds to see if any new stats are available. This year, there is nothing new.

So, if there is something that can be calculated with the data already in Kilg.us, let me know and I’ll be happy to add it! If you want a new counting stat added, though, I can’t add it.

Maybe someday an angel investor will throw a pile of money at Kilg.us and we can afford to purchase a real data feed (~$25k/year). Until then, we just have to make due with what we have.

Hopefully it’s enough to be useful to you!

(That said, don’t forget to click “Select Stats” for each of your teams to see all the stats that are available: currently 32 categories!)

From this end, it looks like all daily boxscore emails were sent out properly. We had a handful of bounce-backs (expected after 6 months of inactivity) but no fatal errors.

If you received an email boxscore that you don’t think you should have, or didn’t receive one that you think you should have, please double-check your account settings and if you see an error, please let me know.

I’m a little late getting this notice out, but Kilg.us will be up and running for the 2012 season. This evening I intend to enable daily boxscore emails so those should start showing up in inboxes tomorrow morning. Be sure to update your subscriptions (Manage My Settings) so you receive emails for the teams that matter to you!

Best of luck with your 2012 season!