Kilg.us – Fantasy Stat Tracker …Tracker

A blog about the development of Kilg.us – The Fantasy Baseball Stat Tracker

This looks like the end of the road…

April 18th, 2018

First of all, thanks to everyone that filled out the survey I sent out a few weeks ago. Over 130 people completed it, which was well more than I ever would have guessed. It was wonderful to hear all the different ways you all have been using Kilg.us, and fun to see how long many of you have been using it!

Unfortunately, while I was looking for ideas and opportunities to make Kilg.us more reliable, it looks like the data behind-the-scenes is leaving us.

As many of you have noticed, the season-to-date stats on Kilg.us haven’t been working this year. I dug into the issue a little over a week ago to see if there was something I could do to fix it. What I found was that the data feed Kilg.us has been using for almost a decade now has undergone some changes. It was never an official data source. My best guess is that it is some XML files that MLBAM published to run some of their Gameday apps. I don’t think it was ever meant to be public, but one day about a decade ago I stumbled across it in a Google search and realized I could build Kilg.us on that data. The data disappeared from Google within a couple days, but continued to be published and unrestricted for years. For 2018, it looks like the feeds have undergone some changes. Some of the data (like where the season-to-date numbers were from) doesn’t seem to be there anymore. The other data–today’s stats–looks like it is generally requiring some kind of token system to authentic. I can’t view it in a browser anymore, but for some reason the server process I built to gather it still seems to be getting in. Even that process can only gather players that were in MLB in years prior, though. It looks like they aren’t creating new files for new players.

I’ve been checking in on the data off-and-on the last week to see if this change was a temporary blip or a real, intentional change. It sure looks like they meant to do it.

So, as best I can tell, the current state of Kilg.us is that it can’t get season-to-date stats, it can’t get stats for rookies, and the stats it can get, it is very clear the publishers don’t want it to get.

On top of all the other issues Kilg.us has developed over the years: failures with daily boxscore emails, finicky user search, slow input of new players, limited statistical categories available, poor utility on mobile devices, lack of encryption, and a degrading server infrastructure…it feels like the time has come to shut it down.

I know a lot of people have offered to contribute–either in time or money–to keep Kilg.us alive. As much as I would like to believe that was a feasible approach, the cost of a reliable data feed alone is about $25k. When you layer on the cost of re-building the data digestion, creating a new interface that works on mobile devices, and refactoring all of the daily boxscores, we’re probably realistically looking at $100-200/user rather than the $10-20/user/season most people indicated they would contribute to Kilg.us. I’ve kicked around the idea of forming a Kickstarter (or something similar) to see if there is a realistic appetite, but I don’t think I personally have the time to make it happen right now.

If someone else gets the bug to try such an endeavor, I’m happy to help in any way I can: offering insight into running such a site, code snippets (not pretty), database designs, lessons learned. You can reach me at matthew@superhumane.com

For all of those that are going to be hurting with the loss of Kilg.us, I think the best replacement I’ve found for most people is a tool from MLB: their Prospect Tracker. Despite the name, it lets you track major leaguers, too. It has daily stats (and yesterday, last 10 days, etc…a lot of things people have requested over the years), year-to-date, and categories Kilg.us could never get. You can “share” a roster which should let you track your team (and any of your league-mates teams). It doesn’t sum stats for the “team” or have daily emails. I hope it makes the transition easier for some of you, though!

My current intention is to keep Kilg.us running in its current state through the end of the month. At that point, I’ll backup the database, make a copy of the cloud instance, and shut down the server. I don’t suspect anyone has a lot of unique data in Kilg.us, but if you do have dozens of teams and hundreds of players built out that you don’t have inventoried elsewhere, give me a shout and I’ll try to get you a database dump of your stuff.

I’ve been holding off writing this post for a few days now because I know it is going to disappoint a lot of people. While I don’t use Kilg.us anymore, I love that so many of you have found value in this silly little site I built. I’m going to be sorry to see it go (though I can only imagine the replacements you all find will be more reliable!), but it has significantly out-lived the usefulness I ever expected of it.

Thank you to all of you that have used Kilg.us, shared Kilg.us with your friends, and sent nice notes of appreciation over the years. If you find a good replacement that you would like to share with others, please let me know and I will try to pass it along to others that could use it. Best of luck with your teams!

 

2018 Season

March 19th, 2018

Nothing new to report since last year, so I’ll link to the 2017 Season post for reference.

That said, I do need to figure out what to do with Kilg.us in the future, but don’t want to leave people hanging! Please take a few minutes and fill out this survey to help me understand how you all are using Kilg.us.

To address a suggestion many people have had (“charge for it!”): last I checked a legit, licensed data feed starts about $25k/year. I wouldn’t feel good about charging for a service in which I couldn’t guarantee the data will be available any given day in the future, so this is a pretty big initial cost to try to cover. That said, if anyone knows of a good, cheap/free way to source MLB stats, please let me know in the survey!

2017 Season

March 8th, 2017

First, let me apologize for Kilg.us being offline for so long. I’ve heard from many, many of you asking about the status of the tool for 2017.

The good news: I rebooted the server and it came back online, so I expect Kilg.us to run through 2017.

The less good news: I have no time to manage, maintain, or update Kilg.us. It is, unfortunately, at the bottom of the priorities in life at this point. So, as has been the case the last few years, I will make an effort to keep Kilg.us online but I don’t have time for bug fixes or offering support to users. That means when features fail (like the daily boxscores which relied on a 3rd-party email service that stopped offering an appropriate free tier for us to use; or the Invite a Friend function) I don’t have the bandwidth to diagnose and fix them. As long as the data source for Kilg.us keeps working as it has in the past, I’ll try to keep Kilg.us up. But, it is almost a decade old at this point and is in desperate need of  lot of technical cleanup, so I can’t promise how long it will stay available.

That said, I need to think about the future of Kilg.us. If it is going to stay available long-term I either need to find a financial model for Kilg.us that allows me to dedicate work hours to it, or find another group or individual interested in taking Kilg.us off my hands. If anyone with a deep technical knowledge (Rackspace Cloud, PHP, MySQL, XML) is interested in taking over Kilg.us, you can reach me at admin@kilg.us to discuss.

For now, hopefully daily stats start flowing into Kilg.us as you all get back to using it. Remember that there are always funny glitches around Spring Training stats (and how some of those might carry into the regular season for players sent to the minors).

Good luck with your teams in 2017!

Daily Boxscore Email Update

May 26th, 2015

I apologize for the delay on getting this issue addressed.

As noted previously, daily boxscore emails stopped going out earlier this month. I’ve finally had time to dig into the issue.

First, if you are subscribed for daily emails, you should have received a boxscore email around 9:30am Mountain time today. If you didn’t receive such an email (and confirmed it isn’t in a SPAM folder), please comment here or email me.

To publish daily boxscore emails, Kilg.us goes through a few steps. First, it makes sure all the latest stats have been collected. It then starts a queue for composing the emails. Each team is checked for subscribers and email content is saved to a database table. Once complete, the queue is closed (the stages of opening and closing this queue are recorded). The next step is to systematically go through that DB table and send the emails. As each email is sent, it is deleted from the table. The emails are sent in batches, 50 at a time.

A few observations:

  • The database shows that the composition queue was completed each day of the outage.
  • The database is stuffed full of all the emails that haven’t gone out in the last few weeks.
  • SendGrid (the 3rd party that actually sends the emails at Kilg.us’ request) shows that a full batch (~550) emails went out on May 1. May 2 dropped to 50, followed by 200, 50, 100, and then to 0 in subsequent days. These increments of 50 suggest something is going wrong with our send batches.
  • I used the exact same scripts to successfully send the emails today (manually executed, rather than scheduled).

All told, I haven’t identified a specific cause of the outage. The send process can take a while so it is possible we ran into a timeout along the way. The server has been updated, scrubbed, and rebooted so it should be in tip-top shape for tomorrow. Fingers crossed that emails will go out as scheduled. If not, I at least know I can manually trigger the process daily until I have time to dig back in deeper.

Email Boxscore Issues

May 18th, 2015

Numerous people have notified me of an issue with daily email boxscores. It seems that since early May, many have not been delivered properly. This is on my radar. Unfortunately, I have not had a chance to dive in to resolve it. Trust that as soon as I find time, I will get it fixed and post an update here about what happened.

2015 Season

April 6th, 2015

The new season is upon us! I’ve had a number of people checking in to see if Kilg.us will be functioning again this year. It looks like our data feeds are still working, so at this point I don’t have plans to shut Kilg.us down.

Daily boxscore emails should resume tomorrow. If you would like to change your settings for which teams you receive, please access your account settings.

As a quick reminder: Kilg.us is in maintenance mode. If something goes down, I’ll try to fix it in a timely manner. At this point I don’t actively use Kilg.us, though, so it isn’t a high priority. Last year I posted some things to keep in mind about the operations of Kilg.us. Please take a look.

Good luck with your 2015 season!

Log-in Issues (User Sessions)

May 11th, 2014

A number of people have reported issues logging in to Kilg.us today. After a little exploration, I found that the database table that records user session data was corrupted. After repairing the table, Kilg.us seems to be back to normal and properly logging users in.

This is the first time we’ve encountered this issue, so I’m not going to dig further into the cause of the corruption. If it happens again, though, I’ll dig a bit deeper.

For now, you should be able to log in normally. If you have any problems, please let me know.

Email Boxscores and 2014 Update

April 9th, 2014

As of tomorrow morning (4/10), Kilg.us should resume sending daily boxscore emails for teams that have requested them.

In order to avoid the flood of unsubscribe and SPAM reports with the first emailings of the new season, I discontinued daily boxscore emails for all teams that haven’t been updated since the start of 2014. If you think you should be seeing a daily email but don’t, check your settings and confirm the team has been signed up. I don’t believe any teams that have been updated since the start of the year have been affected, but it’s always possible.

Obviously, I’ve been a little slow getting things up and running for the new season. I’ll blame a perfect storm of vacations, the constant distraction/entertainment of my first sons’ birth, and starting up a new business (Super Humane. If you have a company that needs some help making the world better through technology—the mobile web in particular—I’d love to chat!). On top of it all, I’m not actually participating in a fantasy league this year for the first time in recent memory. Nonetheless, my intention is to continue operating Kilg.us for the 2014 season.

A few reminders of what the continued operation of Kilg.us involves:

  • We use a free but limited stats feed in order to have up-to-the-second stats. Sometimes it has issues (like Spring stats trailing into the season…), but 99% of the time it works.
  • The stats already in Kilg.us are all we have. We can’t add games played/pitched, WAR, or anything else that isn’t there now.
  • Kilg.us was built to display real-time stats. We added “season-to-date” later, then tacked on the daily boxscores to help people who couldn’t log in every night. Kilg.us was never intended to help slice and dice stats.
  • When things go wrong (it seems to happen once or twice a year), I’ll try to fix it as quick as I can. It usually takes a day or two.
  • If you have a problem, email me. I don’t regularly use Kilg.us anymore, so I need people to point out issues when they arise. I’ll respond as soon as I can.
  • You’re welcome to manage your team, all your teams, all the teams in your league, or whatever you like. We’ve got over 6000 teams being managed by 3400 users right now.
  • Kilg.us is free! Which is great! Especially when generous folk are willing to donate to offset the hosting costs associated with it!

Best of luck for your 2014 season!

Server Issues this Morning

August 25th, 2013

This morning a lot of people awoke to find Kilg.us was kicking out errors and was showing all players had been lost from all team. Frankly, not a nice way to start the morning!

Luckily, the issue was resolved with a quick reboot of the server. It seems to have been an issue with one of the database tables, but I’ll be performing some additional diagnostics to try to determine what caused all the errors this morning. For now things should be back up and running.

Sorry for the inconvenience!

Spring Training Stats

April 5th, 2013

I’ve been getting a flood of emails about spring training statistics showing up for players that haven’t played in the big leagues yet this year.

Kilg.us collects all of its stats for an XML feed that MLB publishes. Every time you request the stats for your team(s), Kilg.us checks to see what the latest stats are and loads them into the system.

Unfortunately, it seems that MLB leaves springs stats listed for all players that appear in the spring but are then sent to the minors (or DL). Until they appear in a big league game, MLB will continue to show springs stats in their feed.

This is just how it works.

If you have a player on your team that is showing spring stats, please move that player to your bench. The player’s individual stats will still show from spring, but this will remove the player’s stats from your team totals.