Kilg.us – Fantasy Stat Tracker …Tracker

This morning the boxscore email script failed while running. I believe I’ve tracked the problem to a time-out error.

As many of you have surely noticed, Kilg.us is taking MUCH longer to render team pages than it did last year–2-6 times as long in my experience. Each time a team page is rendered, Kilg.us checks to make sure it has the latest stats for each player. If it doesn’t, it pulls in the latest and updates the database. This same process occurs when the daily boxscores are generated. This process is taking much longer than it used to. What used to take a few minutes is now trending over 90.

I’m not sure what I will do about this, yet. As I see it now, there are four options:

1. optimize the current process
2. replace the continuous look-up mechanism with a scheduled process
3. change hosting provider
4. change stats provider

I will likely start with #1, but I hold little hope of its success. #2 would suck as it could mean Kilg.us stats are a few minutes out of date at any given time. I’m also not sure it would actually decrease the load. I would love to do #3 as I’ve never been impressed with Kilg.us’s current host, but better options cost more money and the time to make the transition isn’t insignificant. Number 4 is also a fanciful option as I am yet to find another free source of live stats and paid options cost tens of thousands of dollars.

When I have a solution–or at least a path forward–selected I will post again. Until then…sorry.

Daily email boxscores should resume tomorrow morning–they send about 6am Eastern time. The emails contain your team’s stats for the previous day’s games. If you’d like to receive the daily boxscore emails for one or more of your teams, visit the Manage my Settings page and check out the Kilg.us Boxscores section.

If you think you’ve signed up for the emails and don’t receive one for your team tomorrow morning, please post a comment to this thread so I can investigate.

Happy baseball season!

With Spring Training upon us, I’m about to start gearing up Kilg.us for the new season. No big plans for upgrades currently. Kilg.us seemed to settle into a nice place mid-way through last year, so I’m hoping that will continue to be the case for 2011.

The one upgrade I am considering is converting/extending Kilg.us to work as a Facebook App. I’m not entirely sure of the constraints or repercussions of such a move yet, but it’s the only item leaping out at me as a fun upgrade. If you have any other ideas, send them my way.

Remember to follow Kilgus on Twitter (@KilgusTracker) and Facebook for all the latest news and updates.

I’m still working to diagnose the issues with daily boxscore emails. Based on my research, I think I have narrowed the issue down to daily boxscore emails for friends’ teams (although I still don’t know why). Is there anyone who has signed up for daily boxscore emails for their own teams that is not receiving them?

Yesterday I started receiving notes that players’ season at-bats in Kilg.us were topping out at 127, even when MLB was reporting a higher value. The reason was a simple oversight in database design. All the numeric values were defined as “tinyint” within the database. This is what is used throughout the daily stats, so it was copied over when the season stats tables were created.

Tinyint is a data-type that only allows for values from -127 to 127. This is great for daily stats (the only daily stat I could imagine passing 127 would be a pitch-count), but doesn’t work so well for season stats. I have changed the data-type to “smallint” for all fields that could exceed 127 in a full-season. Smallint will allow values up to 32767.

Season-to-date stats are collected on a scheduled basis, every 15 minutes. In the next few minutes we should see season-to-date at-bat values correcting themselves. I’ll be monitoring this to ensure it happens so I can diagnose further problems if it doesn’t work.

Thanks to all of you that chimed in with this issue and for providing details so I could correct it quickly!

I noticed the other day that Kilg.us had recorded a save for one of my pitchers that, in fact, had blown a save. Reviewing the code that collects these stats, I think I’ve identified the issue.

Unlike most stats which come from a dedicated attribute in the XML files, saves, wins, losses, and holds need to be parsed out of a generic “notes” attribute. The logic performing this parsing was looking for “S,” to identify a save. While this matches the pattern (S,9)–a pitcher who recorded his 9th save–it also matches the pattern (BS,3)–a pitcher who has blown his 3rd save. I have updated this parsing pattern to look for “S,” in the absense of “BS,”. Hopefully this will fix the problem. I’ll be keeping an eye on this throughout the evening, but ping me if you notice any issues.

I jumped into the Find Friends section of Kilg.us this evening, planning to update the logic to auto-accept all friend requests to “Major League Baseball” (at present, I manually check the account daily and accept requests). Hitting the page, though, I encountered a Lucene search error, so that derailed my attention.

It looks like the search logic was getting confused when the page was sent an empty query (just clicking the “Find Friends” link rather than using the form on the Dashboard). This wasn’t happening a month or so ago when I set it up, so I’m guessing it has something to do with the size of the index that is being queried now.

Anyway, I updated the code to handle empty queries more gracefully. While testing that, I noticed an issue when requesting Friends that don’t have a name on their profile. When making the request, the prompt would ask if you wanted to request ” ” to be your friend. That isn’t very helpful. I updated that script to pull in the user’s email address if a name isn’t on the profile. Now when you ask someone to be your friend, you will either be prompted with their name or their email address. Much better.

In the process, my JavaScript errored out a couple times. This, unfortunately, means I requested to be friends with a couple people I don’t know. So if you got a random friend request from me, I won’t be offended if you decline!

There has been an unforeseen development since adding drag-and-drop to Team pages on Kilg.us. The purpose of drag-and-drop was to easily move players on an off the bench. Drag-and-drop also lets you change the order in which players are listed on your team page. A number of people have noticed that, after they re-order their players, that order isn’t maintained when they reload the page.

Kilg.us does not keep a record of the order you place your players in–it only tracks whether they are on or off the bench. When you load your team page, your players will always display in scorecard order–Catcher, 1st Base, 2nd Base, 3rd Base, Shortstop, Left Field, Center Field, Right Field. If you have multiple players at a position, they will be in alphabetical order based on last name, then first name. The position for a player is based on what MLB lists as the player’s primary position. This may or may not match up with the position qualifications for players in your league.

MLB doesn’t differentiate relief and starting pitchers, so they are all listed alphabetically–again by last name, then first name.

As time allows, I will look into letting Kilg.us monitor the order players are dragged into. For the immediate future, other features are priority.

Since the release of season-to-date stats, a lot of people have been asking for games played and games started to be included. I’m doing some research to find a reliable place to find such a statistic. If I can find it–and figure out how it plays into a paradigm with daily stats, too–I will incorporate it. The level of effort to include it is pretty minimal, assuming it can be found. More details to come.

Has anyone noticed a problem receiving some of their daily email boxscores? If so, please add a comment or email me. I think some might be being missed, but would like to hear from any people actually experiencing problems.

Thanks for your help!