Kilg.us – Fantasy Stat Tracker …Tracker

I have moved Kilg.us back to referencing the database on the new servers. Once again, things seem to be going well. Hopefully that will still be the case in a couple hours!

I did not attempt to copy over the old database. There is a lot of complexity involved in merging databases and I just didn’t have the patience after yesterday. As a result, if you added or removed players, moved them on or off the bench, or manipulated a team or your account in the last 24 hours, those changes will be lost. You will need to re-apply any changes.

To the couple people that signed up for Kilg.us today–I’m sorry, but you’ll need to do it again! I promise this isn’t a regular occurrence. In fact, I’ve never deleted an account before!

The only outstanding issue that I am aware of from the migration is the appearance of players that are not on the bench. They are rendering as gray and “benched” right now, but the team is still calculating their stats in totals. I’m working on that now and assume it will be a quick fix.

If you notice any other anomalies, please comment here, email me, or post to Facebook or Twitter (@kilgustracker).

**UPDATE 10:40PM**

It appears that batter stats are being collected properly for individuals, but tallied improperly for teams.

It appears that pitcher stats are not being collected properly for individuals, but tallies are correct for teams (based on day-old data).

Still investigating.

**UPDATE 11:19PM**

The culprits seem to be the stored procedures that Kilg.us tries to use for storing stats. All the procedures seem to be failing with the move to the new server. Luckily, Kilg.us included fallback string concatenation mechanisms for exactly this situation. I have disabled all the stored procedure calls and everything now seems to be working.

This morning on Facebook I mentioned a few goals for fixes and improvements to Kilg.us this weekend. Those items were:

1. update MLB player/team lists
2. get email boxscores working
3. speed up stat collection
4. trim players that show up on the “Major League Baseball” team rosters (right now it includes players throughout all the minors, with no respect to whether they might actually appear in a real game…

I’m pleased to report that numbers 1 and 4 are soundly completed! As of this morning, all active MLB players should be listed in Kilg.us and associated with their correct teams. If you follow any of the “Major League Baseball” teams, those rosters should also be cleaned up to only show players on that team’s 40-man roster.

Unfortunately, 2 and 3 are stills works in progress. I have made some changes to the model for how player objects are created. The result of this change is that player stats are only refreshed from MLB.com when needed, rather than any time a player is referenced. In practice, loading a team page requires that a player’s stats be up-to-the-second. When a daily boxscore email is generated in the wee hours of the morning, though, there is no need to go gather latest stats–they haven’t changed in the hours since the games all ended.

In my testing so far, team pages are loading 10-40% faster now. I’m still not happy with their performance, but at least it is progress. When I try to generate all the daily emails, those are running significantly faster. Before today, the process was trending towards 90 minutes, timing out, and failing. Now the process is hammering through 80 (of ~360) emails in the first minute and sending emails properly. At that point, the server slows to a crawl and, while it continues to output the email content, it refuses to send the emails.

I considered reaching out to the hosting company for help in identifying why their system was doing this, but I’ve given up. The company (AMS Computer Services) and infrastructure that hosts Kilg.us sucks. I’m going to try moving Kilg.us to a cloud hosting solution from Rackspace. I’ve spent a good part of the day getting the new server up and configured. Tomorrow I should be able to start moving Kilg.us over, testing, troubleshooting, and seeing how it performs. It’s going to be a long, tedious, frustrating process but it can’t possibly be worse than sticking with the status quo.

In the meantime, my fingers are crossed that our daily boxscore emails will fire off properly in the morning…but I’m not holding my breath.

The reason I’ve mentioned in the past for not making the move to a new hosting solution (besides the time and effort) is that it will cost more than the current option. Because Kilg.us is going to cost me more to host now, I’ve added a PayPal donation button to give anyone interested the option in helping to support Kilg.us. I’m not hoping to make anything off Kilg.us, but it would be great it is wasn’t a financial burden. If you like using Kilg.us, I hope you’ll consider giving a little bit to help offset my costs. $5, $10, $1, anything will help!

I hope to have an update with good news by tomorrow evening as I start to migrate Kilg.us.

There has been an unforeseen development since adding drag-and-drop to Team pages on Kilg.us. The purpose of drag-and-drop was to easily move players on an off the bench. Drag-and-drop also lets you change the order in which players are listed on your team page. A number of people have noticed that, after they re-order their players, that order isn’t maintained when they reload the page.

Kilg.us does not keep a record of the order you place your players in–it only tracks whether they are on or off the bench. When you load your team page, your players will always display in scorecard order–Catcher, 1st Base, 2nd Base, 3rd Base, Shortstop, Left Field, Center Field, Right Field. If you have multiple players at a position, they will be in alphabetical order based on last name, then first name. The position for a player is based on what MLB lists as the player’s primary position. This may or may not match up with the position qualifications for players in your league.

MLB doesn’t differentiate relief and starting pitchers, so they are all listed alphabetically–again by last name, then first name.

As time allows, I will look into letting Kilg.us monitor the order players are dragged into. For the immediate future, other features are priority.

Players are added and removed individually from each of your teams. Kilg.us doesn’t try to enforce any specific league rules for when players can be added or removed from teams. You can do so at any time.

Find a Player

To find a player to add to your roster, you must first navigate to your team page. Across just below your team’s name there is a series of links including “Add Player”. Clicking this link will take you to the page to search for and add a player.

On the Add Player screen you will see a group of search fields in the right column. As their labels describe, you can search for players based on first name, last name, and position.

Kilg.us doesn’t use Lucene search for player searching yet (it does for friend searching), but this functionality will be added later in 2010.

Add Player page with result for a search on "Young"

All of the search fields you direct string matches. This means they will find sub-strings within longer names, but won’t find variations on the terms you entered. So, a last-name search for “you” will return “Young” and “Youkilis” but not “Lyon”. If your search doesn’t return the player you were expecting, trying using a shorter string of characters.

Review a Player’s Stats

After you have some search results returned, you can check out the stats of the players in the list. Clicking on a player’s name will open a layer that shows the player’s picture, total season stats, and a link to the player’s MLB.com profile page.

Add Player page displaying Michael Young player layer

Add a Player

To add a player to your team, click on the green “plus” sign (+) next to his name. This will bring up a prompt to confirm that you wish to add the player to your team. If you accept, the player will be added to your team. This is handled by an AJAX request which means the page won’t need to reload. A new prompt will appear once the player has been added.

You can add as many players to your team as you would like in this session.

Activating your Player

After being added to your team, your new player will be placed on your bench. This means his stats won’t be counted when your team’s stats are calculated. To activate the player, click the gray-bars handle to the left of the player’s name on your team page and drag him above the “Bench” bar. To return him to the bench, just drag and drop him below the “Bench” bar.

Players riding the pine on a Team page

Remove a Player

If you have added a player to your team accidentally or just want to remove someone, you can do so at any time. On your team page, there is a red “X” icon on the right end of each player’s stat line. Clicking this icon will prompt you to confirm that you wish to remove the player from the team. If you confirm, the page will reload and the player will be removed.

You can always add a player back to your team by using the same process described above for adding a player.

Tonight I updated the mechanism in which Kilg.us gathers all the players for the Major League Baseball Teams. The old process involved some manual intervention. The new process is entirely programmatic. At the moment, I need to kick off the process manually, but I hope to get it scheduled here in the near future. I want to optimize the process a bit before tossing it in a CRON job, though. Right now the process takes about 2 minutes to run and generates about 3200 PHP Warning messages.

All the warning messages are generated by a series of HTTP requests. The way the process determines if a player is on a major league roster is by checking if the player has an XML file on the MLB server for the current day. If the file exists, Kilg.us ensures that player is on the proper Kilg.us team. If the file doesn’t exist, Kilg.us deletes any records of that player from the proper Kilg.us team. This also generates 2 Warning messages per failed HTTP request, though.

I’ll keep an eye on things, see if I can clean it up, and run the script every few days to keep the MLB teams up-to-date. Depending on how things go, I might go ahead and schedule the task or I might re-write it.

Another update today was in the Javascript that calculates the number of innings your pitchers have thrown on a given day. In the past, if your team had thrown an increment of innings that included 1/3 of an inning, your total innings probably rounded that up to 2/3.  This was because of a basic flaw in my logic for calculating those partial innings. I’ve corrected it (actually re-writing the entire function) and now innings pitched should be tallied properly.

I quit working too late last night to post about the updates, so I’ll cover a couple days’ work now.

The biggest visible change was an update to the top site navigation. Previously, after logging-in, if you visited the home page or Tracker blog, there was no way to return to the system without logging-in again. Obviously, that was inconvenient. The home page and blog now both recognize an authenticated session and will display a new navigation link (that exists throughout the site now) to visit the Dashboard. This will get you back “into” Kilg.us without needing to log-in again.

The other big functional change is on the Add Player page. After receiving a report that adding a player was causing an alert in Firefox, I dug in to see what was happening. Unfortunately the alert was simply a slow script notice, not an actual error. What I ultimately determined was that the cumulative stat bubbles that I added the other day were causing an odd JQuery loop when the page is unloaded. Rather than trying to dig into JQuery to see what the problem was, I determined it could be alleviated by reducing the number of instances of the stat bubbles on the page. So the Add Player page now features pagination! Search results will be displayed 50 at a time with pagination links at the bottom of the page. This leads to quicker load times of the page, as well, so it seems like an improvement all-round.

Complicating the new Add Player pagination is the position filter. That filter literally filters down the players shown on the page, based on position.  All it really does is turn off the display of players that don’t match the filter. So, previously, changing the filter to a specific position would cause a display of all players at that position. Now, changing the filter will only show the players among the currently displayed 50 that match. This seems confusing to me, so I think I will need to re-write how it works.

On a non-visible vein, I’ve incorporated the functionality to allow Users to view other Users’ teams. This is a precursor to the social-stat-tracking concept I’ve been discussing. There is no interface to share your teams with other Users yet–the relationship has to be manually set in the database. There is a new relationship type, though. Originally a team only had one User related to it: the Owner who had total control to do anything with the team they wished. Now there is a Viewer User type. This User type can only view the team. They cannot change the team name, delete the team, add players, remove players, bench players, or change the team’s stats. I’m hoping to waste a good chunk of time this weekend working on social-stat-tracking. With a lot of luck, I’ll have something working by opening day.

On a reality-strikes front: the iPhone web app won’t be ready for opening day. Once life settles down a bit, I’ll try to get back to it later this Spring or Summer.

A little post-superbowl work tonight involved some improvements to the team creation process. The biggest improvement was addition of instructional text. Kilg.us now guides the User through creating a new team and then adding players to that team.

I also added some simple data checking around the length of team names. The checks come into play both when creating a new team and changing the name of an existing team. The checks count the number of characters in the name and provide a notice if the User enteres more than 24 characters. The only reason for the 24-character recommendation is to ensure the name displays nicely in the layout when viewing the Team page. There is no technical or architectural reason for the recommendation, so the application doesn’t enforce the limit–it simply recommends it. A User is free to define a name of more than 24 characters.

It’s been a little while since I’ve made any changes to the Fantasy Stat Tracker, but I had a few minutes today and thought of a great improvement. Previously, when viewing your team you could identify which players were on the “bench” for a given day. Unfortunately, that designation as a “bench” player was specific to the current, loaded instance of the Team page. When you reloaded the page (to see updated stats), you lost your bench designations and needed to re-check your bench players.

As of today, that is no longer the case. When you tag a player as being on the bench, they will stay on the bench until you explicitly remove them. After benching a player, when you reload the page they will still be on the bench. If you log-out and log back in tomorrow, they’ll still be on the bench. Until you un-check the little “bench” box next to the player’s name, their stats won’t count in your calculated totals.

This, for me, is probably the biggest annoyance I’ve had with the Stat Tracker, so I’m quite pleased it is resolved. Hopefully this will be helpful to everyone else using the tool as well.

As an added bonus, when a player is on the bench they will automatically be dropped to the bottom of the player list so their stats are easily ignored as you scroll through your other players.

I’ve re-written how the Fantasy Stat Tracker is gathering its player list. Previously it was pulling data that was created before the season started. This left the application without a few dozen players and some players with incorrect information (players on the wrong teams or with incorrect positions). The application is now capable of pulling player data from daily game stats. This means that I can run the process to grab player data and have it pull in everything updated to the current day. Rookies and other new players will be added, and positions will be updated if they change.

In most fantasy leagues, as players play a new position they ALSO become available at that position rather than replacing their old position with the new one. Because the Fantasy Stat Tracker isn’t as much about your roster and league rules as it is about the stats for the players, I don’t see the behavior of replacing positions as being too troublesome. When I run out of things to work on, though, I’ll look into listing players with all their applicable positions.

It is now possible to sign up for an account on the Fantasy Stat Tracker, build your team, and track your stats! I’ve finished the functionality just in the last few minutes, so there is A LOT of work yet to do–rough edges to smooth, data to validate, workflows to finesse. But…the basic functionality works! There is no warning if your email duplicates an existing one (it will fail silently and won’t create your account). It also isn’t checking whether user names exist already, so it’s possible to duplicate users. That means down the road I might flush the database to get rid of crap data. So, if you sign up now, I don’t promise I won’t erase your account as development continues. I’ll try not to, though.

I was able to create an account and build a 25-man roster in less than 5 minutes. Please, give it a shot, and let me know what you think. Please post comments on the Fantasy Stat Tracker back to this blog post.

Enjoy!