Kilg.us – Fantasy Stat Tracker …Tracker

Over the last couple weeks, I have received numerous emails from people having trouble signing-in to their accounts. With a little digging I was able to find a consistent theme: they were all using Internet Explorer 8 or 9.

Specifically what was happening was that a person would type their existing username into the sign-in form and would then be prompted to create an account. As everyone is probably aware, Kilg.us uses the same form for logging in and signing up. If you enter an existing username, it asks you to log-in. If you enter a username that doesn’t exist, it asks you to sign-up.

To manage this functionality, Kilg.us generates an AJAX request after you enter your username then move focus to the password field. This is creates a “change” event on the username field. When that event happens, Kilg.us asks the database whether the username you entered exists. The database returns a response in the form of XML. The XML has one message: the username is valid (it exists in the database) or it is invalid (it doesn’t exist in the database).

Once the XML response is returned, some JavaScript parses it to read whether it says “valid” or “invalid”. If the value is “invalid” the form asks you to create an account. The JavaScript to read this follows standard practices to navigate the document object model (DOM) and find the value in question. This works in Firefox, Chrome, Safari, Konqueror, Mobile Safari, Opera, Android Browser, Blackberry Browser, etc., etc. Apparently, how Internet Explorer handles the XML is different. It doesn’t create all the parameters that the standard approach would.

This shouldn’t come as a surprise to anyone.

What I ultimately found is that Internet Explorer will parse the value out and save it in a “text” parameter for the XML object. Rather than navigating the DOM in a standard fashion, I’m now taking a shortcut and checking if that “text” parameter exists on the XML object. If it does, I read it in as the returned value. If it doesn’t (obviously, all the other browsers to generate this rogue parameter), I continue to navigate the tree and get the value in the proper manner. This only works because the only value of the XML is the “valid” or “invalid” message.

The end result: the log-in/sign-up form logic should now be working again in IE. And still working in every other browser. If you find that not to be the case, please let me know (comment, email, Facebook, Tweet).

I got a couple emails yesterday that some daily boxscore email stats were incorrect. Looking into the matter, it was immediately obvious what had happen. As I worked on the transition of Kilg.us to its new servers, I had accidentally left a database connection pointing to the new dev database, instead of the old, production database. As such, the stats weren’t up-to-date when the daily emails were generated. I simply pointed the connection back to the correct database and figured all was well.

This morning there have again been reports of incorrect stats. I double-checked that the database connection was pointing to the correct IP address–it was. Then I remembered one of the many reasons I’m moving Kilg.us: the hosting company uses a non-standard port for database connections! Why? Who the hell knows. But I had forgotten to explicitly change back to their port. I’ve done that now. I’ve tested the connection. All seems to be well.

On the migration topic, Kilg.us is up and fully functional on its new servers. I’m testing manually with day-to-day usage to try to identify any aspects or configurations I may have missed. Right now, my simple tests are showing that Kilg.us runs between 2.2 and 5 times as fast on the new servers (adjusting as best I can for load considerations). I’m working on some minor changes that will decrease the bandwidth Kilg.us requires because the new servers are billed by data transfer. I’ll give at least 24 hours notice before I make the migration. Depending on how quickly the domain propagates, Kilg.us may be down for some of you for up to 24 hours during the migration. More details to come.

Tomorrow morning Kilg.us will use a entirely new process for generating and sending daily boxscore emails! So let’s all cross our fingers and hope it works…

I have spent all of this week re-designing and re-building how emails are generated, stored, and sent. Suffice to say, the new system is significantly more complex than the old one. When I’ve had a chance to breath and am not so tired, I’ll go into more detail on how it works.

My expectations of the new system are that it will:

• Scale to accommodate as many users/emails as desired (the old system failed in the 200-300 range)
• Allow of future portability if Kilg.us needs to change hosting
• Implement all components to identify HTML vs. Text email so boxscores render properly on all clients
• Allow for side-by-side HTML and Text versions (text version is forthcoming)
• Provide redundancy of data, so if the process fails it can be re-started without losing a day’s data
• Reduce likelihood of Kilg.us mail servers being identified as SPAM and black-listed
• Lower computation overhead

It sounds almost too good to be true! Tomorrow’s push will be the first full run trying to send out to everyone’s unique addresses. All the testing so far has been delivering to a kilg.us address (or small tests to outside domains). I’ll be keeping an eye on things in the morning to try to ensure everything goes smoothly. If you do not receive your normal boxscore in an acceptably legible format, please post comments here with details (DETAILS–without them I can’t do anything–what email client, what team name, what was wrong, screenshots, any little detail).

Yesterday I started receiving notes that players’ season at-bats in Kilg.us were topping out at 127, even when MLB was reporting a higher value. The reason was a simple oversight in database design. All the numeric values were defined as “tinyint” within the database. This is what is used throughout the daily stats, so it was copied over when the season stats tables were created.

Tinyint is a data-type that only allows for values from -127 to 127. This is great for daily stats (the only daily stat I could imagine passing 127 would be a pitch-count), but doesn’t work so well for season stats. I have changed the data-type to “smallint” for all fields that could exceed 127 in a full-season. Smallint will allow values up to 32767.

Season-to-date stats are collected on a scheduled basis, every 15 minutes. In the next few minutes we should see season-to-date at-bat values correcting themselves. I’ll be monitoring this to ensure it happens so I can diagnose further problems if it doesn’t work.

Thanks to all of you that chimed in with this issue and for providing details so I could correct it quickly!

I noticed the other day that Kilg.us had recorded a save for one of my pitchers that, in fact, had blown a save. Reviewing the code that collects these stats, I think I’ve identified the issue.

Unlike most stats which come from a dedicated attribute in the XML files, saves, wins, losses, and holds need to be parsed out of a generic “notes” attribute. The logic performing this parsing was looking for “S,” to identify a save. While this matches the pattern (S,9)–a pitcher who recorded his 9th save–it also matches the pattern (BS,3)–a pitcher who has blown his 3rd save. I have updated this parsing pattern to look for “S,” in the absense of “BS,”. Hopefully this will fix the problem. I’ll be keeping an eye on this throughout the evening, but ping me if you notice any issues.

I jumped into the Find Friends section of Kilg.us this evening, planning to update the logic to auto-accept all friend requests to “Major League Baseball” (at present, I manually check the account daily and accept requests). Hitting the page, though, I encountered a Lucene search error, so that derailed my attention.

It looks like the search logic was getting confused when the page was sent an empty query (just clicking the “Find Friends” link rather than using the form on the Dashboard). This wasn’t happening a month or so ago when I set it up, so I’m guessing it has something to do with the size of the index that is being queried now.

Anyway, I updated the code to handle empty queries more gracefully. While testing that, I noticed an issue when requesting Friends that don’t have a name on their profile. When making the request, the prompt would ask if you wanted to request ” ” to be your friend. That isn’t very helpful. I updated that script to pull in the user’s email address if a name isn’t on the profile. Now when you ask someone to be your friend, you will either be prompted with their name or their email address. Much better.

In the process, my JavaScript errored out a couple times. This, unfortunately, means I requested to be friends with a couple people I don’t know. So if you got a random friend request from me, I won’t be offended if you decline!

Has anyone noticed a problem receiving some of their daily email boxscores? If so, please add a comment or email me. I think some might be being missed, but would like to hear from any people actually experiencing problems.

Thanks for your help!

A comment yesterday brought to my attention that Kilg.us was not showing WHIP values for individual players on their Season-to-Date stats. This was an oversight in an assumption I made about the data Kilg.us was collecting and storing.

Each time Kilg.us renders stats, it checks to see if the latest stats are in the database or if they need to be collected. If the latest ones are in the database, it displays those. If not, it collects the new ones, saves them, and displays them. The XML data from MLB contains the season WHIP stat for each pitcher. So I assumed I was also storing that value in the database.

As it turns out, I wasn’t. The database stores the counting values (innings, hits, walks, etc). Kilg.us  then calculates the average stats (WHIP, ERA, k/9, etc.) based on the counting stats. In the Pitcher object, it was attempting to pull the WHIP value from the dataset returned, which works when collecting new stats from MLB. When the latest stats are in the database, though, that WHIP value doesn’t exist, so “-.–” is displayed instead.

I have updated the Pitcher object to calculate the WHIP value whenever it is needed. This means WHIP should be displaying in Season-to-Date stat views now. If you find otherwise, let me know!

I haven’t posted about any Kilg.us changes for a couple days, but there has been a little going on. In my defense, I started a “Road to the Show” career in MLB 2010 The Show, so I MIGHT have been spending a little more time honing by batting skills than publishing about Kilg.us changes.

On the fixes side, there were some issues around gathering players and adding them to Kilg.us. Those have been revised now, so any player on a 40-man roster should be showing up (and then some). If you don’t see someone that you want on your team, post a comment or let me know.

There is an outstanding issue with rookies, though. I think I’ve narrowed it to the stored procedure that saves player stats, but all rookie pitchers show a blank (no zeros, no stats, completely blank) stat line regardless of their performance. This is next on the list of things to fix.

Next up, when players were being collected from MLB.com, they were added to the official “Major League Baseball” owner’s teams if they didn’t currently exist on the appropriate roster. They weren’t, however, being removed from their previous team if they had been traded or were a free agent (rather than a rookie). This never seemed to come up last year, but there code I changed to fix the problem hasn’t been touched in nearly a year, so I’m not quite sure the root of the problem. Hopefully it is resolved going forward.

On that note, keep in mind that you can follow any MLB team’s stats by adding the owner “Major League Baseball” to your friends. That particular owner has all 30 teams with rosters that are automatically updated to have all the latest players.

Among new features for 2010, there are three more items I hope to get out by opening day (or very shortly thereafter):

1. Apply the new theme to the …Tracker blog
2. Implement Lucene for Add Player searches
3. Create a Season-to-date view of a team’s stats (cumulative for the season, not respective of adds/removals during the year; this was a request from a Strat-o-matic owner–hopefully others will find use in it)

Opening Day is barely a week away!

When clicking on the red “X” associated with a player on a team page, Kilg.us is supposed to prompt you to confirm that you wish to remove the player from your roster then, if you accept, actually perform the removal. Due to a duplication of the class name associated with the red “X”, a duplicate prompt was being generated. Upon acceptance of both prompts, the proper player removal was being performed. This duplicate prompt has been removed.

The duplicate prompt was being generated for the first player in the table from which you tried to remove someone. For example, if you tried to remove an outfielder from your team, you would be asked “Are you sure you wish to remove OUTFIELDER X from TEAMNAME’s roster?”. This is correct behavior. After clicking “OK”, though, you would be given the same prompt for your catcher (or whichever player was at the top of the table): “Are you sure you wish to remove CATCHER Y from TEAMNAME’s roster?”. This shouldn’t happen. If you had nonetheless accepted that prompt, Kilg.us would have properly removed the player you wanted to remove and not altered the other. If you declined that second prompt, though, the player you originally requested to have removed would not be removed.

This was happening because a class used to identify the red “X” button was duplicated onto the table-cell containing that button. I have updated the script so it is more specific in where it is looking for that class. This prevents the duplicate prompt from being generated and returns us to the proper flow to remove players.