Kilg.us – Fantasy Stat Tracker …Tracker

A blog about the development of Kilg.us – The Fantasy Baseball Stat Tracker

Archive for the ‘Fixes’ Category

Daily Boxscore Email Update

Tuesday, May 26th, 2015

I apologize for the delay on getting this issue addressed.

As noted previously, daily boxscore emails stopped going out earlier this month. I’ve finally had time to dig into the issue.

First, if you are subscribed for daily emails, you should have received a boxscore email around 9:30am Mountain time today. If you didn’t receive such an email (and confirmed it isn’t in a SPAM folder), please comment here or email me.

To publish daily boxscore emails, Kilg.us goes through a few steps. First, it makes sure all the latest stats have been collected. It then starts a queue for composing the emails. Each team is checked for subscribers and email content is saved to a database table. Once complete, the queue is closed (the stages of opening and closing this queue are recorded). The next step is to systematically go through that DB table and send the emails. As each email is sent, it is deleted from the table. The emails are sent in batches, 50 at a time.

A few observations:

  • The database shows that the composition queue was completed each day of the outage.
  • The database is stuffed full of all the emails that haven’t gone out in the last few weeks.
  • SendGrid (the 3rd party that actually sends the emails at Kilg.us’ request) shows that a full batch (~550) emails went out on May 1. May 2 dropped to 50, followed by 200, 50, 100, and then to 0 in subsequent days. These increments of 50 suggest something is going wrong with our send batches.
  • I used the exact same scripts to successfully send the emails today (manually executed, rather than scheduled).

All told, I haven’t identified a specific cause of the outage. The send process can take a while so it is possible we ran into a timeout along the way. The server has been updated, scrubbed, and rebooted so it should be in tip-top shape for tomorrow. Fingers crossed that emails will go out as scheduled. If not, I at least know I can manually trigger the process daily until I have time to dig back in deeper.

Email Boxscore Issues

Monday, May 18th, 2015

Numerous people have notified me of an issue with daily email boxscores. It seems that since early May, many have not been delivered properly. This is on my radar. Unfortunately, I have not had a chance to dive in to resolve it. Trust that as soon as I find time, I will get it fixed and post an update here about what happened.

Log-in Issues (User Sessions)

Sunday, May 11th, 2014

A number of people have reported issues logging in to Kilg.us today. After a little exploration, I found that the database table that records user session data was corrupted. After repairing the table, Kilg.us seems to be back to normal and properly logging users in.

This is the first time we’ve encountered this issue, so I’m not going to dig further into the cause of the corruption. If it happens again, though, I’ll dig a bit deeper.

For now, you should be able to log in normally. If you have any problems, please let me know.

Server Issues this Morning

Sunday, August 25th, 2013

This morning a lot of people awoke to find Kilg.us was kicking out errors and was showing all players had been lost from all team. Frankly, not a nice way to start the morning!

Luckily, the issue was resolved with a quick reboot of the server. It seems to have been an issue with one of the database tables, but I’ll be performing some additional diagnostics to try to determine what caused all the errors this morning. For now things should be back up and running.

Sorry for the inconvenience!

Spring Training Stats

Friday, April 5th, 2013

I’ve been getting a flood of emails about spring training statistics showing up for players that haven’t played in the big leagues yet this year.

Kilg.us collects all of its stats for an XML feed that MLB publishes. Every time you request the stats for your team(s), Kilg.us checks to see what the latest stats are and loads them into the system.

Unfortunately, it seems that MLB leaves springs stats listed for all players that appear in the spring but are then sent to the minors (or DL). Until they appear in a big league game, MLB will continue to show springs stats in their feed.

This is just how it works.

If you have a player on your team that is showing spring stats, please move that player to your bench. The player’s individual stats will still show from spring, but this will remove the player’s stats from your team totals.

Stat Updates

Thursday, July 19th, 2012

Over the last few weeks I’ve received a number of messages that some players that haven’t played in the big leagues are showing season stats for 2012. All of that data comes from MLB, so if they show season stats in their data feeds, Kilg.us shows it as well. Usually these are spring training stats that just haven’t been updated.

I have made an effort to go through and delete any season stats for players that have been reported. If you see this behavior for any additional players, please let me know via email or in comments here.

A quick apology…

Wednesday, April 18th, 2012

As I mentioned yesterday, the old Kilg.us system for generating and sending email boxscores would run up against usage rates on a daily basis, leaving some messages unsent. With the move to SendGrid, our cap is raised, which is a good thing. I removed the limits from Kilg.us. What I failed to identify was that all of those unsent email boxscores were still sitting in queue (almost a full year’s worth…). When I removed the limit in Kilg.us and told it to send, boy did it ever!

I logged into SendGrid this morning expecting to see ~470 boxscores sent this morning. Instead the count was nearly 4000! I frantically shut down the process to try to reduce the irritation any more than had already been caused. But a lot of messages went out to people that may not have been expecting them. For that, I am very sorry!

For some reason, these email boxscores have been the single biggest headache with Kilg.us over the years.

Anyway, I’m back in the code tweaking things up in hopes of avoiding another disaster tomorrow.

For all those that have emailed me over the last year about a missed boxscore email, I hope it showed up today! /badjoke

Seriously, though, sorry about the extra messages. I’ll try to make sure it doesn’t happen again.


Update

I’ve made three changes to try to avoid this happening again:

1) I’ve cleared the email queue so nothing old even exists now.

2) I’ve capped the number of email boxscores Kilg.us will send tomorrow at 500. This is slightly higher than the number I think should be sent, so hopefully everyone gets their proper messages. If something goes wrong, though, the impact will be minimal.

3) I’ve updated the script that selects the emails to send to have it double-check that it is only pulling email boxscores generated in the current day.

Hopefully these three actions will prevent this from happening tomorrow. Nonetheless, I’ll be keeping a very close eye on things in the morning.

Internet Explorer Sign-up or Log-in Bug

Saturday, May 7th, 2011

Over the last couple weeks, I have received numerous emails from people having trouble signing-in to their accounts. With a little digging I was able to find a consistent theme: they were all using Internet Explorer 8 or 9.

Specifically what was happening was that a person would type their existing username into the sign-in form and would then be prompted to create an account. As everyone is probably aware, Kilg.us uses the same form for logging in and signing up. If you enter an existing username, it asks you to log-in. If you enter a username that doesn’t exist, it asks you to sign-up.

To manage this functionality, Kilg.us generates an AJAX request after you enter your username then move focus to the password field. This is creates a “change” event on the username field. When that event happens, Kilg.us asks the database whether the username you entered exists. The database returns a response in the form of XML. The XML has one message: the username is valid (it exists in the database) or it is invalid (it doesn’t exist in the database).

Once the XML response is returned, some JavaScript parses it to read whether it says “valid” or “invalid”. If the value is “invalid” the form asks you to create an account. The JavaScript to read this follows standard practices to navigate the document object model (DOM) and find the value in question. This works in Firefox, Chrome, Safari, Konqueror, Mobile Safari, Opera, Android Browser, Blackberry Browser, etc., etc. Apparently, how Internet Explorer handles the XML is different. It doesn’t create all the parameters that the standard approach would.

This shouldn’t come as a surprise to anyone.

What I ultimately found is that Internet Explorer will parse the value out and save it in a “text” parameter for the XML object. Rather than navigating the DOM in a standard fashion, I’m now taking a shortcut and checking if that “text” parameter exists on the XML object. If it does, I read it in as the returned value. If it doesn’t (obviously, all the other browsers to generate this rogue parameter), I continue to navigate the tree and get the value in the proper manner. This only works because the only value of the XML is the “valid” or “invalid” message.

The end result: the log-in/sign-up form logic should now be working again in IE. And still working in every other browser. If you find that not to be the case, please let me know (comment, email, Facebook, Tweet).

Well that’s embarrassing

Sunday, April 24th, 2011

I got a couple emails yesterday that some daily boxscore email stats were incorrect. Looking into the matter, it was immediately obvious what had happen. As I worked on the transition of Kilg.us to its new servers, I had accidentally left a database connection pointing to the new dev database, instead of the old, production database. As such, the stats weren’t up-to-date when the daily emails were generated. I simply pointed the connection back to the correct database and figured all was well.

This morning there have again been reports of incorrect stats. I double-checked that the database connection was pointing to the correct IP address–it was. Then I remembered one of the many reasons I’m moving Kilg.us: the hosting company uses a non-standard port for database connections! Why? Who the hell knows. But I had forgotten to explicitly change back to their port. I’ve done that now. I’ve tested the connection. All seems to be well.

On the migration topic, Kilg.us is up and fully functional on its new servers. I’m testing manually with day-to-day usage to try to identify any aspects or configurations I may have missed. Right now, my simple tests are showing that Kilg.us runs between 2.2 and 5 times as fast on the new servers (adjusting as best I can for load considerations). I’m working on some minor changes that will decrease the bandwidth Kilg.us requires because the new servers are billed by data transfer. I’ll give at least 24 hours notice before I make the migration. Depending on how quickly the domain propagates, Kilg.us may be down for some of you for up to 24 hours during the migration. More details to come.

Cross your fingers!

Friday, April 15th, 2011

Tomorrow morning Kilg.us will use a entirely new process for generating and sending daily boxscore emails! So let’s all cross our fingers and hope it works…

I have spent all of this week re-designing and re-building how emails are generated, stored, and sent. Suffice to say, the new system is significantly more complex than the old one. When I’ve had a chance to breath and am not so tired, I’ll go into more detail on how it works.

My expectations of the new system are that it will:

  • Scale to accommodate as many users/emails as desired (the old system failed in the 200-300 range)
  • Allow of future portability if Kilg.us needs to change hosting
  • Implement all components to identify HTML vs. Text email so boxscores render properly on all clients
  • Allow for side-by-side HTML and Text versions (text version is forthcoming)
  • Provide redundancy of data, so if the process fails it can be re-started without losing a day’s data
  • Reduce likelihood of Kilg.us mail servers being identified as SPAM and black-listed
  • Lower computation overhead

It sounds almost too good to be true! Tomorrow’s push will be the first full run trying to send out to everyone’s unique addresses. All the testing so far has been delivering to a kilg.us address (or small tests to outside domains). I’ll be keeping an eye on things in the morning to try to ensure everything goes smoothly. If you do not receive your normal boxscore in an acceptably legible format, please post comments here with details (DETAILS–without them I can’t do anything–what email client, what team name, what was wrong, screenshots, any little detail).