Kilg.us – Fantasy Stat Tracker …Tracker

A blog about the development of Kilg.us – The Fantasy Baseball Stat Tracker

Posts Tagged ‘email’

Daily Boxscore Email Update

Tuesday, May 26th, 2015

I apologize for the delay on getting this issue addressed.

As noted previously, daily boxscore emails stopped going out earlier this month. I’ve finally had time to dig into the issue.

First, if you are subscribed for daily emails, you should have received a boxscore email around 9:30am Mountain time today. If you didn’t receive such an email (and confirmed it isn’t in a SPAM folder), please comment here or email me.

To publish daily boxscore emails, Kilg.us goes through a few steps. First, it makes sure all the latest stats have been collected. It then starts a queue for composing the emails. Each team is checked for subscribers and email content is saved to a database table. Once complete, the queue is closed (the stages of opening and closing this queue are recorded). The next step is to systematically go through that DB table and send the emails. As each email is sent, it is deleted from the table. The emails are sent in batches, 50 at a time.

A few observations:

  • The database shows that the composition queue was completed each day of the outage.
  • The database is stuffed full of all the emails that haven’t gone out in the last few weeks.
  • SendGrid (the 3rd party that actually sends the emails at Kilg.us’ request) shows that a full batch (~550) emails went out on May 1. May 2 dropped to 50, followed by 200, 50, 100, and then to 0 in subsequent days. These increments of 50 suggest something is going wrong with our send batches.
  • I used the exact same scripts to successfully send the emails today (manually executed, rather than scheduled).

All told, I haven’t identified a specific cause of the outage. The send process can take a while so it is possible we ran into a timeout along the way. The server has been updated, scrubbed, and rebooted so it should be in tip-top shape for tomorrow. Fingers crossed that emails will go out as scheduled. If not, I at least know I can manually trigger the process daily until I have time to dig back in deeper.

Email Boxscore Issues

Monday, May 18th, 2015

Numerous people have notified me of an issue with daily email boxscores. It seems that since early May, many have not been delivered properly. This is on my radar. Unfortunately, I have not had a chance to dive in to resolve it. Trust that as soon as I find time, I will get it fixed and post an update here about what happened.

2015 Season

Monday, April 6th, 2015

The new season is upon us! I’ve had a number of people checking in to see if Kilg.us will be functioning again this year. It looks like our data feeds are still working, so at this point I don’t have plans to shut Kilg.us down.

Daily boxscore emails should resume tomorrow. If you would like to change your settings for which teams you receive, please access your account settings.

As a quick reminder: Kilg.us is in maintenance mode. If something goes down, I’ll try to fix it in a timely manner. At this point I don’t actively use Kilg.us, though, so it isn’t a high priority. Last year I posted some things to keep in mind about the operations of Kilg.us. Please take a look.

Good luck with your 2015 season!

Email Boxscores and 2014 Update

Wednesday, April 9th, 2014

As of tomorrow morning (4/10), Kilg.us should resume sending daily boxscore emails for teams that have requested them.

In order to avoid the flood of unsubscribe and SPAM reports with the first emailings of the new season, I discontinued daily boxscore emails for all teams that haven’t been updated since the start of 2014. If you think you should be seeing a daily email but don’t, check your settings and confirm the team has been signed up. I don’t believe any teams that have been updated since the start of the year have been affected, but it’s always possible.

Obviously, I’ve been a little slow getting things up and running for the new season. I’ll blame a perfect storm of vacations, the constant distraction/entertainment of my first sons’ birth, and starting up a new business (Super Humane. If you have a company that needs some help making the world better through technology—the mobile web in particular—I’d love to chat!). On top of it all, I’m not actually participating in a fantasy league this year for the first time in recent memory. Nonetheless, my intention is to continue operating Kilg.us for the 2014 season.

A few reminders of what the continued operation of Kilg.us involves:

  • We use a free but limited stats feed in order to have up-to-the-second stats. Sometimes it has issues (like Spring stats trailing into the season…), but 99% of the time it works.
  • The stats already in Kilg.us are all we have. We can’t add games played/pitched, WAR, or anything else that isn’t there now.
  • Kilg.us was built to display real-time stats. We added “season-to-date” later, then tacked on the daily boxscores to help people who couldn’t log in every night. Kilg.us was never intended to help slice and dice stats.
  • When things go wrong (it seems to happen once or twice a year), I’ll try to fix it as quick as I can. It usually takes a day or two.
  • If you have a problem, email me. I don’t regularly use Kilg.us anymore, so I need people to point out issues when they arise. I’ll respond as soon as I can.
  • You’re welcome to manage your team, all your teams, all the teams in your league, or whatever you like. We’ve got over 6000 teams being managed by 3400 users right now.
  • Kilg.us is free! Which is great! Especially when generous folk are willing to donate to offset the hosting costs associated with it!

Best of luck for your 2014 season!

That’s a wrap!

Monday, October 29th, 2012

The 2012 baseball season has come to a close. As such, Kilg.us will no longer be sending out daily box score emails.

It’s been a tumultuous year for Kilg.us. In particular, we experienced a lot of accuracy issues with our data source (free MLB.com published XML feeds) and I heard more than ever about the lack of this stat or that stat.

I stopped using Kilg.us for my personal needs a couple years ago (I originally built it as a personal tool). Going into 2013, I’m on the fence as to whether to continue maintaining Kilg.us. Obviously there are costs associated with keeping the servers up and running (thanks to all of you that have donated to help cover hosting costs!), but even more important is the time it takes to field email questions and comments, fix things when they go wrong, and monitor the performance of servers, databases, and email services. We’ll see how that goes in a few months when the excitement of a new season ramps up.

In the meantime, keep your chins up. Baseball is only a few cold months away!

A quick apology…

Wednesday, April 18th, 2012

As I mentioned yesterday, the old Kilg.us system for generating and sending email boxscores would run up against usage rates on a daily basis, leaving some messages unsent. With the move to SendGrid, our cap is raised, which is a good thing. I removed the limits from Kilg.us. What I failed to identify was that all of those unsent email boxscores were still sitting in queue (almost a full year’s worth…). When I removed the limit in Kilg.us and told it to send, boy did it ever!

I logged into SendGrid this morning expecting to see ~470 boxscores sent this morning. Instead the count was nearly 4000! I frantically shut down the process to try to reduce the irritation any more than had already been caused. But a lot of messages went out to people that may not have been expecting them. For that, I am very sorry!

For some reason, these email boxscores have been the single biggest headache with Kilg.us over the years.

Anyway, I’m back in the code tweaking things up in hopes of avoiding another disaster tomorrow.

For all those that have emailed me over the last year about a missed boxscore email, I hope it showed up today! /badjoke

Seriously, though, sorry about the extra messages. I’ll try to make sure it doesn’t happen again.


Update

I’ve made three changes to try to avoid this happening again:

1) I’ve cleared the email queue so nothing old even exists now.

2) I’ve capped the number of email boxscores Kilg.us will send tomorrow at 500. This is slightly higher than the number I think should be sent, so hopefully everyone gets their proper messages. If something goes wrong, though, the impact will be minimal.

3) I’ve updated the script that selects the emails to send to have it double-check that it is only pulling email boxscores generated in the current day.

Hopefully these three actions will prevent this from happening tomorrow. Nonetheless, I’ll be keeping a very close eye on things in the morning.

SendGrid Roll-out Successful

Tuesday, April 17th, 2012

Last night I upgraded the email sending mechanism in Kilg.us to use SendGrid. This morning all email boxscores look to have gone out as planned.

As a number people have noted, “as planned” may be less that what we all would like. One of the things SendGrid will help me do is see how many emails are being generated, sent, delivered, read, etc. This will allow me to better manage how daily boxscore emails work.

The first issue to address are those users that have signed up for boxscore emails but haven’t received them. The old email provider had a much lower limit on the number of messages that could be sent per batch and per day. This lead to some less than ideal compromises in designing the system. Basically, rather than just flooding the email server with all the boxscore emails, I had to throttle the process, building a queuing system and sending out small batches of emails. As the system neared its daily limits, some people’s emails would go un-sent.

In addition to the migration to SendGrid last night, this morning I have updated the queuing and sending system. It now queues everything the same, but tomorrow will send one large batch of messages (using an anti-flood function to ensure the email server doesn’t get overwhelmed). This SHOULD ensure that everyone receives every email they have requested! We’ll check back in tomorrow morning to see if it worked.

As a tack on, I switched this WordPress installation to also use SendGrid which should make it less likely that I miss comments and questions people post on here (sorry if I missed your question or comment!).

Moving to SendGrid

Monday, April 16th, 2012

In the next day or so Kilg.us will be moving its email handling to SendGrid. This should give me better transparency into what messages are being send, received, and bounced back. Right now things are a bit of a manual process which makes it difficult to diagnose one-off errors (particularly with daily boxscore emails).

If all goes well, I’ll probably make the change tonight. Theoretically, there should be no change for anyone but me. Let’s hope it works out that way!

First 2012 Boxscore Emails

Friday, April 6th, 2012

From this end, it looks like all daily boxscore emails were sent out properly. We had a handful of bounce-backs (expected after 6 months of inactivity) but no fatal errors.

If you received an email boxscore that you don’t think you should have, or didn’t receive one that you think you should have, please double-check your account settings and if you see an error, please let me know.

2012 Plans

Thursday, April 5th, 2012

I’m a little late getting this notice out, but Kilg.us will be up and running for the 2012 season. This evening I intend to enable daily boxscore emails so those should start showing up in inboxes tomorrow morning. Be sure to update your subscriptions (Manage My Settings) so you receive emails for the teams that matter to you!

Best of luck with your 2012 season!