home
 
 

 
Some scoring thoughts

Actually, limiting the distance is a good idea... maybe even less than 1500 miles away, especially if it's in an east-west direction, because of the different system thing and the fact that you can only draw one polygon.

If this was a contest really intent on predicting tornado locations, there would also be negative points for tornadoes that occur when no outlook was made at all, but I know this is no such contest, and many people don't dedicate their lives to it either!  haha.  Take this week for example - tornadoes are occurring across the southern Plains and southeast today and tomorrow, but I'm at a conference this week and have had no time to look at this recently.  I could do an outlook for tomorrow, but I have found it more beneficial point-wise to do the outlooks further in advance due to the lead-time factor.

One thing I did think about was single tornadoes vs. outbreaks.  If you draw a small outlook polygon and a single tornado falls in the area - for example, my February 1 polygon if it was moved one state east, or my May 7 outlook from last year, you get 100% of the tornadoes in the polygon and a very good score with a small area.  Fine, that makes sense.  But lets say you expect a large number of tornadoes, like an outbreak.  If it's something like the Super Outbreak of April 3-4, 1974, you would have had to draw a large polygon from the Gulf Coast to Michigan to capture every single tornado, and you would get a lot less points because your polygon was so big.  But it had to be big.  Even with not so large an outbreak, take my most recent polygon from April 4.  I expected a significant event, and I got the main area of tornadoes pretty accurately, but in order to have captured all of them, I would have had to draw a fairly large polygon from Mississippi/Louisiana north to central Ohio and east to North Carolina, a polygon that would have covered parts of at least 11 states and likely a few more unless I knew exactly where to cut the line to the west to avoid Indiana and Arkansas.  I have to wonder if the points lost for increase in area and perimeter would have been worth the points gained for percentage of events and dst missed.  Probably, but even so, it's really tough to draw such a large area knowing you are starting at at a lower value.

On a similar note, I'm guessing it's probably a lot of work to work tornado strength in the scoring.  Many of the Enhanced Fujita ratings aren't determined until NWS survey teams go out into the field to do the surveying, I know, and are often not added to the local storm reports themselves.  But there could be one isolated "weak" EF0 tornado within a small outlook polygon that could give a very high score, while in an outbreak, you could have drawn a polygon in an area that captures every single EF2, EF3, EF4, and EF5 tornado, with some weaker ones missing the polygon (for example), that would score lower even though you were able to pinpoint the strongest tornadoes during a large outbreak that may have been difficult to cover the entire extent.

That said, I worked for the National Weather Service briefly in the past, and from experience I know that though most are, not 100% of tornado reports that are sent out are 100% reliable.  There are also multiple reports sent out for the same tornadoes, especially the longer-track ones, near larger population centers, or where several chasers have converged on the same tornado.  There are tornadoes that go by with no reports.  There are funnel clouds that never touch the ground, or maybe even barely come out of a cloud, that are called in by the public to NWS offices as tornadoes, and thus are sent out as reports (those are not too common because of the training of storm spotters, however).  Of course, most of those are things that wouldn't affect the scores too much, and are things I try to take into account when I do outlooks (for example, on my most recent outlook, I intentionally cut out most of the entire southern Louisiana marshland/Mississippi delta south of the New Orleans area because of the scarcity of population there).

Those are some of the thoughts I've had.  When I remember more I'll include them.


↑ UP Powered by Quick Disclosure Lite
© 2010~2021 SCS-INC.US
UP ↑