Tuesday, March 4, 2014

Sorry...

I've totally ignored this blog for so very long. There is just so much work to be done, and to be quite honest... blogging takes too much time away from real work. So for those who may have stumbled across this or pay attention in any way; We're all working quite a lot on a plethora of projects. I am currently swamped, but excited with my full plate. Lot's of cool things down the road for NEXLAB.

The big news is that we have some serious plans to give the site another big upgrade. I probably shouldn't say too much at this point because it's a long way off still, but expect far greater user-end customization, better mobile support, cleaner graphics, improved navigation, improved "linking" (easier to share and bookmark products) and as always... more products.

Like I said though, there is a lot going on right now and beginning an overhaul is difficult. But the plotting and scheming is well underway. We continue to take great pride in creating a comprehensive forecasting and teaching tool for access to anyone, anywhere, anytime for free.

Saturday, February 16, 2013

State of NEXLAB Address

My fellow weather enthusiasts, met geeks, forecasters, nowcasters, storm chasers (hopefully the respectable lot), and various other sorts of people who just so happen to frequent http://weather.cod.edu...

Let's get down to business.

A couple weeks ago I launched all of our new analysis data, complete with redesigned pages, layout and new products to boot. This includes the birth of our very own RAP Mesoanalysis data and a vastly improved page for accessing a complete and customized set of raw, un-analyzed plotting maps (upper air + sfc + composite) all in a single PDF. On top of this I gave our navigational menus a cosmetic touch-up, bringing in some more appropriate color choices to help balance out the overall look of the site. As it so happened I managed to lump in some improvements to the models page such that now we can see the progress of the newest model runs of the NAM, GFS and UKMET. It's a minor improvement at this point, but the item that made it possible will mean some future improvements to our model comparison products... but more on that later.

Getting back to our analysis data; Barring a few touch-ups and some overlooked glitches to fix, I feel that area of the site is done for now. However mobile friendly versions of these pages do not exist yet. That is a priority at the moment. Our satellite and radar products don't seem to need much attention just yet. We have been having some problems with overlays coming up completely blank, which essentially ruins our meso-analysis feature for satellite. But to my knowledge, that problem has been fixed... hopefully for good. Our models page seems as good as it has ever been, yet as I said before there are some improvements I'm working on. The noticeable change that is to come will be some links that will make it VERY easy to jump to our model comparison products from any given model image. The rest of the improvements to the models page are really all behind-the-scenes work. Cleaning up code, moving some pages into more appropriate locations and... a few alterations to links... it has to happen.

With all of that said, I and the rest of the staff/faculty have been reflecting on the progress of the site since we launched "version 2.0" back in April of 2011. We feel like at this point the site is operating much closer to what we had originally been striving for. As such, we have been joking that the what we have now is "NEXLAB 2.5". That's nothing official, of course. However curiously enough, we have been batting around the phrase "NEXLAB 3.0". That concept has become less of an inside joke and more of an actual plan that we will follow through with... at some point.

To give a sense of what we expect for "3.0"; It all seems to be centered around 2 or 3 core elements. We want to re-work the design of the site such that it will favor larger and higher resolution images. Basically, we want to break out of the 800x600 box we've put ourselves in. We also are eagerly awaiting news from Bill Bellon and Tom Whittaker about HANIS (HTML5 Animation Software), the expected successor to Flanis and JSAni for creating animated loops of images; among other features. Once we have an idea of the capabilities of HANIS, that might actually be the point where we start moving forward with work for "3.0". The only other goal to achieve with "3.0" would be a fully tailored NEXLAB Mobile site.

But before we make that jump; I wonder if any of our users have been scratching their heads wondering when in the world we are going to give any attention to our text products. We've been avoiding working on them for a pretty simple reason; They work, they are exactly what everyone is used to and other areas of the site have needed more attention. Nevertheless it's the only untouched corner of our data site and it's time will come. However it is February and this time of year starts to become a whirlwind (quite literally) for our program. Our 2013 storm chases start at the end of April and there are many things to be taken care of before then. Then April through late-June we will all be occupied with running the trips. With that in mind we have opted to start work on text in July. Again, breaking ground on Text Products in July 2013. No ETA on launch.

Even before and after Text is started/finished, we have a lot of other plans to move on before "3.0" (again) becomes a realistic project. A project for our Models page that has slowly but steadily been progressing is an effort to switch our images from being generated by WXP to GEMPAK. We also expect to improve the coverage of our model "sectors" which will include at least 1, perhaps a couple western CONUS sectors. An exciting new idea I've had recently is to setup some user-defined "Global Settings" for the entire data site. For example; You may have noticed that we are making wide use of image "rollovers". In order to make these rollovers function fluidly on the site, we pre-load all of the images on page-load rather than having each image download "on-rollover". This means that a good chunk of data is being downloaded all at once. This (as we are aware of ourselves) is not always a desirable thing with a mobile connection that is touch-n-go out in the plains. A global setting I'd like to implement is a way to disable pre-loaded rollovers site-wide. Other settings along these lines will likely be included as well. I'd also like to make it just a "smidge" easier to "bookmark" specific sections of our data site. It's been brought to my attention that that would be desired among some users.

Continuing a laundry list of plans and schemes; We've also been talking about adding a handful of Terminal Doppler Radar sites to our single site radar page. On a related note; we've been expecting a pair of brand new servers to replace our existing ones (hopefully before April, although, no ETA on that). With these powerful new servers and increased overhead, we've been hoping to push them as much as we can. This would include upgrading a few more NEXRAD sites to feature full looping for all products.

I believe that's just about all we have planned at the moment. Although I must admit, now that I have it all written down it does seem like an ambitious and daunting workload. Nevertheless, these are our plans and now you know as much as I do. As always, feel free to comment here, our feedback page, or tweet us @CODMeteorology or @CoDWXData.

Wednesday, December 5, 2012

Breaking Silence

So much for that promise of weekly updates, eh?

There has been a reason for that. Most of my attention has (had) shifted to a separate project related to course material for our Forecasting classes. I had to take some time to work on that project while students were (are) using it.

That project now seems to be more or less out of the way. As such I now can devote my attention back to some other projects that have been in the works.

The grand "re-working" of our analysis products is still in progress. There are a few hurdles to jump through with that. I'm not out to make any promises about when that will all be done... beyond saying that our hope is to have it all done by the end of the year... but who knows, that might be a bit of an under-estimate. We may take until mid-January to finish that stuff up.

In the mean time I plan on working on a couple other "minor" projects. I will (finally) be creating a way for mobile users to switch between the mobile catered "JSAni" satellite pages, to the desktop catered "Flanis" pages. That is mainly a fix for strict Android users running a version of Gingerbread that may not properly load overlays.

Speaking of our satellite data; Since GOES 14 has shifted it's orbit into the position of GOES 13, and has now permanently taken over the role as GOES East, a strange thing has occurred. I started to notice a familiar pattern return to the diurnal change in filesize for our satellite image. Short story version of that; I'm re-introducing "night-time removal" of images to our visible satellite loops. It's not perfect, however. It only seems to work with sectors that are covered almost entirely by GOES East (a.k.a. GOES 14). Many of our sectors along the WRN CONUS will not remove night-time images from loops. But I feel something is better than nothing in this case.

I also have a whole host of random (but COOL!) ideas to inject into our model data. Apart from purely cleaning up code (!), I'm working on something that should help our model comparison products work better. Without going into too much detail, I'm hoping to create a more seamless experience with using a comparison product, and then looking at model products for a given model/run/sector/hour. In addition, I've already started toying with some "run status" type data. Meaning you, the user, should have more of a cue when new model runs are coming in, and just how "done" they are. Mike and Victor have also been plotting and scheming some improvements to the images themselves. Perhaps even some new products. Last but not least... Improved sector coverage! Hopefully we will delight some west coast users.

That's all for now. Back to work. Please pardon any typos or grammatical errors in this post.

/throws-proof-reading-out-the-window

Monday, October 22, 2012

Analysis Work...

I've been quiet lately. But this doesn't mean I (we) have been lazy. We are still in the throes of major reconstruction and restructuring of our analysis data. In some cases we are merely making some cosmetic changes to the pages that display the data. But in most cases we have been working to generate completely new and improved images. In addition we are attempting to add as many functional and navigational niceties as possible. All the while I'm attempting to keep the back end of our site as tidy as possible. This means URLs to specific products will change because I'm attempting to better organize our directory structure.

With the simple changes and updates; I've pushed those out as quickly as possible. However the biggest challenge has been rethinking organization and navigation. I've been quite pleased with navigational concepts I've brought to other products on our site. But bringing those changes to our analysis products in a successful and intuitive manner has proved itself to be a tad complicated.

Nevertheless in the past couple weeks I've had a couple break-throughs that have brought us much closer to completion. The best place to show off these new features is with our new Upper Air data page. The following *should be* the final product and new location for this page.


What's new?
  • Raw data; both in PDF and GIF format, as well as contoured images are all accessible from this page
  • All images are now generated by GEMPAK rather than WXP
  • New menu system with improved functionality as seen with other product pages across our site
  • BOTH Time AND Height rollover features are available for Raw Data and Jet Stream Analysis. Temporal rollover will remember what Height you are on, Height rollover will remember what Time you are on.

A feature I'm still debating whether or not to add to this page is the ability to load Sounding(RAOB) data by clicking sites on the Raw Data plots. This is a feature we used to have long ago and *might* be possible to implement again. Obviously this data is now displayed in a much more complicated manner, so it's not entirely obvious if reinstating such a feature is possible.

Moving on; Surface data is next in line to be worked on. I haven't really approached it yet, but off the top of my head I imagine it to be an easier mountain to scale than this new Upper Air page.

With the flurry of improvements I was able to fire off this summer, I feel this current task with our Analysis products has moved slowly. If you have been anticipating these improvements and feel that they've been delayed in anyway... I assure you I am just as anxious to deliver, but we want to get everything right first.

Until next time...

Tuesday, September 25, 2012

GOES Down! GOES Down!

GOES-13, also known as GOES-East went into a state of hardware failure a couple days ago. GOES-14 which has been on standby has come online and taken it's place as GOES-East.

A couple weeks ago I rolled out the replacement satellite/radar pages, touting that visible satellite loops would now remove a healthy chunk of "night-time" images. Following that I realized that a handful of west coast regions for 1km, 2km and 6km resolution were not properly removing night-time images.

This removal of night-time images was accomplished by taking advantage of the fact that when our images get generated, the file size directly correlates to the average brightness of the image. This means that with visible satellite images, there should be a very natural drop-off in file size for night-time images. Within the code that grabs the most recent set of images, I simply added a file size check for visible images and filtered out the images that fell below a certain file size. This effectively removed night-time images from mix without using any information on sunrise, sunset, time of year, or latitude. I was quite happy with that approach. It meant I didn't have to over think the scripting or tailor the code to each region. Basically; less work, more reward.

Getting back to those west coast regions not removing night-time images; It turns out that GOES-West (GOES-15) changes how it operates when taking images of the dark side of the Earth. It attempts to bring in as much light as possible, which gradually increases the brightness of an otherwise dark image. This results in the minimum file size occurring right around sunrise and sunset, with a relative maximum in file size overnight. Here are some highly sophisticated graphs to help further describe this situation.

Images from GOES-13 would have created a File Size vs. Time graph similar to the one above.

GOES-15 on the other hand creates a graph similar to the one below.

Because of this, I can't set a lower threshold for file size to remove night-time images because it would also be removing valid day-time images.

Initially, I was willing to just let it be since the overwhelming majority of the sectors we provide are covered by GOES-East. But now GOES-East is GOES-14 and wouldn't you know it... GOES-14 operates the same way GOES-15 does. Both satellites now variably adjust the amount of light coming in to help grab as much valid data as possible from the visible channel. But this means that when our images are generated... well, hopefully you get the idea by now.

So... for what it's worth, I got 2 weeks out of a pretty neat feature for our visible satellite loops. But at the moment, I'm not interested in getting excessively cute with the script that grabs the most recent set of images for a loop and make it remove night-time images. I will also not write it in some way that involves me editing the code constantly to adjust for variances in time of year and latitude. Waste of time.

When other, larger tasks are taken care of, I may return to working on the scripts for these pages and try to get clever. For now, "Sorry folks. Can't do it."

Monday, September 24, 2012

Short post...

I'm going to attempt to update this blog at least on a weekly basis. Tuesdays will likely be the day. So since today is not tuesday, this will only be a short entry.

I've held off on the idea of allowing mobile users to switch back to FLANIS based satellite loops. Yes, for those who have devices that aren't properly loading the overlays, JSAni is not very pleasant to deal with. But neither is making the necessary changes to the pages to allow a user-friendly re-route to the FLANIS pages and back again. This would only be a temporary fix anyways. Instead I've been in contact with Bill Bellon about the issue, and while he is busy, the issue is on his to-do list. Until either a more definitive dead-end or conclusive fix is reached, I will leave the pages as is. I apologize for the inconvenience, however there is too much else at the moment that needs attention. I never like leaving a product semi-broken. But we have limited time and a lot that needs to be worked on.

Monday, September 17, 2012

Dual Pol Sites and Tiny Fixes

This should be a week to get a lot done... a lot of little things.

The current mode of operation in my head is to take care of a few scattered issues with a few of the projects that have been worked on recently. Notably this means issues with mobile satellite products, a few numerical model quirks, and a few random other items. These little tasks need to be knocked out so that all of our attention can be put toward Surface and Upper Air products.

One of the bigger, "little things" I've taken care of today is allowing several sites to display Dual Pol data that have recently been upgraded. These dual pol upgrades tend to happen for only 2 - 4 sites at a time, often take a week or two go into effect, and then roughly another week before our scripts start properly creating the images. Because of this I have the php scripts on the web end restrict access to these products until we know the upgrade has taken effect. But it has been several weeks since I've been able to peruse our sites to find out what sites are now ready with dual pol data. Turns out many have come online recently, making me look like a slouch.

Here is a complete list of recently added Dual Pol sites; KRGX, KLRX, KNKX, KDAX, KMAX, KGGW, KHDX, KEPZ, KABR, KDLH, KMPX, KSRX, KLZK, KLTX, KGYX, KCBW, KTYX, KCCX, KILN, KFTG, and KDMX. A total of 21 sites.

If you ever catch a site that has recently received the Dual-Pol upgrade but we aren't allowing access to the products, let us know using our Feedback page. Never hurts to give us a heads up. I often have a dozen other things on my mind that might be distracting me from making that small change.

Other tiny fixes; I'm going to finally work up a way for mobile users to switch between flanis/janis loops. At the very least for ones that use overlays. Which basically means satellite and radar products (excluding NEXRAD data). I'll be fixing some broken links in our Links page.

Then, as mentioned, we start major work on analysis products.

A note about our staff: It may not be readily apparent, but our staff consists two full-time professors, two official part-time staffers, a (contracted? fuzzy on the details) part-time remote staff member, and a couple IT consultants and administrators. Of this small staff, only myself has been tasked with the primary priority of designing and maintaining the website. The rest of the staff have other priorities that at times take precedence over product maintenance and development. Faculty needs to teach. Only one of our system administrators is technically paid, the other graciously offers his time and help (sparingly, of course) for free. Needless to say, we all have our plates very full. But we all remain considerably excited about this website. Since our site offers access to vast amounts of meteorological data for free, where others charge the user, our work feels very rewarding. We do this because most of us are forecasters at heart and we enjoy having fun tools at our disposal. We also firmly believe that giving students access to this data is a critical part of their education.

Long story short, we all scramble like crazy to do work quickly and do it well to maintain a unique, multi-faceted, undergraduate meteorology program along with a one-of-a-kind meteorological data and academic website. It's a tall order. But we do it every day.

With that said... sometimes... an occasional burn-out is to be expected. ;)

That's why I attempt to keep communication with users open. That way we give you, for a moment, the opportunity to be part of our staff. Your feedback is always a welcome contribution.