Unidata's Vision for Providing Comprehensive and End-to-end Data Services
This paper presents Unidata's vision for providing comprehensive, well-integrated, and end-to-end data services for the geosciences. These include an array of functions for collecting, finding, and accessing data; data management tools for generating, cataloging, and exchanging metadata; and submitting or publishing, sharing, analyzing, visualizing, and integrating data. When this vision is realized, users —no matter where they are or how they are connected to the Internet—will be able to find and access a plethora of geosciences data and use Unidata-provided tools and services both productively and creatively in their research and education. What that vision means for the Unidata community is elucidated by drawing a simple analogy. Most of users are familiar with Amazon and eBay e-commerce sites and content sharing sites like YouTube and Flickr. On the eBay marketplace, people can sell practically anything at any time and buyers can share their experience of purchasing a product or the reputation of a seller. Likewise, at Amazon, thousands of merchants sell their goods and millions of customers not only buy those goods, but provide a review or opinion of the products they buy and share their experiences as purchasers. Similarly, YouTube and Flickr are sites tailored to video- and photo-sharing, respectively, where users can upload their own content and share it with millions of other users, including family and friends. What all these sites, together with social-networking applications like MySpace and Facebook, have enabled is a sense of a virtual community in which users can search and browse products or content, comment and rate those products from anywhere, at any time, and via any Internet- enabled device like an iPhone, laptop, or a desktop computer. In essence, these enterprises have fundamentally altered people's buying modes and behavior toward purchases. Unidata believes that similar approaches, appropriately tailored to meet the needs of the scientific community, can be adopted to provide and share geosciences data and actively collaborate in the future. For example, future case-study data access systems, in addition to providing datasets and tools, will provide services that allow users to provide commentaries on a weather event, say a hurricane, as well as provide feedback on the quality, usefulness, and interpretation of the datasets through integrated blogs, forums, and Wikis, along with uploading and sharing products they derive, ancillary materials that users might have gathered (such as photos and videos from the storm), and publications and curricular materials they develop, all through a single data portal. In essence, such case study collections will be "living" or dynamic, allowing users to be also contributors as they add value to and grow existing case study collections.
High Availability Applications for NOMADS at the NOAA Web Operations Center Aimed at Providing Reliable Real Time Access to Operational Model Data
The NOAA Operational Modeling Archive Distribution System (NOMADS) is now delivering high availability
services as part of NOAA's official real time data dissemination at its Web Operations Center (WOC). The
WOC is a web service used by all organizational units in NOAA and acts as a data repository where public
information can be posted to a secure and scalable content server. A goal is to foster collaborations among
the research and education communities, value added retailers, and public access for science and
development efforts aimed at advancing modeling and GEO-related tasks. The services used to access the
operational model data output are the Open-source Project for a Network Data Access Protocol (OPeNDAP),
implemented with the Grid Analysis and Display System (GrADS) Data Server (GDS), and applications for
slicing, dicing and area sub-setting the large matrix of real time model data holdings. This approach insures
an efficient use of computer resources because users transmit/receive only the data necessary for their tasks
including metadata. Data sets served in this way with a high availability server offer vast possibilities for the
creation of new products for value added retailers and the scientific community. New applications to access
data and observations for verification of gridded model output, and progress toward integration with access to
conventional and non-conventional observations will be discussed. We will demonstrate how users can use
NOMADS services to repackage area subsets either using repackaging of GRIB2 files, or values selected by
ensemble component, (forecast) time, vertical levels, global horizontal location, and by variable, virtually a 6-
Dimensional analysis services across the internet.
NOAA National Operational Model Archive and Distribution System (NOMADS) Access to the Twentieth Century Reanalysis Project
Using a state-of-the-art data assimilation system and surface pressure observations, the Twentieth Century Reanalysis Project is generating a six-hourly, four-dimensional global atmospheric dataset spanning 1891- 2008 to place current atmospheric circulation patterns into a historical perspective. An important consequence of climate change is the altered likelihood of weather extremes. To have confidence in projected changes of such extremes in the twenty-first century, it is first necessary to assess model simulations of such changes throughout the twentieth century, using daily data. Any such daily verification data must also have quantified estimates of uncertainty to allow a fair quantitative assessment of the simulations. This paper will describe the data and NOMADS access paths to these data from the National Climatic Data Center in Asheville, NC.
Research and operational applications in multi-center ensemble forecasting
The North American Ensemble Forecast System (NAEFS) was built up in 2004 by the Meteorological Service of Canada (MSC), the National Meteorological Service of Mexico (NMSM), and the US National Weather Service (NWS) as an operational multi-center ensemble forecast system. Currently it combines the 20-member MSC and NWS ensembles to form a joint ensemble of 40 members twice a day. The joint ensemble forecast, after bias correction and statistical downscaling, is used to generate a suite of products for CONUS, North America and for other regions of the globe. The THORPEX Interactive Grand Global Ensemble (TIGGE) project has been established a few years ago to collect operational global ensemble forecasts from world centers, and distribute to the scientific community, to encourage research leading to the acceleration of improvements in the skill and utility of high impact weather forecasts. TIGGE research is expected to advise the development of the operational NAEFS system and eventually the two projects are expected to converge into a single operational system, the Global Interactive Forecast System (GIFS). This presentation will review recent developments, the current status, and plans related to the TIGGE research and NAEFS operational multi-center ensemble projects.
Advances in Drought Data and Services Distribution Through The U.S. Drought Portal
The U.S. Drought Portal (USDP, http://www.drought.gov) is part of the National Integrated Drought Information System (NIDIS), and has been developed as a national resource for data, models, risk information and impacts of drought. The USDP has the responsibility for integrating, archiving, and disseminating drought data, information, and forecasts from multiple federal, state, and other agencies. It provides this information via the internet, in support of drought early warning, mitigation, and adaptation. The USDP is also a registered component of the Group on Earth Observations, Global Earth Observing Systems of Systems (GEOSS) and part of the GEOSS Architecture Implementation Pilot. Data and information on the USDP is primarily disseminated in two forms. The first is through Open Geospatial Consortium-compliant web services. A custom Map Viewer was developed and integrated into the USDP to display cartographic information. This viewer works in conjunction with the National Climatic Data Center's (NCDC) Weather and Climate Toolkit and Geodata Portal to provide access to these Geographic Information System and Google Earth applications. The second form of information dissemination is through a dynamic graphical display. This tool leverages NCDC databases to provide drought and drought-related parameters for time series analysis. This tool was developed to work within a number of web-based environments. Both of the tools mentioned are constantly evolving, both in terms of functionality and in terms of the amount of information provided through them. Future plans for the USDP, including the tools mentioned, include expansion of information, data, and forecasts beyond the U.S. to include the international drought community. Additional effort will go into developing and providing region-specific information for the U.S. and in linking to non-NCDC databases.
Using Unidata's THREDDS and RAMADDA Services to Support Two Disparate Projects
This presentation will provide an overview of the implementation and use of THREDDS Data
Server ( TDS) and RAMADDA services in support of two disparate projects: the U.S. Antarctic Program
(USAP) community and application of lessons learned to the UCAR Africa Initiative meningitis effort.
For the past several years real-time meteorological numerical model (AMPS, NCAR/ESSL/MMM), satellite (UW-
Madison SSEC/AMRC and SPAWAR), and observation (SPAWAR) data have been shared among USAP
researchers in a project-specific Internet Data Distribution system, the Antarctic-IDD. Like its Unidata
community counterparts, the North American IDD and the South American IDD-Brazil, this network
was built using Unidata's Local Data Manager Version 6 ( LDM-6) data sharing engine. Recently,
collaboration among USAP and Unidata personnel has resulted in establishing a UCAR-based facility that
provides programmatic and human-interactive access to the rich dataset available in the Antarctic-IDD.
This effort employs the Unidata-developed TDS and RAMADDA server technologies. Transparent,
programmatic access to the data served by TDS and RAMADDA is freely available through a variety of
data analysis and visualization applications including the Unidata Integrated Data Viewer ( IDV) and
UW/SSEC's next generation McIDAS, McIDAS-V.
A second project, the UCAR Africa Initiative, is aimed at developing a prototype Earth-gauging system to
investigate the utility of integrating weather and health data to help manage meningitis vaccination programs in
Africa. The overarching goal of this effort is to contribute to saving lives and enhancing livelihood in Ghana
through integration of health and environmental data, and by using those data in health-related decision-
making. TDS and RAMADDA services will be employed to facilitate accumulating, distributing, and
remotely accessing numerical model, weather observation, and epidemiological data that will comprise the
core of the project database. Unidata's LDM-6 will be used for routine data collection and dissemination
tasks, and the Unidata IDV will be used for data analysis and in the production of project-specific products
such as risk maps that will be made available through TDS/ RAMADDA and Google Earth.
Training on Eastern Pacific tropical cyclones for Latin American students
Tropical cyclones are one of the most impressive atmospheric phenomena and their development in the
Atlantic and Eastern Pacific basins has potential to affect several Latin-American and Caribbean countries,
where human resources are limited. As part of an international research project, we are offering short courses
based on the current understanding of tropical cyclones in the Eastern Pacific basin. Our main goal is to train
students from higher-education institutions from various countries in Latin America. Key aspects are tropical
cyclone formation and evolution, with particular emphasis on their development off the west coast of Mexico.
Our approach includes lectures on tropical cyclone climatology and formation, dynamic and thermodynamic
models, air-sea interaction and oceanic response, ocean waves and coastal impacts as well as variability and
climate-related predictions. In particular, we use a best-track dataset issued by the United States National
Hurricane Center and satellite observations to analyze convective patterns for the period 1970-2006. Case
studies that resulted in landfall over northwestern Mexico are analyzed in more detail; this includes systems
that developed during the 2006, 2007 and 2008 seasons.
Additionally, we have organized a human-dimensions symposium to discuss socio-economic issues that are
associated with the landfall of tropical cyclones. This includes coastal zone impact and flooding, the link
between cyclones and water resources, the flow of weather and climate information from scientists to policy-
makers, the role of emergency managers and decision makers, impact over health issues and the viewpoint of
the insurance industry.