a light-weight MODIS L2 to GRASS transformer: a preliminary version

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

        What is described below is an alpha version of a free software
        tool which allows importing of some of the MODIS L2 datasets
        into GRASS, performing the necessary reprojection along the way.
        (The use of MODIS L2 datasets is a rather natural prerequisite
        for MODIS-based near-realtime monitoring of the environment.)

        It seems that I'm now in the process of recounting my three
        years long or so experience with HDF-EOS, swaths, grids,
        reprojection, etc. Preasumably, making bits of my experience
        public may improve my own understanding of the tasks involved,
        but hopefully will also be useful to the community.

        The basic ideas beyond this tool are the same as for the MS2GT
        package, but with C and Bourne Shell used in place of IDL. The
        non-free HDF-EOS to GeoTiff tool also provides similar
        functionality. However, I had some issues with the latter using
        zero (0) as the fill value for the resulting rasters,
        effectively mapping the parts of the source dataset filled with
        zeros to become indistinguishable from the =E2=80=9Cno data=E2=80=9D areas.

        This is the work in progress, the usual disclaimer applies.

        The code below is under GNU GPL v3 or later.

* Dependencies

        The tool is written in Bash and relies on the following
        packages:

        * NetCDF tools 0.1-rc1 (namely, ncget-hdf) =E2=80=94 to extract data
         from HDF4;

        * Rawtools, as of the latest version available from its
         Savannah' GNU Arch archive:

$ tla register-archive
rawtools@sv.nongnu.org--rawtools
http://arch.savannah.gnu.org/archives/rawtools/rawtools/=20
$ tla get
rawtools@sv.nongnu.org--rawtools/rawtools--main--0.1--patch-5=20

         with the new rawselect and ygrid tools added as described by
         the diff (attached) =E2=80=94 to perform necessary binary
         representation conversions, =E2=80=9Cgeolocation inversion=E2=80=9D and =
the
         reprojection proper, etc.; (it's expected that these tools
         will become part of the package by the time of its
         long-awaited 0.1 release); (if there ever be one);

        * GNU Awk and bc, Tcl 8.5 =E2=80=94 to perform floating-point
         calculations and to create raw data streams, thanks to the
         binary(3tcl) Tcl built-in command, respectively;

        * GRASS (tested on 6.4.0RC5+39438, as included in Debian
         Squeeze) =E2=80=94 for obvious reasons.

* Usage

GRASS $ r.in.hdf.ygrid [-crop] LATLONG-HDF DATA-HDF PREFIX [SDS]...

        E. g.:

GRASS $ r.in.hdf.ygrid
MOD07_L2.A20100125.05^M500.005.1264429452.15915.hdf
MOD07_L2.A20100125.05^M500.005.1264429452.15915.hdf
2010-01-25T0500.mod07-l2.
Surface_Temperature Surface_Pressure=20
...
GRASS $ g.mlist type=3Drast pattern=3D2010-01-25T0500.mod07-l2.Surface_*=20
2010-01-25T0500.mod07-l2.Surface_Temperature
2010-01-25T0500.mod07-l2.Surface_Pressure
GRASS $=20

* Limitations

        I've tested the tool on some of the MOD07_L2 arrays (namely:
        K_Index, Lifted_Index, Surface_Elevation, Surface_Pressure,
        Surface_Temperature, Total_Ozone, Total_Totals, Water_Vapor,
        Water_Vapor_Direct, Water_Vapor_High, Water_Vapor_Low), and it
        the result seems to be sane. Hopefully, I'll test it more
        thoroughly by the end of the week.

        The known limitations are:

        * the dataset is expected to be 2D, with the dimensions matching
         those of the =E2=80=9CLatitude=E2=80=9D and =E2=80=9CLongitude=E2=80=9D;=
this effectively
         precludes one from using it on data having finer than 1 km
         resolution (unless one has a working modenlarge tool as well);

        * the data is always coerced into =E2=80=9Cfloat=E2=80=9D, even if the sou=
rce
         is, say, integer with the scale_factor and add_offset values
         set to identify transformation.

        The last issue is probably the easiest to resolve. (Indeed, I
        have a version of this tool tweaked to import the MOD02 data
        dating back to Juny 2007; preasumably, it was working then.)

        There're probably more issues with this tool.

        Please report bugs to adir-devel@theory.asu.ru.

* Download

        Due to the Aioe.org message size limits, I've put the relevant
        files on a Web server:

http://waterlily.siamics.net/~ivan/src/r.in.hdf.ygrid-1264541514.sh.gz
http://waterlily.siamics.net/~ivan/src/rawtools-1264548302.diff.gz

        The NetCDF tools package could be obtained from:

http://theory.asu.ru/~ivan/src/netcdf-tools-0.1-rc1.tar.gz
http://waterlily.siamics.net/~ivan/src/netcdf-tools-0.1-rc1.tar.gz

        while the yet to be released Rawtools package could be obtained
        from the public GNU Arch archive as shown above.

        Also of interest may be:

http://waterlily.siamics.net/~ivan/archives/git/gitweb.cgi?p=3Dnctools.git
http://waterlily.siamics.net/~ivan/archives/git/nctools.git/

=2D --=20
FSF associate member #7257
=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAktffGIACgkQMBO2oCMOM0qigACguXO5ncFpqd0nB6Xh4Dd6S3t0
eR0AoIwdOI7dPehKBEaXd2XKRI2/CRH1
=3Dqrj+
=2D----END PGP SIGNATURE-----