JASMES Tutorial

This tutorial shows how to convert MODIS data in binary format provided by JASMES service to GeoTiFF.
By converting from binary format to GeoTiFF, data can be viewed, edited, analyzed, using QGIS.

Please refer to the following page for details of JASMES user registration. JASMES users can use all data published in JASMES services.
In this tutorial, as an example, we will create GeoTiFF that converts MODIS data(filename : '*_le') to physical quantity. JASMES User Registration JASMES Data Access JASMES Data List

Please use this tutorial as follows for your purposes.

・If you want to convert JASMES MODIS data to GeoTiFF
 Execute the sample script according to "1. How to use sample script"

・If you want to know what the sample script does
 See "2. Description of script processing"

・If you want to know how to open GeoTiFF in QGIS for use
 See "3. How to open GeoTiFF with QGIS"

1-1. Sample script download

These are sample scripts used in this tutorial. Please download it from the buttons bellow.
Multiple scripts are provided. Please refer to the list in "1-3. Target Data List" to determine which script to use.
This tutorial is an example of using script(1). Basically, the GeoTiFF conversion method is same except for reading part, and script usage is also the same.
For special instructions for each script, please refer to the "README_*.txt" file included with the downloaded script.

Sample Script(1) (filename:'*_le') Download Sample Script(2) (filename:'*_8b') Download Sample Script(3) (multi band) Download Sample Script(4) (lst) Download Sample Script(5) (wf, wst) Download Sample Script(6) (rgb) Download

1-2. Execution enviroment

The tutorial shows an example of execution under Linux.
The sample scripts use Python3.

The following is the runtime environment and Python libraries used when testing the sample scripts.

1-3. Target data list

To get JASMES data, you need to register as a user here. Please refer to here for how to get data after user registration.

Products for which sample scripts can be used and the script number are listed below.
Please refer to the list of data provided by JASMES for details on the area, resolution, and statistical period (Daily, Monthly, etc.) of each product, if necessary.

No.	Web page	Sensor	category	Product		Script						Note
						(1)	(2)	(3)	(4)	(5)	(6)
1	JASMES	Terra・Aqua /MODIS	Atmoshere	aot	atmosphere optical tickness	○	-	-	-	-	-
2	JASMES	Terra・Aqua /MODIS	Atmoshere	aerosol	aerosol data	-	-	○	-	-	-	multi band
3	JASMES	Terra・Aqua /MODIS	Atmoshere	alph	Angstrom exponent	○	-	-	-	-	-
4	JASMES	Terra・Aqua /MODIS	Atmoshere	cfr	Cloud flag	-	-	-	-	-	-	Not applicable for HDF file.
5	JASMES	Terra・Aqua /MODIS	Atmoshere	dpar	Direct PAR	○	-	-	-	-	-
6	JASMES	Terra・Aqua /MODIS	Atmoshere	par	Photosynthetically Available Radiation	○	-	-	-	-	-
7	JASMES	Terra・Aqua /MODIS	Atmoshere	ptw	Precipitable Water	○	-	-	-	-	-
8	JASMES	Terra・Aqua /MODIS	Atmoshere	tip	Transmittance of Instantaneous PAR at noon	○	○	-	-	-	-
9	JASMES	Terra・Aqua /MODIS	Atmoshere	rpar	surface Reflectance weighted by PAR wavelengths & solar irradiance	○	-	-	-	-	-
10	JASMES	Terra・Aqua /MODIS	Atmoshere	swr	Shortwave Radiation	○	○	-	-	-	-
11	JASMES	Terra・Aqua /MODIS	Atmoshere	ta1	Aerosol optical thickness [WK= 412.46 nm]	○	-	-	-	-	-
12	JASMES	Terra・Aqua /MODIS	Atmoshere	taua	Aerosol abundance (Optical thickness)	○	-	-	-	-	-
13	JASMES	Terra・Aqua /MODIS	Atmoshere	uva	UV-A	○	○	-	-	-	-
14	JASMES	Terra・Aqua /MODIS	Atmoshere	uvb	UV-B	○	○	-	-	-	-
15	JASMES	Terra・Aqua /MODIS	Land	lst	Land Surface Temperature	-	-	-	○	-	-
16	JASMES	Terra・Aqua /MODIS	Land	ndvi	Activity of Vegetation (NDVI)	○	-	-	-	-	-
17	JASMES	Terra・Aqua /MODIS	Land	wf	Wild fire	-	-	-	-	○	-	No header in data.
18	JASMES	Terra・Aqua /MODIS	Land	wst	Plant Water Stress Trend	-	-	-	-	○	-	No header in data.
19	JASMES	Terra・Aqua /MODIS	Ocean	chla	Phytoplankton abundance (Chlorophyll-a)	○	-	-	-	-	-
20	JASMES	Terra・Aqua /MODIS	Ocean	olst	Ocean and Land Surface Temperature	○	-	-	-	-	-
21	JASMES	Terra・Aqua /MODIS	Ocean	sst	Sea Surface Temperature	○	-	-	-	-	-
22	JASMES	Terra・Aqua /MODIS	Ocean	apg	Absorption Coefficient of Particles + CDOM	○	-	-	-	-	-
23	JASMES	Terra・Aqua /MODIS	Ocean	bbp	Backscattering Coefficient of Particles	○	-	-	-	-	-
24	JASMES	Terra・Aqua /MODIS	Cryoshere	snwcfr	Long-term Snow Cover Extent	○	-	-	-	-	-
25	JASMES	Terra・Aqua /MODIS	-	rgb	RGB	-	-	-	-	-	○	RGB
26	JASMES	Terra・Aqua /MODIS	-	multi	par data including subparameters	-	-	○	-	-	-	multi band

1-4. How to run the sample script

Execute the sample script with the input file path, output path and target area as arguments.
The usage of the script for the file name "*_le" is described. For special instructions for each script, please refer to the "README_*.txt" file included with the downloaded script.

$ ./MODIS_le_bin2geotiff.py <Input> <Output> <Area>

Argument 1 Input
 Specify the full path of binary file to be input.

Argument 2 Output
 Specify the name of the GeoTiFF file to be output.
 If only a file name is specified, the output will be in the current directory.

Argument 3 Area
 Specify the area corresponding to the file to be input.(Global, Japan and Thai)


An example of command execution is shown below. The output file name can be any arbitrary file name.
[DIR] : Data input/output directory

$ ./MODIS_le_bin2geotiff.py [DIR]/MDS02SSH_A20230101Av1_v811_7200_3601_CHLA_le [DIR]/MDS02SSH_A20230101Av1_v811_7200_3601_CHLA_le.tif Global

From here, the various processes of the sample scripts are explained.

2-1. Reading binary data with Python

The header is specified according to the format of JASMES MODIS data shown in "1-3. Target Data List".
Since JASMES MODIS data is formatted to include necessary information such as pixel size in the header section, it is necessary to read the header.
Please note that this is specific to JASMES MODIS data.

# Header format
class headerFormat(c.Structure):
  _fields_ = [
    ("npixel" , c.c_char*6 ),
    ("nline"  , c.c_char*6 ),
    ("lon_min", c.c_char*8 ),
    ("lat_max", c.c_char*8 ),
    ("reso"   , c.c_char*8 ),
    ("slope"  , c.c_char*12),
    ("offset ", c.c_char*12)
  ]

arguments (input binary file path, output GeoTiFF file path, target area).

argvs   = sys.argv

# argument ERROR
if len(argvs) != 4:
  print("argument error: 1:input file, 2:output file, 3:area (Global / Japan / Thai)")
  sys.exit(9)
else:
  binfile = argvs[1]
  output  = argvs[2]
  area    = argvs[3]

Binary file reading is performed.
After reading the header information into format_data according to the file, the data portions is read into binarr, skipping the header portion.

# 1 line * 2byte
HEADER_SIZE = X_SIZE*2
format_data = headerFormat()

# Binary file open
with open(binfile, 'rb') as f:
  # Read header
  f.readinto(format_data)
  # Header skip
  f.seek(HEADER_SIZE)
  # Read binary data as unsiged-2byte integer
  binarr = np.fromfile(f, dtype=np.uint16)

The necessary values of the headers read in format_data are retrieved.
The values obtained are as follows.
・SLOPE, OFFSET are used for physical quantity convertion (DN * SLOPE + OFFSET).
・X_SIZE, Y_SIZE, UpperLeft_LON, UpperLeft_LAT, RESO are used for GeoTiFF output.

# Header value set
X_SIZE        = np.uint16(format_data.npixel)
Y_SIZE        = np.uint16(format_data.nline)
UpperLeft_LON = np.float32(format_data.lon_min)
UpperLeft_LAT = np.float32(format_data.lat_max)
RESO          = np.float32(format_data.reso)
SLOPE         = np.float32(format_data.slope)
OFFSET        = np.float32(format_data.offset)

# pixel center -> top left corner of the pixel
UpperLeft_LON = UpperLeft_LON - (RESO * 0.5)
UpperLeft_LAT = UpperLeft_LAT + (RESO * 0.5)

2-2. Convert to physical quantities using slope and offset values with Python

"DN * SLOPE + OFFSET" is converted to a physical quantity and stored in an array(data type is Float).
"65535" is "Error_DN", it is replaced by "nan".(In the case of "*_8b", "255" is "Error_DN".)

# Reshape pixel x line data
orgdat = binarr.reshape(Y_SIZE,X_SIZE).copy()
# value = DN * SLOPE + OFFSET
otemp  = orgdat * SLOPE + OFFSET
# convert error value to nan
otemp[orgdat==65535] = np.nan

2-3. Output GeoTiFF with Python (GDAL library)

The array is output in GeoTiFF format using Python's GDAL library.
Create a GDAL driver that creates GeoTiFF.
・driver.Create(output, cols, rows, band, data type)

# output GeoTiFF
driver = gdal.GetDriverByName('GTiff')

# outputfile, cols, rows, band num, data type
cols      = otemp.shape[1]
rows      = otemp.shape[0]
outRaster = driver.Create(output, cols, rows, 1, gdal.GDT_Float32)

SetGeoTransform with the coordinate values.
・outRaster.SetGeoTransform((upper left longitude, pixel size(x), rotate, upper left latitude, rotate, pixel size(y)))

# upper_left_longitude, pixel size(x), rotate, upper_left_latitude, rotate, pixel size(y)
dx = RESO
dy = RESO*-1
outRaster.SetGeoTransform((UpperLeft_LON, dx, 0, UpperLeft_LAT, 0, dy))

Specify the band.

# band
outband = outRaster.GetRasterBand(1)
outband.WriteArray(otemp)

The projection method is set to EPSG:4326.

# EPSG
outRasterSRS = osr.SpatialReference()
outRasterSRS.ImportFromEPSG(4326)
outRaster.SetProjection(outRasterSRS.ExportToWkt())

Export to file to output GeoTiFF.

outband.FlushCache()
outRaster = None

print("### END output : "+output+" ###")
sys.exit(0)

The GeoTiFF created in this tutorial can be opened directly in QGIS, a free GIS software, for viewing, editing, and analyzing data.
Please refer to the following QGIS usage instruction if necessary.

・Imaging procedure for GCOM-C (SHIKISAI) products using QGIS (G-Portal)
Please refer to Step5 and later and "How convert color lamps" as the physical quantities and GeoTiFF conversion have already been done in this tutorial.
Imaging procedure for GCOM-C (SHIKISAI) products using QGIS (G-Portal)
・Display of Chlorophyll-a Concentration (QGIS) (Shikisai Portal)
Please refer to the logarithmic scale (QGIS) if necessary, as the logarithmic scale may be easier to read in some cases, such as CHLA.
Display of Chlorophyll-a Concentration (QGIS) (Shikisai Portal)