Time series

Overview

A Satellite Image Time Series (SITS) is a set of satellite images taken from the same scene at different times. A SITS makes use of different satellite sources to obtain a larger data series with a short time interval between two images. In this case, it is fundamental to observe the spatial resolution and registration constraints.

Satellite observations offer opportunities for understanding how Earth is changing, for determining the causes of these changes, and for predicting future changes. Remotely sensed data, combined with information from ecosystem models, offers an opportunity for predicting and understanding the behavior of the Earth’s ecosystem. Sensors with high spatial and temporal resolutions make the observation of precise spatio-temporal structures in dynamic scenes more accessible. Temporal components integrated with spectral and spatial dimensions allow the identification of complex patterns concerning applications connected with environmental monitoring and analysis of land-cover dynamics.

Change detection can only provide a “before and after” scenario; a time-series analysis provides an opportunity to study patterns and key changes in the landscape evolution over time.

This SEPAL recipe allows users to create and retrieve SITS based on Landsat and Copernicus programs imagery using the Google Earth Engine datacube.

Warning

You won’t be able to retrieve the images if your SEPAL account and GEE account are not connected. Follow Connect SEPAL to GEE to know more.

Start

Once the time-series recipe is selected, SEPAL will show up the recipe process in a new tab(1), the base map will change to Google high-resolution imagery and the AOI selection window will open itself on the bottom right side (2).

The first step is to change the name of the recipe. This name will be used to name your files and recipes in the SEPAL folders. Use the best-suited convention for your need. Simply double-click the tab and write a new name. It will default to Time_series_<start_date>_<end_date>_<band name>.

Note

The SEPAL team recommends using the following naming convention: <aoi name>_<start-<end>_<measure>_<sensors>. As an example the following is a satisfying recipe name: sgp_2012-2018_ndfi_l78.

Parameters

On the bottom right corner, 4 tabs are available. They will allow you to customize the time series to its needs.

  • AOI: the Area of interest (AOI)

  • DAT: the dates of the time series

  • SRC: the sources datasets of the time series

  • PRC: the pre-processing parameters

AOI Selection

The data exported by the recipe will be cut to the bounds of the AOI. There are multiple ways to select the AOI in SEPAL:

  • Administrative boundaries

  • EE Tables

  • Drawn polygons

Tip

The type choice made at this step is not definitive, simply click on <the selected method> on top of the AOI window and a dropdown will allow you to switch between the mentioned methods.

Administrative boundaries

You can select administrative layers as an AOI. These geometries are extracted from the FAO GAUL Level 1 dataset available on GEE. Select Select country/province.

Note

As GEE does not support non-latin characters, accents and special characters have been removed from country and provinces names.

In the first dropdown menu, you will be able to select a Country (administrative layer 0) from the country list. Optionally one can also select a province (administrative level 1) within the country selected. The dropdown list is updated on the fly according to the country selection. If nothing is selected, the whole country will be considered.

A buffer can be applied on these boundaries, define its size using the provided slider (in km). It is by default set to 0 i.e. disabled.

Note

The area of interest and preview will take longer to show up when buffering is enabled.

Once all the parameters are selected, the AOI will be previewed in the small map at the bottom of the frame. To validate it click on Apply button. Once validated, the map will zoom on the AOI and draw it in grey on the map.

EE table

The user can use custom AOI defined by shapes. These shapes need to be ingested in EarthEngine as a ee.FeatureCollection. Select EE table.

in the first dropdown, provide a completely qualified GEE asset name (e.g. projects/gtfp-fao/assets/aoi_ecowas).

Warning

You must have access to this asset. If that’s not the case ask the owner of the asset to modify the sharing parameters.

  • Select include all and the whole geometries associated with the features will be used as AOI.

  • Select filter and the user will be able to provide a column name and a value to filter within the table. The Aoi will then be reduced to the filtered features of the initial asset.

A buffer can be applied on these boundaries, define its size using the provided slider (in km). It is by default set to 0 i.e. disabled.

Note

The area of interest and the preview will take longer to show up when buffering is enabled.

Once all the parameters are selected, the AOI will be previewed in the small map at the bottom of the frame. To validate it click on Apply button. Once validated, the map will zoom on the AOI and draw it in grey on the map.

Draw polygon

The user can use custom AOI defined by a drawn shape. This shape will be converted into a ee.FeatureCollection on the fly. Select draw a polygon to use this option.

The pointer in the map will be converted into a . Click successively on the map to draw a polygon.

Once the geometry is closed, the AOI will be previewed in the small map at the bottom of the frame. To validate it click on Apply button. Once validated, the map will zoom on the AOI and draw it in grey on the map.

Dates

In the DAT tab, you will be asked to select the starting date and the ending date of the time series. Click on the date tex field to open a date picker popup. Click on the Select button to validate a date. When both dates are selected click on apply button.

Sources

As mentioned in the introduction, A SITS makes use of different satellite sources to obtain a larger data series with a shorter time interval between the images. To meet this objective, SEPAL allows you to select data from multiple entry points. You can select multiple sources in the following list (click on the link to see the corresponding dataset information):

  • Optical datasets: These datasets are based on Optical satellites imagery.

    • L8: Landsat 8 Tier 1. Landsat scenes with the highest available data quality are placed into Tier 1 and are considered suitable for time-series processing analysis. Tier 1 includes Level-1 Precision Terrain (L1TP) processed data that have well-characterized radiometry and are inter-calibrated across the different Landsat sensors. The geo-registration of Tier 1 scenes will be consistent and within prescribed tolerances [<=12 m root mean square error (RMSE)]. All Tier 1 Landsat data can be considered consistent and inter-calibrated (regardless of the sensor) across the full collection.


    • L8 T2: Landsat 8 Tier 2. Scenes not meeting Tier 1 criteria during processing are assigned to Tier 2. This includes Systematic terrain (L1GT) and Systematic (L1GS) processed scenes, as well as any L1TP scenes that do not meet the Tier 1 specifications due to significant cloud cover, insufficient ground control, and other factors. Users interested in Tier 2 scenes can analyze the RMSE and other properties to determine the suitability for use in individual applications and studies.


    • L7: Landsat 7 Tier 1. Landsat scenes with the highest available data quality are placed into Tier 1 and are considered suitable for time-series processing analysis. Tier 1 includes Level-1 Precision Terrain (L1TP) processed data that have well-characterized radiometry and are inter-calibrated across the different Landsat sensors. The geo-registration of Tier 1 scenes will be consistent and within prescribed tolerances [<=12 m root mean square error (RMSE)]. All Tier 1 Landsat data can be considered consistent and inter-calibrated (regardless of the sensor) across the full collection.


    • L7 T2: Landsat 7 Tier 2. Scenes not meeting Tier 1 criteria during processing are assigned to Tier 2. This includes Systematic terrain (L1GT) and Systematic (L1GS) processed scenes, as well as any L1TP scenes that do not meet the Tier 1 specifications due to significant cloud cover, insufficient ground control, and other factors. Users interested in Tier 2 scenes can analyze the RMSE and other properties to determine the suitability for use in individual applications and studies.


    • L4-5: Landsat 4 Tier 1 combined with Landsat 5 Tier 1. Landsat scenes with the highest available data quality are placed into Tier 1 and are considered suitable for time-series processing analysis. Tier 1 includes Level-1 Precision Terrain (L1TP) processed data that have well-characterized radiometry and are inter-calibrated across the different Landsat sensors. The geo-registration of Tier 1 scenes will be consistent and within prescribed tolerances [<=12 m root mean square error (RMSE)]. All Tier 1 Landsat data can be considered consistent and inter-calibrated (regardless of the sensor) across the full collection.


    • L4-5 T2: Landsat 4 TM Tier 2 combined with Landsat 5 TM Tier 2. Scenes not meeting Tier 1 criteria during processing are assigned to Tier 2. This includes Systematic terrain (L1GT) and Systematic (L1GS) processed scenes, as well as any L1TP scenes that do not meet the Tier 1 specifications due to significant cloud cover, insufficient ground control, and other factors. Users interested in Tier 2 scenes can analyze the RMSE and other properties to determine the suitability for use in individual applications and studies.


    • S2: Sentinel-2 Multispectral instrument is a wide-swath, high-resolution, multi-spectral imaging mission supporting Copernicus Land Monitoring studies, including the monitoring of vegetation, soil and water cover, as well as observation of inland waterways and coastal areas.


  • Radar datasets: This dataset is based on Radar satellite imagery.

    • S1: Sentinel-1 SAR GRD. The Sentinel-1 mission provides data from a dual-polarization C-band Synthetic Aperture Radar (SAR) instrument at 5.405GHz (C band). This collection includes the S1 Ground Range Detected (GRD) scenes, processed using the Sentinel-1 Toolbox to generate a calibrated, ortho-corrected product. The collection is updated daily. New assets are ingested within two days after they become available.

One can Optionally Select in the last dropdown a classification recipe and apply the classifier to every acquisition in the time-series, transforming the time series into a land cover time series. To know more about the classification recipe got to Classification.

When all the data are selected click on apply.

Pre-processing

Warning

This section is optional as these parameters are set by default.

  • correction: None

  • cloud detection: QA bands, Cloud score

  • cloud masking: moderate

  • snow masking: on

Multiple pre-processing parameters can be set to improve the quality of the provided images. SEPAL has gathered 4 of them in the form of these interactive buttons. If you think others should be added to hesitate to mention it in our issue tracker.

Correction

  • surface reflectance: Use scenes atmospherically corrected surface reflectance.

  • BRDF correction: Correct for bidirectional reflectance distribution function (BRDF) effects.

Cloud detection

  • QA bands: use pre-created QA bands from datasets

  • Cloud score: use cloud scoring algorithm

Cloud masking

  • Moderate: rely only on image source QA bands for cloud masking

  • Agressive: rely on image source QA bands and a cloud scoring algorithm for cloud masking. This will probably mask out some built-up areas and other bright features.

Snow masking

  • On: mask snow. This tends to leave some pixels with shadowy snow

  • Off: don’t mask snow. Note that some clouds might get misclassified as snow, and because of this, disabling snow masking might lead to cloud artifacts.

Available Bands

Note

The wavelength of each band is dependant on the used satellite.

The time series will use a single observation for each pixel. This observation can be one of the following:

Name

Description

Formula

blue

blue

green

green

red

red

nir

near infrared

swir1

shortwave infrared 1

swir2

shortwave infrared 1

aerosol

aerosol attributes

pan

panchromatic band

cirrus

cirrus cloud detection

thermal

thermal

thermal2

thermal2

brightness

brightness from Tasseled cap bands

\(0.3037 (band 1) + 0.2793 (band 2) + 0.4743 (band 3) + 0.5585 (band 4) + 0.5082 (band 5) + 0.1863 (band 7)\)

greeness

greeness from Tasseled cap bands

\(−0.2848 (band 1) − 0.2435 (band 2) − 0.5436 (band 3) + 0.7243 (band 4) + 0.0840 (band 5) − 0.1800 (band 7)\)

wetness

wetness from Tasseled cap bands

\(0.1509 (band 1) + 0.1973 (band 2) + 0.3279 (band 3) + 0.3406 (band 4) − 0.7112 (band 5) − 0.4572 (band 7)\)

fourth

fourth from Tasseled cap bands

fifth

fifth from Tasseled cap bands

sixth

sixth from Tasseled cap bands

NDVI

Normalized difference vegetation index

\(((nir - red)/(nir + red))\)

NDMI:

Normalized Difference Moisture Index

\(ndmi = (nir - swir1)/(nir + swir1)\)

NDWI

Normalized difference water index

MNDWI

Modified Normalized Difference Water Index

\(mndwi = (green - swir) / (green + swir)\)

NDFI

Normalized Difference Fraction Index

\(ndfi = (GV_shade - (NPV + soil)/(GV_shade + NPV + soil)\)

EVI

Enhanced vegetation index

\(evi =G * (nir - red)/(nir + C_1 * red - C_2 * blue + L)\)

EVI2

Two-band EVI (Enhanced vegetation index)

\(evi_2 = 2.5 * (nir - red) / (nir + 2.4 * red + 1)\)

SAVI

Soil-Adjusted Vegetation Index

\(((nir - red) / (nir + red + L)) x (1 + L)\)

NBR

Normailzed burn ratio

\(nbr = (nir - swir) / (nir + swir)\)

UI

Urban index

\(ui = (swir2 - nir) / (swir2 + nir)\)

NDBI

Normalized Difference Built-up Index

\(ndbi = (swir - nir) / (swir + nir)\)

IBI

Index based built-up index

\(ibi = (ndbi - (savi + mndwi) / 2) / (ndbi + (savi + mndwi) / 2)\)

BUI

Built-up Index

\(bui = (red - swir1) / (red + swir1) + (swir2 - swir1) / (swir2 + swir1)\)

Analysis

Once all the parameters are set, you can generate data from the recipe. Some can be directly generated on the fly from the interface, the rest require retrieving the data to SEPAL folders.

The analysis icons can be found in the top right corner of the sepal window:

  • : plot data

  • : retreive data

Tip

The download icon is only released when the data parameters are complete. If the button is disabled, check your parameters, some might be missing.

Plot

Click on the button to start the plotting tool. Move the pointer to the main map, the pointer will be transformed into a . Click anywhere in the AOI to plot data for this specific location in the following popup window.

The plotting area is dynamic and can be customized by the user.

Using the slider (1), the temporal width displayed can be changed. It cannot exceed the start and/or end date of the time series.

You can also select the observation feature by selecting one of the available measures in the dropdown selector in the top left corner (2). The available bands are the same as the described previously.

On the main graph, each point represents one valid (based on the pre-processing filters) observation. Hover on the point to let the tooltip describe the value and date of the observation (3).

Tip

The coordinates of the point are displayed at the top of the chart window.

Warning

The plot feature is retrieving information from GEE on the fly and serving it in an interactive window. This operation can take time depending on the number of available observations and the complexity of the selected pre-processing parameters. If the popup window displays a spinning wheel, wait up to 2 min to see the data displayed.

Export

The data generated by the recipe can also be used in other workflows but before it needs to be retrieved from GEE to your SEPAL environment.

Parameters

Click on the button, it will open the download parameters window. You will be able to select the measure to use on each observation of the time-series. This measure can be selected in the list of available bands presented in a previous section.

Tip

There is no fixed rule to the measure selection. Each index is more adapted to a set of analyses in a defined biome. The knowledge of the study area, the evolution expected and the careful selection of an adapted measure will improve the quality of the downstream analysis.

You can set a custom scale for exportation by changing the value of the slider (m). Keep in mind that Sentinel data native resolution is 10 m and Landsat is 30 m.

When all the data is selected click on the apply button. Notice that the task tab in the bottom left corner of the screen (1) will change from to which means that the tasks are loading.

Exportation status

Going to the task tab (bottom left corner using the or buttons —depending on the loading status—), you will see the list of the different loading tasks. The interface will provide you with information about the task progress and it will display an error if the exportation has failed. If you are unsatisfied with the way we present information, the task can also be monitored using the GEE task manager.

Tip

This operation is running between GEE and SEPAL servers in the background, you can thus close the SEPAL page without killing the process.

When the task is finished the frame will be displayed in green as shown on the second image.

Access

Once the downloading process is done, you can access the data in your SEPAL folders. The data will be stored in the downloads folder using the following format:

.
└── downloads
    └── <TS name>
        ├── <tile number>
        │   ├── chunk-<start date>_<end date>
        │   │   ├── <TS name>_<tile number>_<start_date>_<end date>-<gee tiling id>.tif
        │   │   ├── ...
        │   │   └── <TS name>_<tile number>_<start_date>_<end date>-<gee tiling id>.tif
        │   ├── ...
        │   ├── chunk-<start date>_<end date>
        │   ├── tile-<gee tiling id>
        │   │   └── stack.vrt
        │   ├── ...
        │   ├── tile-<gee tiling id>
        │   ├── dates.csv
        │   └── stack.vrt
        ├── ...
        └── <tile number>

Danger

Understanding how images are stored in a Time Series is only required if you want to manually use them. The SEPAL applications are bound to this tiling sytem and can digest this information for you.

The data are stored in a folder using the name of the time series as it was set in the first section of this document. SEPAL team was forced to use this folder structure as Google Earth Engine is enable to export an ee.ImageCollection. As the number of data is spatially too big to be exported at once, the data are cut into small pieces and brought back together in a stack.vrt file.

The AOI provided by the user will be split in multiple SEPAL tiles. The AOI is an ee.FeatureCollection, each feature is downloaded in a different tile. if the Tile is bigger than 2°x2° (EPSG:4326) then the feature is split again until all the tiles are smaller than the maximal 2° size. The tiles are identified by their <tile_number>.

To limit the size of the downloaded images, in each SEPAL tile, the time period is splitted in chunks of 3 month. they are identified by their <chunk-<start>_<end>. Chunks are image folders. As a SEPAL tile is still bigger than what GEE can download at once, the images are splitted in GEE tile. This tiling system uses it’s own identification system (000000xxxx-000000xxxx). In consequence chunks contain tile raster images. Each one of these images is composed of 1 band per observation date, with the value of the measure for each pixel. The bands are named with the date.

To gather all theses raster together a first agregation on time is performed. One stack.vrt is created per GEE tile meaning that each stack.vrt`file contains all the :code:`*<gee tiling id>.tif contained in each chunk. reconstituting the full time period on the smallest spatial unit: the gee tile. each file is stored in a folder called tile-<gee tiling id>.

Finally information are gathered at the spatialy at the SEPAL tile level in the main stack.vrt file.

The last file: date.csv gather all the observation dates in chronological order.

Note

The dates contained in date.csv can differ from one SEPAL tile to another due to data availability and pre-processing filters

Tip

The full folder with a concistent treefolder is required to read the .vrt

Here is an example of a real TS folder:

.
└── downloads
    └── tutorial_TS
        ├── 1
        │   ├── chunk-2012-01-01_2012-04-01
        │   │   ├── tutorial_TS_1_2012-01-01_2012-04-01-0000000000-0000000000.tif
        │   │   ├── ...
        │   │   └── tutorial_TS_1_2012-01-01_2012-04-01-0000002560-0000001024.tif
        │   ├── ...
        │   ├── chunk-2018-10-01_2018-12-31
        │   ├── tile-0000000000-0000000000
        │   │   └── stack.vrt
        │   ├── ...
        │   ├── tile-0000002560-0000001024
        │   ├── dates.csv
        │   └── stack.vrt
        ├── ...
        └── 3

Important

Now that you have downloaded the TS to your SEPAL account, it can be retrieved to your computer using FileZilla or used in one of our time-series analysis module.