Quality flags and filtering

This page describes the quality and advisory flags provided with the soil moisture products. These flags allow users to identify pixels affected by surface conditions, processing limitations, or suitability constraints for specific applications.

The available flag layers vary by product:

D-MSSM (6 km): PFLAG, QFLAG (7-bit), AFLAG
D-ESSM (500 m) and D-HSSM (100 m): EFLAG, QFLAG (8-bit), AFLAG

All flags share the same spatial grid and dimensions within a given product.

Overview¶

Common properties:

Dimensions: (y, x)
Units: unitless
Grid mapping: spatial_ref
Data type: ubyte (unsigned 8-bit integer)
Missing value: 255

Pixels with value 255 represent missing or undefined data and should be excluded from analysis.

PFLAG — Processing flag (`pflag`)¶

Applies to: D-MSSM only

Description: Bit flags that record the quality and processing success of the data.

Short name: PFLAG
Long name: Processing flag
Data type: ubyte
Fill value: 255
Interpretation: bitmask

Bit meanings¶

Bit	Condition	Meaning when bit = 1
0	Recommended quality	Quality not recommended
1	Retrieval successful	Retrieval not successful

Recommended quality is determined from the QFLAG. If static and/or dynamic conditions are present, the retrieval is flagged as not recommended. The retrieval-successful state is determined from the DAI (Data Availability Index); when dai = 0, no soil moisture is retrieved and pflag is set to 3 (not recommended + not successful).

QFLAG — Quality flag (`qflag`)¶

Description: Bit flags that record surface conditions.

Short name: QFLAG
Long name: Quality flag
Data type: ubyte
Fill value: 255
Interpretation: bitmask

Each bit represents a condition that may affect retrieval quality. D-MSSM and D-ESSM/D-HSSM use different bit definitions.

D-MSSM qflag (7 bits)¶

Bit	Condition	Meaning when bit = 1	Recommended action
0	Complex topography	Complex terrain present	Use with caution
1	Urban area	Urban area detected	Use with caution or exclude
2	Water body	Water body detected	Typically exclude
3	Precipitation	Heavy precipitation detected	Use with caution
4	Snow and ice	High snow/ice amounts detected	Usually exclude
5	Freeze/thaw	Frozen ground detected	Usually exclude
6	Dense vegetation	Dense vegetation detected	Use with caution

Static conditions (complex topography, urban area, water bodies) do not change over time. Dynamic conditions (precipitation, snow/ice, freeze/thaw, dense vegetation) vary with each observation.

D-ESSM / D-HSSM qflag (8 bits)¶

Bit	Condition	Meaning when bit = 1	Recommended action
0	Terrain ruggedness	Complex terrain	Use with caution
1	Urban area	Built-up surfaces present	Use with caution or exclude
2	Permanent water body	Open water	Typically exclude
3	Periodic water body	Seasonal water	Often exclude depending on application
4	Dense vegetation	Strong canopy effects	Use with caution
5	Patchy vegetation	Heterogeneous land cover	Moderate uncertainty
6	RMSE of downscaling	Elevated downscaling error	Consider thresholding
7	Freeze/thaw	Frozen ground conditions	Usually exclude

Multiple bits may be set simultaneously in both variants.

Interpreting values¶

Because QFLAG is a bitmask, the numeric value encodes a combination of conditions.

Examples (D-ESSM/D-HSSM):

0 → No flags set
4 → Permanent water body (bit 2)
20 → Permanent water (bit 2) + Dense vegetation (bit 4)
255 → Missing / no data

EFLAG — Enhanced flag (`eflag`)¶

Applies to: D-ESSM and D-HSSM only

Description: Indicates where downscaling/enhancement has been applied.

Short name: EFLAG
Long name: Enhanced flag
Data type: ubyte
Fill value: 255

EFLAG values¶

Value	Meaning
0	Enhanced (downscaling applied)
1	Not enhanced
255	Missing / no data

EFLAG usage¶

EFLAG allows users to distinguish between pixels produced directly from source observations and those derived through enhancement or downscaling.

Typical uses:

Analyze enhanced and non-enhanced regions separately
Restrict analysis to enhanced pixels only
Evaluate spatial coverage of enhancement methods

AFLAG — Advisory flag (`aflag`)¶

Applies to: all products (D-MSSM, D-ESSM, D-HSSM)

Description: Binary indicator of whether soil moisture values are recommended for use.

Short name: AFLAG
Long name: Advisory flag
Data type: ubyte
Fill value: 255

AFLAG values¶

Value	Meaning
0	Quality recommended
1	Quality not recommended
255	Missing / no data

AFLAG usage¶

AFLAG provides a high-level recommendation based on internal quality assessments. It is intended as a quick screening tool for users who prefer a simple accept/reject criterion.

Recommended filtering strategy¶

Filtering should depend on the application and tolerance for uncertainty.

Conservative approach¶

Exclude pixels where:

AFLAG = 1
QFLAG indicates water bodies (D-MSSM: bit 2; D-ESSM/D-HSSM: bits 2 or 3)
QFLAG indicates freeze/thaw conditions (D-MSSM: bit 5; D-ESSM/D-HSSM: bit 7)
Flag value is 255

Moderate approach¶

Exclude only major contamination sources:

Permanent water (D-MSSM: bit 2; D-ESSM/D-HSSM: bit 2)
Freeze/thaw (D-MSSM: bit 5; D-ESSM/D-HSSM: bit 7)
Missing values (255)

Application-specific filtering¶

Some applications may retain pixels flagged for vegetation or urban areas, while others may exclude them.

Example: decoding flags in Python¶

import xarray as xr

ds = xr.open_dataset("...")

qflag = ds["qflag"]
aflag = ds["aflag"]

# Mask missing values
valid = (qflag != 255) & (aflag != 255)

# Function to test whether a bit is set
def bit_is_set(data, bit):
    return (data & (1 << bit)) != 0

# Example masks (D-ESSM / D-HSSM)
is_perm_water = bit_is_set(qflag, 2)
is_seasonal_water = bit_is_set(qflag, 3)
is_frozen = bit_is_set(qflag, 7)

# Conservative filtering
mask = valid & (aflag == 0) & ~(is_perm_water | is_seasonal_water | is_frozen)

# Apply to soil moisture variable (replace 'sm' with actual name)
sm_filtered = ds["sm"].where(mask)