How do I analyse a generated texture in Python?

A generated polygon texture is a Client File holding a stack of frames in one blob, with a generation block on the record describing how to read them: the raster size, the geographic extent it covers, the value range the greyscale was quantised into, and where each frame sits in the blob. A simulation writes one per attribute, so a flood model might publish water level, velocity and wave height as separate textures over the same area.

The question this answers is "what did the event touch". The texture says what the values were and where, and a Data Lab query says which of your Entities were there, so the two together say which records the event reached and when.

Reading a texture needs the optional extra, which a base install does not have.

pip install bruce-models[texture]

Finding a texture

Textures are Client Files stored under the Polygon Texture purpose, so they are found the same way as any other file. Each one names the attribute it was generated from in its generation block.

import bruce_models as BM
# Not exported from the package root, so it is imported by path.
from bruce_models.client_file.client_file_texture import ClientFileTexture

# Either a session token from a login or a long-lived access token.
TOKEN = "YOUR_TOKEN"

bruce = BM.BruceApi({
    "account_id": "YOUR_ACCOUNT_ID",
    "session_id": TOKEN
})

# Every generated texture on the account, newest first.
query = (BM.ClientFileQueryBuilder(bruce)
         .purpose(BM.ClientFile.PURPOSE_POLYGON_TEXTURE)
         .newest_first())

print(f"{query.count():,} textures")

for record in query.files():
    generation = (record.get("Data") or {}).get("generation") or {}
    print(f"{record.get('ID')}  "
          f"attribute={generation.get('Attribute')}  "
          f"frames={len(generation.get('Frames') or [])}")

# When you know the source and attribute, ask for that one directly.
one = BM.ClientFile.find_generated_texture(
    bruce,
    entity_type_source_id=YOUR_SOURCE_ID,
    attribute="water_level"
)

Reading what it holds

Opening a texture downloads the blob once and reads the generation block off the record. From there everything is local, so walking every frame costs no requests at all.

Two things are worked out from the frames rather than asked of you. Cells that report the same value in every frame carry nothing a change can be read from, whether that is a permanent feature or an artefact of the model, and they are excluded. The threshold that separates "wet" from "dry" is read off the series too: the level whose selection varies most between frames is the one carrying the event, so a number that suits one archive is not quietly applied to the next.

texture = ClientFileTexture.open(bruce, TEXTURE_FILE_ID)

print(f"{len(texture)} frames of {texture.attribute}")
print(f"{texture.grid.width} x {texture.grid.height} texels")
print(f"values {texture.value_range[0]:.3f} to {texture.value_range[1]:.3f}")

# Read off the series rather than named here.
print(f"separating at {texture.natural_threshold():.4f}")

# The frames worth querying: the event arriving, its worst, and it passing.
chosen = texture.frames_of_interest()
print("frames of interest:", list(chosen))

# Each area of one frame, as a shape ready to query with.
for shape in texture.shapes_above(frame=chosen[0]):
    print("area with", len(shape["Polygon"]), "ring(s)")

What the event reached

Each area a frame holds is a Bruce geometry, so it goes straight into a Data Lab query as a spatial criterion. Walking the frames of interest and asking which Entities each area covers gives you the records the event reached, and the frame's timestamp says when.

An Entity resolves to the texels it covers, and the raster does not move, so that is worked out once and reused for every frame. Reading its own value per frame is then local.

PARCEL_TYPE_ID = "YOUR_ENTITY_TYPE_ID"

parcels = BM.DataLabBuilder(bruce, page_size=1000).entity_type(PARCEL_TYPE_ID)
timestamps = texture.timestamps()

# Each reading is its own edit, so a parcel keeps one per frame.
changes = BM.ChangeSetBuilder(bruce, allow_multi_same_entity=True)

# Consecutive frames trace the same ground, and an unchanged area answers
# with the same Entities, so the answer is kept against its geometry.
cache = BM.LRUCache(100000, weigh=len)

for index in chosen:
    when = timestamps[index]
    values, _ = texture.frame(index)

    for shape in texture.shapes_above(frame=index):
        key = str(shape)
        found = cache.get(key)
        if found is None:
            found = []
            for page in parcels.intersecting(shape).pages():
                for item in page.items:
                    found.append((item, texture.texels_of(item)))
            cache.set(key, found)

        for entity, texels in found:
            # The wettest texel it covers, since a record on a boundary
            # is as wet as its worst part.
            level = texture.peak_at(texels, values=values)
            if level is None:
                continue
            reading = {}
            BM.Entity.set_value(reading, "flood/time", when)
            BM.Entity.set_value(reading, "flood/level", level)
            changes.queue(entity, reading)

    print(f"frame {index} ({when}): {len(changes):,} readings")

print(f"{len(changes.entity_ids):,} Entities reached")

# Written as ordinary updates, since a saved Change Set holds one item
# per Entity and these are a series per record.
report = changes.apply()
print(f"{report['succeeded']:,} readings saved")

Things worth knowing

The outline can never be finer than a texel. Areas are traced off the raster, so the shape you query with steps around whole cells. On a coarse texture a texel can be hundreds of metres across, which is much larger than a building or a parcel.

Values are quantised to 8 bits over the archive's own range. A frame that reaches the top of that range is saturated rather than measured, and two thresholds closer together than one step select the same texels.

A texture is not a depth map unless the attribute is a depth. A water surface elevation is measured against a datum, so a negative value is a real reading rather than dry ground. Checktexture.attribute before deciding what a number means.

Repeated readings on one Entity need allow_multi_same_entity. A Change Set built without it merges edits per Entity, so a series of timestamped readings would collapse to the last one. A builder created with it can only be applied, not saved as a Change Set record, because a record holds one item per Entity.