DataLab secondary queries

A secondary query answers "which of these relate to those". Build a selection per Entity Type, pair them with secondary, and add a criterion comparing each candidate against the primary it is being tested against.

The selection you pass in becomes the secondary, and the secondary is what comes back. Swapping which Type is primary and which is secondary changes the answer, not just the wording.

Basics

Primaries are compared in batches, since the join is done per primary. batch sets how many go into each request.

import bruce_models as BM

buildings = BM.DataLabBuilder(bruce).entity_type(BUILDING_TYPE_ID)
sensors = BM.DataLabBuilder(bruce).entity_type(SENSOR_TYPE_ID)

# Sensors that sit inside a building.
inside = buildings.secondary(sensors, batch=50).within_primary()

# Or ones that merely touch or overlap it.
touching = buildings.secondary(sensors).intersecting_primary()

print(f"{inside.count():,} matches across every primary")
print(f"{inside.count_for(BUILDING_ID):,} in one building")

Reading the matches

Read the matches grouped by the primary they belong to, or on their own when it does not matter which primary produced them.

def on_group(primary_id, secondaries):
    print(f"{primary_id}: {len(secondaries)} matches")

matched = inside.each_group(on_group)
print(matched["primaries"], matched["matches"])

# The same thing as an iterator.
for primary_id, secondaries in inside.groups():
    pass

# Just the matches, without saying which primary they belong to.
for entity in inside.entities():
    pass

A primary that matched nothing is not reported, so the primary count here can be lower than the number of primaries in the selection. An Entity inside two overlapping primaries is answered once per primary, so walking the matches on their own can repeat it.

Criteria against the primary

Spatial criteria are the common case, but any criterion can compare against the primary by using a Ref as the value. Ref("path", on=Ref.PRIMARY) reads that attribute off the matching primary row, which only means anything in a secondary selection.

# Sensors whose owner matches the building's owner.
same_owner = (buildings.secondary(sensors)
              .equals("owner", BM.Ref("owner", on=BM.Ref.PRIMARY)))

# Whether the pairing is tied to the primary at all.
print(inside.is_related())

Action: Set attribute value

Runs against every matched secondary. The value can be a Ref reading the primary, which is how you copy something across the join without moving the records over the network.

# Stamp a constant onto every match.
inside.run_action_set_attribute("inside_building", True, attribute_type="boolean")

# Or copy the building's name onto each sensor it contains.
inside.run_action_set_attribute("building_name", BM.Ref("name", on=BM.Ref.PRIMARY))

Action: Save match count

Writes how many secondaries each primary matched onto the primary itself.

inside.run_action_save_secondary_count("sensor_count", skip_self_match=True)

Action: Relationships

Creates or removes a relationship between each primary and the secondaries it matched, following a Relationship Mapping.

inside.run_action_relate(RELATIONSHIP_MAPPING_ID, copy_location=True)

inside.run_action_remove_relationships(RELATIONSHIP_MAPPING_ID)

Things worth knowing

Actions here answer with a Pending Action, the same as an ordinary DataLab action. Read the ID with DataLabBuilder.action_id and wait with PendingAction.on_completion.

Batch size is a cost, not a limit. It sets how many primaries are joined per request, so a large batch means fewer, heavier requests. The default suits most selections.

Reading before writing. Queue edits during the walk and apply them afterwards through a ChangeSetBuilder, so the join cannot shift underneath the paging when the change would stop a record matching. The overlapping Entities FAQ works through a full example.