DataLab queries

DataLabBuilder builds the same queries the API reference documents, without hand-writing the request body. Every criterion returns the builder, so they chain, and nothing is requested until you read the selection.

Criteria combine with AND by default. Pass logic="OR" to any of them to change that, and use the grouping helpers below when precedence matters.

Basic query / Entity Type criteria

Most queries start with an Entity Type. Pass include_children to take its sub-types too.

import bruce_models as BM

query = BM.DataLabBuilder(bruce, page_size=1000).entity_type(ENTITY_TYPE_ID)

# Including sub-types.
query = BM.DataLabBuilder(bruce).entity_type(ENTITY_TYPE_ID, include_children=True)

Criteria: Attributes

Attributes address by slash separated path. The named helpers cover the common operators, and where takes any operator the API accepts.

query = (BM.DataLabBuilder(bruce)
         .entity_type(ENTITY_TYPE_ID)
         .equals("status", "Active")
         .greater_than("height", 10)
         .between("installed", "2020-01-01", "2024-12-31")
         .starts_with("name", "Pump")
         .is_in("region", ["North", "South"])
         .is_not_null("serial"))

# Anything the named helpers do not cover.
query = query.where("condition", "NOTEQUAL", "Retired")

# Comparing two attributes on the same Entity rather than against a value.
query = query.where_attribute("actual", "GREATERTHAN", "expected")

Criteria: Tags

# Entities carrying every one of these Tags.
query = query.where_tagged([405, 406])

# Or any one of them.
query = query.where_tagged([405, 406], mode="any")

Criteria: User

query = query.where_created_by(USER_ID)

# By when rather than by whom. UpdatedTime is the other column.
query = query.where_created("GREATERTHAN", "2024-01-01")
query = query.where_created("BETWEEN", "2024-01-01", "2024-06-30",
                            column="UpdatedTime")

Criteria: LOD

# Entities that have a Level of Detail.
query = query.where_lod()

# Narrowed to a kind, or to a level.
query = query.where_lod(lod_type="MESH")
query = query.where_lod(level=2, level_operator="GREATERTHAN")

Criteria: Assembly

# The roots of assemblies.
query = query.where_assembly_root()

# Everything under a root, or everything above a child.
query = query.where_under_root(ROOT_ENTITY_ID)
query = query.where_above_child(CHILD_ENTITY_ID)

Criteria: Relations and attachments

# Entities that have any relation, or one of a type.
query = query.where_relation()
query = query.where_relation(relation_type=RELATION_TYPE_ID)

# Related to one specific Entity.
query = query.where_related_to(ENTITY_ID, direction="reverse")

# Entities carrying an attachment, or a comment.
query = query.where_attachment()
query = query.where_comment(contains="inspect")

Criteria: Geometry

The spatial criteria take a Bruce geometry, which GeometryBuilder constructs. A generated texture also answers with geometries, so an area of a simulation goes straight in here.

box = BM.GeometryBuilder.box(174.50, -37.10, 175.05, -36.60)

query = query.within(box)
query = query.intersecting(shape)
query = query.near(point, metres=500)

# Entities that do or do not have a location at all.
query = query.has_geometry()
query = query.missing_geometry()

Criteria: Reference to another DataLab query

# Joins this selection to a saved query on a shared attribute.
query = query.where_attribute_join(
    SAVED_QUERY_ID,
    input_attribute_path="asset_code",
    target_attribute_path="code"
)

Criteria: '()' groups

A group is a branch built from the same builder. group_any joins the branches with OR, group_all with AND, and the group as a whole joins the rest of the query by its own logic.

base = BM.DataLabBuilder(bruce).entity_type(ENTITY_TYPE_ID)

# status == "Active" AND (region == "North" OR region == "South")
query = base.group_any(
    base.branch().equals("region", "North"),
    base.branch().equals("region", "South")
).equals("status", "Active")

Reading the selection

Nothing is requested while the query is being built. These are the calls that go to the API, and each pages underneath so a large selection is never held in full.

# How many match, without transferring any of them.
total = query.count()

# A page at a time, with the position carried on the page.
def on_page(page):
    print(f"page {page.number}: {len(page)} records, {page.seen:,} so far")
    # Returning False stops the walk.

walked = query.each_page(on_page)
print(walked["pages"], walked["records"], walked["stopped_early"])

# A record at a time, without caring about page boundaries.
query.each(lambda entity: print(BM.Entity.get_value(entity, "Bruce/ID")))

# IDs only, which is the cheapest way to stream a large selection.
query.each_id(print)

# Iterators, for when a loop reads better than a callback.
for entity in query.entities():
    pass
for page in query.pages():
    pass

# Ordering matters for anything that pages.
ordered = query.order_by("name").expand("EntityType,Source")

# A Data Schema inferred from the matches, answered in one request.
schema = query.schema()

Things worth knowing

The builder is immutable. Every criterion returns a new builder rather than changing the one you called it on, so a base query can be branched into several without them affecting each other.

Order before you page. Paging an unordered selection can repeat or skip records if the underlying data changes while you walk it. snapshot_ids takes the whole ID list up front when that matters more than memory.

Ask for a count before a walk. count transfers nothing, so it is the cheap way to decide whether a selection is worth reading at all.