Learning › Transformer
Edit this page

iModel Transformation and Data Exchange

The @itwin/imodel-transformer package provides some classes that implement Extract, Transform, and Load (ETL) functionality:

The above classes contain the lower-level functionality required to implement transformation and data exchange services. These classes should be considered a framework and not confused with the actual packaged and deployed services that use the framework.

See Error handling in imodel-transformer for the package's error ownership rules and guidance for handling identified transformer errors.

IModelExporter

The IModelExporter and IModelExportHandler base classes are used when the source data in an ETL workflow is contained within an iModel.

While it is possible to export data from an iModel using the standard IModelDb API, the IModelExporter and IModelExportHandler base classes offer the following capabilities:

Incremental exports

IModelExporter.exportChanges exports changes collected from the selected changesets or supplied through ExportChangesOptions.changedInstanceIds. For inserted and updated elements, the exporter finds each changed element and the parents needed to reach it. It visits only those paths instead of checking every element in each changed model. Deleted element IDs are passed together to IModelExportHandler.onDeleteElements.

Changed elements are visited parent before child. The exporter passes through unchanged ancestors without exporting them. When it reaches a changed element, it calls shouldExportElement once for each unchanged ancestor that has not been checked yet, starting at the top. If an ancestor is rejected, onSkipElement is called for it and its descendants are skipped, as in a full export. A changed element rejected by shouldExportElement also causes its descendants to be skipped. An element excluded by ID triggers onSkipElement even when unchanged, and its descendants are skipped. Because unchanged ancestors are filtered only when a changed element is reached, an excluded element can receive onSkipElement before a rejected ancestor above it does. Configure exclusions before starting an export operation; changing them while an export is in progress is unsupported.

A custom IModelExporter subclass that overrides exportElement or exportChildElements uses the previous per-element traversal. This preserves calls to those overrides, but the subclass does not receive the faster changed-element traversal.

Learn how IModelExporter filters, batches, and exports ElementAspects in the Processing ElementAspects guide. The guide also covers change handling and the owner metadata required for custom deleted aspect changes.

Below is an example of using IModelExporter and IModelExportHandler to export all Code values from an iModel:

import { Code, CodeSpec } from "@itwin/core-common"; import { Element, IModelJsFs as fs, IModelDb, SnapshotDb } from "@itwin/core-backend"; process.env.TRANSFORMER_NO_STRICT_DEP_CHECK = "1"; // allow this monorepo's dev versions of core libs in transformer import { IModelExporter, IModelExportHandler } from "@itwin/imodel-transformer"; /** CodeExporter creates a CSV output file containing all Codes from the specified iModel. */ class CodeExporter extends IModelExportHandler { public outputFileName: string; /** Initiate the export of codes. */ public static async exportCodes(iModelDb: IModelDb, outputFileName: string): Promise<void> { const exporter = new IModelExporter(iModelDb); const exportHandler = new CodeExporter(outputFileName); exporter.registerHandler(exportHandler); await exporter.exportAll(); } /** Construct a new CodeExporter */ private constructor(outputFileName: string) { super(); this.outputFileName = outputFileName; } /** Override of IModelExportHandler.onExportElement that outputs a line of a CSV file when the Element has a Code. */ public override onExportElement(element: Element, isUpdate: boolean | undefined): void { if (!Code.isEmpty(element.code)) { // only output when Element has a Code const codeSpec: CodeSpec = element.iModel.codeSpecs.getById(element.code.spec); fs.appendFileSync(this.outputFileName, `${element.id}, ${codeSpec.name}, ${element.code.value}\n`); } super.onExportElement(element, isUpdate); } }

IModelImporter

The IModelImporter base class is used when the target in an ETL workflow is an iModel.

While it is possible to import data into an iModel using the standard IModelDb API, the IModelImporter class offers the following capabilities:

Incremental element deletion callbacks

IModelExporter.exportChanges passes all deleted source element IDs to one IModelExportHandler.onDeleteElements callback. Custom export handlers and IModelTransformer subclasses must use this callback because there is no singular deletion callback.

IModelTransformer maps the source IDs before IModelImporter deletes the target elements as one batch. The importer keeps elements that something outside the batch still uses, and deletes the rest. A custom importer can inspect, count, or audit the requested elements by overriding onDeleteElements(elementIds: ReadonlySet<Id64String>). Finish any work that needs the elements to exist before calling super.onDeleteElements(), which does the deletion. The public deleteElement() method goes through the same hook with a one-element set.

See Deleting elements for what a deletion removes, which elements it keeps, and how to handle deletion errors.

IModelImportOptions.autoExtendProjectExtents

IModelImportOptions.autoExtendProjectExtents provides different options for handling the projectExtents of the target iModel. See the following for more information about projectExtents:

autoExtendProjectExtents = false

This setting should be used when the target iModel projectExtents are being set directly. For example:

  • If the target iModel projectExtents will be the same as the source iModel, then it can just be copied over.
  • If the target iModel projectExtents are known ahead of time, then it can be directly set.

autoExtendProjectExtents = true

This setting causes the target iModel projectExtents to be extended to include the range box of every element that is imported. This includes potential outliers (one/few elements that are located far away from the main collection of elements). Outliers tend to suggest a user modeling problem or a Connector problem, but it is difficult for a program to know for sure what the intent was. This setting assumes every Element is there for a reason.

autoExtendProjectExtents = { excludeOutliers: true }

This setting causes the projectExtents to be extended to include the range box of every element that is imported except for outliers. In this case, outliers are assumed to be a mistake and IModelImporter tries to detect them using fuzzy logic from the IModelDb.computeProjectExtents method in order to exclude them from the projectExtents calculation.

Either of the non-false autoExtendProjectExtents options are useful for consolidation cases or filtering cases where the target iModel will have different optimal projectExtents than the source iModel(s).

IModelElementCloneContext

The IModelElementCloneContext class provides the core cloning capability required for iModel transformation. It also maintains the sourceId --> targetId mapping which is required to successfully clone Entity instances from the source iModel into the target iModel. iModel entities are highly related to each other. Therefore, cloning an entity means copying a graph of objects and remapping their source references (Ids) to other target entities.

IModelTransformer

The IModelTransformer base class is used when the source and target in an ETL workflow are both/different iModels and some sort of data transformation is needed in the middle. An instance of IModelTransformer holds instances of IModelExporter, IModelImporter, and IModelElementCloneContext. This means that customization is possible at the export stage, the transformation stage, and the import stage of the overall ETL process.

Potential transformations include:

  • Cloning - copying an entity from the source and remapping its internal Ids for the target
  • Filtering - excluding data from the target that is contained in the source
  • Augmenting - generating data during transformation for the target that is not part of the source
  • Schema Mapping - mapping classes and properties to a new schema during transformation
  • Change Squashing - each iModel has its own change ledger, so multiple changesets from the source could be squashed into a single changeset to the target

Filtering during change processing

The export filter is IModelExporter.shouldExportElement. It applies the exporter's exclusions, such as excludeElement, excludeElementClass, and excludeElementsInCategory, then calls the handler's shouldExportElement, which IModelTransformer subclasses override. When the filter rejects an element, its descendants are skipped during both full and change processing.

During change processing, a changed element can require an unchanged element that has no mapping in the target, such as its parent or category. If the filter rejects an unchanged parent, the changed element is skipped along with the parent's other descendants, and no error is thrown. In every other case, if the required element passes the export filter, the transformer looks for it in the target by FederationGuid and then by Code. Change processing does not insert unchanged elements, so the transformer throws ITwinError with key DependencyMappingMissing when the required element is rejected or can't be found. This happens in two cases:

  • The filter rejects a required element other than the parent, such as the category of an element that the filter accepts. Accept the required element, or reject every element that requires it.
  • The required element passes the filter but is missing from the target. This happens when the element was deleted from the target, or when the filter starts accepting elements that it rejected in an earlier run, for example after the categories or views it filters by change. To insert the element, override addCustomChanges and add it with ChangedInstanceIds.addCustomElementChange.

A changed required element is exported before the element that requires it. If it is still not mapped afterwards, because the filter rejects it or one of its ancestors, the transformer throws the same error. Accept the required element and its ancestors, or reject the element that requires it.

Processing a subset

IModelTransformer.process() finalizes its importer automatically. The subset methods processElement, processChildElements, processModel, processModelContents, processRelationships, and processSubject are composable and do not finalize after each call. After the last subset operation, call IModelImporter.finalize before saving target changes:

try { await transformer.processElement(sourceElementId); await transformer.processRelationships(relationshipClassName); transformer.importer.finalize(); targetEditTxn.saveChanges(); } finally { transformer.dispose(); }

See schema processing for schema selection, dynamic schema unions, conflict handling, and the schema-processing workflow.

Logging

With batch processes like iModel transformation and data exchange, logging is often the only way to figure out what is actually happening. The following logger categories are provided for use with the Logger:

Implementing iModel branching workflows through the transformer

Using the transformer as a framework, it is possible to implement the necessary operations for a branching workflow, where branch iModels from a master iModel form a tree-like change history as branches synchronize at distinct points to transfer data to the master. The implementation with examples and terminology by the transformer is elaborated in the branching iModels article.

ETL examples

More samples of using the transformer to export iModels or subsets of iModels to different formats can be found in the ETL (Extract-Transform-Load) samples repository

Last Updated: 01 October, 2026