Skip to content

Home/Data transformation & ETL

05 / Data transformation & ETL

Transform data.
Standardize formats.

Develop repeatable data transformations and ETL pipelines. Data is read from supported sources, mapped to a target model, validated and loaded into the required system.

Discuss your projectView project scope
ETLJSON / XML / CSVValidation
INTEGRATION DIAGRAM / EXAMPLESOURCE / TRANSFORM / TARGET
SOURCE / JSON

Different formats.

{
  "customer_id": "001042",
  "country": "at",
  "total": "21,90"
}
MAPPING / NORMALIZATION
TRANSFORM / RULES

customer_id→externalId:string
country→uppercase()
total→decimal(de-AT)
VALIDATE / TARGET SCHEMA
TARGET / JSON

Validated data types.

{
  "externalId": "001042",
  "countryCode": "AT",
  "total": 21.9
}

Leading zeros are preserved. Numbers and codes follow the target schema.

Starting point

The format is known. The meaning often is not.

Two CSV files may describe the same field differently. Dates can be ambiguous, decimal separators vary by locale and leading zeros in a product ID can be essential. We define the meaning of fields and document their conversion.

Applications

Bring formats into a shared model.

Automate imports and exports

Replace recurring manual conversions with a data pipeline for CSV, XML, JSON or supported database extracts.

Reporting and consolidation

Bring data from multiple systems into a consistent reporting model with documented lineage and update rules.

Partner-specific mapping

Map internal records to the schema required by a supplier, marketplace or platform. Validation takes place before delivery.

Project scope

Mapping.
Validation. Target contract.

Scope depends on your systems and the desired outcome. We agree on these points with you.

Invalid data needs visible handling.

Invalid records are isolated, corrected or rejected according to agreed rules. Missing details are not silently filled in. Ambiguities remain visible and can be resolved by the responsible business team.

  • Source analysis and a versioned target schema
  • Mappings for data types, IDs, dates, units and locale-specific formats
  • Required-field validation and rules for rejected records
  • Repeatable extraction, transformation and loading steps
  • Execution logs and traceable data lineage
  • Representative test data and acceptance criteria for subsequent changes

Service scope and responsibilities are agreed for each project. Our approach: acceptance and operations

Technical questions

Data types and mapping rules.

Data migrationData synchronization
How does ETL differ from data migration?
ETL describes extraction, transformation and loading, which can be performed regularly. Migration moves data into a new system, usually involving a cutover. An ETL pipeline may be used for this.
Can you convert CSV, XML and JSON?
Yes, where source and target schemas are defined. Preserving meaning, data types and relationships is essential. A file extension alone does not define a complete conversion rule.
How are invalid records handled?
We define validation rules and an exception path. Rejected records remain visible with enough context for correction. The project specifies whether an error stops the entire run or is handled individually.
Can mappings be changed later?
Mappings are versioned and tested with representative data. Changes can then be validated against the existing target schema before release.

Your next connection

What should your systems
achieve together?

You bring the requirements. We clarify the connections.

Discuss your project

Practical integration challenges

Technical topics in this area.

Data processes

Automate CSV imports

The file arrives every day. Someone corrects columns, uploads it and then checks error messages. DataSource turns this recurring task into a documented import with clear rules for incomplete and invalid data.

Data processes

JSON / XML data transformation

The data is technically readable but still rejected. Field names, types or structures do not match the target application. We develop traceable mappings that also handle empty values, edge cases and changes to the source format.

Data processes

Automate product data workflows

The ERP supplies master data, the PIM supplies text and a supplier provides technical attributes. Manual lists connect this information only temporarily. We create a defined data flow from each authoritative field to a validated channel export.