CWL Workflow (Six-Step Topology)
A CWL Workflow carrying the same six-step ETL topology as the Airflow and Argo fixtures, with dependencies expressed as step output references and a multi-source input resolved by pickValue. For cross-format DAG conversion tests.
cwlVersion: v1.2
class: Workflow
id: orders-etl
label: Orders ETL workflow
doc: |
Inert fixture carrying the same six-step topology as the Airflow, Argo, Graphviz and Mermaid
fixtures in this category, so a cross-format converter has one documented answer to check.
requirements:
SubworkflowFeatureRequirement: {}
inputs:
raw_extract:
type: File
window_hours:
type: int
default: 24
outputs:
notice:
type: File
outputSource: notify_owner/report
steps:
ingest_orders:
run: cwl-command-line-tool.cwl
in:
extract: raw_extract
out: [record_count]
validate_orders:
run: cwl-command-line-tool.cwl
in:
extract: ingest_orders/record_count
out: [record_count]
transform_orders:
run: cwl-command-line-tool.cwl
in:
extract: validate_orders/record_count
out: [record_count]
load_warehouse:
run: cwl-command-line-tool.cwl
in:
extract: transform_orders/record_count
out: [record_count]
refresh_dashboard:
run: cwl-command-line-tool.cwlSpecifications
- Cwl Version
- v1.2
- Class
- Workflow
- Steps
- 6
- Edges
- 6
- Has Pick Value
- true
- Topology
- ingest_orders, validate_orders, transform_orders, load_warehouse, refresh_dashboard, notify_owner
Testing contract
Expected to pass- Scenario
- Derive a workflow graph from CWL step input sources rather than from an explicit edge list
- Expected result
- Six steps and six edges resolve, and the final step's two-source input is recognised as a join requiring MultipleInputFeatureRequirement
What is a .cwl file?
CWL (Common Workflow Language) is an open standard for describing command-line analysis workflows portably. A .cwl document is YAML (or JSON) declaring a cwlVersion and a class of CommandLineTool, Workflow, or ExpressionTool, with typed inputs and outputs, input bindings that map values onto command-line arguments, and — for workflows — steps wired together by source references. Requirements and hints declare the container image and resources a step needs.
How to use this file
Use an example .cwl file to test workflow parsers, validators, and execution engines, verifying that step dependencies form the expected graph, that input bindings produce the documented command line, and that an unsupported cwlVersion is rejected.
How to use this file for testing
“CWL Workflow (Six-Step Topology)” is a deterministic Novus Examples fixture for Graph data, Data engineering, Conversion testing. Node/edge datasets in GraphML and GEXF (directed and undirected, with attributes and weights) — for testing network importers, layout tools, and graph converters.
Documented properties for this file: 6 edges. Compare results against paired or grouped companions on this page when present (clean↔damaged, searchable↔scanned, or format twins) so scores stay reproducible across runs.
Download the file once, keep the path stable in CI or local scripts, and treat the spec table as the contract: dimensions, seeds, field lists, and roles are intentional. Corrupt or invalid samples are labelled as such — expect parsers to fail loudly rather than silently accept them.
Pipeline and infrastructure fixtures are inert configuration: steps reference fictional images and scripts, and nothing here executes. Run your linter, schema validator, migrator, or policy engine against them, and expect the deprecated-syntax and intentionally invalid variants to be rejected.
Related files
- cwlCWL Scatter and Cross-ProductCWL scatter in both forms: a single-parameter scatter over an array input, and a two-parameter scatter with scatterMethod flat_crossproduct. The scatter dimensionality is what distinguishes correct CWL engines from approximate ones.
- pyAirflow DAG Definition (Operator Style)An Airflow DAG definition in the classic operator style, using only the no-op EmptyOperator so the file describes a topology and performs no work. Carries the six-task ETL graph shared across the Airflow JSON, Argo, Graphviz and Mermaid fixtures.
- pyAirflow TaskFlow API DAGA decorator-based Airflow TaskFlow DAG where dependencies are implied by function calls rather than >> operators — the shape static DAG extractors most often get wrong. Every task returns a literal, so nothing performs I/O.
- jsonAirflow-Shaped Serialized DAG (JSON)The serialized-DAG shape Airflow stores in its metadata database, as standalone JSON: per-task metadata with explicit downstream_task_ids. Carries the same six-task topology as the Python, Argo, Graphviz and Mermaid fixtures.
- yamlArgo Workflow DAG TemplateAn Argo Workflows DAG template carrying the same six-task ETL topology as the Airflow, Graphviz and Mermaid fixtures in this category, so a converter or visualiser can be scored against one known answer across four formats.
- yamlArgo Workflow Steps TemplateArgo's steps template, whose double-nested list is a genuine parser trap: the outer list is sequential and the inner list is parallel, so a reader that flattens it reports four sequential steps instead of three groups.
Generated by generation/pipelines.py. Free for any use, no attribution required — license.