DaCe v2.0.0 Pre-Release (alpha 4)

What's Changed

• Symbolic serialization format changed to a type-safe representation, see #2366
• Bug fixes

Full Changelog: v2.0.0-alpha3...v2.0.0-alpha4

DaCe v2.0.0 Pre-Release (alpha 3)

What's Changed

• More documentation
• Minor bug fixes

Full Changelog: v2.0.0-alpha2...v2.0.0-alpha3

DaCe v2.0.0 Pre-Release (alpha 2)

This is a second pre-release version of DaCe, with the following two major changes:

• Machine Learning Integration for DaCe (Autodiff - ONNX - PyTorch) by @affifboudaoud in #2164
• Extract FPGA code generation to separate repository by @tbennun in #2252

For all changes, see the Full Changelog: v2.0.0-alpha1...v2.0.0-alpha2

DaCe v2.0.0 Pre-Release (alpha)

This version is a pre-release tracker for dace v2.0

v1.0.2

This release contains backports of further minor fixes.

What's Changed

• Fix typos (backport) by @romanc in #1918
• Fix typo by @romanc in #1945
• Fix: DDE removing read from access_set in read/write nodes by @romanc in #1955
• StateFusion misses read-write conflict due to early return by @FlorianDeconinck in #1954

Full Changelog: v1.0.1...v1.0.2

v1.0.1

This release contains backports of minor fixes following the release of v1.0.0.

Full Changelog: v1.0.0...v1.0.1

v1.0.0

We are happy to announce DaCe version 1.0!

It is a major release milestone, and we went over many of the known issues over the years to ensure that this is the most stable version we can release without making fundamental changes to the framework. The Stateful DataFlow multiGraph (SDFG) intermediate representation used in this version is faithful to the original paper, which was published in 2019.

On a fundamental level, this release is no different from a minor version release (this version could have been DaCe 0.17), so there are no breaking changes from v0.x.

We would like to thank everyone who contributed to DaCe over the years and helped reach this milestone! It would not have been possible without you.

Release Notes

In addition to many issues and bugfixes courtesy of @acalotoiu, @tim0s, @htorst, @tbennun, @phschaad, @BenWeber42, @philip-paul-mueller, @luigifusco, @ThrudPrimrose, @FlorianDeconinck, @pratyai, @edopao, @kotsaloscv, and @iBug, several new features for quality of life and future development were added.

New features introduced into the SDFG IR and builder API:

• Add GUIDs to SDFG elements and SDFG diff support (by @phschaad)
• Added can_be_applied_to() to Transformation API (by @philip-paul-mueller)
• SDFG.auto_optimize, SDFG.regenerate_code, and SDFG.as_schedule_tree are now easily accessible as API methods and fields

New Python frontend features

• You can now specify the storage location of expressions inline using the @ operator or type hints. Examples:
  • a = np.ones(M) @ dace.StorageType.CPU_ThreadLocal
  • b: dace.float64[M, N] @ dace.StorageType.GPU_Global = np.zeros(...)

New transformations

• WCRToAugAssign transformation (by @alexnick83)

New code generation features

• clang-format can now be configured to be called on generated code (by @ThrudPrimrose)

Experimental features

• Control flow (loop, conditional, named) regions (by @phschaad and @luca-patrignani). Stay tuned for more updates in the next development releases!

Other changes and bugfix highlights

• Support for SymPy 1.13 (by @BenWeber42)
• Rename misleading topological_sort to bfs_nodes by @BenWeber42 in #1590
• Add multidimensional maps to GPU docs by @tbennun in #1608
• Improve SDFG work-depth analysis and add SDFG simulated operational intensity analysis by @phschaad in #1607
• Scalar return values are now disallowed by @philip-paul-mueller in #1609
• Fixed RedundantArray's handling of "reshaping" Memlets by @philip-paul-mueller in #1603
• Loop Region Code Generation by @phschaad in #1597
• Bump certifi from 2023.7.22 to 2024.7.4 by @dependabot in #1614
• Fix incorrect input/output of nested dace programs by @phschaad in #1615
• Return correct state in nest_sdfg_subgraph by @tbennun in #1627
• Made TransientReuse Less Verbose by @philip-paul-mueller in #1622
• Improving the Usage of #pragma unroll by @philip-paul-mueller in #1621
• Added PatternNode to dace.transformation imports. by @philip-paul-mueller in #1618
• Implement user regions and function call regions by @luca-patrignani in #1623
• Add UUIDs to SDFG elements by @phschaad in #1631
• framecode: Fix missing BasicCFBlock argument by @iBug in #1630
• Specified behaviour of Subset.covers() for different dimensionality by @philip-paul-mueller in #1637
• More robust loop detection by @tbennun in #1646
• Fix missed exploration of edges in constant propagation by @luigifusco in #1635
• Fix infinite loop with control flow blocks by @tbennun in #1634
• Print out exception on parsing fail early by @FlorianDeconinck in #1651
• Reworked Optional Serializing by @philip-paul-mueller in #1647
• Modified SetProperty by @philip-paul-mueller in #1653
• Made CompiledSDFG in the main namespace available. by @philip-paul-mueller in #1567
• SDFG Diff Tool by @phschaad in #1632
• Made the SDFGState.add_mapped_tasklet() more convenient by @philip-paul-mueller in #1655
• Maps With Zero Parameters by @philip-paul-mueller in #1649
• Bug in constant propagation with multiple constants by @tbennun in #1658
• Fixed PruneConnectors by @philip-paul-mueller in #1660
• Fix array indirection to memlet subset promotion by @BenWeber42 in #1406
• Renamed graph.bfs_edges to edge_bfs by @BenWeber42 in #1604
• Inter-state edge assignment race test by @tbennun in #1672
• Fix race conditions in Constant Propagation and Reference-To-View by @tbennun in #1679
• Improve memlet label and string initialization by @tbennun in #1680
• Control Flow Raising by @phschaad in #1657
• Updated InlineMultistateSDFG by @philip-paul-mueller in #1689
• Extend TrivialTaskletElimination for map scope by @edopao in #1650
• Fix to Read and Write Sets by @philip-paul-mueller in #1678
• Make is_empty() and propagate_subset() not unnecessarily rely on the src and dst by @pratyai in #1699
• fix(codegen/prettycode): Use base_indentation as intended by @iBug in #1697
• Warn on potential data races by @phschaad in #1712
• Python frontend stability and inline storage specification by @tbennun in #1711
• infer_symbols_from_datadescriptor : modification to infer offset by @kotsaloscv in #1525
• Add CFG to generate_scope in tutorials by @ThrudPrimrose in #1706
• Better CopyToMap by @philip-paul-mueller in #1675
• More NumPy operation implementations by @tbennun in #1498
• Fix jupyter's version of SDFV by @phschaad in #1714
• Fix broken codegen tutorial by @romanc in #1720
• CI: Update checkout and setup-python actions by @romanc in #1718
• Bump version and update dependencies by @tbennun in #1722
• Various Cutout Fixes by @phschaad in #1662
• Various stability improvements and convenience APIs by @tbennun in #1724
• Rename FORTRAN frontend tests by @pratyai in #1729
• Add back clang-format support by @ThrudPrimrose in #1732
• Fix problem with struct reads on interstate edges by @phschaad in #1512
• Quality of life: Improved error messages by @romanc in #1731
• Cherry-picked a handful of intrinsic related commits out of multi_sdfg branch. by @pratyai in #1728
• Used valid FORTRAN test program for a couple frontend tests + Made floatlit2string() convert the FORTRAN real literal strings into python floats. by @pratyai in #1733
• Fix pure reduce expansion for squeezed output memlets. by @pratyai in #1709
• Make the import of typing.Literal portable between python versions 3.7 and 3.12 by @pratyai in #1700
• Fix type inference and code generation for typeclasses and numpy types by @tbennun in #1725
• SDFG API additions for version 1.0 by @tbennun in #1740
• Replace another FORTRAN test program with gfortran -Wall certified test program. by @pratyai in #1736
• Unskip unit tests and provide reasons for skipped tests by @tbennun in #1742
• Fix OpenMP dynamic loop bounds that use persistent memory by @tbennun in #1746
• Fixes for SDFGState._read_and_write_sets() by @philip-paul-mueller in #1747
• Fix temporary transient counter during Python parsing of nested calls by @tbennun in #1745
• Fix pystr_to_symbolic not correctly interpreting constants as boolean values in boolean comparisons by @phschaad in #1756
• Fixed dace::math::pi and dace::math::nan on GPU by @philip-paul-mueller in #1759
• Make scalar to symbol promotion robust to node order in state by @tbennun in #1766

Full Changelog: v0.16.1...v1.0.0

v1.0.0rc1

We are happy to announce the first release candidate of DaCe version 1.0!

This version uses the SDFG intermediate representation as published in the original Stateful Dataflow Multigraphs paper, which has been stable for quite some time.

On a fundamental level, this release is no different from a minor version release (this version could have been DaCe 0.17). However, with this release we would like to emphasize stability rather than new features.

If you are using DaCe and have a critical or blocking issue that makes it unstable, please create an issue and refer to it in the release discussion, so that we can add it to our release plan. Thank you for using DaCe!

Release Notes

New features:

• Add GUIDs to SDFG elements and SDFG diff support (by @phschaad)
• Added can_be_applied_to() to Transformation API (by @philip-paul-mueller)
• Support SymPy 1.13 (by @BenWeber42)
• New WCRToAugAssign transformation (by @alexnick83)
• (Experimental) Control flow (loop, conditional, named) regions (by @phschaad and @luca-patrignani). Stay tuned for more updates in the next development releases!

Bugfixes:

• Inter-state edge assignment race condition test in validation (by @tbennun)
• Improve memlet label and string initialization (by @tbennun, @philip-paul-mueller)
• Minor updates to documentation and internal APIs (by @tbennun, @phschaad, @philip-paul-mueller, @BenWeber42)
• Minor fixes to the following transformations and passes: RedundantArray, TransientReuse, DetectLoop, ConstantPropagation, PruneConnectors (by @philip-paul-mueller, @tbennun, @luigifusco)
• Minor frontend improvements (by @FlorianDeconinck, @BenWeber42)
• Minor improvements to the code generator (by @iBug, @philip-paul-mueller)

See Full Changelog: v0.16.1...v1.0.0rc1

New Contributors

• @iBug made their first contribution in #1630
• @luigifusco made their first contribution in #1635

v0.16.1

What's Changed

The main purpose of this release is to require NumPy < 2 for DaCe, since NumPy 2.0.0 contains breaking changes which aren't compatible with DaCe currently.

Recently, NumPy 2.0.0 has been released: https://numpy.org/news/#numpy-200-released

The release comes with documented breaking changes. Unfortunately, DaCe is currently not compatible with these changes. This also affects the recent 0.16 release of DaCe. Hence, we adjust our dependency requirements to use NumPy < 2 as a temporary work-around in this PR:

Fix numpy version to < 2.0 by @phschaad in #1601

Long term, we are tracking adding support for NumPy 2 in DaCe in this issue: #1602

Fix constant propagation failing due to invalid topological sort by @phschaad in #1589

This changeset has also landed in DaCe's development branch earlier. It fixes an issue where the ConstantPropagation pass can fail for certain graph structures.

Full Changelog: v0.16...v0.16.1

v0.16

What's Changed

CI/CD pipeline for NOAA & NASA weather and climate model by @FlorianDeconinck & @BenWeber42 in #1460, #1478 & #1575

Our collaborators NOAA & NASA have successfully used DaCe as an optimization framework and back-end for some of the components of their climate and weather model. Particularly, the FV3 dycore and GFS physics parametrization have been ported to a combination of GT4Py Python DSL and DaCe. DaCe is used within their stack as a stencil backend and as a full-program optimizer integrating stencils and glue-code together.

With this CI/CD pipeline, we run various checks for those components on every change to DaCe. This is an important step for DaCe to ensure stability for real-world applications that utilize DaCe. We are very grateful for this contribution and the collaboration with NOAA & NASA.

Changed default of serialize_all_fields to False by @BenWeber42 in #1564

This feature was already implemented in the previous 0.15.1 release in #1452, but not enabled by default. In this release, we are changing the default so that only fields with non-default values are serialized. This generally leads to a reduction in file size for SDFGs.

Since each DaCe version stores the default values of each field, it is still possible to recover these missing values. Default values should rarely change across different DaCe versions. Nevertheless, we want to caution users & developers when using SDFG files with different DaCe versions.

Analysis passes for access range analysis by @tbennun in #1484

Adds two analysis passes to help with analyzing data access sets: access ranges and Reference sources. To enable constructing sets of memlets, this PR also reintroduces data descriptor names to memlet hashes.

Reference-to-View pass and comprehensive reference test suite by @tbennun in #1485

Implements a reference-to-view pass (converting references to views if they are only set to one particular subset). Also improves the simplify pipeline in the presence of Reference data descriptors and adds multiple tests that use references.

Ndarray strides by @alexnick83 in #1506

The PR adds support for custom strides to dace.ndarray. Furthermore, the stride unit is number of elements, in contrast to NumPy/CuPy, where it is number of bytes. Custom strides are not supported for numpy.ndarray and cupy.ndarray.

Structure Support to NestedSDFGs and Python Frontend by @alexnick83 in #1366

Adds basic support for nested data (Structures) to the Python frontend. It also resolves issues with the use of Structures in nested SDFG scopes (mostly code generation).

Generalize StructArrays to ContainerArrays and refactor View class structure by @tbennun in #1504

This PR enables the use of an array data descriptor that contains a nested data descriptor (e.g., ContainerArray of Arrays). Its contents can then be viewed normally with View or StructureView.
With this, concepts such as jagged arrays are natively supported in DaCe (see test for example).
Also adds support for using ctypes pointers and arrays as arguments to SDFGs.

This PR also refactors the notion of views to a View interface, and provides views to arrays, structures, and container arrays. It also adds a syntactic-sugar/helper API to define a view of an existing data descriptor.

Add support for distributed compilation in DaceProgram by @kotsaloscv in #1551 & #1555

Adds configurable support for distributed compilation (MPI) to the Python front-end (via mpi4py). Distributed compilation can be enabled with the distributed_compilation parameter in the dace.program decorator.

Fixes and other improvements:

• Remove unused deps by @jack-mcivor in #1459
• Small fix for debuginfo that can be None by @kotsaloscv in #1469
• Make dynamic map range docs more explicit by @tbennun in #1474
• Added nan to the DaCe math namespace by @philip-paul-mueller in #1437
• Fix for floordiv on GPU target by @edopao in #1471
• Add merge_group to CI for merge queues by @tbennun in #1482
• Fix SymPy dependency (again) by @tbennun in #1483
• Fix for CUDA codegen by @edopao in #1442
• Complete coverage for reference-to-view pass by @tbennun in #1488
• CMakeLists.txt Improvements for CUDA by @kylosus in #1337
• Faster Call for CompiledSDFG by @philip-paul-mueller in #1467
• Evaluate dtype_to_typeclass at use time by @tbennun in #1494
• Fix redefinition of interstate edge type in code generator by @tbennun in #1490
• CuPy fixes and special cases for HIP by @tbennun in #1492
• CI Update by @tim0s in #1502
• FPGA CI Update by @tim0s in #1508
• Bump jinja2 from 3.1.2 to 3.1.3 by @dependabot in #1503
• Jupyter fix by @phschaad in #1489
• Modernize HIP CMake commands, fix corner cases by @tbennun in #1518
• Remove the long-deprecated symbol.get/set methods by @tbennun in #1523
• Support output indirection in numpy frontend by @tbennun in #1509
• Fix for const references by @alexnick83 in #1522
• DeadDataFlowElimination will add type hint when removing a connector by @luca-patrignani in #1499
• Fixed an issue in the Memlet duplication verification. by @philip-paul-mueller in #1526
• Refactor SDFG List to CFG List by @phschaad in #1511
• Dependency Edge Hotfix by @Berke-Ates in #1513
• Remove Property.from_string and Property.to_string by @luca-patrignani in #1529
• Fixed the {in,out}_edges() function of the DiGraph class. by @philip-paul-mueller in #1527
• Fixes for structures nested in (nested) struct-arrays by @alexnick83 in #1534
• Updated and fixed the MapExpansion transformation. by @philip-paul-mueller in #1532
• Updated and fixed the MapDimShuffle tranformation. by @philip-paul-mueller in #1531
• Use State Fissioning to Generalize Transformations by @lukastruemper in #1462
• Fixed edge consolidation by @philip-paul-mueller in #1546
• Fix Profiler + Minor improvements by @JanKleine in #1548
• Add dtype for numpy.uintp which is compatible with C uintptr_t by @kotsaloscv in #1544
• Fix bug in map_fusion transformation by @edopao in #1553
• Updated the add_state_{after, before}() function. by @philip-paul-mueller in #1556
• Bump idna from 3.4 to 3.7 by @dependabot in #1557
• Fix infinite loops in memlet path when a scope cycle is added by @tbennun in #1559
• Adds support for ArrayView to the Python Frontend by @alexnick83 in #1565
• It is now possible to suppress output in view() by @philip-paul-mueller in #1566
• Bump jinja2 from 3.1.3 to 3.1.4 by @dependabot in #1569
• Correction in the docstring of the SDFG class's init method by @alexnick83 in #1571
• Fix Subscript literal evaluation for List by @FlorianDeconinck in #1570
• SDFG.save() now performs tilde expansion. by @philip-paul-mueller in #1578
• Control Flow Block Constraints by @phschaad in #1476
• Updated SDFV and Corresponding HTML Template by @phschaad in #1580
• Changed Xilinx C++11 flag to C++14 by @BenWeber42 in #1585
• Made dace::math::pow forward to std::pow more generic by @Berke-Ates @philip-paul-mueller @phschaad @BenWeber42 in #1580

New Contributors

• @jack-mcivor made their first contribution in #1459
• @kylosus made their first contribution in #1337
• @luca-patrignani made their first contribution in #1499

Full Changelog: v0.15.1...v0.16