The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Data-Overlay-0.54
River stage zero No dependents
/ Data::Overlay

NAME

Data::Overlay - merge/overlay data with composable changes

VERSION

Data::Overlay version 0.54 - ALPHA, no compatibility promises, seriously

SYNOPSIS

#!perl -s #line 31

    use strict; use warnings;
    use Data::Overlay qw(overlay compose);
    use Data::Dumper;
    $Data::Dumper::Sortkeys = 1;

    my $data_structure = {
        a => 123,
        b => {
            w => [ "some", "content" ],
            x => "hello",
            y => \"world",
        },
        c => [ 4, 5, 6],
        d => { da => [], db => undef, dc => qr/abc/ },
    };

    my %changes = (
        f  => 0,                    # add top level key
        a  => '1, 2, 3',            # overwrite key
        b  => { z => '!'  },        # nested operation
        c  => { '=unshift' => 3.5 },# prepend array
        c  => { '=push' => 7 },     # append array
        d  => { da => [ "DA" ],     # replace w/ differing type
                db => {
                    '=defor' => 123,  # only update if undef
                },
              },
    );

    # apply %changes to $data_structure (read-only ok),
    # returning a new data structure sharing unchanged data with the old
    my $new_data_structure = overlay($data_structure, \%changes);

    # Note sharing shown by Dumper
    print Dumper($data_structure, \%changes, $new_data_structure);

__END__

Running SYNOPSIS

Once Data::Overlay is installed, you can run it with either:

    perl -x -MData::Overlay `pmpath Data::Overlay`

    perl -x -MData::Overlay \
        `perl -MData::Overlay -le 'print $INC{"Data/Overlay.pm"}'`

DESCRIPTION

Basic Idea

The overlay functions can be used to apply a group of changes (also called an overlay) to a data structure, non-destructively, returning a shallow-ish copy with the changes applied. "Shallow-ish" meaning shallow copies at each level along the path of the deeper changes.

  $result = overlay($original, $overlay);

The algorithm walks the overlay structure, either taking values from it, or when nothing has changed, retaining the values of the original data structure. This means that the only the overlay fully traversed.

When the overlay is doesn't use any special Data::Overlay keys (ones starting with "="), then the result will be the merger of the original and the overlay, with the overlay taking precedence. In particular, only hashes will really be merged, somewhat like %new = (%defaults, %options), but recursively. This means that array refs, scalars, code, etc. will be replace whatever is in the original, regardless of the original type (so an array in the overlay will take precedence over an array, hash or scalar in the original). That's why it's not called Data::Underlay.

Any different merging behaviour needs to be marked with special keys in the overlay called "actions". These start with an "=" sign. (Double it in the overlay to have an actual leading "=" in the result). The actions are described below, but they combine the original and overlay in various ways, pushing/unshifting arrays, only overwriting false or undefined, up to providing ability to write your own combining callback.

Reference Sharing

Data::Overlay tries to minimize copying and so will try to share references between the new structure and the old structure and overlay. This means that changing any of these is likely to produce unintended consequences. This sharing is only practical when the data structures are either read-only or cloned immediately after overlaying (Cloner not included).

GLOSSARY

  overlay
       v : put something on top of something else
       n : layer put on top of something else

  $ds, $old_ds, $new_ds - arbitrary, nested Perl data-structures

INTERFACE

overlay

    $new_ds = overlay($old_ds, $overlay);

Apply an overlay to $old_ds, returning $new_ds as the result.

$old_ds is unchanged. $new_ds may share references to parts of $old_ds and $overlay (see "Reference Sharing"). If this isn't desired then clone $new_ds.

overlay_all

    $new_ds = overlay_all($old_ds, $overlay1, $overlay2, ...);

Apply several overlays to $old_ds, returning $new_ds as the result. They are logically applied left to right, that is $overlay1, then overlay2, etc. (Internally compose is used, see next)

compose

    $combined_overlay = compose($overlay1, $overlay2, ..);

Produce an overlay that has the combined effect of applying $overlay1 then $overlay2, etc.

combine_with

    $combiner_sub = combine_with { $a && $b };
    my $conf = { action_map => {
                    and => $combiner_sub
                 } };
    my $new = overlay($this, $that, $conf);

Utility to build simple overlay actions. For example, the included "or" is just:

    combine_with { $a || $b};

$a is the old_ds parameter, $b is the overlay parameter. @_ contains the rest of the arguments.

Actions

config
overwrite

"Overwrite" the old location with the value of the overwrite key. This is the default action for non-HASH overlay elements. (The actual action used in this implicit case and also the explicit case is $conf->{action_map}{overwrite}, so it can be overriden).

You can also use overwrite explicitly, to overlay an empty hash (which would otherwise be treated as a keyless, no-op overlay):

  overlay(undef, {});                   # -> undef
  overlay(undef, {'=overwrite'=>{}});   # -> {}
defaults
delete
defor

Defined or (//) is used to combine the original and overlay

or
push
unshift
pop
shift
run
      '=run' => {
        code => sub {
            my ($old_ds, @args) = @_;
            ...;
            return $result;
        },
        args => [ ... ], # optional argument list
      }
foreach

Apply one overlay to all elements of an array or values of a hash (or just a scalar). Often useful with =run if the overlay is a function of the original value.

seq

Apply in sequence an array of overlays.

DIAGNOSTICS

Error message here, perhaps with %s placeholders

[Description of error here]

Another error message here

[Description of error here]

[Et cetera, et cetera]

Cookbook and Serving Suggestions

Some made up use-cases.

Configuration Data Merging

 overlay_all($defaults, $host_conf, $app_conf, $user_conf, $cmd_line_conf);

List of Undoable Edits

Use the memory sharing to keep a sequence of persistent data structures. "Persistent" in the functional programming sense, you can access (read-only) old and new versions.

Circular References in Overlays

There is no protection against reference cycles in overlays.

Devel::Cycle or Data::Structure::Util may help.

Unsharing Data with Clone

If you don't want any sharing of data between the result and source or overlay, then use a clone. Either Storable's dclone or Clone

    $new_clone = dclone(overlay($old, $overlay));

Escaping "=" Keys

Rmap

Writing Your Own Callbacks

Note that while most of the names of core actions are based on mutating perl functions, their implementation is careful to do shallow copies.

Readonly for Testing

The Readonly module is useful for testing that nothing is changing data that is supposed to be Readonly.

Sharing State in Callbacks

Shared lexical variables.

Where Are The Objects?

This is a bit of an experiment in using data immutability, persistence and sharing instead of using OO conventions to manage changing state. (This approach doesn't hit all of the OO targets, but Data::Overlay's subset may be useful).

Blessed references are treated as opaque object by default (not overlaid). (Encapsulation was ignored in developer release 0.53 and earlier).

DEPENDENCIES

List::MoreUtils, Sub::Name

BUGS AND LIMITATIONS

I'm happy to hear suggestions, please email them or use RT in addition to using cpan ratings or annocpan (I'll notice them faster).

No bugs have been reported.

Please report any bugs or feature requests to bug-data-edit@rt.cpan.org, or through the web interface at http://rt.cpan.org.

SEE ALSO

Merging of nested data structures:

  • Hash::Merge merge with global options

  • Data::Utilities (Data::Merger) merge with call-time options

  • Data::Nested merging (per application options), paths and schemas

  • Data::ModeMerge

    "Mode" (overwrite/add/default) is in the data merged, like Data::Overlay. Uses special characters to indicate the action performed. Also permits local config overrides and extensions.

Potential overlay builders, modules that could be used to build overlays for data:

  • CGI::Expand

    Build nested hashes from a flat path hash:

        use CGI::Expand qw(expand_hash);

    my $overlay = expand_hash({"a.b.c"=>1,"a.b.d"=>[2,3]})
    # = {'a' => {'b' => {'c' => 1,'d' => [1,2,3]}}};

    Hash::Flatten also has an unflatten function.

Lazy deep copying nested data:

Data structure differences:

autovivification can avoid nested accesses creating intermediate keys.

There some overlap between what this module is trying to do and both the darcs "theory of patches", and operational transforms. The overlap is mainly around composing changes, but there's nothing particularly concurrent about Data::Overlay. Also, patches and operations have more context and are invertible.

KEYWORDS

Merge, edit, overlay, clone, modify, transform, memory sharing, COW, operational transform, patch.

So an SEO expert walks into a bar, tavern, pub...

AUTHOR

Brad Bowman <cpan@bereft.net>

LICENCE AND COPYRIGHT

Copyright (c) 2011, Brad Bowman <cpan@bereft.net>. All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

DISCLAIMER OF WARRANTY

BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.