NAME

Data::Clean - Clean data structure

VERSION

This document describes version 0.508 of Data::Clean (from Perl distribution Data-Clean), released on 2022-08-28.

SYNOPSIS

 use Data::Clean;

 my $cleanser = Data::Clean->new(
     # specify how to deal with specific classes
     'DateTime'     => [call_method => 'epoch'], # replace object with its epoch
     'Time::Moment' => [call_method => 'epoch'], # replace object with its epoch
     'Regexp'       => ['stringify'], # replace $obj with "$obj"

     # specify how to deal with all scalar refs
     SCALAR         => ['deref_scalar'], # replace \1 with 1

     # specify how to deal with circular reference
     -circular      => ['clone'],

     # specify how to deal with all other kinds of objects
     -obj           => ['unbless'],

     # recurse into object
     #'!recurse_obj'=> 1,

     # generate cleaner with debugging messages
     #'!debug'      => 1,
 );

 # to get cleansed data
 my $cleansed_data = $cleanser->clone_and_clean($data);

 # to replace original data with cleansed one
 $cleanser->clean_in_place($data);

DESCRIPTION

This class can be used to process a data structure by replacing some forms of data items with other forms. One of the main uses is to clean "unsafe" data, e.g. clean a data structure so it can be encoded to JSON (see Data::Clean::ForJSON, which is a thin wrapper over this class).

As can be seen from the example, you specify a list of transformations to be done, and then this class will generate an appropriate Perl code to do the cleansing. This class is faster than the other ways of processing, e.g. Data::Rmap (see Bencher::Scenarios::DataCleansing for some benchmarks).

METHODS

new(%opts) => $obj

Create a new instance.

Options specify what to do with certain category of data. Option keys are either reference types (like HASH, ARRAY, SCALAR) or class names (like Foo::Bar), or -obj (to match all kinds of objects, a.k.a. blessed references), -circular (to match circular references), -ref (to refer to any kind of references, used to process references not handled by other options). Option values are arrayrefs, the first element of the array is command name, to specify what to do with the reference/class. The rest are command arguments.

Note that arrayrefs and hashrefs are always walked into, so it's not trapped by -ref.

Default for %opts: -ref => 'stringify'.

Option keys that start with ! are special:

• !recurse_obj (bool)

  Can be set to true to to recurse into objects if they are hash- or array-based. By default objects are not recursed into. Note that if you enable this option, object options (like Foo::Bar or -obj) won't work for hash- and array-based objects because they will be recursed instead.

• !clone_func (str)

  Set fully qualified name of clone function to use. The default is to get the value of the environment PERL_DATA_CLEAN_CLONE_FUNC or use the default Clone::PP::clone.

  The clone module (all but the last part of the !clone_func value) will automatically be loaded using require().

• !debug (bool)

  If set to true, will generate code to print debugging messages. For debugging only.

Available commands:

• ['stringify']

  This will stringify a reference like {} to something like HASH(0x135f998).

• ['replace_with_ref']

  This will replace a reference like {} with HASH.

• ['replace_with_str', STR]

  This will replace a reference like {} with STR.

• ['call_method' => STR]

  This will call a method named STR and use its return as the replacement. For example: DateTime->from_epoch(epoch=>1000) when processed with [call_method => 'epoch'] will become 1000.

• ['call_func', STR]

  This will call a function named STR with value as argument and use its return as the replacement.

• ['one_or_zero']

  This will perform $val ? 1:0.

• ['deref_scalar_one_or_zero']

  This will perform ${$val} ? 1:0.

• ['deref_scalar']

  This will replace a scalar reference like \1 with 1.

• ['unbless']

  This will perform unblessing using Function::Fallback::CoreOrPP::unbless(). Should be done only for objects (-obj).

• ['die']

  Die. Only for testing.

• ['code', STR]

  This will replace with STR treated as Perl code.

• ['clone', INT]

  This command is useful if you have circular references and want to expand/copy them. For example:

   my $def_opts = { opt1 => 'default', opt2 => 0 };
   my $users    = { alice => $def_opts, bob => $def_opts, charlie => $def_opts };

  $users contains three references to the same data structure. With the default behaviour of -circular => [replace_with_str => 'CIRCULAR'] the cleaned data structure will be:

   { alice   => { opt1 => 'default', opt2 => 0 },
     bob     => 'CIRCULAR',
     charlie => 'CIRCULAR' }

  But with -circular => ['clone'] option, the data structure will be cleaned to become (the $def_opts is cloned):

   { alice   => { opt1 => 'default', opt2 => 0 },
     bob     => { opt1 => 'default', opt2 => 0 },
     charlie => { opt1 => 'default', opt2 => 0 }, }

  The command argument specifies the number of references to clone as a limit (the default is 50), since a cyclical structure can lead to infinite cloning. Above this limit, the circular references will be replaced with a string "CIRCULAR". For example:

   my $a = [1]; push @$a, $a;

  With -circular => ['clone', 2] the data will be cleaned as:

   [1, [1, [1, "CIRCULAR"]]]

  With -circular => ['clone', 3] the data will be cleaned as:

   [1, [1, [1, [1, "CIRCULAR"]]]]

$obj->clean_in_place($data) => $cleaned

Clean $data. Modify data in-place.

$obj->clone_and_clean($data) => $cleaned

Clean $data. Clone $data first.

ENVIRONMENT

• PERL_DATA_CLEAN_CLONE_FUNC

  String. Set default for !clone_func option.

• LOG_CLEANSER_CODE => BOOL (default: 0)

  Can be enabled if you want to see the generated cleanser code. It is logged at level trace using Log::ger.

• LINENUM => BOOL (default: 1)

  When logging cleanser code, whether to give line numbers.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Data-Clean.

SOURCE

Source repository is at https://github.com/perlancar/perl-Data-Clean.

SEE ALSO

Related modules: Data::Rmap, Hash::Sanitize, Data::Walk.

AUTHOR

perlancar <perlancar@cpan.org>

CONTRIBUTING

To contribute, you can send patches by email/via RT, or send pull requests on GitHub.

Most of the time, you don't need to build the distribution yourself. You can simply modify the code, then test via:

 % prove -l

If you want to build the distribution (e.g. to try to install it locally on your system), you can install Dist::Zilla, Dist::Zilla::PluginBundle::Author::PERLANCAR, Pod::Weaver::PluginBundle::Author::PERLANCAR, and sometimes one or two other Dist::Zilla- and/or Pod::Weaver plugins. Any additional steps required beyond that are considered a bug and can be reported to me.

COPYRIGHT AND LICENSE

This software is copyright (c) 2022, 2020, 2019, 2018, 2017, 2016 by perlancar <perlancar@cpan.org>.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Data-Clean

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

cpanm Data::Clean

perl -MCPAN -e shell
install Data::Clean