NAME

grep-url - Print lines having URL(s) (optionally of certain criteria) in them

VERSION

This document describes version 0.004 of grep-url (from Perl distribution App-grep-url), released on 2021-08-01.

SYNOPSIS

Usage:

% grep-url [--color=str] [--count|-c] [--debug|--log-level=level|--quiet|--trace|--verbose] [(--file=filename)+] [--format=name|--json] [--host-contains=str] [--host-matches=re] [--host-not-contains=str] [--ignore-case|-i|--no-ignore-case|--noignore-case] [--invert-match|-v|--no-invert-match|--noinvert-match] [--line-number|-n] [--max-urls=int] [--min-urls=uint] [--(no)naked-res] [--page-result[=program]|--view-result[=program]] [--path-contains=str] [--path-matches=re] [--path-not-contains=str] [(--query-param-contains key=s)+] [(--query-param-matches key=s)+] [(--query-param-not-contains key=s)+] [--quiet-arg] [--scheme-contains=str] [--scheme-matches=re] [--scheme-not-contains=str] [(--scheme=str)+] [--files-json=json] [--query-param-contains-json=json] [--query-param-matches-json=json] [--query-param-not-contains-json=json] [--schemes-json=json] [-q|--silent] -- [file] ...

Examples:

Show lines that contain at least 2 URLs:

 % grep-url --min-urls 2 file.txt

Show lines that contain URLs from google:

 % grep-url --host-contains google file.txt

Show lines that contain search URLs from google where query contains the keyword "mortal":

 % grep-url --host-contains google --query-param-contains q=mortal file.txt

DESCRIPTION

This is a grep-like utility that greps for URLs of certain criteria.

OPTIONS

* marks required options.

Main options

--file=s@

Can also be specified as the 1st command-line argument and onwards.

Can be specified multiple times.

--files-json=s

See --file.

Can also be specified as the 1st command-line argument and onwards.

Filtering options

--max-urls=s

Default value:

 -1

--min-urls=s

Default value:

 1

General output control options

--color=s

Default value:

 "auto"

Valid values:

 ["never","always","auto"]

--count, -c

Supress normal output, return a count of matching lines.

--quiet-arg, --silent, -q

Logging options

--debug

Shortcut for --log-level=debug.

--log-level=s

Set log level.

--quiet

Shortcut for --log-level=error.

--trace

Shortcut for --log-level=trace.

--verbose

Shortcut for --log-level=info.

Matching control options

--ignore-case, -i

--invert-match, -v

Invert the sense of matching.

Output line prefix control options

--line-number, -n

Output options

--format=s

Choose output format, e.g. json, text.

Default value:

 undef

--json

Set output format to json.

--naked-res

When outputing as JSON, strip result envelope.

Default value:

 0

By default, when outputing as JSON, the full enveloped result is returned, e.g.:

    [200,"OK",[1,2,3],{"func.extra"=>4}]

The reason is so you can get the status (1st element), status message (2nd element) as well as result metadata/extra result (4th element) instead of just the result (3rd element). However, sometimes you want just the result, e.g. when you want to pipe the result for more post-processing. In this case you can use `--naked-res` so you just get:

    [1,2,3]

--page-result

Filter output through a pager.

--view-result

View output using a viewer.

Url criteria options

--host-contains=s

--host-matches=s

--host-not-contains=s

--path-contains=s

--path-matches=s

--path-not-contains=s

--query-param-contains-json=s

See --query-param-contains.

--query-param-contains=s%

Each value is a name-value pair, use key=value syntax. Can be specified multiple times.

--query-param-matches-json=s

See --query-param-matches.

--query-param-matches=s%

Each value is a name-value pair, use key=value syntax. Can be specified multiple times.

--query-param-not-contains-json=s

See --query-param-not-contains.

--query-param-not-contains=s%

Each value is a name-value pair, use key=value syntax. Can be specified multiple times.

--scheme-contains=s

--scheme-matches=s

--scheme-not-contains=s

--scheme=s@

Default value:

 ["http","file"]

Can be specified multiple times.

--schemes-json=s

See --scheme.

Other options

--help, -h, -?

Display help message and exit.

--version, -V

Display program's version and exit.

COMPLETION

This script has shell tab completion capability with support for several shells.

bash

To activate bash completion for this script, put:

 complete -C grep-url grep-url

in your bash startup (e.g. ~/.bashrc). Your next shell session will then recognize tab completion for the command. Or, you can also directly execute the line above in your shell to activate immediately.

It is recommended, however, that you install modules using cpanm-shcompgen which can activate shell completion for scripts immediately.

tcsh

To activate tcsh completion for this script, put:

 complete grep-url 'p/*/`grep-url`/'

in your tcsh startup (e.g. ~/.tcshrc). Your next shell session will then recognize tab completion for the command. Or, you can also directly execute the line above in your shell to activate immediately.

It is also recommended to install shcompgen (see above).

other shells

For fish and zsh, install shcompgen as described above.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/App-grep-url.

SOURCE

Source repository is at https://github.com/perlancar/perl-App-grep-url.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=App-grep-url

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

SEE ALSO

grep-email.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2021 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

cpanm App::grep::url

perl -MCPAN -e shell
install App::grep::url