NAME
CAPE::Utils - A helpful library for with CAPE.
VERSION
Version 2.8.0
SYNOPSIS
Quick summary of what the module does.
Perhaps a little code snippet.
use CAPE::Utils;
my $cape_util=CAPE::Utils->new();
my $sub_results=$cape_util->submit(items=>@to_detonate,unique=>0, quiet=>1);
use JSON;
print encode_json($sub_results)."\n";METHODS
new
Initiates the object. One argument is taken and that is the path to the INI config file. The default is '/usr/local/etc/cape_utils.ini' and if not found, the defaults will be used.
my $cape_util=CAPE::Utils->new('/path/to/some/config.ini');connect
Return a DBH from DBI->connect for the CAPE SQL server.
This will die with the output from $DBI::errstr if it fails.
my $dbh = $cape->connect;fail
Set one or more pending tasks to failed as below.
UPDATE tasks SET status = 'failed_processing' WHERE status = 'pending'The following options are also supported.
- where :: Additional SQL where statements to add. Something must
be specified for this, unless fail_all in the config is
true. Otherwise this method will die.
my $rows=$cape_util->fail( where=>"target like '%foo%'");get_pending_count
Get pending count pending tasks.
- where :: And additional where part of the SQL statement. "and" will
automatically be used to join it with the rest of the
statement. May not contain a ';'.
- Default :: undef
my $count=$cape_util->get_pending_count;get_pending
Returns a arrah ref of hash refs of rows from the tasks table where the status is set to pending via "select * from tasks where status = 'pending'"
- where :: And additional where part of the SQL statement. "and" will
automatically be used to join it with the rest of the
statement. May not contain a ';'.
- Default :: undef
- where :: Additional SQL where statements to add.get_pending_table
Generates a ASCII table for pending.
The following config variables can are relevant to this and may be overriden.
table_border
table_color
pending_columns
pending_target_clip
pending_time_clipThe following options are also supported.
- where :: And additional where part of the SQL statement. "and" will
automatically be used to join it with the rest of the
statement. May not contain a ';'.
- Default :: undef
print $cape_util->get_pending_table( pending_columns=>'id,package');get_running
Returns a array ref of hash refs of rows from the tasks table where the status is set to pending.
select * from tasks where status = 'running'The statement above is used to find running tasks.
- where :: And additional where part of the SQL statement. "and" will
automatically be used to join it with the rest of the
statement. May not contain a ';'.
- Default :: undef
use Data::Dumper;
my $running=$cape_utils->get_running;
print Dumper($running);get_running_count
Get pending count running tasks.
select * from tasks where status = 'running'The statement above is used to find running tasks.
- where :: And additional where part of the SQL statement. "and" will
automatically be used to join it with the rest of the
statement. May not contain a ';'.
- Default :: undef
my $count=$cape_util->get_running_count;get_running_table
Generates a ASCII table for pending.
The following config variables can are relevant to this and may be overriden.
table_border
table_color
running_columns
running_target_clip
running_time_clipThe statement below is used to find running tasks.
select * from tasks where status = 'running'The following options are also supported.
- where :: And additional where part of the SQL statement. "and" will
automatically be used to join it with the rest of the
statement. May not contain a ';'.
- Default :: undef
print $cape_util->get_pending_table( pending_columns=>'id,package');get_tasks
Returns a array ref of hash refs of rows from the tasks table where the status is set to pending.
- where :: The where part of the SQL statement. May not contain a ';'.
- Default :: undef
- order :: Column to order by.
- Default :: id
- limit :: Number of items to return.
- Default :: 100
- direction :: Direction to order in.
- Default :: desc
use Data::Dumper;A small example showing getting running, ordering by category, and limiting to 20.
my $tasks=$cape_utils->get_tasks(where=>"status = 'running'", limit=>20, order=>"category", direction=>'desc');
print Dumper($running);get_tasks_count
Gets a count of tasks.
- where :: The where part of the SQL statement. May not contain a ';'.
- Default :: undef
use Data::Dumper;A small example showing getting running, ordering by category, and limiting to 20.
my $count=$cape_util->get_tasks_count(where=>"status = 'running'", limit=>20, order=>"category", direction=>'desc');get_tasks_table
Generates a ASCII table for tasks.
The following config variables can are relevant to this and may be overriden.
table_border
table_color
task_columns
task_target_clip
task_time_clipThe following options are also supported.
- where :: Additional SQL where statements to add.
- Default :: undef
- order :: Column to order by.
- Default :: id
- limit :: Number of items to return.
- Default :: 100
- direction :: Direction to order in.
- Default :: desc
print $cape_util->get_tasks_table( where => "status = 'reported'");munge
Munges the specified report file.
$cape_utils->munge(file=>$report_file);search
Searches the list of tasks. By default everything will be return ed.
- where :: Additional SQL where statements to use for searching.
May not contain a ';'.
- Default :: undefAddtionally there are also helpers for searching. These may not contain either a /\'/ or a /\\/. They will be joined via and.
The following are handled as a simple equality.
- timeout
- memory
- enforce_timeout
- timedoutThe following are numeric. Each one may accept multiple comma sperated values. The equalities =, >,>=, <=, and ! are supported. If no equality is specified, then = is used.
- id
- timeout
- priority
- dropped_files
- running_processes
- api_calls
- domains
- signatures_total
- signatures_alert
- files_written
- registry_keys_modified
- crash_issues
- anti_issues
- sample_id
- machine_id
# will result in id >= 3 and id < 44
id => '>=3,<44'
# either of these will be id = 4
id => '=4'
id => '4'The following are string items. As is, they are evaluated as a simple equality. If ending with ending in '_like', they will be evaluated as a like.
- target
- category
- custom
- machine
- package
- route
- tags_tasks
- options
- platform
# becomes... target = 'foo'
target => 'foo'
# becomes... target like 'foo%'
target_like => 'foo%'submit
Submits files to CAPE.
- clock :: Timestamp to use for setting the clock to of the VM for
when executing the item. If left undefined, it will be
autogenerated.
- Format :: mm-dd-yyy HH:MM:ss
- items :: A array ref of items to submit. If a directory is listed in
here, it will be read, but subdirectories will not be recursed. They
will be ignored.
- name_regex :: Regex to use for matching items in a submitted dir.
Only used if the a submitted item is a dir.
- Default :: undef
- mime_regex :: Array ref of desired mime types to match via
regex. Only used if the a submitted item is a dir.
- Default :: undef
- timeout :: Value to use for timeout. Set to 0 to not enforce.
- Default :: 200
- machine :: The machine to use for this. If not defined, first
available will be used.
- Default :: undef
- package :: Package to use, if not letting CAPE decide.
- Default :: undef
- options :: Option string to be passed via --options.
- Default :: undef
- random :: If it should randomize the order of submission.
- Default :: 1
- tags :: Tags to be passed to the script via --tags.
- Default :: undef
- platform :: What to pass to --platform.
- Default :: undef
- custom :: Any custom values to pass via --custom.
- Default :: undef
- enforce_timeout :: Force it to run the entire period.
- Default :: 0
- unique :: Only submit it if it is unique.
- Default :: 0
-quiet :: Do not print the results.
- Default :: 0The retuned value is a hash ref where the keys are successfully submitted files and values of those keys is the task ID.
my $sub_results=$cape_util->submit(items=>@to_detonate,unique=>0, quiet=>1);
use JSON;
print encode_json($sub_results)."\n";timestamp
Creates a timestamp to be used with utils/submit. localtime is used to get the current time.
print $cape_util->timestamp."\n";shuffle
Performa a Fisher Yates shuffle on the passed array ref.
check_remote
Checks the remote connection.
Two variablesare required, API key and IP.
$results=$cape_utils->check_remote(apikey=>$apikey, remote=>$remote_ip);
if (!$results){
print "unauthed\n";
return;
}eve_process
Process the finished tasks for CAPEv2.
$cape_utils->eve_process;CONFIG FILE
The default config file is '/usr/local/etc/cape_utils.ini'.
The defaults are as below, which out of the box, it will work by default with CAPEv2 in it's default config.
# The DBI dsn to use
dsn=dbi:Pg:dbname=cape
# DB user
user=cape
# DB password
pass=
# the install base for CAPEv2
base=/opt/CAPEv2/
# 0/1 if poetry should be used
poetry=1
# 0/1 if fail should be allowed to run with out a where statement
fail_all=0
# colums to use for pending table show
pending_columns=id,target,package,timeout,ET,route,options,clock,added_on
# colums to use for runniong table show
running_columns=id,target,package,timeout,ET,route,options,clock,added_on,started_on,machine
# colums to use for tasks table
task_columns=id,target,package,timeout,ET,route,options,clock,added_on,latest,machine,status
# if the target column for running table display should be clipped to the filename
running_target_clip=1
# if microseconds should be clipped from time for running table display
running_time_clip=1
# if the target column for pending table display should be clipped to the filename
pending_target_clip=1
# if microseconds should be clipped from time for pending table display
pending_time_clip=1
# if the target column for task table display should be clipped to the filename
task_target_clip=1
# if microseconds should be clipped from time for task table display
task_time_clip=1
# default table color
table_color=Text::ANSITable::Standard::NoGradation
# default table border
table_border=ASCII::None
# when submitting use now for the current time
set_clock_to_now=1
# default timeout value for submit
timeout=200
# default value for enforce timeout for submit
enforce_timeout=0
# how to auth for mojo_cape_submit
# ip = match against subnets
# apikey = use apikey
# both = require both to match
# either = either may work
auth=ip
# the api key to for with mojo_cape_submit
#apikey=
# comma seperated list of allowed subnets for mojo_cape_submit
subnets=192.168.0.0/16,127.0.0.1/8,::1/128,172.16.0.0/12,10.0.0.0/8
# incoming dir to use for mojo_cape_submit
incoming=/malware/client-incoming
# directory to store json data files for submissions recieved by mojo_cape_submit
# this directory is also used for storing run specific eves
incoming_json=/malware/incoming-json
# Location to write the eve log to.
eve=/opt/CAPEv2/log/eve.json
# how far to go back for processing eve
eve_look_back=360
# malscore for changing the event_type for eve from potential_malware_detonation to alert
malscore=0Report Munge Section
INI sections matching /^munge\_/ will be used for report munging. This requires two values for that sections, 'check' and 'munge'.
'check' is a path to a Perl script that will wrapped in a eval and require to check if the file should be munged or not.
'munge' is a path to a Perl script that will wrapped in a eval and require to do the munging.
Below is a example showing the setup for a single script.
[munge_pdf]
check=/usr/local/etc/cape_utils_munge/pdf_check
munge=/usr/local/etc/cape_utils_munge/pdf_mungeIf more than one munge section exists, they are ran in sorted order.
If the paths specied do not start with a '/', './', or '../', then '/usr/local/etc/cape_utils_munge/' is applied to the start.
The scripts are read as evaled strings.
The relevant variables are as below.
- $munge_it :: Perl boolean for if it should be munged or not. Should be set by the check script.
- $report :: The hash ref containing the parsed JSON report.
- $changed :: Perl boolean for if it changed or not.For some examples see the directory 'munge_examples'.
CAPEv2 lite.json to EVE handling
Tasks are found by looking back X number of seconds in the tasks table for tasks that have reported. The amount of time is determined by the config value 'eve_look_back'.
It will check if a task has been processed already or not be seeing if a task specified EVE JSON has been created under the 'incoming_json' directory. This is in the format $task_id.'eve.json'. If not, it will proceed.
It reads the 'lite.json' report for task as well as the incoming JSON. It then copies the keys 'signatures' and 'malscore' into the hash for the incoming JSON and writes it out to $task_id.'eve.json' and appending it to the file specified via the config value 'eve'.
The are two possible values for 'event_type', 'potential_malware_detonation' and 'alert'. 'potential_malware_detonation' is changed to alert when 'malscore' goves over the value specified via config value 'malscore'.
'row' is the full row for the task in question from the task table as a hash.
'signatures' is copied from '.signature' in the report JSON.
AUTHOR
Zane C. Bowers-Hadley, <vvelox at vvelox.net>
BUGS
Please report any bugs or feature requests to bug-cape-utils at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=CAPE-Utils. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
SUPPORT
You can find documentation for this module with the perldoc command.
perldoc CAPE::UtilsYou can also look for information at:
RT: CPAN's request tracker (report bugs here)
Search CPAN
Git
Web
ACKNOWLEDGEMENTS
LICENSE AND COPYRIGHT
This software is Copyright (c) 2022 by Zane C. Bowers-Hadley.
This is free software, licensed under:
The Artistic License 2.0 (GPL Compatible)